Text chat with AI is powerful, but it's still typing. The next wave is voice — talking to AI the way you'd talk to a friend, with real-time responses, natural intonation, and no typing delay. It's already partially here, and by mid-2027 it'll be the default way most people interact with AI. Here's the state of play.
What's Available Right Now
ChatGPT Voice Mode
OpenAI's Advanced Voice Mode is the current benchmark. The latency is 300-500ms (feels nearly real-time), the voices sound natural, and it handles interruptions well. The huge downside: it's censored. OpenAI's content filter is even more aggressive in voice mode than in text — it'll cut you off mid-sentence if the conversation drifts toward anything remotely sensitive. For general assistant tasks (cooking timers, brainstorming, trivia), it's excellent. For anything you'd need uncensored AI for, it's useless.
ElevenLabs
ElevenLabs isn't a chatbot — it's a text-to-speech API that produces the most natural-sounding AI voices available. Developers use it to add voice to their own apps. The quality is stunning — cloned voices are nearly indistinguishable from the original. At $5/mo for 30,000 characters (~30 minutes of audio), it's affordable for integration. The limitation is that it's a building block, not a product — you need to wire it up to an LLM yourself.
Character.AI Voice
Character.AI added voice chat for its characters. The implementation is decent — you can talk to your character and hear them respond in a chosen voice. The problem is the same as Character.AI's text mode: the content filter. If your conversation goes anywhere interesting, the AI clams up. Users report that voice mode is even more restrictive than text mode, presumably because voice interactions feel more "real" and Character.AI is terrified of liability.
SillyTavern + TTS Pipeline
The self-hosted option. SillyTavern supports TTS plugins — you can wire up ElevenLabs, Coqui, or XTTS as a voice backend. With a local LLM for text generation and a local TTS model for voice, you get a fully uncensored voice AI companion running on your own hardware. The setup is non-trivial (expect 4-8 hours for first-time configuration), latency depends on your hardware (1-3 seconds on a good GPU), and voice quality varies wildly depending on which TTS model you use. But it works, it's private, and no one can take it away.
What Makes Voice AI Hard
- Latency — Humans notice delays over 500ms in conversation. Getting an LLM to generate text, convert it to speech, and stream it back under 500ms is an engineering challenge.
- Interruption handling — Real conversations involve interruptions, overlapping speech, and mid-sentence corrections. Most voice AI handles this poorly.
- Emotional range — Text AI can describe emotions. Voice AI needs to convey them through tone, pacing, and inflection. Current TTS models are getting better but still sound flat during emotionally complex exchanges.
- Cost — Running a TTS model alongside an LLM doubles your compute costs. For cloud providers, this means voice mode is expensive to offer.
- Censorship — Voice interactions feel more "real" to both users and regulators. Every platform tightens its content filter when voice is involved.
What to Expect in 2027
The pieces are falling into place. Open-source TTS models (Kokoro, XTTS-v2, Kitten-TTS) are approaching ElevenLabs quality. Streaming LLM inference is getting faster. The hardware costs are dropping. Within a year, expect to see uncensored voice AI companions that sound indistinguishable from a real person, respond in under 500ms, and run either locally on a decent GPU or through a cloud service that doesn't censor the conversation.
Aether is building toward this. Our current focus is text and image — getting those right before adding voice. But the architecture supports it, and the demand from our users is clear. When we ship voice, it'll be uncensored like everything else we do.
I want to build a voice AI companion using open-source tools. Walk me through the full stack — which LLM, which TTS model, how to wire them together, and what hardware I need. Give me actual model names and config, not vague suggestions.Open this in Aether →
The Voice AI Landscape at a Glance
- Best quality (censored): ChatGPT Advanced Voice Mode — 300ms latency, natural voices, heavy content filter
- Best TTS engine: ElevenLabs — $5/mo, stunning quality, API-only (not a chatbot)
- Best open-source TTS: Kokoro / Kitten-TTS 2 — free, runs locally, approaching ElevenLabs quality
- Best uncensored (self-hosted): SillyTavern + local LLM + XTTS — full control, 4-8hr setup
- Best uncensored (cloud): Coming soon — Aether voice is on the roadmap