Give Your Text LLM Ears and a Mouth

What it does

Unmute is a self-hosted voice interface for text LLMs. It pipes real-time user audio through Kyutai’s speech-to-text model, feeds the transcription to an LLM of your choice, and streams the response through Kyutai’s text-to-speech. The whole loop—listen, think, speak—happens via WebSocket with enough speed that it feels conversational.

The interesting bit

The project treats voice not as an afterthought but as infrastructure: it splits STT, LLM, and TTS across separate GPUs to shave latency, dropping TTS response time from roughly 750 ms on a single GPU to about 450 ms. It is also deliberately model-agnostic—the backend speaks to any OpenAI-compatible server, so you can swap in local VLLM, OpenRouter, or Ollama without touching the voice stack.

Key highlights

• Runs fully offline with local models (default is Gemma 3 1B) or plugs into remote APIs like OpenRouter.
• Multi-service architecture (frontend, backend, STT, LLM, TTS) deployable via Docker Compose or manually.
• Optimized for low-latency real-time conversation; STT handles end-of-speech detection to trigger LLM inference.
• Supports custom voices and characters through a voices.yaml config and includes a community voice-donation repository.
• Requires an NVIDIA GPU with at least 16 GB VRAM and runs on Linux or WSL; macOS and native Windows are explicitly unsupported.

Caveats

• No HTTPS in the default Docker Compose or dockerless setups; you will need to BYO TLS or use Docker Swarm.
• The README warns that debugging multi-node Docker Swarm deployments is “hard” and Kyutai will not help you with it.
• Voice and prompt config files are cached at backend startup, so changes require a restart.

Verdict

Worth a look if you want a self-hosted, low-latency voice assistant and have the GPU hardware to spare. Skip it if you are on macOS, running native Windows, or hoping for a single-binary install.