A local voice stack that mimics OpenAI’s API down to the endpoint

What it does Speaches is a self-hosted server that exposes OpenAI-compatible endpoints for speech-to-text, text-to-speech, and real-time audio generation. It bundles faster-whisper for transcription and translation, and pipes output through the kokoro or piper TTS engines. The idea is to let you swap OpenAI’s voice API for a container on your own hardware without touching client code.

The interesting bit The project borrows Ollama’s playbook for LLMs but applies it to voice: dynamic model loading and unloading means you can request any supported model by name and the server will fetch it, then evict it after idle time. That laziness saves GPU memory without forcing you to manage model lifecycles manually.

Key highlights

• OpenAI API compatibility across transcription, translation, chat completions, and the realtime audio endpoint.
• Streaming transcription over SSE, so partial results arrive before the audio finishes.
• TTS via kokoro—which the README notes ranks first on the Hugging Face TTS Arena—and the lighter piper engine.
• GPU and CPU support, deployable as Docker containers.
• Dynamic model loading: specify a model in the request and the server handles the rest.

Caveats

• The streaming transcription demo is marked TODO, so that feature may be less documented or polished than the realtime API.
• README typos and an apologetic note about demo audio quality suggest the project is moving fast and presentation still trails function.

Verdict Developers who want a private, drop-in replacement for OpenAI’s speech endpoints without retooling their clients should look here. If you need managed multi-tenant scaling or enterprise SLAs, you’re still better off with the cloud API.