One C++ binary that speaks, listens, and knows when to do which
speech.cpp packs TTS, ASR, VAD, diarization and speech enhancement into a single ggml-based executable with an OpenAI-compatible server — no Python, no CUDA.

What it does
It’s one command, speech, and one shared library behind a C API, running speech models on Metal, Vulkan or the CPU via ggml. It synthesizes speech (Qwen3-TTS, Irodori-TTS, Drama 3), transcribes recordings or the live microphone (Qwen3-ASR, ReazonSpeech, Parakeet, Nemotron), detects speech regions with Silero VAD, diarizes up to eight speakers, and even removes noise with the RE-USE models. speech serve exposes OpenAI’s audio API over HTTP, so existing OpenAI clients work by pointing them at localhost.
The interesting bit
The whole voice pipeline — the boring glue that usually lives in a Python dependency swamp — is a single self-contained executable that fetches GGUF models by name and streams audio from the first frame. The README claims time-to-first-audio of 0.029 s for Qwen3-TTS 0.6B on an Apple M5, with the rest generated while it plays. Every model stage is checked against its official implementation from the same inputs, which is more diligence than most conversion projects admit to.
Key highlights
- One binary, no Python, no CUDA; runs on macOS (Metal), Windows/Linux (Vulkan or CPU), and even WebAssembly in the browser
- OpenAI-compatible audio API plus Realtime transcription over HTTP, with a built-in web page to try models
- Live microphone transcription and
speech talk, a spoken conversation with a model that listens and speaks simultaneously - Broad model catalog: 83-language multi-speaker synthesis (Drama 3), hour-long verbatim transcription (Nemotron 3.5), voice cloning from reference recordings
- A worker mode (
speech worker) that keeps models loaded between requests for other programs
Caveats
- The headline models are heavy: Drama 3 is a 20.8 GB file, Nemotron Transcribe 19 GB, Haru 10.56 GB — the small Qwen3 models are the practical entry point
- Haru-4B is described as a research base model that “breaks easily”
- Linux needs glibc 2.34+; Intel Macs require building from source
Verdict
Anyone building voice features who’s tired of stitching whisper.cpp, a VAD library and a TTS server together should look hard at this. If you need CUDA or training, it’s not that project — and the big models demand serious disk space.
Frequently asked
- What is nyosegawa/speech.cpp?
- speech.cpp packs TTS, ASR, VAD, diarization and speech enhancement into a single ggml-based executable with an OpenAI-compatible server — no Python, no CUDA.
- Is speech.cpp open source?
- Yes — nyosegawa/speech.cpp is open source, released under the MIT license.
- What language is speech.cpp written in?
- nyosegawa/speech.cpp is primarily written in C++.
- How popular is speech.cpp?
- nyosegawa/speech.cpp has 512 stars on GitHub.
- Where can I find speech.cpp?
- nyosegawa/speech.cpp is on GitHub at https://github.com/nyosegawa/speech.cpp.