OpenAI-compatible TTS you can run on a CPU, Jetson, or gaming GPU

What it does Kokoro-FastAPI wraps the Kokoro-82M text-to-speech model in a FastAPI service with prebuilt Docker images. It mimics OpenAI’s speech endpoint, so existing client libraries drop in without changes. The project targets CPU, NVIDIA CUDA, and experimental AMD ROCm on both x86_64 and arm64, including Jetson and GH200 boards.

The interesting bit The server can blend multiple voices using weighted ratios like af_bella(2)+af_sky(1), automatically normalize the mix, and save the resulting voicepack for reuse. It also offers phoneme-level endpoints and per-word timestamped captions, which are unusual conveniences for a lightweight wrapper.

Key highlights

• OpenAI-compatible speech endpoint with support for English, Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, and Mandarin Chinese.
• Weighted voice combination with automatic normalization and persistent .pt voicepacks.
• Prebuilt Docker images for CPU, NVIDIA CUDA (including Blackwell/RTX 50-series), and experimental AMD ROCm on both x86_64 and arm64.
• Streaming output with first-token latency around 300 ms on GPU and under one second on an Apple M3 Pro CPU, though an older i7 can take roughly 3.5 seconds.
• Phoneme endpoints let you generate phonemes from text or synthesize audio directly from phonemes.

Caveats

• AMD ROCm support is experimental and limited to x86_64; the README does not claim it is production-ready.
• Apple Silicon MPS acceleration requires running natively via UV; the Docker image is CPU-only on Macs.
• The project leans on espeak-ng as a fallback for unknown words, but the README notes results “have varied.”

Verdict A solid choice for developers who want a private, self-hosted TTS endpoint that plugs into existing OpenAI client code or frontends like SillyTavern and OpenWebUI. Less compelling if you are looking for a fully managed, hardware-agnostic TTS platform rather than a model-specific wrapper.