Open-weight TTS that claims to beat the frontier — and streams in 40 ms

What it does Breeze TTS 2 is a bilingual (English/Chinese) text-to-speech model with open weights and a PyTorch inference stack. It does three things: clone a voice from reference audio plus its exact transcript, design a voice purely from a natural-language description, and direct a cloned voice’s tone, pace, and delivery with instructions. It also supports inline vocal events like (laugh) or [咳嗽] written directly into the text.

The interesting bit Voice Design needs no reference audio at all — you describe the speaker in plain language and CFG steering (--cfg-scale 4) makes the model follow it. The engineering is also unusually transparent: the batch API exposes a model_digest SHA-256 over the checkpoint, and CUDA Graph capture, bucketing, and memory behavior are documented in detail rather than hand-waved.

Key highlights

• Claims #1 among open-weight models on the Artificial Analysis TTS leaderboard, ahead of proprietary systems
• Under 40 ms time-to-first-audio and ~3.1× real-time streaming (0.32 RTF) on a warmed-up H100 fast path
• Eager inference fits in ~7.7 GiB; a 12 GB GPU is the stated minimum
• Batch API handles up to 128 texts per request, with CUDA Graph acceleration for CFG batches
• Streaming API serves mono 24 kHz PCM over HTTP, single-concurrency

Caveats

• The code is Apache 2.0, but the model weights, derivative models, and self-hosted outputs are research/non-commercial only — read the license before building anything
• The fast path needs ~14.4 GiB (24 GB GPU recommended) and adds cold-start warmup time
• Linux + NVIDIA GPU only; the tested environment is a Docker build targeting H100/A100

Verdict Worth a look if you want a local, low-latency, instruction-following TTS for English or Chinese — the latency numbers and voice design are genuinely interesting. Skip it if you need commercial use or non-CUDA hardware.