Llama-3B, but it speaks in full paragraphs

What it does

Orpheus TTS is a text-to-speech system built on a Llama-3B backbone. It generates speech by treating audio synthesis as a language modeling task, and claims natural intonation and rhythm superior to closed-source alternatives. The project offers pretrained and finetuned English models plus a research preview of multilingual variants.

The interesting bit

Instead of dedicated acoustic models, Orpheus uses an LLM to predict speech tokens, which yields emergent prosody and the ability to insert verbal tics like laughter or sighs via simple tags such as <laugh> or <sigh>. Zero-shot voice cloning works by conditioning the pretrained model on text-speech pairs in the prompt, though the README notes reliability improves with more examples.

Key highlights

• Built on Llama-3B and trained on 100k+ hours of English speech data
• Supports emotive tags including <laugh>, <chuckle>, <sigh>, <yawn>, and <gasp>
• Claims ~200ms streaming latency, reducible to ~100ms with input streaming
• Research preview of multilingual models (7 language pairs) alongside English production models
• Fine-tuning supported with as few as ~50 examples per speaker, with best results around 300

Caveats

• The realtime streaming package has a known glitch that occasionally skips frames, per the project’s checklist
• The PyPI release currently lags behind the repository; users may need the local package to avoid KV cache errors
• Zero-shot voice cloning with the pretrained model requires multiple prompt pairs for reliable results, and the voice cloning Colab is marked as needing a fix

Verdict

Try it if you want an open-source TTS engine that can sigh, laugh, and clone voices from a handful of samples. Look elsewhere if you need guaranteed frame-perfect streaming or a mature packaging pipeline.