Open-source TTS that takes stage directions

What it does Parler-TTS generates speech from text using transformer models guided by natural language descriptions—think “a female speaker with moderate pitch and very clear audio.” It ships inference and training code alongside two public checkpoints (880M and 2.3B parameters) trained on 45k hours of audiobook data. The project reproduces research from Stability AI and Edinburgh University, but unlike the original paper, releases everything—weights, datasets, and preprocessing—under a permissive license.

The interesting bit You do not need a reference audio clip to clone a voice; you prompt the model like a casting director, specifying gender, speaking rate, reverberation, or choosing one of 34 named speakers baked into the checkpoint. The repository is also designed to work with the companion Data-Speech tool for annotating custom datasets, so you can build and fine-tune your own voices from scratch.

Key highlights

• Fully open-source release: model weights, training data, preprocessing scripts, and training recipes are all public.
• Two ready-to-use checkpoints: Mini (880M parameters) and Large (2.3B parameters).
• Inference optimizations include SDPA, Flash Attention 2, and torch.compile support.
• 34 named speakers available for consistent voice generation across sessions.
• Includes training and fine-tuning guides, plus a Colab notebook for single-speaker datasets.

Caveats

• Apple Silicon users currently need a nightly PyTorch build for bfloat16 support.
• PEFT/LoRA fine-tuning and multilingual training are listed as future work, not yet implemented.

Verdict Worth a look if you want an open, hackable TTS pipeline where voice control is text-first rather than audio-first. Skip it if you need a drop-in multilingual solution or immediate LoRA fine-tuning out of the box.