Who spoke when? This toolkit actually knows.

What it does

pyannote.audio figures out who is speaking when in an audio recording. It handles the whole messy pipeline: detecting speech, spotting speaker changes, finding overlapping talkers, and clustering voices into distinct speakers. Everything ships as pretrained PyTorch models you can run locally or finetune on your own data.

The interesting bit

The project straddles open source and commercial API without being sneaky about it. The community-1 pipeline runs entirely offline with a Hugging Face token, while precision-2 offloads to pyannoteAI servers—or self-hosts if you want the speedup. Benchmarks are refreshingly specific: diarization error rates across a dozen datasets, plus wall-clock seconds per hour of audio on an H100.

Key highlights

• Pretrained pipelines and models hosted on Hugging Face, ready to use
• Local GPU inference via Pipeline.from_pretrained() with progress hooks
• Multi-GPU training support through pytorch-lightning
• Telemetry is opt-in, anonymized, and granularly configurable
• Extensive tutorial and video archive, though some tutorials lag behind current versions

Caveats

• Community tutorials note they “should be updated” for newer versions
• Local pipeline requires accepting Hugging Face model terms and token setup
• Premium precision-2 needs separate pyannoteAI API key

Verdict

Worth a look if you need speaker diarization in a Python stack and want pretrained models that actually benchmark well. Skip if you’re expecting a polished GUI or turnkey deployment without token wrangling.