Google's speaker diarization clustering, rebuilt in Python

What it does

SpectralCluster re-implements, in Python, the spectral clustering algorithms from Google’s speaker diarization research — the ICASSP 2018 LSTM diarization paper, the Turn-to-Diarize constrained clustering paper, and the multi-stage streaming clustering paper. You feed it an embedding matrix (typically speaker embeddings) and get cluster labels back via SpectralClusterer.predict(). It’s the boring math between “we extracted embeddings” and “we know who spoke when” — which is exactly where diarization systems live or die.

The interesting bit

The refinement sequence applied to the affinity matrix before eigendecomposition is what made the 2018 paper work, and it’s fully exposed here: Gaussian blur, row-wise thresholding, and the quirky CropDiagonal trick that mimics a Laplacian while being more blur-tolerant. There’s also auto-tuning of the p_percentile threshold, constrained clustering driven by speaker-turn scores, and a MultiStageClusterer whose streaming_predict revises earlier labels as each new embedding arrives — corrections included.

Key highlights

• Four Laplacian variants (affinity, unnormalized, graph cut, random walk) plus pluggable affinity functions and K-Means distances (cosine by default, any scipy distance works)
• Constrained spectral clustering via affinity integration or constraint propagation from speaker-turn scores
• Streaming multi-stage clustering with a Hungarian-based Deflicker to stop labels flapping between frames
• Fallback clusterers for degenerate cases (too few embeddings, single-cluster decisions)
• max_spectral_size caps the Laplacian for speed by pre-clustering hierarchically — at some quality cost

Caveats

• The author is explicit: this is a demonstration, not a reproduction — not the original C++ code, not a Google product, and some features may be missing or incomplete
• Speaker turn detection itself is not implemented; the multi-stage code covers only fallback, main, pre-clusterer, and dynamic compression
• The API changed when the newer algorithms landed; anyone on the old interface is pointed back to version 0.1.0

Verdict

If you’re building diarization or any embedding-clustering pipeline and want to try the exact techniques from Google’s papers without reading C++, this is the reference. If you just need generic clustering, scikit-learn already has you covered.