The Whisper diet: 49% smaller, 6× faster

What it does

Distil-Whisper is a family of student models trained to mimic OpenAI’s Whisper for English automatic speech recognition. The checkpoints handle both short clips under 30 seconds and long-form audio, and they plug directly into the Hugging Face Transformers library. It is essentially a drop-in replacement for Whisper when you only need English and would rather not burn GPU cycles.

The interesting bit

The project applies knowledge distillation to a large speech-to-text encoder-decoder, shrinking the model by 49% and boosting relative latency by up to 6.8× while staying within 1% word error rate of the teacher on out-of-distribution sets. The latest distil-large-v3 checkpoint is also compatible with OpenAI’s own sequential long-form algorithm, so it can slot into existing buffered-transcription pipelines without rewriting your inference strategy.

Key highlights

• 49% smaller and up to 6.8× faster than Whisper large-v3, per the README benchmark table.
• distil-large-v3 is the recommended default; distil-small.en drops to 166M parameters for on-device or mobile use, performing within 4% WER of the full-size model.
• Supports both short-form and long-form English audio, with two long-form algorithms: sequential (accuracy-focused, compatible with OpenAI) and chunked (latency-focused).
• Training code and the distillation paper are included in the repository.
• Runs inside the standard Hugging Face transformers pipeline using safetensors and half-precision loading.

Caveats

• English only. The README explicitly states there is no multilingual support and points users to OpenAI’s Whisper Turbo for other languages.
• Short-form accuracy is slightly worse than the original, as shown in the benchmark table (e.g., 9.7% vs. 8.4% WER for the large-v3 counterpart).
• Long-form transcription requires choosing between sequential and chunked algorithms, which have different latency and accuracy trade-offs.

Verdict

Worth a look if you are shipping English-only transcription at scale or squeezing ASR onto edge hardware. Skip it if your users speak anything other than English.