Whisper, but actually fast: a drop-in speed demon

What it does

faster-whisper is a Python reimplementation of OpenAI’s Whisper speech-to-text model that swaps the original inference stack for CTranslate2, a dedicated fast inference engine for Transformers. It supports GPU and CPU execution, 8-bit quantization, batched inference, and word-level timestamps. The project also bundles a VAD filter (Silero) to skip silent audio segments and works with distilled Whisper variants.

The interesting bit

The speedup isn’t from a new model architecture—it’s from aggressively optimized inference. The benchmarks claim a large-v2 model on GPU drops from 2m23s to 1m03s versus OpenAI’s implementation, and batched int8 inference hits 16 seconds for 13 minutes of audio. On CPU it’s more modest, but int8 quantization still roughly halves memory use. The project has become infrastructure: WhisperX, speaches, and a half-dozen other tools build on it.

Key highlights

• Up to 4× faster than openai/whisper at the same accuracy, per project benchmarks
• 8-bit quantization on both CPU and GPU for lower memory footprint
• Batched inference pipeline (BatchedInferencePipeline) for throughput over latency
• No system FFmpeg required—audio decoding via bundled PyAV libraries
• Drop-in API compatibility with the original Whisper model interface
• Active ecosystem: used by WhisperX, whisper-ctranslate2, speaches, and others

Caveats

• GPU setup requires specific NVIDIA library versions (CUDA 12 + cuDNN 9); mismatches force awkward ctranslate2 downgrades
• CPU performance without batching or quantization is actually slower than whisper.cpp in the project’s own benchmarks
• The generator-based transcribe() return means transcription doesn’t start until you iterate—easy to footgun

Verdict

If you’re already running Whisper in Python and need more throughput without touching C++, this is the pragmatic upgrade. If you’re on CPU-only and want absolute minimal dependencies, whisper.cpp still wins on simplicity and single-thread speed.