A CLI that makes Whisper devour hours of audio in seconds

What it does

insanely-fast-whisper is a command-line tool that runs OpenAI’s Whisper speech-recognition model entirely on your own hardware. It wraps Hugging Face Transformers, Optimum, and Flash Attention 2 into a single utility that transcribes or translates audio files, with optional speaker diarization via pyannote.audio. The goal is to squeeze maximum throughput out of local GPUs without touching a cloud API.

The interesting bit

The project began as a benchmarking demo for the Transformers library and accidentally became a community-driven CLI. Rather than rewriting Whisper, it gains speed by layering well-known optimizations—fp16, batching, and Flash Attention 2—on top of the standard model. The README claims this stack can transcribe 150 minutes of audio in under 98 seconds on an Nvidia A100.

Key highlights

• On-device only: runs on NVIDIA GPUs or Apple Silicon Macs (mps), keeping audio data local.
• Optional diarization: can label speakers and constrain the expected speaker count with min/max limits.
• Flexible timestamps: outputs either chunk-level or word-level timestamps.
• Model swaps: defaults to Whisper Large v3 but supports Distil-Whisper variants via a flag.
• Published benchmarks: includes A100 comparisons against Faster Whisper and various optimization combinations.

Caveats

• Windows support is shaky; the README flags an unknown-root-cause CUDA error and offers only a manual workaround.
• Apple Silicon users face higher memory pressure than on CUDA, often needing to reduce batch size to avoid OOMs.
• The CLI is explicitly “highly opinionated” and limited to NVIDIA GPUs and Macs.

Verdict

Try it if you routinely batch-process long audio locally and want a pre-tuned, optimization-heavy pipeline. Look elsewhere if you need a polished cross-platform GUI or hassle-free Windows GPU support.