Whisper glue that actually stays stuck

What it does

transcribe-anything is a frontend that feeds local files or YouTube/Rumble URLs into OpenAI Whisper and spits out subtitles, raw text, and optionally a speaker.json file that groups transcript chunks by who was talking. It runs each backend in an isolated environment so dependency hell stays theoretical.

The interesting bit

The project is basically well-organized glue — but it’s honest glue. It auto-detects hardware and dispatches to insanely-fast-whisper on CUDA, lightning-whisper-mlx on Apple Silicon, or vanilla Whisper on CPU. The speaker.json output is genuinely unusual: it de-chunkifies diarized output into speaker-labeled paragraphs with timestamps, which the README claims no other Whisper frontend does.

Key highlights

• Hardware dispatch: --device insane for CUDA, --device mlx for Mac ARM, --device cpu as fallback
• Speaker diarization via pyannote.audio behind Hugging Face token gating; produces speaker.json with speaker switches and timestamps
• Optional subtitle burn-in with --embed
• Python API exposed alongside CLI
• Docker image available (niteris/transcribe-anything)
• Sister project transcribe-everything for batch/remote-path operations

Caveats

• --device insane with large-v3 + batching is experimental: README warns of repeated text patterns and misaligned SRT/VTT timestamps
• insane mode is memory-hungry; a 12 GB NVIDIA 3060 OOMs on large content without --batch-size 8 or smaller
• distil-whisper/distil-large-v2 is explicitly blacklisted by the author for “extremely bad stuttering” and non-deterministic output
• Diarization requires agreeing to pyannote.audio licensing on Hugging Face; otherwise you get runtime exceptions

Verdict

Worth a look if you need transcription with speaker labels and want someone else to manage the backend zoo. Skip it if you already have a working Whisper pipeline or need production-grade reliability on the fast paths — the README itself flags those as experimental.