A 431 KB Windows Whisper Port That Ditches CUDA for Direct3D

What it does

This is a Windows-specific implementation of OpenAI’s Whisper automatic speech recognition model. It transcribes audio files or live microphone input by running inference on your GPU through Direct3D 11 compute shaders, shipping as a tiny native DLL with an optional desktop GUI.

The interesting bit

The author skipped the usual CUDA and PyTorch stack entirely, opting for Windows’ built-in DirectCompute to make GPU acceleration vendor-agnostic on Windows. On the author’s test hardware, this cut a 3-minute transcription from 45 seconds down to 19 while shrinking the runtime footprint from 9.63 GB of dependencies to a 431 KB DLL.

Key highlights

• Runs on any Direct3D 11.0-capable GPU, which in practice means nearly any discrete or integrated GPU from 2011 onward
• Includes WhisperDesktop.exe for file transcription and live audio capture with built-in voice activity detection
• Exposes a COM-style API, plus a C# NuGet wrapper and PowerShell 5.1 scripting support
• Ships with a built-in profiler that measures execution time of individual compute shaders
• Mixed F16/F32 precision with low overall memory usage

Caveats

• Realtime microphone capture carries 5–10 seconds of latency because the model struggles with short audio segments
• Automatic language detection is not implemented
• Performance is specifically tuned for a handful of tested cards (e.g., GTX 1080Ti, some AMD APUs); discrete AMD and Intel integrated GPUs may not perform as well

Verdict

Grab it if you want offline speech-to-text on Windows without installing a Python ML stack. Avoid it if you need low-latency live transcription, automatic language detection, or a non-Windows OS.