ASR that runs offline from iPhone to POWER9

What it does whisper.cpp is a plain C/C++ inference engine for OpenAI’s Whisper automatic speech recognition model. It swaps the original Python/PyTorch stack for a self-contained implementation that delegates tensor math to the companion ggml library. The idea is to embed speech-to-text in applications without dragging along a runtime or dependencies.

The interesting bit Apple Silicon gets the VIP treatment—Metal, Core ML, and the Neural Engine are all wired in—but the project also goes to bat for oddball hardware like POWER9/10, Ascend NPUs, and Moore Threads GPUs. It is aggressively portable, running in a browser via WebAssembly or on an iPhone fully offline.

Key highlights

• Zero runtime memory allocations, mixed F16/F32 precision, and integer quantization keep the footprint small enough for phones and Raspberry Pis.
• Broad hardware acceleration: Metal on Apple Silicon, Vulkan, NVIDIA, AMD ROCm, OpenVINO, and even Moore Threads GPUs.
• Ships with a C-style API and bindings for Java, plus examples for iOS, Android, and WebAssembly.
• Core ML encoder offload can exceed 3× CPU speed on Apple Silicon, though the first run incurs a compilation delay on the Neural Engine.

Caveats

• The bundled CLI example only accepts 16-bit WAV input, so plan on converting audio beforehand.
• Core ML and OpenVINO integrations need companion encoder model files, and the first run on a device is slow while the accelerator compiles the graph.
• Older macOS versions may experience transcription hallucination when using Core ML; the README recommends macOS Sonoma or newer.

Verdict Grab this if you need offline, embeddable speech-to-text across a zoo of devices and accelerators. Skip it if you are content running Whisper in a Python environment and do not care about edge deployment or binary size.