Your Ryzen AI NPU finally gets an Ollama of its own

What it does FastFlowLM is a lightweight runtime that runs large language models, vision models, audio processing, and embeddings entirely on AMD Ryzen AI NPUs with XDNA2 architecture—specifically Strix, Strix Halo, Kraken, and Gorgon Point chips. It exposes a CLI and local server with REST and OpenAI-compatible APIs, keeping the GPU and CPU free for other work. The README claims over 10× better power efficiency than GPU-based inference and supports context lengths up to 256k tokens.

The interesting bit At just 17 MB, the runtime installs quickly, but the real curiosity is its split personality: the orchestration layer and CLI are open-source MIT, while the actual NPU kernels are proprietary binaries free only for companies with annual revenue under $10 million. It is essentially an Ollama-shaped interface over AMD-specific IRON and AIE-MLIR kernels, which is a pragmatic way to unlock silicon that most inference stacks ignore.

Key highlights

• Runs fully on AMD Ryzen AI NPUs (XDNA2), leaving the GPU and CPU idle
• Supports LLMs, vision models, audio, embeddings, and MoE architectures
• Claims over 10× power efficiency versus GPU inference and context windows up to 256k tokens
• 17 MB runtime with Windows and Linux support
• Open-source CLI and orchestration (MIT), but NPU kernels are proprietary closed binaries

Caveats

• Requires specific NPU driver versions (>= 32.0.203.304, with .311 recommended) and downloads optimized kernels from HuggingFace, which can occasionally corrupt and may be inaccessible in some regions
• Free-tier users must include a “Powered by FastFlowLM” attribution in their projects
• Only supports Ryzen AI Series chips with XDNA2 NPUs, so older AMD hardware is incompatible

Verdict Worth a look if you own a recent AMD Ryzen AI laptop and want local inference without waking the GPU. Everyone else—Intel users, desktop Ryzen owners, and open-source purists who flinch at proprietary kernel blobs—can safely keep walking.