AMD's stealth play to own local AI inference

What it does

Lemonade is a local AI inference server with two modes: a system service that exposes OpenAI, Anthropic, and Ollama-compatible APIs, and an embeddable binary you can ship inside your own app. It runs LLMs, image generation, speech synthesis, and transcription across CPU, GPU, and NPU — with explicit optimizations for AMD’s Ryzen AI NPUs, Radeon GPUs, and the new Strix Halo APUs.

The interesting bit

The project is community-built but AMD engineers contribute the performance-critical paths. That makes it something of a reference implementation disguised as grassroots software — AMD gets to tune for its own silicon without maintaining a closed SDK. The vllm backend for Strix Halo is already marked experimental, suggesting they’re using this project to iterate on next-chip support before silicon ships broadly.

Key highlights

• Multi-modal out of the box: text (llamacpp, vllm, FLM), images (Stable Diffusion), speech (Whisper, Kokoro)
• Hardware abstraction across Vulkan, ROCm, CUDA, Metal, and XDNA2 NPU backends
• Model manager with lemonade pull for GGUF, FLM, and ONNX formats plus Hugging Face imports
• Packaged for every platform short of BSD: Windows MSI, macOS pkg, Linux debs/rpms/Snaps/PPAs, Docker
• Mobile companion apps on iOS and Android

Caveats

• The README’s supported configurations table cuts off mid-row for speech-to-text OS support, so NPU whisper on Linux is unclear
• vllm backend is experimental and locked to Strix Halo iGPU only
• AMD hardware gets first-class tuning; Intel and NVIDIA users get functional but not optimized paths

Verdict

Worth a look if you’re running recent AMD hardware and want a drop-in local replacement for OpenAI API calls. Skip it if you’re on Intel graphics or need battle-tested production SLAs — this is still moving fast.