Zig-Powered LLM Server for Mac Ditches Python and Electron

What it does

mlx-serve is a single Zig binary that runs MLX and GGUF models natively on Apple Silicon. It exposes OpenAI-compatible, Anthropic-compatible, and Ollama-compatible HTTP APIs from the same localhost:11234 port, so existing clients like Claude Code, the OpenAI SDK, and Open WebUI can repoint without reconfiguration. Beyond text, the bundled MLX Core macOS app also handles image generation, video, music, speech with voice cloning, and 3D model output.

The interesting bit

Most local inference stacks are built in Python or wrapped in Electron; mlx-serve does neither, opting for Zig and a signed native menu-bar app. It also bundles a full generative media pipeline—diffusion, video, audio, and 3D—alongside its LLM engine, as if determined to replace an entire cloud API suite from a single Mac menu bar.

Key highlights

• Geomean decode speed is claimed to be 26% faster than LM Studio on identical MLX weights, with speculative decoding and continuous batching.
• Serves OpenAI (/v1/chat/completions), Anthropic Messages (/v1/messages), and Ollama (/api/chat) wire protocols from a single endpoint.
• MLX Core app provides native chat, agent mode with MCP tool calling, and resumable model downloads—no Electron, no Python runtime.
• Runs both MLX weights and GGUF models via embedded llama.cpp, including DeepSeek V4 Flash through a dedicated engine.
• LAN model sharing over Bonjour lets other Macs on the network proxy inference to one machine’s local cache.

Caveats

• Apple Silicon only; Intel Macs and other platforms are not supported.

Verdict

A strong choice for Mac developers who want a fast, API-compatible local inference stack with a native GUI. Everyone else should admire it from afar.