A 26B-parameter model that fits in a base-model Mac

What it does

TurboFieldfare is a custom Swift and Metal inference engine built specifically for Google’s Gemma 4 26B-A4B. Instead of loading the full 14.3 GB model into memory, it keeps a 1.35 GB shared core and the KV cache resident, then pulls individual experts from SSD only when the router needs them. The result is a 26-billion-parameter model that generates tokens on an 8 GB M2 MacBook Air.

The interesting bit

This is not a wrapper around MLX or llama.cpp; it is a model-specific runtime with its own kernels, caching, and streaming installer. The author documented 103 measured experiments tuning kernels, I/O, prefill, and decode. The installer itself is careful: it streams byte ranges from Hugging Face and repacks them directly into a local .gturbo layout without ever materializing a second full checkpoint on disk.

Key highlights

• Runs in ~2 GB of RAM with ~14.3 GB of local storage; decode hits 5.1–6.3 tok/s on an 8 GB M2 Air and 31–35 tok/s on an M5 Pro.
• Ships as a native Mac app, CLI, and an experimental loopback OpenAI-compatible server.
• Uses a 16-slot LFU expert cache per layer; the CPU plans reads while Metal computes the shared-expert branch.
• Prefill works in chunks up to 128 tokens to amortize the cost of fetching experts.
• Requires macOS 26, Metal 4, Swift 6.2, and an Apple Silicon Mac; x86 and older OS versions are not supported.

Caveats

• Text-only: no image, audio, or video support. The loopback server accepts tool declarations but the client must execute them.
• The project is tightly coupled to one model architecture; do not expect it to load your favorite Llama or Qwen checkpoint.
• Measured speeds are reference points, not ceilings; throughput depends on prompt length, page-cache state, and hardware.

Verdict

Owners of base-model Apple Silicon Macs who want to run a modern 26B-parameter model locally should look here. If you need a general-purpose inference server or multimodal support, this is not your runtime.