Your gaming PC as an elastic cluster for 290B-parameter MoEs

What it does

FreeToken is a local inference engine built to serve massive open-weight Mixture-of-Experts models—290B+ parameters—on consumer hardware. It treats GPUs, CPUs, host memory, and interconnects as a single elastic platform, shipping as both a desktop GUI and an API server compatible with Anthropic and OpenAI specs. The target use case is interactive chat and agentic tool-calling with frontier models like DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2.

The interesting bit

Instead of assuming everything lives in VRAM, the engine uses a bandwidth-adaptive $q^ ext{star}$ policy to co-execute layers across CPU and GPU, double-buffering prefill streams and caching experts globally. It also checkpoints semantic anchors so that when an agent edits context—injecting a tool result or a thinking block—the system avoids recomputing the entire KV cache from scratch.

Key highlights

• Targets RTX 30/40/50-series GPUs and scales across laptops, desktops, and workstations.
• Dynamic runtime VRAM re-allocation between expert caches and KV memory without engine restarts.
• Semantic anchor checkpoints for recurrent state and KV caches to skip redundant recomputation.
• Supports MXFP4, NVFP4, FP8, and BF16 quantization formats.
• Anthropic/OpenAI-compatible APIs for integration with agents like Codex, Claude Code, and OpenClaw.

Verdict

Grab it if you are trying to run frontier MoEs locally and need more sophistication than a simple download-and-run wrapper. Pass if your models already fit comfortably in vLLM or llama.cpp without cross-device orchestration.