Your own LLM cluster from spare GPUs and Macs

What it does

Parallax is a decentralized inference framework that treats a scattered set of nodes—Linux boxes with GPUs, Apple Silicon Macs, whatever you have—as one logical cluster for serving LLMs. It handles pipeline-parallel sharding, request routing, and KV-cache management across those nodes, even if they are in different physical locations with different specs. The project wraps existing engines (SGLang and vLLM for GPUs, MLX LM for Macs) and ties them together with its own P2P layer called Lattica.

The interesting bit

Rather than demanding a rack of identical A100s, Parallax leans into heterogeneity: it will apparently pipeline-shard a model across whatever mix of hardware you have handy, including Macs via MLX. That is a different bet from most serving stacks, which assume a uniform datacenter fleet.

Key highlights

• Supports a wide range of open weights: DeepSeek, Qwen, Kimi, MiniMax, GLM, Meta Llama, and OpenAI’s gpt-oss.
• GPU backend uses SGLang and vLLM; Mac backend uses MLX LM, with paged KV cache and continuous batching on Apple Silicon.
• P2P communication is handled by Lattica, Gradient’s own networking layer.
• Dynamic request scheduling and routing across the cluster.
• Pitched for both Linux/WSL GPU hosts and macOS.

Caveats

• The project is at version 0.0.1, so expect early-software sharp edges.
• The README pitches cross-location nodes but does not explain how network latency or security boundaries are handled.

Verdict

Worth a look if you have a pile of mismatched machines—Macs, consumer GPUs, remote boxes—and want to serve large models without renting a cloud cluster. Skip it if you need a battle-tested, production-grade inference platform today; this is clearly early-stage tooling.