NVIDIA PAIR: A Load Balancer for Your Home GPU Cluster

What it does

NVIDIA PAIR is a local inference router that discovers machines on your network and distributes independent LLM requests among them. It exposes Ollama-compatible and OpenAI-compatible endpoints, so any client that speaks those APIs can treat a cluster of home machines as one backend. The idea is to keep prompts and responses on the local network while letting concurrent workloads share the load instead of stacking up on a single device.

The interesting bit

The real insight is what PAIR does not try to do: it does not pool GPU memory, shard a model across nodes, or split one request between machines. It is strictly a request-level router, which keeps it out of the distributed-systems weeds and means it simply forwards each job to whichever node already holds the model and has the headroom.

Key highlights

• Exposes standard Ollama and OpenAI proxy endpoints to any local client or agent
• Routes by engine availability, model presence, and current workload across the cluster
• Supports mixed clusters of Windows, Linux, and macOS on x64 and arm64
• Can install and manage Ollama and LM Studio engines, or connect to existing ones
• Pairs new machines into the cluster with a six-digit PIN

Caveats

• It routes whole requests only: no GPU memory pooling, no model sharding across machines, no splitting an in-flight inference between nodes
• A machine running PAIR is not guaranteed to run an inference engine; GPU, driver, and memory requirements are still dictated by each engine and model
• Windows on ARM support is currently experimental

Verdict

Worth installing if you have multiple home machines and want a single local endpoint for multi-agent or concurrent LLM workloads. Skip it if you need to run a model larger than one machine can hold, because PAIR will not shard it across the cluster.