Self-hosted Copilot for the GPU-rich and support-poor

What it does FauxPilot mimics GitHub Copilot by serving code completions from SalesForce CodeGen models running inside NVIDIA’s Triton Inference Server with the FasterTransformer backend. It exposes an OpenAI-compatible API and works with the Copilot plugin, letting your editor talk to a server you control. A setup script handles downloading and converting the models from Hugging Face into the right format.

The interesting bit The project treats multi-GPU setups as first-class citizens: VRAM requirements are totals, not per-card limits, so you can split a 6B model across two RTX 3080s. It is a refreshingly practical admission that local AI often requires hardware gymnastics.

Key highlights

• Runs fully offline on local NVIDIA GPUs (Compute Capability ≥ 6.0)
• Compatible with existing Copilot plugins and OpenAI API clients
• Supports model sharding across multiple GPUs to fit larger models
• Uses the SalesForce CodeGen family via Triton Inference Server and FasterTransformer
• Ships with a self-aware “lmao” support policy and minimal documentation

Caveats

• Requires a full NVIDIA stack: compatible GPU, nvidia-docker, Docker, and enough VRAM for your chosen model
• No formal support or warranty; the authors are upfront that help is community-only
• README client instructions are vague, pointing to external docs without detail

Verdict Worth a look if you already own high-end NVIDIA GPUs and want to keep code completion local. Skip it if you were hoping for a managed service or AMD support.