Field wiki for running frontier LLMs on PCIe Blackwell without NVLink

What it does This repository is a community field wiki for serving massive language models—GLM-5, DeepSeek-V4-Flash, Kimi, Qwen3.8-27B—on NVIDIA RTX PRO 6000 Blackwell GPUs strung together over PCIe without NVLink. It collects exact Docker builds, vLLM and SGLang runbooks, benchmark tables, quantization fidelity checks using KLD divergence, and hardware topology notes for specific server configurations. Think of it as a lab notebook that happens to be executable.

The interesting bit What sets it apart from typical model cards is its almost forensic obsession with reproducibility: every page locks down exact Docker image tags, source commits, model snapshot IDs, tensor-parallelism layouts, and client commands so another builder on a 4-GPU or 16-GPU PCIe-switch system can replicate the result without tribal knowledge. The community also preserves regression history, so old runbooks are kept rather than overwritten.

Key highlights

• Covers frontier models including GLM-5.3-Flash, DeepSeek-V4-Flash, Kimi-K2.7-Code, and Qwen3.8-27B with specific quantization paths such as NVFP4, MXFP8, and FP8.
• Hardware focus on RTX PRO 6000 Blackwell (SM120, 96 GB GDDR7, PCIe 5.0 x16) across 4/8/16-GPU PCIe-switch topologies from ASUS and ASRockRack.
• Debugging guides for PCIe oneshot all-reduce, NCCL tuning, speculative decoding variants (MTP, DSpark, DFlash), and Blackwell power-limit sweeps.
• Quantization sanity checks via KLD methodology, plus verifier-backed behavioral fidelity comparisons for quantized against BF16 outputs.
• Maintained from live Discord experiments with explicit newcomer onboarding rules to keep technical signal high.

Caveats

• The documentation is dense with acronyms and assumes familiarity with vLLM/SGLang internals; newcomers will need the glossary.
• Most content targets a very specific GPU (RTX PRO 6000 Blackwell), so applicability to other hardware is limited.
• Pages are versioned and historical, meaning the latest answer may be scattered across multiple runbook revisions.

Verdict Anyone building a local inference cluster on RTX PRO 6000 Blackwell PCIe hardware should bookmark this; if you are not running vLLM or SGLang on SM120 GPUs, it is probably too specific to be useful.