vLLM on Ascend NPUs, no fork required

What it does vllm-ascend is a hardware plugin that ports the vLLM inference engine to Huawei Ascend NPUs. It implements the upstream hardware-pluggable interface so Transformer, MoE, embedding, and multi-modal models can run on Atlas A2, A3, and 300I Duo hardware without modifying vLLM core.

The interesting bit Instead of maintaining a fork, the project lives as a decoupled plugin repository that tracks vLLM on its own schedule. The main branch follows vLLM upstream, while release branches like releases/v0.18.0 pin to specific vLLM stable tags, letting Ascend support evolve independently.

Key highlights

• Supports popular open-source model families on Ascend instead of NVIDIA GPUs
• Targets Atlas 800I A2 and A3 inference and training series, with experimental 300I Duo support
• Requires a rigid software stack: CANN 9.0.0, PyTorch 2.10.0, and matching torch-npu and vLLM versions
• Maintains stable branches alongside vLLM releases, plus feature branches for collaboration
• Governed by an active community with weekly meetings, user forums, and public meetups

Caveats

• Dependency versions are strict; Python must be between 3.10 and 3.11, and CANN, PyTorch, and torch-npu must match exactly
• Atlas 300I Duo support is explicitly marked experimental
• Older dev branches from v0.7.1 through v0.11.0 are unmaintained or frozen to bug fixes only

Verdict Essential if you already run Huawei Ascend hardware and want vLLM’s serving features. Skip it if you are not in the Ascend ecosystem; this is strictly a hardware bridge, not a general improvement to vLLM.