NVIDIA’s SegFormer: five transformer sizes, one research-only license

What it does

SegFormer classifies every pixel in an image—road, sky, building—using a transformer-based encoder-decoder. This repository contains the official PyTorch implementation, training and evaluation scripts, and pretrained weights for the five model sizes (B0 to B5) described in the NeurIPS 2021 paper. The entire release is layered on top of MMSegmentation v0.13.0, so it inherits that framework’s data pipelines and multi-GPU orchestration.

The interesting bit

The authors did not rebuild the training pipeline from scratch. By building on MMSegmentation v0.13.0, they inherit data handling and distributed training and focus on releasing their graded family of pretrained transformers and ImageNet-1K backbones.

Key highlights

• Five model variants (B0–B5) offering a speed-accuracy trade-off, though the README does not quantify exact latency or memory figures.
• Pretrained weights available for both the segmentation models and the ImageNet-1K backbones via Google Drive and OneDrive links.
• Supports standard academic benchmarks including ADE20K and Cityscapes.
• Ships with a single-image demo script for quick qualitative tests.
• NVIDIA Research non-commercial license: research and evaluation only, which means no production deployment without a separate business inquiry.

Caveats

• The documented example environment relies on CUDA 10.1, PyTorch 1.7.1, and mmcv-full==1.2.7, which are several generations behind current releases.
• Model weights are distributed through Google Drive and OneDrive folders rather than a versioned artifact store.
• The NVIDIA Source Code License restricts usage to research and evaluation; commercial deployment requires a separate business inquiry.

Verdict

Computer-vision researchers studying transformer-based segmentation or benchmarking on ADE20K and Cityscapes should grab this. Practitioners looking for a commercially permissive, drop-in segmentation model should keep scrolling.