Vision transformer that routes attention in two stages

What it does BiFormer is a vision transformer built around Bi-Level Routing Attention (BRA). The repository supplies ImageNet-1K checkpoints and training code for classification, object detection, and semantic segmentation. It leans on the timm library and borrows structural patterns from ConvNeXt and UniFormer.

The interesting bit The authors candidly note that the released object detection results surpass the paper’s reported numbers thanks to a bug fix. They have refactored the core BRA module for readability while keeping a legacy implementation around so older checkpoints do not break. A pending wishlist for CUDA and cutlass optimizations suggests the current code is still a research work-in-progress.

Key highlights

• ImageNet-1K models from Tiny to Base, plus an Swin-Tiny-Layout ablation variant
• Detection and segmentation code released alongside classification
• Core BRA module refactored for readability; legacy path preserved for backward compatibility
• Built on timm, ConvNeXt, and UniFormer foundations
• MIT licensed

Caveats

• Several roadmap items remain unfinished, including token-labeling code, a visualization demo, and a fused CUDA implementation
• The original paper’s object detection results were under-reported due to a bug since fixed
• A planned Triton optimization was abandoned because of an upstream Triton issue

Verdict Worth a look if you are researching hierarchical attention mechanisms in vision and want a recent baseline to extend. Skip it if you need a polished, production-ready vision backbone without research-code rough edges.