A DINOv2 detector that tops YOLO on COCO accuracy and latency charts

What it does

RF-DETR is a real-time object detection and instance segmentation architecture built on a DINOv2 vision transformer backbone. It packages models from Nano to 2XLarge behind a single Python API, aiming for state-of-the-art trade-offs between accuracy and latency on standard benchmarks like Microsoft COCO and RF100-VL. The project ships pretrained weights and a rfdetr package that handles both bounding-box detection and instance segmentation.

The interesting bit

The README reads like a scorecard: exhaustive benchmark tables pit RF-DETR against YOLO11, YOLO26, LW-DETR, and D-FINE on an NVIDIA T4, showing the model frequently landing higher on the accuracy-latency frontier. The twist is a licensing split: the core open-source package and smaller models wear Apache 2.0, while the top-tier XL and 2XL detection weights sit under a separate PML 1.0 license in the rfdetr_plus package.

Key highlights

• Claims SOTA results on COCO and RF100-VL for both detection and instance segmentation, with latency measured on TensorRT/FP16 on an NVIDIA T4.
• Uses a DINOv2 vision transformer backbone across all model sizes, from 30.5M parameters (Nano, 384×384) up to 126.9M (2XL, 880×880).
• Offers a unified API for detection and segmentation; swapping model sizes is a class-name change in Python.
• Open-source detection and segmentation models are Apache 2.0, but the XL/2XL detection checkpoints require the separately licensed rfdetr_plus under PML 1.0.
• Requires Python ≥ 3.10.

Caveats

• The top-performing XL and 2XL detection models are not Apache 2.0; they fall under PML 1.0 via the rfdetr_plus package, which may complicate commercial use.
• Benchmarks are self-reported by Roboflow and reference a separate reproducibility repo (roboflow/sab) rather than detailing full methodology in the README.

Verdict

Computer-vision engineers tired of YOLO’s licensing or accuracy ceilings should evaluate this, particularly if they need instance segmentation from the same model family. If you require the largest detection weights under a fully open-source license, the PML 1.0 split is a dealbreaker.