Open-vocabulary detection that actually hits 100 FPS

What it does

OmDet-Turbo is a transformer-based object detector that recognizes arbitrary categories from text prompts—open-vocabulary detection—without fine-tuning on a fixed class list. The project provides pretrained weights, ONNX export, and both offline and API-server inference modes. It has also been upstreamed into Hugging Face Transformers.

The interesting bit

Instead of pushing all multimodal work through the heavy encoder, the authors designed an Efficient Fusion Head that handles vision-language blending and ROI computation more cheaply. The Base model reportedly runs at 100.2 FPS on an A100 while scoring 30.1 AP on ODinW and 26.86 AP on OVDEval in zero-shot settings.

Key highlights

• Speed claims: 100.2 FPS on an A100 for the Base model (COCO val2017), and 140 FPS under TensorRT for the Tiny variant.
• Zero-shot performance: Base achieves 30.1 AP on ODinW and 26.86 AP on OVDEval without task-specific training.
• Distribution: integrated into Hugging Face Transformers 4.45.0; weights hosted on Hugging Face and ModelScope.
• Deployment traits: supports ONNX export, though the default export omits post-processing and locks input sizes.
• Inference modes: includes a local demo script and a WSGI API server out of the box.

Caveats

• The README covers inference and export but offers no training code or dataset setup guidance.
• The Model Zoo table only lists the Tiny variant, even though several benchmark claims reference a larger Base model.
• ONNX conversion requires manual config edits and leaves post-processing behind, so production graphs need extra engineering.

Verdict

Good fit for developers who need fast, text-guided detection and are happy to use the provided checkpoints. Less useful if you want to train from scratch or need a fully baked ONNX pipeline without manual assembly.