DETR's queries were just boxes in disguise

What it does

DAB-DETR rethinks the object queries in Facebook’s DETR transformer detector. Instead of learned embeddings, it feeds actual bounding-box coordinates into the decoder and refines them layer by layer. This gives the model explicit spatial priors and cuts the notoriously long training schedule.

The interesting bit

The authors frame the decoder queries as performing soft ROI pooling in a cascade—essentially treating attention as a differentiable region-proposal refinement loop. Box width and height even modulate the positional attention maps directly.

Key highlights

• Hits 45.7 AP on MS-COCO (ResNet50-DC5, 50 epochs), claiming top spot among DETR-like models under the same setting.
• Also provides DAB-Deformable-DETR, a multiscale variant that reaches 48.7 AP.
• Ships with pretrained weights, an inference visualization notebook, and a full model-zoo table.
• Built directly on the original DETR codebase, so the dependencies are familiar.

Caveats

• The authors themselves now point users to their newer detrex toolbox and DINO (63.3 AP) for state-of-the-art results, making this repo partly a historical reference.
• DAB-Deformable-DETR requires manually compiling CUDA operators for deformable attention.
• The README is mostly a quickstart guide and model zoo; architectural details require reading the paper.

Verdict

Good for researchers tracing the evolution of transformer detectors or reproducing the ICLR 2022 baseline. Skip it if you just need the best off-the-shelf model; the authors’ own DINO and detrex have already surpassed it.