A one-stream transformer tracker that trains in a day

What it does

OSTrack is a single-stream visual object tracker built on a Vision Transformer. Instead of the classic Siamese approach that pushes template and search frames through the backbone separately, it concatenates them into one input sequence. The model jointly learns features and models target relations through self-attention, then scores candidate locations. It runs without extra temporal context and claims SOTA numbers on LaSOT, GOT-10k, TrackingNet, and UAV123.

The interesting bit

The real win is the Early Candidate Elimination (ECE) module, which prunes dud locations early so memory and compute don’t scale with the full search grid. Combined with MAE-pretrained ViT weights, the whole thing trains in about a day on four V100s—unusually fast for a transformer tracker.

Key highlights

• One-stream architecture: template and search region share a single forward pass rather than two separate ones.
• Trains OSTrack-256 in roughly 24 hours on 4× V100 GPUs; inference is also sped up by the tight, parallelized structure.
• Early Candidate Elimination reduces memory and time consumption during training and inference.
• SOTA benchmark numbers on LaSOT, GOT-10k, TrackingNet, and UAV123 without relying on temporal information.
• Won the VOT-2022 STb (box GT) and RTb challenges; also available via Modelscope for quick demos.

Caveats

• The repo is essentially a research artifact: training and evaluation scripts assume you have the full LaSOT, GOT-10k, COCO, and TrackingNet datasets sitting in exactly the expected folder layout.
• Speed claims in the paper were measured on an RTX 2080 Ti, so your modern GPU mileage may vary.

Verdict

Worth a look if you’re doing visual tracking research and need a strong, fast-training baseline to hack on. Skip it if you just want a drop-in tracking API without managing datasets and checkpoints.