A 214 FPS reality check for 3D multi-object tracking

What it does

AB3DMOT is a bare-bones 3D multi-object tracking baseline for autonomous driving and robotics. It takes oriented 3D bounding boxes from an off-the-shelf LiDAR detector such as PointRCNN and runs them through a 3D Kalman filter and the Hungarian algorithm for state estimation and data association. The authors also ship a 3D extension of the official KITTI 2D MOT evaluation protocol, including new metrics such as sAMOTA.

The interesting bit

The punchline is that this deliberately simple pipeline—explicitly inspired by the 2D SORT tracker—was state-of-the-art on KITTI 3D MOT at the time of publication, improving 3D MOTA from 72.23 to 76.47 while running at 214.7 FPS, roughly 65× faster than the state-of-the-art 2D MOT system at the time. It is a reminder that standard filtering methods can still set the pace when paired with a decent detector.

Key highlights

• Runs at 214.7 FPS using standard 3D Kalman filtering and Hungarian matching
• Achieved state-of-the-art 3D MOT results on KITTI (3D MOTA 76.47) using only off-the-shelf detectors
• Projects to 2nd place on the KITTI 2D MOT leaderboard when 3D tracks are flattened to the image plane
• Includes evaluation extensions and new metrics (e.g., sAMOTA) for the KITTI benchmark
• Added nuScenes support and refactored libraries in 2022; a separate ROS real-time fork exists

Caveats

• The 2D leaderboard ranking and 65× speedup claim compare 3D tracking against published 2D MOT systems, not an apples-to-apples comparison
• The tracker is not a detector: it requires an external 3D object detector to produce input bounding boxes
• Code was significantly refactored in 2022, so older issues or forks may not match the current structure

Verdict

Worth a look if you need a fast, interpretable 3D tracking baseline for LiDAR-based autonomy, or if you want to benchmark against classic filtering methods. Skip it if you are hunting for an end-to-end learned tracker with built-in detection.