A 3D Detection Garage That Borrows 300 2D Models

What it does

MMDetection3D is a PyTorch toolbox that bundles 3D object detection for both indoor scenes and outdoor autonomous driving. It wraps datasets like KITTI, nuScenes, Waymo, ScanNet, and SUN RGB-D into a common training framework, letting you swap between LiDAR-only, camera-only, and multi-modal detectors without rewriting data loaders.

The interesting bit

The project’s real leverage comes from its family ties: because it sits inside the OpenMMLab ecosystem, it inherits the entire MMDetection model zoo. That means 2D backbones and modules plug directly into 3D heads, saving you from writing the usual adapter code. The README also claims it trains faster than reference implementations—VoteNet reportedly runs at 358 samples/sec here versus 77 in the official repo.

Key highlights

• Supports single- and multi-modal detectors out of the box, including PointPillars, MVXNet, and VoteNet.
• Covers major indoor and outdoor benchmarks: KITTI, nuScenes, Waymo, Lyft, ScanNet, SUN RGB-D, and SemanticKITTI.
• Reuses 300+ models and methods from 40+ papers in MMDetection, letting 2D components drop into 3D pipelines.
• Self-reported benchmarks show higher training throughput than OpenPCDet, VoteNet, and Det3D for several architectures.
• Recent releases add monocular and BEV 3D detection on Waymo, plus a mini Waymo dataset for quick iteration.

Caveats

• Several newer methods—such as DSVT, Nerf-Det, and BEVFusion—currently live in the projects folder, suggesting they are experimental or less tightly integrated than core models.
• The efficiency comparisons are internal benchmarks against specific competing repos; your hardware and configuration will affect actual speed.
• The framework is a research toolbox, not a turnkey product, so expect to spend time with config files and dataset preparation guides.

Verdict

Researchers and engineers building perception stacks for robotics or autonomous driving should look here first to avoid reimplementing baselines. If you just need a quick off-the-shelf inference script without the OpenMMLab config ceremony, this is probably too much toolbox for the job.