Faster sparse convolutions by refusing one-size-fits-all dataflows

What it does

TorchSparse is a PyTorch library for high-performance point cloud processing. It provides specialized GPU kernels for sparse 3D convolutions—the irregular, scattered operations common in LiDAR and autonomous driving pipelines. The library targets both inference and mixed-precision training, with benchmarks reporting measured speedups over MinkowskiEngine, SpConv, and its own earlier MLSys release.

The interesting bit

Most point cloud deep learning libraries commit to one dataflow for an entire model. TorchSparse systematically analyzes different dataflows and switches between them during execution, which the authors credit for up to 2.9× inference speedups on an NVIDIA A100. It also ships with plugins for MMDetection3D and OpenPCDet.

Key highlights

• Claims up to 2.9×, 3.3×, 2.2×, and 1.7× end-to-end inference speedup on an A100 over MinkowskiEngine, SpConv 1.2, TorchSparse-MLSys, and SpConv v2 respectively.
• Training benchmarks report 1.16× speedup on A100 and 1.27× on RTX 2080 Ti over SpConv 2.3.5, and 4.6–4.8× over MinkowskiEngine.
• Supports FP16, TF32, and FP32 across Pascal, Turing, and Ampere GPUs, plus a 1.25× latency improvement over SpConv 2.3.5 on Jetson Orin.
• Provides pre-built Linux wheels and plugins for MMDetection3D and OpenPCDet.

Caveats

• Pre-built wheels are only provided for Linux; other platforms build from source.
• The authors cannot publicly release raw benchmark data from SemanticKITTI, nuScenes, or Waymo due to license requirements.
• The README notes that their custom PyPI server may not work as expected, with manual wheel downloads as the fallback.

Verdict Worth evaluating if you are training or deploying sparse 3D convolutions for LiDAR and need more throughput than MinkowskiEngine or SpConv provide. If your workloads are dense 2D images or you prefer to avoid CUDA version matching, look elsewhere.