Learning 3D geometry matches from RGB-D leftovers

What it does

3DMatch is a learned local geometric descriptor that turns volumetric patches of point clouds, depth maps, or meshes into feature vectors for registration and keypoint matching. The toolbox provides a reference C++/CUDA implementation, pre-trained weights, training code built on the Marvin framework, and utilities for converting data into Truncated Distance Function voxel grids. It also bundles TSDF fusion and evaluation scripts for end-to-end geometric alignment.

The interesting bit

The training strategy harvests correspondence pairs automatically from existing RGB-D reconstructions, turning publicly available scan datasets into supervision without manual labeling. This lets the ConvNet learn descriptors robust to the noise and incompleteness that break traditional histogram-based methods.

Key highlights

• Learns a volumetric patch descriptor using a Siamese network architecture.
• Includes pre-trained weights and a demo that aligns point clouds with RANSAC.
• Provides CUDA utilities for TDF voxel grid generation from meshes, depth maps, and point clouds.
• Ships with a Marvin-based training pipeline and a custom CUDA data layer for sampling RGB-D correspondences.
• Authors report generalization across tasks like surface correspondence and object model alignment.

Caveats

• The codebase is anchored to a 2017 stack: CUDA 7.5, cuDNN 5.1, Ubuntu 14.04, and Matlab 2015b.
• The demo reads only a simple binary point cloud format, so running it on custom data requires modifying demo.cu.
• The authors note that the demo alignment can fail on the first try because keypoints are selected randomly by default.

Verdict

Worth a look if you are studying classical 3D deep learning or need a baseline for local geometric descriptors on legacy RGB-D data. Skip it if you want a modern, plug-and-play point cloud network built on contemporary frameworks.