Neural networks that read 3D point clouds without rasterizing

What it does

PointNet trains deep neural networks to classify and segment 3D geometry represented as point clouds—unordered sets of coordinates—without first converting them into regular voxel grids or image collections. The repository provides training and evaluation pipelines for object classification on ModelNet40 and part segmentation on ShapeNet Part, plus utilities for visualizing misclassified point clouds. It is the reference implementation accompanying the CVPR 2017 paper.

The interesting bit

The architecture treats a point cloud as an unordered set and respects permutation invariance, so shuffling the input points does not change the result. By consuming raw points directly, it sidesteps the “unnecessarily voluminous” representations and associated issues that come with forcing point clouds into regular 3D voxel grids or multi-view images.

Key highlights

• Directly consumes point clouds as input, respecting permutation invariance rather than relying on rasterized proxies.
• Provides unified training code for both 3D object classification and part segmentation tasks.
• Automatically downloads and prepares the ModelNet40 dataset (416 MB) and supports ShapeNet Part segmentation.
• Includes visualization tools that render misclassified point clouds into three-view images for error analysis.
• Released under MIT license; the paper spawned a family of follow-ups including PointNet++ and Frustum PointNets.

Caveats

• The code targets a legacy stack tested only on Python 2.7, TensorFlow 1.0.1, CUDA 8.0, and cuDNN 5.1 on Ubuntu 14.04, so modern environments may need adaptation.
• The README points to a third-party PyTorch implementation, as this repository is TensorFlow-native.

Verdict

Study this if you are researching geometric deep learning or building 3D perception pipelines, since it is the original architecture that consumes point clouds directly. Skip it if you need a modern, drop-in production framework; treat it as reference code rather than an actively maintained library.