NeRF with a depth sensor: fewer floaters, more walls

What it does

Takes an RGB-D image sequence—poses, focal length, and filtered depth maps—and reconstructs a dense 3D surface by blending implicit surface representations with a neural radiance field. It builds directly on Mildenhall et al.’s original NeRF code, adding depth supervision and a Marching Cubes stage to pull out explicit meshes instead of just volumetric color. The authors ship synthetic and real benchmark datasets, plus evaluation meshes, so you can compare outputs against ground truth.

The interesting bit

Most NeRF variants optimize for pretty pictures; this one optimizes for actual walls and floors. It uses BundleFusion-style bilateral-filtered depth maps to anchor the radiance field to real measurements, producing room-scale surface meshes rather than just viewable volumes.

Key highlights

• Marries implicit surface representations with NeRF, using RGB-D sequences rather than RGB alone.
• Expects a rigid data layout—filtered depth, poses in OpenGL convention, and a focal length file—but includes template loaders for ScanNet and custom datasets.
• Ships with a 7.25 GB benchmark dataset and pre-generated evaluation meshes for direct comparison.
• Extracts final geometry via an external Marching Cubes implementation adapted from prior work.

Caveats

• The dataloader is hard-coded to a depth_filtered folder; raw depth works but degrades results, and you will need to reproduce the BundleFusion bilateral filter pipeline if your data lacks it.
• Pose conventions matter: ScanNet data must be manually transformed to OpenGL coordinates before use.
• The codebase is largely a research fork of the original NeRF repository, so expect inherited architectural quirks rather than a clean-slate production library.

Verdict

Grab this if you are reproducing the CVPR paper or benchmarking RGB-D surface reconstruction methods. Skip it if you need a plug-and-play 3D scanning app—this is a research artifact with rigid data expectations.