Teaching cameras depth perception without a ruler

What it does SfMLearner trains neural networks to predict single-view depth and camera ego-motion from nothing but unlabeled monocular video. It learns by synthesizing neighboring frames from predicted depth maps and estimated camera poses, then penalizing photometric error—effectively forcing the model to explain the video through 3D geometry. The repository includes training and evaluation code for the KITTI and Cityscapes datasets, along with pre-trained checkpoints.

The interesting bit The supervision signal is entirely self-generated: correct depth and pose should allow perfect frame reconstruction, so photometric inconsistency becomes the loss function. The authors later found that removing batch normalization and adding data augmentation improved KITTI depth results beyond the original paper, even without the explainability regularization or extra Cityscapes data.

Key highlights

• Implements the CVPR 2017 Oral paper that helped pioneer unsupervised monocular depth estimation.
• Jointly learns depth prediction and visual odometry from raw video sequences.
• Provides pre-trained KITTI models and evaluation scripts that reproduce paper metrics (Abs Rel 0.183 on the Eigen test split).
• Reports pose estimation drift of roughly 0.016 on KITTI odometry sequence 09 and 0.013 on sequence 10.
• A community PyTorch reimplementation exists for those avoiding legacy TensorFlow.

Caveats

• The codebase was developed for TensorFlow 1.0, CUDA 8.0, and Ubuntu 16.04, so expect some dependency archaeology on modern systems.
• The README presents a faithful paper implementation; those wanting active maintenance or a modern framework should look to the community PyTorch port.

Verdict Researchers studying self-supervised depth or chasing historical baselines will find a clean reference. Practitioners needing a modern, drop-in depth-estimation pipeline should keep scrolling.