SLAM built from nothing but Gaussian splats

What it does

MonoGS performs dense simultaneous localization and mapping using 3D Gaussian Splatting as both its tracking and rendering backbone. Fed with a monocular video stream—and optionally stereo or RGB-D—it estimates camera trajectory while growing a photorealistic Gaussian map that can be explored in a live GUI or evaluated headlessly.

The interesting bit

Instead of maintaining separate map and tracking representations, the system treats the Gaussian scene as the map and differentiates through the rasterizer to backpropagate pose gradients. That lets the camera track itself against the same splats the user sees rendered.

Key highlights

• CVPR 2024 Highlight (and Best Demo Award winner) from Imperial College London
• First monocular SLAM pipeline built purely on 3D Gaussian Splatting
• Live RealSense D455 support with a GPU speed-up branch hitting roughly 10 fps on an RTX 4090
• Offline evaluation supported on TUM-RGBD, Replica, and EuRoC MAV datasets
• Stereo mode is available but marked experimental

Caveats

• All experiments were run on an RTX 4090; expect different behavior on other GPUs
• Multi-process execution carries some non-determinism from GPU utilization jitter
• The released code may diverge slightly from the paper’s reported metrics
• Live tracking requires careful, gentle camera motion until the first bundle adjustment finishes

Verdict

Grab it if you are working at the intersection of dense SLAM and novel-view rendering and want a unified Gaussian representation. Skip it if you need battle-tested, hardware-agnostic localization or are allergic to GPU-dependent research code.