Gaussian splats that actually move

What it does Takes the rasterization-friendly 3D Gaussian Splatting representation and adds a fourth dimension—time—so it can model dynamic scenes like bouncing balls or cooking beef. The core idea is to model how Gaussians deform over time rather than rebuilding the scene from scratch each frame. It targets the same real-time rendering niche as the original 3DGS paper, just for things that move.

The interesting bit The authors graft a multi-resolution HexPlane module onto the Gaussian parameter pipeline to handle temporal variation, which is a more structured approach than simply training a separate 3DGS model per frame. They also provide scripts to export per-frame Gaussian clouds and merge multiple trained 4D scenes—acknowledging, perhaps unintentionally, that people will want to hack around the format’s limitations.

Key highlights

• CVPR 2024 oral; training times corrected down to ~8 minutes on D-NeRF synthetic scenes and ~30 minutes on HyperNeRF real scenes after removing debug logging overhead
• Supports synthetic (D-NeRF), real dynamic (HyperNeRF, DyNeRF), and custom multi-view datasets via COLMAP preprocessing
• Includes a SIBR viewer, Colab demos, and checkpointing for coarse-to-fine training stages
• Borrows heavily from 3DGS, K-planes, HexPlane, and TiNeuVox—this is integration research, not a from-scratch renderer
• export_perframe_3DGS.py lets you extract plain 3D Gaussian clouds at any timestamp if you need to drop back to static tooling

Caveats

• Blender format “seems doesn’t work” per the README; the authors invite PRs rather than fixing it themselves
• Custom datasets require nontrivial COLMAP wrangling and config file authoring; the “custom” path is essentially “use nerfstudio and hope”
• Still under active development; hyperparameters have changed post-publication (March 2024 update for Neu3D)

Verdict Worth a look if you’re already invested in the 3DGS ecosystem and need motion without abandoning real-time rasterization. If your scenes are static or you need production-grade stability, the original 3DGS toolchain is the safer bet for now.