Teaching Gaussian Splats to Obey Newton's Laws

What it does

PhysGaussian takes a reconstructed 3D Gaussian Splatting scene and runs physics on it using a customized Material Point Method. It treats each Gaussian kernel as a continuum particle with deformation and stress attributes, letting you animate materials like jelly, metal, sand, or snow without rebuilding the scene as a mesh. The rendering and simulation share the exact same discrete representation.

The interesting bit

The core principle is “what you see is what you simulate” — there is no triangle mesh, tetrahedron grid, or marching cubes step hiding in the pipeline. Because the visual kernels and the physics particles are identical, the geometry you view is literally the geometry being integrated forward in time.

Key highlights

• Supports elastic solids, plastic metals, non-Newtonian fluids, and granular materials (jelly, sand, foam, snow, plasticine, etc.)
• Uses a customized MPM solver that evolves mechanical stress and kinematic deformation per Gaussian
• Skips traditional meshing or cage embeddings entirely; the Gaussian cloud is both render primitive and simulation particle
• Provides pretrained models and a Colab notebook for quick experiments
• Boundary conditions and external forces can be applied directly to the Gaussian kernels or the simulation grid

Caveats

• Custom datasets require manual preprocessing: you must rotate the scene to align with coordinate planes, filter low-opacity kernels, and often use external 3D software like Houdini to inspect Gaussian distributions before simulation
• The optional internal particle-filling algorithm is sensitive to noise and may produce unsatisfying results if the reconstructed Gaussians are messy
• Built as an extension of the original Gaussian Splatting repository, so it inherits that submodule dependency

Verdict

Graphics researchers and VFX tinkerers who want to smash, bend, or melt Gaussian-splat scenes without remeshing should take a look. If you are looking for a drop-in real-time renderer, this is a physics sandbox, not an engine.