Satellite photos to explorable 3D cities, no LiDAR required

What it does

Skyfall-GS builds explorable 3D city scenes from plain satellite imagery using 3D Gaussian Splatting. It sidesteps the usual requirement for expensive ground-level 3D scans or LiDAR by treating overhead photos as a starting scaffold, then hallucinating and refining street-level detail through an iterative training loop. The output is a real-time-renderable gaussian scene you can fly through.

The interesting bit

The project avoids the “no training data” trap with a curriculum-style refinement strategy it calls Iterative Dataset Update (IDU): the model renders views, critiques its own output, and feeds those synthetic angles back into training alongside the original satellite shots. A diffusion model fills in texture where satellites can’t see. It’s essentially bootstrapping a ground-truth dataset from thin air.

Key highlights

• Two-stage pipeline: reconstruction from satellite imagery, then synthesis and refinement via IDU
• Supports custom datasets from COLMAP or satellite-derived SfM (via the companion SatelliteSfM repo)
• Pre-built datasets and fused PLY models for Jacksonville and New York City on Hugging Face
• Evaluation suite computes PSNR, SSIM, LPIPS, CLIP-FID, and CMMD against baselines like Mip-Splatting, Sat-NeRF, and CityDreamer
• Includes scripts for automated training, video rendering from camera paths, and an online PLY viewer

Caveats

• README warns against using raw training-output .ply files directly; you must fuse them first with create_fused_ply.py for visualization
• Custom dataset preprocessing requires external tooling (SatelliteSfM) and careful directory structure
• Stage 2 training is parameter-heavy: the example command lists ~20 flags, suggesting significant tuning surface area

Verdict

Worth a look if you’re working on geospatial visualization, urban digital twins, or any domain where ground-level 3D capture is impractical. Skip it if you already have dense LiDAR or photogrammetry captures — this project’s whole pitch is doing without them.