Your phone video becomes a 3D scene your AI can actually measure

What it does You upload a phone video; a GPU job reconstructs a persistent 3D scene — geometry, detected objects, a report. From there your AI assistant can measure distances and angles, plan paths through free space, run server-side scene agents, and export the scan as LeRobot/GR00T-style training data or an Isaac Sim/MuJoCo scene. Everything is exposed over MCP, so Claude Code, Codex, and Cursor can drive the whole loop conversationally.

The interesting bit The measurement honesty: numbers stay relative until you calibrate against a real-world distance, and server-side uncertainty labels reach the AI unedited — the system is built so the model can’t pretend a relative unit is metres. Also notable: big files go to your disk, never into the AI’s context window, which is the correct instinct.

Key highlights

• 42 MCP tools covering reconstruction, measurement, path planning, agents, and robot-data exports
• Reconstruction core builds on MIT SPARK’s VGGT-SLAM, with Depth Anything V2 for scale-drift correction
• Fully self-hostable on your own GPU box or Modal account — no account with the vendor required
• An offline simulator fakes the entire workflow with fixture data, so you can develop and demo with zero GPU
• BSD-2-Clause for the repo code

Caveats

• The VGGT-1B model a self-hosted server fetches is CC BY-NC 4.0 — non-commercial only. Commercial use means the hosted service or your own license.
• server/ and core/ are hand-synced mirrors of private repos, not developed in the open; the README itself admits the self-host paths are young.
• Tool count is inconsistent in the README (41 vs 42) — trivial, but a hint of how fast this is moving.

Verdict Worth a look if you’re doing robotics, spatial AI, or want your agent to reason about real rooms. If you just want pretty 3D scans with no agent in the loop, lighter tools exist.