Motion capture without the capture: one AI video rigs any creature

What it does

You give it a rigged 3D creature — dragon, spider, wolf, whatever — and it produces engine-ready animation as a GLB. A video model (MiniMax H3 via fal) renders the creature from front and side cameras in one frame, a DINOv2-based joint reader trained on that specific creature finds every joint in both views, and the two views are triangulated into 3D. The rig is then fitted to those points frame by frame. No mocap suit, no keyframing.

The interesting bit

The stereo trick is the clever part: instead of two cameras or two generations, one video contains both views side by side, and since a joint sits at the same height in both, the pair pins it in 3D. The joint reader is also per-creature — trained on ~2,200 random poses within the rig’s joint limits, dropping its average miss from 40 px to 6 px in about 20 minutes on one GPU. It also ships as Claude Code agent skills, so an agent can walk the whole pipeline.

Key highlights

• About $3 per usable move, all in; ~$17 and a couple of hours to onboard a new creature with ~5 moves
• Works on any body plan: wings, tails, eight legs (for gestures at least)
• Exports GLB at 24 fps with no hand edits, under a minute per clip
• Nine ready-made creatures (T-rex, knight, centaur, mech, scorpion…) with 4–7 moves each
• Runs the fitting on CPU; training a reader needs an NVIDIA GPU (12 GB, 24 GB safer)

Caveats

• Honest failure list in the README: eight-legged walking fails (the two views disagree on which legs are up), bird wings don’t fit, tail tips drift, feet slide 1–2 cm per step
• A usable move took ~7 clips on average; only 58% of captured clips passed the checks
• Animations only fit the creature they were made for — no retargeting, and every new model needs a new reader
• Output meshes are 55–60k triangles with 4K textures; heavy for browser games, no lighter versions yet

Verdict

Worth a look for indie game devs and 3D artists who need creature animations on a budget and have the GPU for it. If you want retargetable mocap, walking spiders, or web-friendly file sizes, wait for the next iteration.