Meta’s SAM for bodies: promptable 3D humans from one image

What it does

SAM 3D Body (3DB) is a promptable encoder-decoder model that reconstructs a full-body 3D human mesh—including body, feet, and hands—from a single image. It supports optional 2D keypoint and mask prompts to nudge the reconstruction, similar to how SAM uses prompts for segmentation. The model is built on a new parametric mesh representation called Momentum Human Rig (MHR), which decouples skeleton from surface shape.

The interesting bit

The project borrows SAM’s promptable UX philosophy but applies it to geometry: you can feed in 2D keypoints or masks to correct a pose, rather than blindly trusting a black-box regressor. Under the hood, it uses a custom parametric rig—Momentum Human Rig—that separates skeleton from surface, which the authors claim improves interpretability and accuracy.

Key highlights

• Released checkpoints top out at 840M parameters (DINOv3-H+) and 631M (ViT-H), with published per-benchmark scores on 3DPW, EMDB, RICH, COCO, LSPET, and Freihand.
• Supports optional 2D keypoint and mask prompts for user-guided inference, plus a dedicated hand decoder for finer hand-pose detail.
• Ships with a released dataset, a live web demo, and a companion SAM 3D Objects repo that can align human and object meshes in a shared scene.
• The training pipeline combined differentiable optimization, multi-view geometry, dense keypoint detection, and a data engine to cover rare poses and viewpoints.

Verdict

Worth a look if you’re doing research in human mesh recovery or need a promptable baseline for pose estimation. Probably overkill if you just need a quick 2D pose extractor or are targeting resource-constrained devices—the checkpoints start at 631M parameters and the repo is research-grade, not a slim SDK.