Real-time mocap that outputs BVH instead of raw coordinates

What it does MocapNET takes 2D body joint detections from ordinary monocular RGB images and regresses 3D human pose directly into the Bio Vision Hierarchy (BVH) format used by animation tools. The system runs in real time—version 2 is reported at 70 fps on CPU-only execution—and later iterations add hand pose, facial configuration, and 3D gaze. A companion Blender plugin can drive skinned MakeHuman characters with the output files.

The interesting bit Instead of producing abstract 3D coordinates that must be manually retargeted, the ensemble uses a compact 2D NSRM representation and is tuned to emit animator-ready BVH skeleton data. An orientation classifier splits the body into upper and lower kinematic hierarchies to keep working under occlusion, and an efficient Inverse Kinematics solver refines limb lengths to match a target person. Version 4 rewrote the stack in Python and added one-click Google Colab demos.

Key highlights

• Compact NSRM encoding: a novel 2D pose representation fed to an orientation-tuned ensemble that classifies body facing direction and routes regression to specialized sub-networks.
• Direct BVH output: produces standard animation files compatible with Blender and other 3D tools.
• Occlusion recovery: decomposes the body into upper and lower kinematic chains to estimate pose when joints are hidden.
• Real-time on modest hardware: v2 claims 70 fps CPU-only; v4 reportedly sustains 30 Hz on a laptop webcam handling body, hands, and face simultaneously.
• Blender integration: includes a plugin that pairs with MakeHuman’s MPFB2 addon to animate custom characters.

Caveats

• Features are fragmented across version-specific branches (mnet1 through mnet4), so the default branch does not reflect the latest Python-based release.
• The v3 branch shipped without an integrated 2D hand estimator; the README suggests pairing it with MediaPipe Holistic for full hand coverage.
• Performance and feature claims vary by branch, and the repository history includes periods of delayed maintenance during thesis work.

Verdict A solid choice for animators and CV researchers who want BVH-native output from a webcam without a motion-capture studio. Probably overkill if you just need 2D keypoints or a single lightweight model.