The kinematic middleman for Mediapipe and Vtubers

What it does Kalidokit is a JavaScript library that consumes 3D landmark arrays from Mediapipe and TensorFlow.js tracking models—FaceMesh, BlazePose, Hands, and Holistic—and computes simple Euler rotations and facial blendshape values. The results are purpose-built to rig 3D VRM models and Live2D avatars, making it the underlying math layer for browser-based Vtuber apps. It is explicitly not a standalone application; it is coordinate translation and trig wrapped in three solver classes: Face, Pose, and Hand.

The interesting bit The library smooths over the subtle but annoying differences between the Mediapipe and TensorFlow.js runtimes, letting you specify which backend produced the landmarks so the trigonometry stays correct. It also includes a stabilizeBlink utility that tries to keep winks and asymmetric blinks from glitching your avatar’s eyes, which is exactly the kind of micro-problem that separates a tech demo from a usable puppet.

Key highlights

• Consumes landmarks from Facemesh, Blazepose, Handpose, and Holistic models.
• Outputs named Euler rotations and blendshapes (mouth shapes A/E/I/O/U, brow raises, pupil positions, joint angles).
• Provides separate Face, Pose, and Hand solver classes with runtime-aware correction for mediapipe vs tfjs.
• Ships with remixable Glitch templates for both Three.js/VRM and Pixi.js/Live2D pipelines.
• Includes a blink stabilizer that interpolates obscured eyes during extreme head rotation.

Caveats

• The project is officially deprecated; the README states its solutions are being integrated directly into MediaPipe.
• Leg tracking is still marked as a work in progress.
• The library is described by its own author as a work in progress, with plans for better facial blendshapes and improved efficiency.

Verdict Grab it if you are maintaining a browser-based Vtuber rig or need a reference for how to turn Mediapipe landmarks into avatar-friendly rotations. Skip it if you are starting fresh—MediaPipe’s newer integrated solutions are the intended replacement, and this repo is no longer moving forward.