Turn a static PNG into a puppet with your webcam

What it does This research demo animates a single, static anime portrait through two local tools: a manual poser driven by UI sliders and a puppeteer that maps your webcam head motion onto the drawing. The catch is that your input image must be a surgically prepared 256×256 transparent PNG with the head dead-center; the model does not forgive casual screenshots.

The interesting bit Instead of relying on layered art or a conventional rig, the PyTorch network infers head rotation and facial deformation directly from a flat image. It is a neat trick, though the author admits it is already eclipsed by a much more capable Version 2.

Key highlights

• Two interaction modes: manual slider posing and real-time webcam puppeteering.
• Ships with five sample characters and a Google Colab path for the manual poser, so you can experiment without owning a high-end GPU.
• Runs fully offline after the initial model download.
• Uses an external face-tracker implementation by KwanHua Lee to bridge webcam input to anime output.

Caveats

• The author explicitly recommends Version 2 instead; this repo is essentially a legacy academic snapshot.
• Local use demands a recent, powerful Nvidia GPU (tested on a GTX 1080 Ti and Titan RTX).
• Input images must adhere to strict formatting rules—256×256, RGBA, transparent background, head contained in the center 128×128—or the results will likely break.

Verdict A useful reference if you are studying single-image neural animation or prototyping a minimalist VTuber stack. If you want a modern, polished tool, follow the author’s own link to Version 2.