What it does

This repo shows how to run TensorFlow.js inside a WeChat mini-program, using the device’s camera for live ML inference. The author also ships a real product called “AI Pocket” (AI Pocket) that evolved from this demo.

The interesting bit

The project has lived two lives. Early on, the author had to fork and patch tfjs-core to make it run in WeChat’s sandboxed environment. Now TensorFlow.js supports custom platforms cleanly, and WeChat offers an official tfjs plugin — but the author still had to reverse-engineer how onCameraFrame crops and formats raw camera data differently across devices (and even front vs. back cameras on the same phone). The README notes this frame-mangling “真叫人头大” — roughly, “it was a genuine headache.”

Key highlights

• Uses the official TensorFlow.js WeChat plugin rather than a patched core library
• Targets WeChat base library ≥ 2.29.0 and developer tools ≥ v1.06.2210310
• Requires ES6-to-ES5 transpilation and “enhanced compilation” enabled in WeChat DevTools
• Includes a build step (npm run build) before the mini-program can run
• The author claims frame data cropping has “趋于一致” (mostly stabilized) across platforms now

Caveats

• The README warns that npm i sometimes needs --force to install dependencies
• Setup is MacOS-recommended and involves multiple WeChat-specific toolchain settings
• The actual model architecture, performance numbers, and supported model formats are not specified

Verdict

Worth a look if you’re building ML features for WeChat’s ecosystem and need a working reference for camera-to-tensor plumbing. Skip it if you’re not committed to WeChat mini-programs — the value is almost entirely in platform-specific glue code.