Three ML models and a shader walk into a browser tab

What it does This demo replaces your real background with a virtual one during live video calls, entirely in the browser. It grabs frames from your webcam, runs person segmentation through one of three ML models, then composites you over a new backdrop using either Canvas 2D or custom WebGL 2 shaders. The whole pipeline stays client-side; no server required.

The interesting bit The author didn’t just import TensorFlow.js and call it a day. For Google’s Meet segmentation model—available only as a TensorFlow Lite file—they compiled a custom WASM inference runtime with XNNPACK and SIMD support, then built a full WebGL 2 rendering pipeline with joint bilateral filtering and light wrapping. It is essentially a from-scratch reconstruction of Google Meet’s background pipeline, running in a single browser tab.

Key highlights

• Side-by-side comparison of BodyPix, MediaPipe Meet, and ML Kit Selfie Segmentation.
• Custom TFLite WASM runtime built with Emscripten and Bazel, calling C++ directly from JavaScript.
• WebGL 2 pipeline handles resizing, softmax, edge-preserving smoothing, background blur, and light wrapping in GLSL.
• On a Pixel 3, the Meet model reaches 60 FPS at 160×96 resolution when using WASM SIMD and the WebGL 2 renderer.
• BodyPix uses standard TensorFlow.js but is described as “not really usable” on mobile and less precise than Meet.

Caveats

• The Meet segmentation model’s license shifted from Apache 2.0 to Google’s Terms of Service in January 2021, so its legal status for reuse is unclear.
• BodyPix performance on mobile is poor, and the ML Kit model underperforms Meet despite similar architecture because it requires a higher input resolution.
• The repository is explicitly a demo and research testbed, not a production library.

Verdict Study this if you’re building browser-based video effects or comparing ML inference strategies across WASM and WebGL. Look elsewhere if you need a polished, drop-in virtual background SDK.