Making LivePortrait actually live, via TensorRT and stubbornness

What it does

FasterLivePortrait takes the LivePortrait face-animation model and wraps it in an optimized inference stack built on ONNX and TensorRT. It drives a static portrait with a video, audio clip, or text prompt, syncing facial motion in real time through a Gradio web UI or a webcam feed. The project also layers in JoyVASA for audio-driven motion and Kokoro-82M for text-to-speech-driven video.

The interesting bit

The author did not simply swap the runtime. They maintain a custom TensorRT plugin for 5D grid sampling—because native TensorRT does not support it—and rely on a patched ONNXRuntime branch for CUDA grid sampling. That is the kind of dependency surgery usually left as an exercise for the reader.

Key highlights

• Claims 30+ FPS end-to-end on an RTX 3090 under TensorRT, counting pre- and post-processing, not just model forward passes.
• Supports simultaneous multi-face inference, animal models, and region-specific animation controls.
• Bundles a Windows all-in-one package, a Docker image, and a Gradio UI with hotkeys for live webcam tweaking.
• Exposes an API server and supports audio-driven generation via JoyVASA and text-driven generation via Kokoro-82M.

Caveats

• TensorRT 10.x is explicitly incompatible; you are locked to TensorRT 8.x, and the ONNXRuntime GPU path requires compiling from a specific feature branch.
• The README notes model licenses are separate from the MIT-licensed code, so commercial use depends on third-party model terms.

Verdict

Worth a look if you need a self-hosted, real-time deepfake puppet booth or want to animate pets without sending data to the cloud. Skip it if you were hoping for a plug-and-play CPU solution; the project is very clear that CPU inference is “extremely slow and not recommended.”