Turn one photo into a real-time, infinite-length avatar

What it does

PersonaLive animates a single reference portrait using a driving video, generating talking-head videos of arbitrary length. It is built as a streamable diffusion framework, processing video in chunks rather than loading entire sequences into memory at once. The project ships with both an offline mode for batch generation and a browser-based Web UI for live streaming.

The interesting bit

The authors solve the memory wall by treating portrait animation as a streaming problem: their temporal module fine-tuning stage explicitly optimizes for chunk-wise generation, letting the model run on consumer GPUs with 12GB VRAM. It is also one of the few diffusion-based avatar systems to ship with a full training pipeline and TensorRT acceleration hooks.

Key highlights

• Generates “infinite-length” animations via a streaming strategy that works on 12GB VRAM.
• Real-time inference mode with an optional Web UI and ComfyUI node support.
• Three-stage training pipeline (image warm-up, adversarial refinement, temporal streaming) with configs for 8x H100 rigs.
• Optional TensorRT conversion for roughly 2× speedup, though the authors warn it may slightly dent quality.
• Weights and paper are already available; the authors note it is strictly for academic research.

Caveats

• xFormers is incompatible with RTX 50-series (Blackwell) GPUs, forcing a fallback to standard attention.
• The bundled TensorRT engine was built on an H100; the README explicitly tells everyone to rebuild it locally for compatibility.
• Academic research only, with a strict disclaimer against harmful or defamatory use.

Verdict

Worth a look if you are building real-time avatar pipelines or researching efficient video diffusion. Skip it if you want a polished consumer app—this is a research release with hardware-specific sharp edges.