Alibaba's audio-to-portrait diffusion model has a very quiet repo

What it does This is the official companion repository for an ECCV 2024 paper by Alibaba’s Institute for Intelligent Computing. The work presents a diffusion model that synthesizes expressive portrait videos driven by audio input, operating under what the authors call “weak conditions.” The README itself contains only the title, author list, venue, and outbound links.

The interesting bit The “weak conditions” framing implies the system may forgo the dense facial landmarks, 3D face models, or reference videos that usually constrain audio-driven generation. That would be a genuine simplification, though the repository explains none of the architecture or training specifics.

Key highlights

• Accepted at ECCV 2024; authored by researchers at Alibaba Group.
• 7,617 GitHub stars despite offering no code, weights, or implementation notes in the README.
• Points to an arXiv preprint, a dedicated project page, and a YouTube demonstration.
• Repository language is undetected, suggesting the repo is effectively a documentation landing zone.

Caveats

• The README is a bare stub: no source files, model details, or usage guidance is visible.
• What “weak conditions” actually entails—whether weak supervision, sparse inputs, or reduced control signals—is not clarified in the repository.

Verdict Researchers tracking audio-driven video generation should check the paper and demo links. Practitioners hunting for trainable code or a ready-to-run model will leave empty-handed.