Making open video diffusion fast enough to stream and edit live

What it does

FastVideo is a toolkit for accelerating open video diffusion transformers. It covers both sides of the lifecycle: post-training recipes (full/LoRA fine-tuning, DMD2 distillation, and sparse distillation) and optimized inference with sequence parallelism and multiple attention backends. It also ships Dreamverse, a real-time streaming video editor that lets you “vibe direct” generation as it plays.

The interesting bit

The project treats speed as a first-class research output, not just an engineering afterthought. It implements custom sparse attention kernels (Video Sparse Attention) and claims >50× denoising speedups through sparse distillation, with published recipes and synthetic datasets for Wan2.1 and Wan2.2 models. A live demo generates 5 seconds of 1080p video in 4.5 seconds on a single GPU.

Key highlights

• End-to-end post-training stack: full/LoRA fine-tuning, DMD2 stepwise distillation, causal distillation via Self-Forcing, and FSDP2 with sequence parallelism.
• Sparse distillation and Video Sparse Attention (VSA) for aggressive speedups; pre-built recipes available for FastWan2.1 and FastWan2.2.
• Real-time inference and editing through Dreamverse, deployable locally, on a B200 server, via Docker, or serverless Modal.
• Supports consumer and datacenter hardware: H100, A100, and RTX 4090 across Linux, Windows, and macOS.
• Already forked by SGLang for its diffusion inference backend; adopted by downstream projects including Hunyuan Video and Kandinsky.

Caveats

• VSA kernels require a separate manual installation step before the sparse attention backends work.
• Performance claims like “5 s of 1080p in 4.5 s” and “>50× speedup” are tied to specific distilled checkpoints and high-end GPUs; see the hardware support matrix before assuming your setup qualifies.

Verdict

Researchers and engineers who need to take open video diffusion models from training to deployed, interactive generation should look here. If you just want a simple CLI to generate cat videos without touching distillation recipes or kernel installs, this is more toolkit than appliance.