Teaching video editors to follow orders, with a million synthetic examples

What it does

Ditto is a data-generation pipeline and trained model for instruction-based video editing — think “make this cloudy” or “turn it into the real domain.” The authors built Ditto-1M, a synthetic dataset of one million video editing examples, by combining an image editor’s creative diversity with an in-context video generator. The resulting model, Editto, is trained on this data with curriculum learning.

The interesting bit

The core bet is that data scarcity, not architecture, is the bottleneck. They burned over 12,000 GPU-days to prove it. An agent crafts diverse editing instructions and filters outputs, making the pipeline self-scaling rather than hand-curated. A distilled model plus temporal enhancer keeps compute costs from exploding.

Key highlights

• One million synthetic video editing pairs (Ditto-1M), fully released on Hugging Face
• Trained model (Editto) available via DiffSynth-Studio or ComfyUI workflows
• ComfyUI quantized path runs on ~11GB VRAM, though with noted quality degradation
• Denoising enhancement add-on using Wan2.2 for post-processing cleanup
• CVPR 2026 Highlight paper; code and weights are public

Caveats

• CC BY-NC-SA 4.0 license: academic/research only, no commercial use
• ComfyUI fast path trades quality for speed; DiffSynth path is the quality default
• “State-of-the-art” claims are the authors’; no independent benchmarks shown in README

Verdict

Worth a look if you’re researching video diffusion, synthetic data scaling, or instruction-following generation. Skip if you need a production-ready commercial tool — the license forbids it, and the setup is still research-grade.