Diffusion models, but you paint with text prompts

What it does

SemanticDraw is a research framework—previously called StreamMultiDiffusion and rebranded for CVPR 2025—that turns Stable Diffusion into an interactive painting tool. You paint semantic masks on a canvas, attach a text prompt to each mask, and the model generates or updates the image in real time. It supports SD 1.5, SDXL, and Stable Diffusion 3, and can load custom .safetensors checkpoints.

The interesting bit

The project treats text prompts as paint colors: each masked region is governed by its own prompt, and the system prevents them from visually bleeding together while still updating the canvas continuously. That marriage of region-based semantic control and streaming generation is the core idea behind its CVPR 2025 paper.

Key highlights

• Supports SD 1.5, SDXL, and SD3, plus custom .safetensors checkpoints.
• Provides both a streaming web demo and a simplified “Semantic Palette” interface.
• Offers a Python API and Jupyter notebooks for programmatic control.
• Runs fully offline on a local GPU; minimum 8 GB VRAM required, 11 GB recommended.
• Built on top of StreamDiffusion, MultiDiffusion, and LCM.

Caveats

• The README still contains stale references to the old StreamMultiDiffusion repository name in its installation snippets.
• You need a local GPU with at least 8 GB VRAM just to get started at 512×512 resolution.

Verdict

Worth a look if you want to experiment with region-guided diffusion or build an interactive generative art tool. Skip it if you are looking for a lightweight CPU-friendly image generator; this is strictly GPU territory.