← all repositories

cloneofsimo/paint-with-words-sd

A Stable Diffusion implementation of the paint-with-words technique from eDiff-I that lets users generate images from text-labeled segmentation maps.

646 stars Jupyter Notebook Image · Video · Audio
paint-with-words-sd
Velocity · 7d
+0.5
★ / day
Trend
steady
star history

This repository implements the paint-with-words method from NVIDIA’s eDiff-I paper using Stable Diffusion. The technique allows users to control image generation by providing a segmentation map where each colored region is associated with a text label. The implementation modifies cross-attention scores to spatially guide the diffusion process, enabling fine-grained control over object placement without changing the seed. It demonstrates before/after comparisons showing how objects like clouds or roads can be selectively included or positioned.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.