cloneofsimo/paint-with-words-sd
A Stable Diffusion implementation of the paint-with-words technique from eDiff-I that lets users generate images from text-labeled segmentation maps.
Not currently ranked — collecting fresh signals.
star history
This repository implements the paint-with-words method from NVIDIA’s eDiff-I paper using Stable Diffusion. The technique allows users to control image generation by providing a segmentation map where each colored region is associated with a text label. The implementation modifies cross-attention scores to spatially guide the diffusion process, enabling fine-grained control over object placement without changing the seed. It demonstrates before/after comparisons showing how objects like clouds or roads can be selectively included or positioned.
Frequently asked
- What is cloneofsimo/paint-with-words-sd?
- A Stable Diffusion implementation of the paint-with-words technique from eDiff-I that lets users generate images from text-labeled segmentation maps.
- Is paint-with-words-sd open source?
- Yes — cloneofsimo/paint-with-words-sd is open source, released under the MIT license.
- What language is paint-with-words-sd written in?
- cloneofsimo/paint-with-words-sd is primarily written in Jupyter Notebook.
- How popular is paint-with-words-sd?
- cloneofsimo/paint-with-words-sd has 646 stars on GitHub.
- Where can I find paint-with-words-sd?
- cloneofsimo/paint-with-words-sd is on GitHub at https://github.com/cloneofsimo/paint-with-words-sd.