Generate or edit images by pointing at other images

What it does

DreamOmni2 handles two distinct visual tasks: generating new images guided by reference photos and text instructions, and editing existing images while preserving the unmodified regions. It accepts multimodal prompts—concrete objects, abstract attributes like texture or hairstyle, and natural language—to steer the output. The project provides separate inference paths and LoRAs for generation and editing because the two tasks impose different consistency constraints.

The interesting bit

Most open-source editing models rely purely on text instructions; DreamOmni2 lets you point to a reference image when words fail. The authors claim the same unified checkpoint surpasses commercial alternatives on abstract-attribute generation and matches them on editing tasks—an unusual combination for an open-source release.

Key highlights

• Unified generation and editing in a single open-source model.
• Supports concrete object references and abstract attributes (material, texture, style, posture).
• Separate LoRAs and inference paths for generation (full aesthetic regeneration) and editing (strict preservation of non-edited areas).
• Hugging Face demos, benchmark dataset, and a ComfyUI workflow are available.
• Highlighted as a CVPR 2026 paper according to the repository.

Caveats

• Example inference scripts contain hard-coded absolute paths, so expect to adapt them before running locally.
• The README warns that editing tasks require the source image to be placed first due to training-data formatting—a sharp edge that is easy to miss.
• Minor typos such as “Instriction” appear in the documentation, suggesting a quick polish pass is still needed.

Verdict

Researchers and practitioners building visual content tools should look here if they need a single open-source checkpoint for both synthesis and editing. If you only need plain text-to-image generation without reference images, this is overkill.