An adapter that stops Stable Diffusion from merging your cats

What it does

MIGC is an adapter that sits on top of Stable Diffusion to generate images containing multiple specific objects in defined positions with distinct attributes. You provide bounding boxes or masks plus text descriptions, and it renders each instance separately rather than merging them into a single ambiguous blob. MIGC++ extends this with simultaneous box-and-mask control and an iterative editing mode called Consistent-MIG that lets you tweak individual instances while keeping the rest of the scene intact.

The interesting bit

The project treats multi-instance generation as a control problem rather than a prompt-engineering problem, using lightweight adapter weights instead of retraining the whole model. On their COCO-MIG benchmark, the adapter outperforms full-model tuning approaches like ReCo, which is somewhat unexpected given it leaves the base model untouched. The authors also had to re-implement the MIGC++ weights from scratch because the originals are locked behind collaborator agreements.

Key highlights

• Plug-and-play adapter for SD 1.4 (and compatible with 1.5), not a full model fine-tune
• Achieves higher instance success rate (66% average) and MIOU (0.56 average) than GLIGEN and InstanceDiffusion on the COCO-MIG benchmark
• MIGC++ supports specifying instance locations with both masks and boxes at the same time
• Includes a web GUI and a Colab demo for quick experimentation
• Can be combined with LoRA and swapped across different base models like RV60B1 or Cetus-Mix

Caveats

• Training code is unavailable due to company policy; only data-preparation scripts are provided
• The released MIGC++ weights are re-implemented and trained with a smaller batch size, not the original checkpoint
• Weights for SD 1.5, SD 2, and SDXL remain on the roadmap, though the SD 1.4 checkpoint reportedly works with SD 1.5

Verdict

Worth a look if you generate complex scenes with Stable Diffusion and need objects to stay in their lanes. Skip it if you need to train your own controller from scratch or if you require native SDXL support today.