Latent diffusion for people who prompt in more than English

What it does

Kandinsky is a suite of text-to-image latent diffusion models. It handles text-to-image, image-to-image, inpainting, and outpainting, with explicit support for multilingual inputs—version 2.0 was trained on a 1 billion sample multilingual dataset. The repository provides inference code and example notebooks across three model generations: 2.0, 2.1, and 2.2.

The interesting bit

Instead of a single monolithic model, Kandinsky runs a two-stage pipeline: a diffusion prior first maps text embeddings into an image-embedding space, and a latent U-Net decoder renders the final pixels. Version 2.1 adds a weighted image-fusion mode that blends text and multiple images, while 2.2 upgrades to a larger CLIP-ViT-G encoder and adds ControlNet-depth conditioning.

Key highlights

• Native multilingual training: Kandinsky 2.0 uses mCLIP-XLMR and mT5 text encoders and was trained on a 1B multilingual image-text dataset.
• Two-stage architecture with a 1B-parameter diffusion prior and a 1.22B-parameter latent U-Net decoder.
• 2.1 introduces image fusion, allowing weighted mixing of text and image inputs in a single generation.
• 2.2 swaps in a 1.8B-parameter CLIP-ViT-G image encoder and supports ControlNet-depth conditioning.
• LoRA fine-tuning notebooks are provided for Kandinsky 2.2.

Verdict

Grab it if you need non-English text-to-image generation or want a DALL-E-style prior/decoder stack to run locally. Skip it if you need a lightweight single-file model or comparative benchmarks—the README keeps the focus on architecture and quickstart examples.