Inpainting by jumping around in diffusion time

What it does

RePaint takes an image with a masked-out hole and fills it using a pre-trained denoising diffusion probabilistic model. The method never fine-tunes the model; it only manipulates the inference process, adding noise to the known region and denoising the whole canvas step by step until the missing pixels are reconstructed. It ships with configs and pre-trained checkpoints for faces, generic ImageNet scenes, and Places2.

The interesting bit

The method uses a resampling schedule that deliberately jumps back to higher noise levels during generation—essentially undoing and redoing steps—to force coherence between the inpainted region and the original image. It can handle extreme masks where most methods collapse, such as every second line missing or three-quarters of the image erased for super-resolution.

Key highlights

• Outperformed autoregressive and GAN-based state-of-the-art methods in a user study across 42 of 44 cases, with 95% statistical significance on most masks.
• Handles extreme masks: every-second-line strips, half-image deletions, thick random strokes, and sparse super-resolution inputs.
• Pure inference scheme: conditions pre-trained unconditional diffusion models at test time without training or fine-tuning.
• Built on OpenAI’s guided-diffusion codebase and provides ready-to-use configs for CelebA-HQ, ImageNet, and Places2.

Caveats

• The authors note they did not reevaluate all experiments after refactoring the code.
• The bundled ImageNet model is heavily biased toward inpainting dogs because of ImageNet’s class distribution.
• Inference involves resampling and many steps, so expect to trade speed for quality unless you trim t_T or jump_n_sample.

Verdict

Worth a look if you research generative inpainting or need a strong baseline for hole-filling with diffusion. Skip it if you need a lightweight, real-time inpainter or a training framework for custom domains.