Image restoration using SDXL, LLaVA, and serious hardware

What it does SUPIR is a CVPR 2024 research release that restores and upscales real-world photographs using a diffusion pipeline built on SDXL. It ships with two trained checkpoints—SUPIR-v0Q for general image quality and SUPIR-v0F for higher fidelity on lightly degraded inputs—and exposes stage-wise controls that let you balance output faithfulness against visual polish.

The interesting bit The README is light on architectural specifics, but the dependency list makes clear this is not a lightweight wrapper: it pairs an SDXL backbone with LLaVA v1.5 13B and multiple CLIP encoders, then adds a dense set of EDM sampler knobs—including s_stage1 and s_stage2—that let you dial the tension between reconstructing the original and generating new detail.

Key highlights

• Two trained checkpoints: SUPIR-v0Q for generalization and SUPIR-v0F for preserving detail under light degradation
• Built on SDXL with LLaVA v1.5 13B and multiple CLIP encoders listed as dependencies; optional Juggernaut XL variants supported
• Fine-grained sampler controls including EDM steps, linear CFG scheduling, and wavelet color fixing
• Gradio demo included, plus a hosted online app at suppixel.ai
• Strictly non-commercial; commercial use requires prior written permission from the authors

Caveats

• Demanding hardware footprint; the README cites 12 GB VRAM for diffusion and 16 GB for LLaVA even with half-precision, tiled VAE, and 8-bit LLaVA optimizations
• Non-commercial license only, with no commercial use permitted without contacting the authors
• Checkpoint setup is fragmented across Hugging Face, Baidu Netdisk, and Google Drive

Verdict A solid playground for researchers and hobbyists experimenting with diffusion-based restoration who have the GPU headroom. Commercial teams and edge-device developers should look elsewhere.