Teaching Stable Diffusion to respect individual pixels

What it does

PASD is a diffusion-based image super-resolution system built on Stable Diffusion. It upscales low-resolution images realistically, restores old photos, colorizes grayscale inputs, and applies personalized stylization using community fine-tunes like ToonYou or modern Disney style. The project provides both SD 1.5 and SDXL variants, plus a lighter pasd_light checkpoint.

The interesting bit

The model initializes its latent representation directly from the low-resolution input rather than from scratch, tethering the diffusion process to the original image. It also uses tiled VAE processing to keep memory usage in check when pushing through ultra-high-resolution images.

Key highlights

• Four distinct tasks: realistic super-resolution, old photo restoration, personalized stylization, and grayscale colorization
• SDXL support added in September 2024, which the authors note surpasses the SD 1.5 version significantly
• Tiled latent decoding for GPU-friendly processing of very large images
• Adjustable noise levels to balance detail between “extremely-detailed” and “over-smoothed”
• Compatible with personalized community models such as majicMIX realistic for different aesthetic results

Caveats

• Architectural details beyond a single diagram are sparse; the README is mostly quickstart material and dataset links
• Training requires carefully curating WebDataset pipelines and multiple high-resolution datasets
• The authors warn that default settings often need manual tuning of seeds, prompts, and conditioning scales to get good results

Verdict

A solid choice for researchers and developers who need diffusion-based upscaling with an emphasis on pixel fidelity. Everyone else should wait for a friendlier wrapper.