Training on fake data to fix real photos (and anime)

What it does

Real-ESRGAN is a practical image and video restoration toolkit that extends the ESRGAN architecture. It ships pre-trained PyTorch models and portable ncnn/Vulkan executables for Windows, Linux, and macOS that run without a CUDA or PyTorch environment. The project covers general photo upscaling, specialized anime image and video models, and optional face enhancement via GFPGAN integration.

The interesting bit

The whole system is trained on pure synthetic data, yet targets real-world blind super-resolution. The authors also provide portable binaries—an unusual courtesy in the research-to-tool pipeline—though they achieve this by tiling images, which can introduce block inconsistency.

Key highlights

• Portable executables for Windows, Linux, and macOS via ncnn/Vulkan; no CUDA or PyTorch installation required
• Specialized models for anime images (RealESRGAN_x4plus_anime_6B) and anime videos (AnimeVideo-v3)
• Optional GFPGAN integration for face enhancement alongside general upscaling
• Inference supports tiled processing, alpha channels, grayscale, and 16-bit images
• Training and finetuning code released for custom datasets

Caveats

• The ncnn portable executable processes images in tiles, which may cause block inconsistency and slightly different results versus the PyTorch implementation
• Portable executables lack some Python script features, such as arbitrary output scaling via outscale

Verdict

Worth exploring if you need offline, practical upscaling for photos or anime with both research flexibility and ready-to-run binaries. Look elsewhere if you want a fully managed service; the core deliverable is models and code, not a hosted API.