StyleGAN inversion that learns from its mistakes, iteratively

What it does

ReStyle attacks the StyleGAN inversion problem: finding the latent code that represents a real image so you can manipulate it with GAN semantics. Rather than the standard encoder approach of predicting that code in a single forward pass, this method loops. Each iteration feeds the current latent guess back into the encoder, which predicts a residual correction. The repository contains training and evaluation code, pretrained models stacked on pSp and e4e backbones, and supports domains ranging from human faces to cars, churches, and horses.

The interesting bit

The insight is that asking an encoder to predict a small delta is easier than asking it to nail the full latent vector from scratch. By treating inversion as iterative refinement, ReStyle tightens reconstruction quality without a severe inference tax. It is essentially a meta-layer that can be wrapped around existing encoder architectures.

Key highlights

• Iterative residual prediction over latent estimates instead of single-pass regression.
• Pretrained models for multiple domains—FFHQ, Stanford Cars, LSUN Church, LSUN Horse, AFHQ Wild—using both ReStyle-pSp and ReStyle-e4e.
• Ships with inference notebooks, a Replicate demo, and auxiliary models (generators, IR-SE50, MoCov2) for training from scratch.
• Built on rosinality’s PyTorch StyleGAN2; includes guidance for converting official NVIDIA TensorFlow and PyTorch-ADA checkpoints.
• Supports encoder bootstrapping for further architectural tinkering.

Caveats

• CPU training or inference is not inherently supported; an NVIDIA GPU with CUDA CuDNN is effectively mandatory.
• Bringing your own StyleGAN generator requires converting checkpoint formats using external scripts.
• The README claims a “negligible increase in inference time” but does not quantify it in the provided documentation.

Verdict

A solid resource if you are already inside the StyleGAN ecosystem and need higher-fidelity inversion. If you are looking for a general-purpose image encoder, this is too domain-specific.