Ditch the two-step: direct image-to-image via StyleGAN encoding

What it does

pSp is an encoder network that swallows an input image—a segmentation map, a blurry thumbnail, a side-profile face—and spits out a set of style vectors. Those vectors flow straight into a frozen, pretrained StyleGAN generator, which reconstructs or translates the image. The authors package it as a generic image-to-image translation engine, with pretrained checkpoints for frontalization, sketch-to-portrait, super-resolution, and toonification.

The interesting bit

Most StyleGAN workflows force you to invert an image into latent space before you can edit it; pSp skips that separate optimization step entirely. By treating translation as a direct encoding problem, it can handle inputs that never lived in the StyleGAN domain to begin with—like a hand-drawn sketch—and still produce a coherent face. It also inherits multi-modal synthesis for free by resampling style vectors, so one sketch can yield several plausible portraits.

Key highlights

• Directly encodes real images into StyleGAN’s extended W+ latent space with no additional optimization.
• Handles tasks lacking pixel-to-pixel correspondence, such as sketch or segmentation-map-to-face synthesis.
• Supports multi-modal output via style-mixing and style resampling.
• Training requires no adversarial component; the encoder and frozen StyleGAN generator do the work.
• Pretrained models cover inversion, frontalization, super-resolution (up to ×32), and toonification.

Caveats

• GPU dependency is strict: the README states CPU support is not inherent and requires modifications.

Verdict

Researchers and generative-art tinkerers who want a single, reusable StyleGAN encoder will find pSp a solid starting point. If you need a lightweight, CPU-friendly face filter, look elsewhere.