One model, any face: swapping identities in images and video

What it does SimSwap is the official PyTorch release of an ACM Multimedia 2020 paper for high-fidelity face swapping. It swaps identities in images and videos using a single generalized model, so you do not need to retrain for every new source face. The repository covers training scripts, inference code, and a higher-resolution variant, SimSwap-HQ.

The interesting bit Instead of the usual patch discriminator, the training pipeline uses a projected discriminator that the authors say saves hardware overhead and improves results slightly. A masking upgrade also smooths the borders around swapped faces, which resolves the “obvious border abruptness” the team flagged in earlier versions.

Key highlights

• One checkpoint handles arbitrary identities the model never saw during training.
• Supports image and video inference, including multi-face clips where you target specific individuals.
• SimSwap-HQ trains at 512×512 resolution on the companion VGGFace2-HQ dataset.
• Colab notebooks and a hosted image on Replicate are available for browser-based or cloud experimentation.
• Strictly academic and non-commercial: the project carries a CC-BY-NC 4.0 license and multiple ethical-use warnings.

Caveats

• User-trained models will look subtly different from the pre-trained weights the authors provide, so matching their exact dataset and batch-size guidance matters.
• Training demands real GPU memory: batch sizes above 16 are recommended for convergence, and the 512×512 HQ path requires the deeper generator configuration.
• Dependency pins like timm==0.5.4 suggest the environment can be finicky if you deviate from the listed stack.

Verdict A solid starting point for researchers and generative-computer-vision tinkerers; commercial teams and casual users should look elsewhere because of the non-commercial license and hardware expectations.