What it does

FaceSwap grabs frames from your webcam, detects facial landmarks, fits a 3D morphable face model to them, and renders that model textured with a target face (Brad Pitt and Einstein are bundled as examples). The rendered face is then alpha-blended back over your own. The result: you see yourself wearing someone else’s face in real time.

The interesting bit

The fitting step is the meat. The code minimizes landmark reprojection error using the Gauss-Newton method, solving simultaneously for blendshape weights, scale, rotation, and translation. No deep learning in the main branch — just old-school nonlinear least squares on a Candide-derived mesh with blendshapes for expressions like mouth opening and eyebrow raising. The author later built a faster GPU version using his own Deep Alignment Network, but that lives on Dropbox, not in the repo.

Key highlights

• Pure Python pipeline: dlib landmarks → 3D model fitting → pygame rendering → feathered compositing
• 3D model includes neutral shape, expression blendshapes, and index mappings between dlib’s 68 landmarks and mesh vertices
• Entry point is literally zad2.py — Polish for “exercise 2” — because this was coursework
• MIT licensed; includes a YouTube demo and a requirements.txt for dependencies
• A faster, more stable DAN-based rewrite exists but is not yet merged

Caveats

• The “faster and more stable version” is off-repo on Dropbox; the GitHub code uses the slower dlib-based tracker
• You must manually download and unpack a 68-point dlib shape predictor (~60MB bz2 from SourceForge)
• The code targets Python 3 but the repo hasn’t seen the DAN rewrite integrated despite the author’s stated hope to do so

Verdict

Worth a look if you want to understand how 3D face fitting works under the hood without neural-network opacity, or if you’re teaching computer vision. Skip it if you need production-grade real-time performance out of the box — the Dropbox fork or modern deepfakes tools will serve you better.