Turning one photo into a 3D face by brute-force voxel regression

What it does

VRN takes a single 2D face image and regresses a 3D volumetric grid spatially aligned to the input. A mesh is extracted by taking the isosurface of that volume. The repo provides a MATLAB pipeline—which handles isosurface extraction and rudimentary per-vertex colouring—and a Python 2.7 fallback that renders the regressed volume via OpenGL. Face detection is handled by an included alignment submodule, though the README warns that dlib can struggle with side poses.

The interesting bit

Instead of fitting a 3D Morphable Model or regressing pose parameters, the network treats reconstruction as a segmentation-like problem and directly outputs voxel occupancy. That trades the traditional iterative correspondence dance for a single forward pass, albeit one that needs enough memory to hallucinate a full 3D grid at once.

Key highlights

• Direct volumetric regression from a single image to a spatially aligned 3D volume
• MATLAB path is the canonical implementation for reproducing paper error metrics
• CPU-only Docker image and an online demo let you try outputs without installing legacy dependencies
• Python 2.7 fallback exists but lacks texture mapping and has known OpenGL rendering quirks
• Includes Bulat’s face-alignment code as a submodule for automatic face detection

Caveats

• The dependency stack is frozen around 2017: Torch7, CUDA 7.5/8.0, CuDNN 5.1, and Python 2.7, so expect friction on modern systems
• The Python/OpenGL path can glitch if background images are not power-of-two or if OpenGL is not fully configured
• Side-profile images may fail detection with the bundled dlib pipeline, requiring manual bounding boxes

Verdict

A solid reference if you are researching volumetric face reconstruction or need a baseline that avoids 3DMM fitting. Look elsewhere if you want a modern, actively maintained pipeline with up-to-date PyTorch and CUDA support.