DeepMind's hierarchical image codec, rebuilt for PyTorch tinkerers

What it does Implements the VQ-VAE-2 architecture in PyTorch for generating 256px images through hierarchical vector quantization. The pipeline splits work across two stages: first a VQ-VAE encodes images into discrete top- and bottom-level codes, then a PixelSNAIL autoregressive model learns to generate those codes. The author provides a pretrained checkpoint on the FFHQ face dataset so you can skip straight to stage two or inspect reconstructions.

The interesting bit The project treats image generation as a compression problem with two timezones—stage one builds a discrete codebook, stage two predicts entries from it. Intermediate codes are cached in LMDB between stages, keeping the pipeline modular and avoiding repeated re-encoding of the dataset.

Key highlights

• Ships with a pretrained VQ-VAE checkpoint trained on FFHQ (vqvae_560.pt)
• Supports distributed training across multiple GPUs for the VQ-VAE stage
• Splits generation into top and bottom hierarchical priors at 256px resolution
• Stores intermediate discrete codes in LMDB for efficient stage-two training
• Pure PyTorch implementation with minimal dependencies

Caveats

• The bundled PixelSNAIL model is intentionally smaller than the paper’s spec; the author notes this is due to GPU memory constraints
• Only 256px resolution is currently supported
• The sample shown is explicitly labeled as a training sample, not a novel generation

Verdict Useful if you want a practical PyTorch implementation of VQ-VAE-2 with pretrained weights for faces. Look elsewhere if you need the original paper’s full model capacity out of the box.