Self-supervised ResNets with a fixed predictor

What it does This repository implements the SimSiam paper for unsupervised pre-training of ResNet-50 on ImageNet. It trains a siamese network to learn visual representations without labels, then evaluates them by fitting a linear classifier on frozen weights. The codebase is explicitly built as a thin layer over PyTorch’s standard ImageNet training example and the MoCo release.

The interesting bit The entire twist is a predictor whose learning rate refuses to decay during the 100-epoch pre-training run, matching the paper’s final Table 1 row. Otherwise, the authors change almost nothing in a standard PyTorch ImageNet pipeline—deliberately boring, and therefore easy to audit.

Key highlights

• Pre-trained ResNet-50 checkpoints and full logs are provided for 100-epoch runs, with linear evaluation reaching 68.1% (batch size 512) and 68.3% (batch size 256) top-1 accuracy.
• The training scripts reuse the standard PyTorch ImageNet example and the MoCo v2 augmentation recipe with only minimal modifications.
• Only multi-GPU DistributedDataParallel is supported; single-GPU or DataParallel training is explicitly disabled.
• Linear evaluation requires the LARS optimizer via NVIDIA Apex and defaults to a batch size of 4096.
• Everything is under a CC-BY-NC 4.0 license, so commercial use is off the table.

Caveats

• You need a multi-GPU machine; single-GPU or DataParallel setups are explicitly unsupported.
• Linear evaluation depends on NVIDIA Apex installed solely for its LARS implementation.
• The CC-BY-NC 4.0 license blocks commercial applications.

Verdict Grab this if you are reproducing ImageNet self-supervised baselines or auditing how siamese architectures behave. Look elsewhere if you need a single-GPU workflow, a commercially permissive license, or a framework that abstracts away standard PyTorch training details.