How to train a ResNet by making it argue with its own reflection

What it does

SimCLR is a self-supervised learning framework for computer vision. It trains ResNets to produce useful image embeddings by contrasting differently augmented versions of the same picture, treating matching pairs as positives and everything else as negatives. The repo contains the official TensorFlow implementation for both SimCLRv1 and v2, along with pretrained checkpoints, fine-tuning scripts, and evaluation notebooks.

The interesting bit

The real payoff is semi-supervised learning. The project ships pretrained checkpoints and fine-tuned variants for ImageNet subsets using only 1% or 10% of the labels, alongside fully supervised baselines for comparison. It also includes distilled self-trained checkpoints and a TF2 rewrite, suggesting the original research code has been dusted off for broader use.

Key highlights

• 65 SimCLRv2 pretrained models ranging from 24M to 795M parameters, with linear evaluation and fine-tuning results on ImageNet.
• Native support for CIFAR-10 and ImageNet pretraining, plus ready-made subsets for 1% and 10% semi-supervised experiments.
• Checkpoints include pretrained, fine-tuned, supervised baselines, and distilled variants stored in Google Cloud Storage.
• A TF2 reimplementation lives in the tf2/ folder, and the codebase supports both TensorFlow 1.x and 2.x.
• Includes Colab notebooks for fine-tuning and for exploring “Intriguing Properties of Contrastive Losses”.

Caveats

• Does not support multi-GPU training; global BatchNorm and cross-core contrastive loss require TPUs or a single GPU.
• Pretraining on ImageNet at paper scale demands 32 TPU v3s and a batch size of 4096, so reproducing from scratch is cloud-heavy.
• SimCLRv1 checkpoints lack the projection head, which limits compatibility with some v2 fine-tuning workflows.

Verdict

Researchers and practitioners experimenting with self-supervised or semi-supervised vision should grab the checkpoints and Colabs. If you are looking for a plug-and-play PyTorch model or need to train on a commodity multi-GPU rig, you will want to look elsewhere or use the third-party converters.