A foundation model that actually went to med school — for retinas
RETFound packages a Nature-published retinal foundation model plus Meta's DINOv2/v3 into one fine-tuning pipeline for eye-disease detection.
What it does
RETFound is a self-supervised vision transformer pre-trained on 1.6 million retinal images (colour fundus photos and OCT), published in Nature in 2023. This repo is the official home for the weights and a fine-tuning pipeline: you bring a labelled dataset of retinal images, and the model adapts to your disease-detection task. It also folds in Meta’s general-purpose DINOv2 and DINOv3 as alternative backbones, so you can benchmark domain-specific pre-training against internet-scale generalists on the same code.
The interesting bit
The repo is quietly a controlled experiment: does pre-training on 1.6M retinal images beat DINOv2/v3 trained on enormous general corpora? A September 2025 preprint benchmarks exactly that, and the follow-up Nature Communications paper examines how pre-training data choice shapes downstream performance. The plumbing is the science.
Key highlights
- Six RETFound variants on HuggingFace (MAE- and DINOv2-based, ~300M params), pre-trained on different cohorts (Nature CFP/OCT, MEH, Shanghai)
- DINOv3 support spanning ~21M to ~6.7B parameters, all wired into the same fine-tuning script
- Both full fine-tuning and linear-probe modes via the
ADAPTATIONflag - Public benchmark data splits and model checkpoints documented in
BENCHMARK.md - Colab notebook and a feature-vector notebook for extracting latent embeddings
Caveats
- RETFound weights are gated behind a HuggingFace access form — not a plain public download
- DINOv2/v3 weights must be fetched separately from Meta’s repos and dropped into the folder manually
- Fine-tuning is single-GPU-oriented (
nproc_per_node=1in the examples); multi-node scaling isn’t demonstrated
Verdict
Worth a look if you work in ophthalmology AI or want a ready-made testbed for domain-specific vs. general-purpose foundation models. If your images aren’t of eyes, the general DINO backbones are the only part that concerns you.
Frequently asked
- What is rmaphoh/RETFound?
- RETFound packages a Nature-published retinal foundation model plus Meta's DINOv2/v3 into one fine-tuning pipeline for eye-disease detection.
- Is RETFound open source?
- Yes — rmaphoh/RETFound is an open-source project tracked on heatdrop.
- What language is RETFound written in?
- rmaphoh/RETFound is primarily written in Python.
- How popular is RETFound?
- rmaphoh/RETFound has 697 stars on GitHub.
- Where can I find RETFound?
- rmaphoh/RETFound is on GitHub at https://github.com/rmaphoh/RETFound.