Mammoth: the continual learning benchmark that refuses to forget anything
A PyTorch framework bundling 70+ continual learning methods and 20+ datasets so researchers can stop reimplementing DER++ and start comparing results.
What it does
Mammoth is a benchmarking framework for continual learning — the subfield where neural networks try to learn new tasks without catastrophically forgetting old ones. It ships with more than 70 methods (DER++, iCaRL, LUCIR, GEM, L2P, RanPAC, and friends) across 23 datasets, from MNIST variants to medical imaging. It’s also the official codebase of the NeurIPS 2020 “Dark Experience Replay” paper, which explains the name on the door.
The interesting bit
The design goal isn’t raw speed or elegance — it’s being easy to debug, which is rare honesty in research code. Everything is modular: new models drop into models/, new datasets into datasets/, and a --model_config=best flag pulls known-good hyperparameters from YAML files instead of making you guess learning rates. It also captures Ctrl+C to save a resumable checkpoint, a small mercy for anyone running multi-day experiments.
Key highlights
- 70+ continual learning methods, including recent pre-trained-model approaches like SLCA, ZSCL, and an ICLR 2026 Task Arithmetic variant with KFAC regularization
- 23 datasets spanning toy, natural-image, fine-grained, aerial, and medical domains
- Installable as a library (
pip install -e .) with usage examples inexamples/— though not yet on PyPI - Checkpoint resumption via
--loadcheck, including loading arguments saved with the checkpoint - Full documentation covering how to build your own models, datasets, and backbones
Caveats
- Requires PyTorch >= 2.1.0 for
scaled_dot_product_attention; older setups need manual edits tobackbone/vit.py - Some models need extra pinned dependencies (
timm==0.9.8,kornia), andgemdoesn’t work on Windows - The planned public results dashboard is explicitly “may take a time” — compute isn’t free, as they note
Verdict
If you do continual learning research and want apples-to-apples comparisons without rebuilding every baseline yourself, this is the repo. If you just want to train one model once, it’s a very large hammer for a very small nail.
Frequently asked
- What is aimagelab/mammoth?
- A PyTorch framework bundling 70+ continual learning methods and 20+ datasets so researchers can stop reimplementing DER++ and start comparing results.
- Is mammoth open source?
- Yes — aimagelab/mammoth is open source, released under the MIT license.
- What language is mammoth written in?
- aimagelab/mammoth is primarily written in Python.
- How popular is mammoth?
- aimagelab/mammoth has 852 stars on GitHub.
- Where can I find mammoth?
- aimagelab/mammoth is on GitHub at https://github.com/aimagelab/mammoth.