← all repositories
aimagelab/mammoth

Mammoth: the continual learning benchmark that refuses to forget anything

A PyTorch framework bundling 70+ continual learning methods and 20+ datasets so researchers can stop reimplementing DER++ and start comparing results.

★852 stars Python ML Frameworks
mammoth
Too new to rank. Velocity needs a few days of history.
star history

What it does

Mammoth is a benchmarking framework for continual learning — the subfield where neural networks try to learn new tasks without catastrophically forgetting old ones. It ships with more than 70 methods (DER++, iCaRL, LUCIR, GEM, L2P, RanPAC, and friends) across 23 datasets, from MNIST variants to medical imaging. It’s also the official codebase of the NeurIPS 2020 “Dark Experience Replay” paper, which explains the name on the door.

The interesting bit

The design goal isn’t raw speed or elegance — it’s being easy to debug, which is rare honesty in research code. Everything is modular: new models drop into models/, new datasets into datasets/, and a --model_config=best flag pulls known-good hyperparameters from YAML files instead of making you guess learning rates. It also captures Ctrl+C to save a resumable checkpoint, a small mercy for anyone running multi-day experiments.

Key highlights

  • 70+ continual learning methods, including recent pre-trained-model approaches like SLCA, ZSCL, and an ICLR 2026 Task Arithmetic variant with KFAC regularization
  • 23 datasets spanning toy, natural-image, fine-grained, aerial, and medical domains
  • Installable as a library (pip install -e .) with usage examples in examples/ — though not yet on PyPI
  • Checkpoint resumption via --loadcheck, including loading arguments saved with the checkpoint
  • Full documentation covering how to build your own models, datasets, and backbones

Caveats

  • Requires PyTorch >= 2.1.0 for scaled_dot_product_attention; older setups need manual edits to backbone/vit.py
  • Some models need extra pinned dependencies (timm==0.9.8, kornia), and gem doesn’t work on Windows
  • The planned public results dashboard is explicitly “may take a time” — compute isn’t free, as they note

Verdict

If you do continual learning research and want apples-to-apples comparisons without rebuilding every baseline yourself, this is the repo. If you just want to train one model once, it’s a very large hammer for a very small nail.

Frequently asked

What is aimagelab/mammoth?
A PyTorch framework bundling 70+ continual learning methods and 20+ datasets so researchers can stop reimplementing DER++ and start comparing results.
Is mammoth open source?
Yes — aimagelab/mammoth is open source, released under the MIT license.
What language is mammoth written in?
aimagelab/mammoth is primarily written in Python.
How popular is mammoth?
aimagelab/mammoth has 852 stars on GitHub.
Where can I find mammoth?
aimagelab/mammoth is on GitHub at https://github.com/aimagelab/mammoth.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.