Uncertainty quantification without the Bayesian hand-waving

What it does

This is an awesome-list that catalogs tutorials, papers, libraries, and courses for conformal prediction — a framework that wraps any model and outputs prediction sets or intervals with finite-sample coverage guarantees. The maintainer, Dr. Valeriy Manokhin, studied under Vladimir Vovk, who created the field.

The interesting bit

Conformal prediction’s core trick is almost insultingly simple: score a calibration set with a “nonconformity” function, then at test time keep anything less weird than the (1 − α) quantile of those scores. The resulting intervals work for random forests, neural networks, LLMs — anything — with no assumptions about the data distribution beyond exchangeability.

Key highlights

• Curated since 2015; includes Python, R, and Julia libraries, plus Kaggle notebooks
• Covers specialized domains: time series forecasting, anomaly detection, LLM uncertainty
• Lists dedicated conference tracks (ICML 2021–2022, NeurIPS 2022 keynote) and the long-running COPA conference
• Maintained by a researcher with direct lineage to the field’s creator
• CC BY-NC-ND 4.0 licensed; carries a Zenodo DOI for academic citation

Caveats

• The list itself is a reference index, not a library — you’ll still need to evaluate which implementation fits your stack
• The README promotes the maintainer’s paid course and book editions prominently
• Some open research questions are flagged (distribution shift, transformer compatibility) without resolution

Verdict

Worth bookmarking if you’re shipping models where overconfidence costs money or safety. Skip it if you already have a conformal prediction workflow and just need a specific package — go to those repos directly.