Wrap sklearn models in prediction intervals with a warranty

What it does

crepes wraps standard scikit-learn classifiers and regressors to provide conformal prediction. It augments point predictions with well-calibrated p-values, cumulative distribution functions, or prediction sets and intervals that carry explicit coverage guarantees. The library covers standard, normalized, and Mondrian conformal classifiers and regressors, plus conformal predictive systems.

The interesting bit

Instead of forcing you to retrain models inside a specialized framework, it treats your existing sklearn estimator as a black-box learner and layers statistical rigor on top through a calibration set. It also includes modules for difficulty estimation and martingale-based exchangeability testing, so you can diagnose when your data drifts.

Key highlights

• Drops in on top of any scikit-learn classifier or regressor via WrapClassifier and WrapRegressor.
• Emits prediction sets, intervals, p-values, or CDFs with guaranteed coverage levels.
• Supports Mondrian categories to control error rates conditionally across subgroups.
• Semi-online mode updates the calibration set after each prediction to maintain independence.
• Bundles crepes.extras for difficulty-aware normalization and crepes.martingales for exchangeability checks.

Caveats

• Normalized regression intervals can balloon unpredictably if the difficulty estimator is uninformative or poorly tuned.

Verdict

A solid choice for tabular-data practitioners who want rigorous uncertainty quantification without abandoning their sklearn pipelines. Those working outside the scikit-learn ecosystem will need to find a different conformal wrapper.