Your model made a prediction. Here's the itemized bill.

What it does

SHAP computes Shapley values—borrowed from cooperative game theory—to allocate exact credit among input features for any machine learning model’s prediction. It tells you which features pushed a specific output higher or lower, and by how much, relative to a baseline average. The library wraps these calculations in model-specific explainers and a suite of visualization tools that turn raw attributions into waterfall charts, beeswarm plots, and pixel-level heatmaps.

The interesting bit

The library treats each prediction as a coalitional game where features are players and the payout is the model output. That theoretical foundation lets it offer exact, C++-backed algorithms for tree ensembles while using approximation schemes like DeepExplainer and GradientExplainer for deep learning and NLP models, all backed by the same Shapley formalism.

Key highlights

• Exact, fast C++ explainers for XGBoost, LightGBM, CatBoost, scikit-learn, and PySpark tree models.
• Native support for Hugging Face transformers, including text-specific visualizations that highlight which tokens drive a prediction.
• DeepExplainer and GradientExplainer for TensorFlow/Keras; DeepExplainer notes preliminary PyTorch support.
• Model-agnostic KernelExplainer that works on any function, though the README example is truncated.
• Optional GPU acceleration for Tree SHAP, available when building from source against the CUDA toolkit.

Caveats

• The README’s model-agnostic KernelExplainer section is cut off mid-sentence, so the full scope of that explainer is unclear from the source.
• DeepExplainer’s PyTorch support is labeled preliminary.
• GPU acceleration for Tree SHAP requires building from source with the CUDA toolkit; it is not enabled in the standard PyPI/conda install.

Verdict

Worth a look if you need to justify individual model decisions to humans—regulators, clinicians, or just your future self. If you already trust your black box and don’t care why it guesses, skip it.