A contextual bandits toolkit honest enough to warn you off itself

What it does Contextual bandits sit between classification and reinforcement learning: you pick an action based on features, but you only see the reward for the choice you made, never the “correct” label. This package implements online, off-policy, and evaluation algorithms—mostly meta-heuristics that wrap binary classifiers like logistic regression or XGBoost—to help you prototype policies, reproduce papers, or just learn the trade-offs. It focuses strictly on binary rewards and shared covariates across all arms.

The interesting bit The author is refreshingly blunt: many of these methods fail to beat simpler baselines, and the docs tell you exactly which algorithms are the “safest bets” if you refuse to tune hyperparameters. That honesty is rarer than a perfectly tuned Thompson sampler.

Key highlights

• Covers three workflows: online learning (contextualbandits.online), off-policy learning from historical data (contextualbandits.offpolicy), and policy evaluation (contextualbandits.evaluation).
• Most methods are classifier-agnostic meta-heuristics; plug in anything with predict_proba or decision_function.
• Supports streaming/batch updates via partial_fit when the base classifier allows it, and includes an incremental linear regression class that matches the closed-form solution.
• Ships with implementations of LinUCB, BootstrappedUCB, Offset Tree, Doubly-Robust estimators, and others drawn directly from academic papers.
• Requires a C compiler for the Cython-accelerated parts; standard pickle serialization is explicitly discouraged in favor of cloudpickle or dill.

Caveats

• The README explicitly warns that many algorithms often lose to naïve benchmarks and should be used for comparison rather than relied upon blindly.
• It handles only discrete {0,1} rewards and assumes all arms share the same feature space; continuous rewards and social-network side information are out of scope.
• Several methods assume probabilistic classifier outputs, so raw SVM scores will misbehave without manual recalibration.

Verdict Grab this if you are a researcher or student trying to reproduce a paper or compare bandit strategies without writing boilerplate. Skip it if you need a battle-tested production recommender system or continuous reward support.