What it does

HyperparameterHunter wraps your existing ML libraries—scikit-learn, Keras, XGBoost, LightGBM, CatBoost, RGF—and automatically records every experiment’s hyperparameters, scores, and predictions to disk. When you later run optimization, it feeds all that accumulated history into the search instead of starting from zero.

The interesting bit

The pitch is behavioral: use it for every experiment from day one, not just when you decide to “do hyperparameter optimization.” The more you use it, the smarter its optimization gets. It’s a bet that your one-off baselines and failed attempts contain signal that standard optimizers throw away.

Key highlights

• Wraps familiar APIs without changing their calling conventions—Keras build_fn, sklearn constructors, etc.
• Supports multiple optimization backends: Bayesian, Random Forest, Extra Trees, gradient-boosted regression trees, and a dummy baseline
• Automatically generates leaderboards, prediction files, and structured JSON experiment descriptions
• Records default hyperparameters you didn’t explicitly set, so you know exactly what ran
• Environment object centralizes data, CV strategy, and metrics across experiments

Caveats

• README is vague on how exactly past experiments inform optimization (is it warm-starting surrogate models, or just avoiding re-evaluation?)
• 707 stars and a Travis CI badge suggest modest maintenance; no clear signal on active development pace

Verdict

Worth a look if you’re running dozens of experiments across multiple libraries and tired of losing track of what you already tried. Skip if you want a lightweight, drop-in optimizer for a single afternoon of tuning—this wants to be your long-term notebook.