Bayesian optimization that races configs head-to-head

What it does SMAC3 is a Python framework for Bayesian hyperparameter optimization. You define a configuration space and a target function; it searches for well-performing hyperparameters using a surrogate model (now sklearn’s Random Forest by default) paired with an intensifier that quickly weeds out losers. It plugs into standard ML pipelines via a simple optimize() call.

The interesting bit The “aggressive racing mechanism” is the twist: instead of evaluating every candidate fully, SMAC3 runs a statistical race to decide which of two configurations actually performs better, cutting budget for obvious duds. The v2.0 rewrite also added native multi-objective, multi-fidelity, and multi-threading support across all facades—plus an ask-and-tell interface for embedding in larger loops.

Key highlights

• Ask-and-tell interface plus run resumption for interactive or distributed workflows
• Multi-objective and multi-fidelity optimization supported natively in all facades
• Switched to sklearn Random Forest as default surrogate in v2.4.0; legacy C++-based pyrfr still available via smac[pyrfr]
• DeepCAVE companion tool for visualizing optimization runs
• BSD-3 licensed, backed by AutoML groups at Hannover and Freiburg

Caveats

• v2.0 dropped the command-line interface and runtime optimization; if you need those, stay on v1.4
• Python 3.7 support removed; tested on 3.8–3.10
• pyrfr path requires installing gcc, gxx, and swig manually

Verdict Worth a look if you’re doing serious hyperparameter search beyond grid/random baselines and want fine-grained control over the optimization loop. Skip if you just need a one-liner wrapper around a sklearn estimator.