Bayesian optimization that actually runs on your GPU

What it does

BoTorch is a low-level toolkit for Bayesian optimization built natively on PyTorch. It gives you modular primitives—Gaussian process models, acquisition functions, optimizers—that you compose into custom BO loops. The library leans heavily on GPyTorch for probabilistic modeling and is explicitly designed as the research substrate for Meta’s higher-level Ax platform.

The interesting bit

The reparameterization trick for Monte Carlo acquisition functions is the quiet hero here. Instead of wrestling with closed-form derivations for every new acquisition function, you get gradient estimates via auto-differentiation—meaning your custom acquisition ideas can run on a GPU without rewriting the math. The README also notes, almost as an aside, that double precision is “highly recommended for GPs,” which is the kind of unglamorous detail that saves hours of silent numerical misery.

Key highlights

• Native PyTorch: auto-diff, GPU parallelism, and dynamic computation graphs out of the box
• First-class GPyTorch integration, including multi-task GPs, deep kernel learning, and deep GPs
• Monte Carlo acquisition functions via reparameterization—new ideas don’t need restrictive model assumptions
• Explicitly positioned as a low-level API; end-users are steered toward the Ax platform instead
• Active beta with a 2020 NeurIPS paper backing the core framework

Caveats

• Still in beta; the README warns of active development and occasional bleeding-edge bugs
• The target audience is researchers and “sophisticated practitioners,” not casual end-users
• Official Anaconda publishing stopped after v0.12; conda-forge is the remaining channel

Verdict

Worth a look if you’re implementing novel acquisition functions or need GPU-accelerated Bayesian optimization in a PyTorch ecosystem. If you just want to tune some hyperparameters, the README itself tells you to use Ax instead—and you should probably listen.