Boosting trees for data that refuses to be independent

What it does

GPBoost is a C++ library that merges gradient tree boosting with latent Gaussian models—Gaussian processes and grouped random effects. It learns a non-linear predictor as an ensemble of trees while simultaneously estimating covariance parameters for spatial, grouped, or longitudinal dependencies. The result is a model that can quantify uncertainty on tabular data where samples are not conditionally independent.

The interesting bit

Classical boosting assumes your rows are independent, which turns spatial maps into patchy discontinuities and wastes information on high-cardinality categorical variables. GPBoost treats the tree ensemble as the fixed-effect mean function inside a mixed-effects or GP framework, so the model learns non-linear trends while respecting the actual correlation structure of the data.

Key highlights

• Supports Gaussian likelihoods (GPBoost algorithm) and non-Gaussian likelihoods via link functions (LaGaBoost algorithm).
• Handles grouped, nested, crossed, and random-coefficient effects alongside Gaussian processes, including combinations thereof.
• Exposes Python and R APIs over a C++ core, with scalable GP approximations for larger datasets.
• Methodology is published in JMLR and IEEE TPAMI.

Caveats

• Several features are still on the roadmap: no multiclass classification, no CI environment, no ONNX export, and Python test coverage lags behind R.
• Non-Euclidean distances for GPs and sample weights for Gaussian likelihoods are not yet supported.

Verdict

Worth a look if you are hitting the ceiling of classical tree boosting on spatial, panel, or grouped data and need probabilistic outputs. If your data is i.i.d. tabular and you just want raw prediction speed, the covariance modeling is likely overkill.