What it does OpenSTL is a benchmarking framework for spatio-temporal predictive learning — essentially, predicting future video frames or weather maps from past ones. It bundles 13 methods (from 2015’s ConvLSTM to 2024’s WaST) and 10 datasets spanning Moving MNIST, traffic flow, human motion, and weather forecasting. The code splits into three layers: core training logic, algorithm implementations, and a user-facing API, plus a PyTorch Lightning variant if you prefer that flavor.

The interesting bit The real work here isn’t the models — most are reimplementations — but the standardization. The authors decompose every method into interchangeable methods, models, and modules, so you can swap a ConvLSTM backbone for a Swin Transformer without rewriting the training loop. It’s the kind of boring infrastructure that makes or breaks reproducible research.

Key highlights

• 13 prediction methods, 11 MetaFormer variants for SimVP, 10 datasets with download scripts and configs
• Three-layer abstraction: core → algorithm → UI, with unified experiment API
• PyTorch Lightning branch recommended; vanilla PyTorch available
• Colab tutorial and Jupyter notebook for custom data
• NeurIPS 2023 Dataset and Benchmark Track acceptance

Caveats

• Benchmark results are still being collected; the README notes they are “working on training benchmarks and will update results synchronizingly”
• Some datasets require Baidu Cloud access with extraction password
• Python capped at ≤3.10.8, and xarray pinned to 0.19.0

Verdict Grab this if you’re doing spatio-temporal prediction research and tired of reimplementing baselines. Skip if you need production-ready video generation — this is a research benchmark, not a deployment framework.