One transformer for forecasting, gaps, and anomalies

What it does

MOMENT is a family of transformer models pre-trained on the Time-series Pile, a large collection of public time-series datasets. It splits sequences into fixed-length patches, randomly masks them, and learns to reconstruct the signal—applying the masked-autoencoder approach to temporal data. Released checkpoints handle forecasting, classification, anomaly detection, imputation, and embedding extraction through lightweight heads or linear probes.

The interesting bit

Zero-shot, the model already outperforms statistical imputation baselines, beats ARIMA on some short-horizon forecasts, and scores second-best F1 on anomaly detection. With only a linear probe, it achieves best F1 on anomaly detection and beats baselines on four of six imputation datasets. PCA on synthetic sinusoids suggests the embeddings encode trend, scale, frequency, and phase, implying the model learns a structured vocabulary for time rather than dataset-specific noise.

Key highlights

• Pre-trained via masked patch reconstruction on the Time-series Pile
• Zero-shot inference for imputation, anomaly detection, classification, and short-horizon forecasting
• Linear probing yields best F1 on anomaly detection and competitive long-horizon forecasting
• Embeddings cluster by trend, scale, frequency, and phase in PCA space
• Small, base, and large sizes; reproducible on a single NVIDIA A6000 with 48 GiB RAM

Caveats

• This repo is an intentionally “extremely lightweight” release; full research code lives elsewhere.
• Multi-channel classification support was only recently fixed.
• Python 3.11 is the sole supported version for now.

Verdict

Worth a look if you need a general-purpose time-series backbone and prefer not to train from scratch. Skip it if you require a mature, all-in-one framework with broad Python version support.