Attention takes on time-series forecasting, inverted

What it does

iTransformer is an unofficial PyTorch implementation of the Tsinghua/Ant Group paper on time-series forecasting with attention networks. The repository provides the core iTransformer module, an improvised iTransformer2D variant that also attends across patched time tokens, and an experimental iTransformerFFT that mixes in Fourier tokens, all returning predictions for multiple future horizons in one pass. It is strictly a model-building block, not a full training framework; the README shows model definitions but leaves data pipelines and loss functions to you.

The interesting bit

The README wryly notes that only tabular data remains—where xgboost still reigns—before the field can truly declare “Attention is all you need.” Beyond the joke, the repository treats the architecture as a testbed, improvising 2D and Fourier variants that go beyond the original paper’s design.

Key highlights

• Returns simultaneous multi-horizon forecasts (e.g., 12, 24, 36, 48 steps) from a single forward pass.
• iTransformer2D improvises attention across both variates and patched time tokens.
• iTransformerFFT is an experimental variant that adds Fast Fourier Transform tokens to the mix.
• Optional reversible instance normalization is included, though the author muses it may be redundant.
• Ships as bare PyTorch modules with no training loop or data pipeline attached.

Caveats

• This is an unofficial implementation; the authors’ official code is linked in the README.
• iTransformer2D, iTransformerFFT, and num_tokens_per_variate are explicitly labeled experimental or improvised.
• The README contains no benchmarks, training recipes, or reproducibility details.

Verdict

Grab this if you want an unofficial, hackable PyTorch module for experimenting with recent time-series attention architectures. Skip it if you need an official reference implementation or a batteries-included training framework.