Swap your PyTorch dataloader and train ImageNet for 98¢

What it does

FFCV is a drop-in replacement for torch.utils.data.Dataloader that converts your dataset into a custom .beton format, then feeds training with compiled, asynchronous pipelines. It keeps your model code untouched and claims to train an ImageNet model on one GPU in 35 minutes for 98¢ on AWS, and a CIFAR-10 model in 36 seconds for 2¢.

The interesting bit

Instead of hacking your model for speed, FFCV attacks the data bottleneck: it uses Numba to compile simple Python augmentations down to machine code, fuses them into pipelines, and automatically handles caching across RAM, SSD, or networked disk. The bet is that data loading, not architecture, is what keeps GPUs underutilized.

Key highlights

• Claims to train an ImageNet model in 35 minutes and CIFAR-10 in 36 seconds on a single GPU by removing data bottlenecks
• Drop-in replacement for standard PyTorch data loaders; model training code stays identical
• Compiles and fuses augmentation pipelines via Numba, scheduling them asynchronously to hide latency
• Adapts to hardware constraints, using OS-level RAM caching when possible and minimizing disk reads otherwise
• Ships with premade benchmark scripts and configs for ImageNet and CIFAR-10

Caveats

• Requires converting your dataset to a custom .beton format before training
• Windows setup involves manual PATH tweaks and editing a pthread.h header file, which looks brittle
• Linux installation is tightly coupled to conda and specific CUDA toolkit versions (11.3 mentioned)

Verdict

If you’re burning cloud credits waiting on DataLoader, FFCV is worth the format conversion. If your model is already compute-bound or you live on Windows, it may cause more friction than relief.