Video Loading for Neural Nets Without the Seeking Pain

What it does Decord is a C++ library with Python bindings that reads video and audio for machine learning. It wraps hardware-accelerated decoders like FFMPEG and Nvidia’s NVDEC to expose frame-level slicing, batching, and synchronized audio-video access through a simple Python API. The goal is to make video behave more like a random-access image dataset when you’re training models.

The interesting bit The project treats video seeking as an optimization problem. Its VideoLoader hides C++ tricks that minimize redundant seeks during shuffling, and it will even deduplicate frame indices in a batch internally so you don’t decode the same frame twice. There is also a unified AVReader that slices audio and video together, which is rarer than it should be in ML tooling.

Key highlights

• Hardware decoder support for FFMPEG/LibAV and Nvidia Codecs; Intel Codecs are listed but not marked complete.
• VideoLoader offers multiple shuffle modes (sequential, random file order, random frame access) to match different I/O patterns.
• Native tensor bridges for Apache MXNet, PyTorch, and TensorFlow (≥2.2.0) via a one-line set_bridge() call.
• VideoReader accepts file-like objects for in-memory decoding, not just file paths.
• Duplicate frame indices in a batch are accepted and handled internally to avoid redundant decoding.

Caveats

• GPU-accelerated decoding is not available in PyPI builds; enabling NVDEC requires compiling from source.
• The “smart shuffle” mode (shuffle=-1) is documented but not yet implemented.
• Intel Codecs support is listed on the roadmap without a completion marker, so its current status is unclear.

Verdict Worth a look if you’re building video-based training pipelines and need batched random access without writing your own decoder logic. Skip it if you just need sequential frame extraction or a general-purpose media player.