Matching pixels globally instead of iterating locally

What it does

GMFlow estimates optical flow—the motion of pixels between two consecutive images—by treating the task as a global matching problem instead of the usual iterative refinement. The framework splits work into five swappable stages: feature extraction, enhancement, matching, propagation, and optional refinement. A pretrained model can infer bidirectional flow and occlusion maps from a sequence of frames.

The interesting bit

The project’s central bet is that matching features globally is more efficient than RAFT-style sequential iteration. That bet pays off on the Sintel benchmark, where GMFlow with one refinement pass outperforms RAFT running thirty-one refinement steps. Because the computation is less sequential, the gap widens on high-end GPUs: a basic GMFlow model hits 26 ms on an A100 versus 57 ms on a V100 for 436×1024 frames.

Key highlights

• Outperforms 31-iteration RAFT on Sintel with a single refinement pass
• Modular five-component design lets you swap in custom feature extractors or matchers
• Bidirectional flow requires no second network forward pass, enabling occlusion detection via forward-backward consistency checks
• Basic inference runs at 57 ms on a V100 or 26 ms on an A100 for standard Sintel resolution
• Training demands heavy hardware: 4× 16 GB V100s for the base model, or up to 8× V100s / 4× A100s for the refined version

Caveats

• The authors now direct users to their newer UniMatch project for stereo and depth extensions, though additional GMFlow pretrained models are still released there
• Training hardware requirements are steep and you may need to tune batch size and iterations to fit your setup

Verdict

Study this if you want a modern, non-iterative optical flow baseline with strong accuracy-efficiency tradeoffs. Look elsewhere if you are hunting for a lightweight training recipe on consumer GPUs.