CVPR 2017's optical flow network, still compiling in 2024

What it does

FlowNet 2.0 takes two images and outputs a dense optical flow field — essentially a map of where every pixel moved between frames. The repo is a full Caffe fork with trained models, Python scripts for batch or single-pair inference, and training templates for three standard datasets (FlyingChairs, ChairsSDHom, FlyingThings3D).

The interesting bit

This is research code with the guardrails off. The authors ship raw .prototxt.template files you must manually edit to enable loss weights, set learning rates, and select datasets before training even starts. The README warns explicitly about path conflicts with other Caffe installs — a tell that this was built for a specific lab environment, not general distribution.

Key highlights

• Ships with pre-trained networks and a model downloader script
• run-flownet-many.py keeps Caffe loaded across batches, avoiding repeated initialization overhead
• Training templates include built-in affine augmentation on cropped regions
• Batch sizes are hard-tuned per dataset resolution (8 for 448×320, 4 for 768×384)
• Research-only license; commercial use requires explicit consent from the authors

Caveats

• Requires compiling a Caffe fork from source; no modern framework support (PyTorch, TensorFlow, etc.)
• Training setup demands hand-editing network definitions with no higher-level configuration
• “No other caffe version in your python and system paths” — dependency isolation is your problem

Verdict

Worth a look if you’re reproducing 2017 optical flow baselines or studying Caffe-era CNN architectures. Everyone else should probably search for a PyTorch reimplementation unless you enjoy Makefile archaeology.