A neural matcher that knows when to quit

What it does

LightGlue is a deep network that takes keypoints and descriptors from two images and returns which points correspond. It ships with pretrained weights for SuperPoint, DISK, ALIKED, and even old-school SIFT. The repo contains only inference code; training lives separately in the authors’ glue-factory library.

The interesting bit

The architecture adapts both its depth (how many transformer layers to run) and width (how many keypoints to keep) per image pair. Easy pairs get early-exited; difficult ones see iterative point pruning. The authors claim this hits 150 FPS at 1024 keypoints on an RTX 3080 with compilation enabled—a 4-10× speedup over SuperGlue. On CPU, it manages 20 FPS at 512 keypoints.

Key highlights

• FlashAttention and PyTorch 2.0 compilation are automatically exploited when available
• Now wrapped into Hugging Face Transformers for one-liner inference
• Exported ONNX variants exist (community-maintained) for TensorRT and OpenVINO
• Also exposed through kornia and usable via the hloc visual-localization toolbox
• Apache-2.0 licensed, though bundled SuperPoint weights carry a separate restrictive license

Caveats

• Point pruning carries overhead that can exceed its benefit for small keypoint counts; the default threshold is tuned for RTX 30-series GPUs and may need tuning for your hardware
• PyTorch compilation disables pruning below 1536 keypoints, which is a somewhat brittle heuristic

Verdict

Worth a look if you’re building SfM, visual localization, or any stereo-matching pipeline where latency matters. Skip it if you need end-to-end training in one repo or if you’re wedded to a feature extractor LightGlue doesn’t support.