A double-take engine for your image folders

What it does

imagededup is a Python library that finds exact and near-duplicate images hiding in a directory. It bundles four perceptual hashing algorithms—PHash, DHash, WHash, and AHash—with a CNN method that can run prepackaged or custom PyTorch models. The package also encodes images, plots duplicate clusters, and ships an evaluation framework to score deduplication quality against ground truth.

The interesting bit

The practical value is pairing fast perceptual hashes with a neural network that catches resized or transformed lookalikes. The README notes that CNNs win on near duplicates, while Difference hashing is fastest for exact copies—so you can trade accuracy for speed depending on how messy your dataset is.

Key highlights

• Combines four perceptual hashing algorithms with a CNN approach using prepackaged or custom PyTorch models.
• Ships an evaluation framework to measure deduplication quality against labeled ground truth.
• Plots duplicate clusters for visual inspection of results.
• Runs on Linux, macOS, and Windows under an Apache 2.0 license.

Caveats

• The published benchmarks are explicitly flagged as stale: they only hold through version 0.2.2, and the README warns that later releases changed the methods enough to invalidate prior numbers.
• Windows users are directed to separate documentation for platform-specific workflow quirks.

Verdict

Worth a look if you’re wrangling large image collections where manual review isn’t viable. Skip it if you already have a bespoke embedding pipeline and don’t need another abstraction layer.