Teaching SAM3 to count sheep, cells, and satellites

What it does Count Anything is a vision-language model built on SAM3 that takes an image and a natural-language query—think “nuclei,” “tractor,” or “tree”—and returns a set of discrete points, one per detected instance. The cardinality of that set is your count. It is trained on CLOC, a benchmark of roughly 220K images and 619 categories that spans general scenes, remote sensing, histopathology, cellular microscopy, agriculture, and microbiology.

The interesting bit Rather than emitting opaque density maps, the model produces actual instance points, so you can audit its work visually. It juggles both sparse, bulky objects and tiny, crowded ones by running two parallel counters—a Region-level Sparse Counter and a Pixel-level Dense Counter—and then merges their outputs with a parameter-free Complementary Count Fusion layer that suppresses duplicate counts.

Key highlights

• Cross-domain scope: trained and evaluated on six visual domains, including satellite imagery and microscopy.
• Annotation agnostic: digests heterogeneous labels—boxes, polygons, masks, rotated boxes, and label maps—and distills them into unified point supervision.
• Benchmark lead: the README notes it outperforms LocateAnything-3B on the full CLOC-v1.1 test set and across all six domains.
• Interpretable output: returns instance-grounded points instead of a single scalar or heatmap.
• Built atop SAM3 with LoRA fine-tuning; the inference checkpoint is hosted on Hugging Face.

Caveats

• The complete CLOC dataset cannot be directly redistributed due to source-dataset licenses; obtaining a ready-to-use copy requires emailing the authors or manually assembling raw sources.
• Training requires the official SAM3 pretrained weights, which are not included in the repository due to redistribution restrictions.

Verdict Worth exploring if you need open-vocabulary counting in scientific, agricultural, or satellite imagery where scale and density vary wildly. Skip it if you just need an off-the-shelf detector for everyday consumer photos.