Track cheetahs, flies, and bacteria with the same toolbox

What it does DeepLabCut is a Python toolbox for markerless pose estimation. You label a few frames of video to mark the features you want to follow, and it trains a deep-learning model to find those same keypoints across the rest of your footage. The same codebase has been used on mice, cheetahs, fruit flies, fish, leeches, bacteria, horses, humans, and even robots.

The interesting bit The project began by borrowing the feature detectors from DeeperCut, a human-pose algorithm, and re-purposing them for the rest of the animal kingdom. Version 3.0 now ships with foundation models like SuperAnimal-Quadruped and SuperAnimal-TopViewMouse that aim to work out-of-the-box on animals the developers have never met, alongside a menu of backbones from ResNet to RTMPose.

Key highlights

• Tracks single or multiple animals in real time from standard video, with no special hardware or markers.
• Foundation pretrained models (SuperAnimal-Quadruped, SuperAnimal-TopViewMouse) are provided for out-of-distribution testing on unseen animals.
• Supports multiple deep-learning engines and backbones; PyTorch is the current default, while TensorFlow is supported but slated for deprecation by 2027.
• Offers both a GUI and a Python API, targeting life scientists and computer-vision researchers alike.
• Backed by extensive documentation, a free online course, a Nature Protocols paper, and Docker support.

Caveats

• Requires Python 3.10+, and TensorFlow support is capped at version 2.10 on Windows and 2.12 on other platforms.
• The README warns that there are “many different models to select from in DeepLabCut 3.0,” so expect a learning curve when choosing an architecture.

Verdict Life scientists studying animal behavior and computer-vision researchers needing a battle-tested, species-agnostic tracker should take a look. If you only need human pose estimation, narrower tools will suffice.