Instance segmentation that keeps up with your webcam

What it does YOLACT is a fully convolutional instance segmentation model built to run in real time on video streams and webcams. The PyTorch implementation ships with pretrained weights and evaluation tools for COCO, still images, and live video. The project unapologetically trades absolute accuracy for frame rate.

The interesting bit The name—You Only Look At CoefficienTs—telegraphs the paper’s central shortcut. The README frames it as a fully convolutional alternative to heavier instance segmentation pipelines, and the payoff is speed: the base ResNet101 variant runs at 33.5 fps on a Titan Xp while posting 29.8 mAP on COCO’s test-dev. In other words, it treats latency as a bug.

Key highlights

• Speed first: the base ResNet50 model hits 42.5 fps at 28.2 mAP on COCO test-dev, and even the deeper ResNet101 keeps 33.5 fps at 29.8 mAP (550 px input).
• YOLACT++ adds deformable convolutions via DCNv2, lifting ResNet50 to 34.1 mAP at 33.5 fps, though you must compile custom CUDA layers to use it.
• Evaluation covers COCO benchmarks, folder batches, single images, video files, and live webcam feeds.
• Training scales across multiple GPUs with automatic hyperparameter adjustment based on total batch size.
• Pretrained weights have been re-uploaded to HuggingFace after the original hosting links expired.

Caveats

• Original pretrained weights links expired; you now need to fetch them from the author’s HuggingFace collection.
• YOLACT++ requires compiling DCNv2 deformable convolution layers, so it is not a pure-Python upgrade.
• A guide for visualizing training logs is listed as “coming soon,” leaving LogVizualizer in utils/logger.py as the only reference.

Verdict Grab this if you need instance segmentation that won’t drop frames on a video stream; look elsewhere if you are chasing maximum mAP regardless of speed. It is built for the real-time crowd, not the leaderboard crowd.