Medical image annotation that learns while you click

What it does

MONAI Label is a server-based annotation system for medical imaging that pairs human experts with AI models. It serves interactive segmentation tools to familiar clinical viewers—3D Slicer and OHIF for radiology, QuPath and Digital Slide Archive for pathology, CVAT for endoscopy—so users can label CT, MRI, whole-slide images, and video frames from a single local server. The system runs fully offline on GPU hardware, with the server and client able to live on the same machine or across a network.

The interesting bit

The project treats annotation as a continuous feedback loop: user corrections are fed back to adapt the models, meaning the AI theoretically improves as it watches you work. It also bundles SAM2 by default for zero-shot organ segmentation, which is a pragmatic admission that sometimes you just want a foundation model to guess the boundaries before the human cleans them up.

Key highlights

• Supports radiology, pathology, and endoscopy through one server architecture with viewer plugins for 3D Slicer, OHIF, QuPath, DSA, and CVAT
• Ships with active-learning workflows and interactive models like DeepEdit and DeepGrow for iterative refinement
• Includes SAM2 (2D and 3D) out of the box for Python 3.10+ installations
• Connects to clinical PACS infrastructure via DICOMWeb
• Runs fully offline on local GPU hardware; server and client can reside on the same machine or separate ones

Caveats

• The supported viewer and modality matrix is explicitly described as limited to what the team has explicitly tested, so your specific file format or workflow may require troubleshooting.
• Switching from the bundled SAM2 to SAM-2.1 requires manual package uninstallation and reinstallation if you used pip rather than Docker.

Verdict

A solid choice for medical imaging teams that need an offline, GPU-powered bridge between research models and clinical annotation tools. If you are not working with DICOM, whole-slide images, or endoscopy frames, this is not your tool.