An open-source labeler that brings its own AI models to work

What it does LabelU is a browser-based annotation platform for image, video, and audio data. It offers standard labeling tools—bounding boxes, segmentation masks, polylines, and keypoints for images; segmentation and classification for video and audio—and exports finished work to JSON, COCO, or MASK formats. The stack can run locally or be accessed through a hosted demo.

The interesting bit The project bundles three optional model servers—Florence-2, GroundingDINO + EfficientSAM, and SAM 3—for one-click AI auto-annotation of images, with VRAM requirements listed down to the gigabyte. That is unusually concrete for an open-source labeling tool, and it means you can batch-process object detection or segmentation locally rather than calling a cloud API or pre-labeling elsewhere.

Key highlights

• Multimodal support: image, video, and audio annotation in one interface.
• Built-in AI auto-annotation with three reference models, including a lightweight CPU-friendly option (~4 GB VRAM) and a high-end GPU option (~8 GB VRAM).
• Imports data directly from S3-compatible object storage such as AWS S3 or MinIO.
• Exports to standard formats: JSON, COCO, and MASK.
• Runs fully offline via a local Python server; the frontend is based on the separate labelU-Kit JavaScript library.

Caveats

• AI auto-annotation is currently limited to images; video and audio still require fully manual labeling.
• The bundled AI model servers require separate setup and discrete GPU resources, up to roughly 8 GB VRAM for the SAM 3 option.

Verdict Worth a spin for teams who need a self-hosted, multimodal labeling tool and want local AI models to handle the first pass on images. Purely text or tabular annotators should look elsewhere.