Vision model training from raw pixels to edge deployment

What it does

LightlyTrain wraps the full vision model lifecycle: self-supervised pretraining of DINOv2 and DINOv3 foundation models on your unlabeled images, then fine-tuning for object detection, instance segmentation, panoptic segmentation, and semantic segmentation. It also handles distillation to shrink models for edge deployment, with ONNX and TensorRT FP16 export built in. The API is a handful of lightly_train.train_* calls rather than a pile of YAML configs.

The interesting bit

The framework ships its own detection head (LTDETR) and integrates EoMT from CVPR 2025 for segmentation, claiming to edge out the original paper’s numbers on COCO—by 0.1 PQ on panoptic and 0.3 mask mAP on instance segmentation with the same DINOv3/ViT-L backbone. It also goes the other direction: PicoDet variants with 1.17M parameters run at 2.2ms on a T4 for embedded use.

Key highlights

• Pretraining and distillation for DINOv2/v3; custom teacher support added in v0.15.0
• Object detection with DINOv3 backbones (ConvNeXt and ViT variants), plus tiny PicoDet models for low-power devices
• Panoptic, instance, and semantic segmentation via EoMT; benchmarks provided on COCO 2017
• ONNX/TensorRT FP16 export for all tasks; latency numbers measured on NVIDIA T4
• Python 3.8–3.13, Windows/Linux/macOS

Caveats

• Commercial use requires contacting the company for a license; the repo does not state an open-source license in the README
• PicoDet results are marked “preliminary” and “in preview”
• The README is heavy on benchmark tables and light on architecture or training methodology details

Verdict

Worth a look if you need to go from unlabeled pretraining to deployed detection or segmentation without stitching together three separate frameworks. Skip it if you need fully open licensing clarity or want to roll your own training loops from scratch.