SAM sees everything; this teaches it to care about your pixels

What it does

This project repackages SAM’s image encoder, prompt encoder, and mask decoder into swappable adapters. You bring a dataset and a task name—semantic segmentation is the only fully checked-out option right now—and the framework handles the fine-tuning loop while letting you freeze individual modules or tune their learning rates independently. The idea is to turn SAM’s generalist “segment any object” reflex into a specialist habit for your particular data.

The interesting bit

Rather than treating SAM as a frozen backbone, it exposes the internals: MaskDecoderAdapter, for instance, splits into a DecoderNeck and DecoderHead so you can modify the model’s guts instead of just bolting on a new classification layer. The README also pitches an end-to-end flow covering training, testing, and deployment, though several of those boxes remain unchecked.

Key highlights

• Adapter-based wrappers for SAM’s three core sub-networks.
• Granular control over which parameters are frozen and per-module lr or weight-decay settings.
• Semantic segmentation training and evaluation are live; matting, instance segmentation, and detection are on the roadmap.
• Dataset loaders provided for TorchVOCSegmentation and a generic BaseSemantic format.
• Future plans include MobileSAM support and ONNX export.

Caveats

• Semantic segmentation lacks a test pipeline, and every other task is still unchecked.
• Dataset support is currently limited to semantic formats; instance and matting loaders are pending.
• ONNX export is listed but not yet implemented.

Verdict

Grab it if you want a structured way to surgically tweak SAM’s internals for a narrow domain. Pass if you need mature, out-of-the-box support for detection, matting, or one-click deployment today.