Shrinking SAM3 by 99.85% without losing the plot

What it does

EfficientSAM3 takes SAM3 — Meta’s latest “promptable concept segmentation” model that understands text prompts and tracks objects through video — and distills it into tiny student networks. The goal is to run on phones, embedded boards, and anywhere a 462M-parameter backbone is a non-starter. It supports nine image encoder variants (RepViT, TinyViT, EfficientViT) and three text encoder options (MobileCLIP S0/S1, MobileCLIP2 L).

The interesting bit

The training curriculum is deliberately staged: Stage 1 distills image and text encoders using SAM1 data, Stage 2 grafts on a Perceiver-based memory module using SAM2 video data, and Stage 3 fine-tunes the full pipeline on SAM3’s own dataset. The authors call this “Progressive Hierarchical Knowledge Distillation” — PhD, naturally. The teaser EfficientViT-S model drops to 0.68M parameters, a 99.85% reduction from the teacher.

Key highlights

• Nine image encoder + three text encoder combinations, mix-and-matchable
• Text encoder compression up to 88% parameters with “similar performance” per the README
• Perceiver-based memory bank replaces SAM3’s dense memory for video tracking
• Same API contract as upstream SAM3; checkpoints on Hugging Face
• CoreML and ONNX export paths supported

Caveats

• Stage 2 and 3 weights are still in progress; only Stage 1 image/text encoders are fully released
• The data_engine branch for Stage 3 appears to be active development territory
• “Similar performance” claims are stated but no detailed benchmark tables are visible in the provided README excerpt

Verdict

Worth watching if you’re building mobile AR, drone vision, or any product where SAM3’s capabilities matter but its footprint doesn’t. Skip it if you need the bleeding-edge SAM3 feature set today — the full three-stage pipeline isn’t fully baked yet.