Metrics for the SAM3 Encoder Distillation

[sqc2290318555]

Are there any metrics for the SAM3 Encoder Distillation stage? What specific accuracy can be achieved through distillation?

[SimonZeng7108]

That's a really good and important question. Currently, we have only benchmarked the Stage 1 unsupervised distillation image encoder performance on COCO, reporting mIoU; you can see the results at the bottom of the Model Zoo section.

The COCO dataset generally gave us a good baseline for how the image encoder performs in downstream geometry-prompt-only tasks. It showed good indications for us to proceed to the next stages, as all trained image encoders surpassed 60% mIoU (previous generation EfficientSAM models performed in the 55–60% range for the same prompts).

We have not been able to quantitatively benchmark the efficient text encoders yet, as we found it challenging to distill a high-performance text encoder at this stage due to the geometry prompt conditioning in the fusion module. This challenge is caused by our decoupled training strategy.

It will be our priority to develop thorough benchmarks after the training code for most stages is finalized and we see signs that all models have competitive performance. For the image model, we will definitely report the mIoU on standard benchmarks like COCO, DAVIS, and LVIS. For the text and combined models, the metrics used will likely be consistent with SAM 3 tasks.