transformers - 💡(How to fix) Fix difference in performance of sam3 video model compared to original sam3 implementation

Hi, i ran the original sam3 video model (from meta) and the transformers sam3 video model on the same video with the same text prompt. I noticed that the original sam3 model detections were more accurate, and even that in the transformers' version some frames didn't have detections at all (whereas the original sam3 did correctly detect the object). i attached an example frame.

<img width="1436" height="1077" alt="Image" src="https://github.com/user-attachments/assets/0f67ae17-fc7a-45c8-9ddd-bebfaecbd69f" /> <img width="1436" height="1077" alt="Image" src="https://github.com/user-attachments/assets/ada8e09f-1b44-4607-b172-127890e42865" />

i also noticed that the transformers model takes less GPU memory, and runs faster, so maybe it is initialized with different hyper-parameters compared to the original?

bottom line, i expected the results to be identical. any reason for such a difference? i use the same code for video loading\processing in both cases.

help would be greatly appriciated,.