ollama - 💡(How to fix) Fix Expose `max_soft_tokens` (image token budget) as a runtime parameter for Gemma 4 models [1 participants]

TL;DR

Increase the max_soft_tokens value to improve accuracy in fine-grained visual tasks, such as OCR.

Guidance

• Review the Google documentation for Gemma 4 to understand the recommended approach for tuning the variable-resolution token budget.
• Consider overriding the hardcoded max_soft_tokens value of 280 in model/models/gemma4/process_image.go to a higher value, such as 560, to improve accuracy.
• Test the impact of different max_soft_tokens values on your specific use case, such as license plate recognition, to determine the optimal value.
• Refer to the discussion in https://github.com/ollama/ollama-python/issues/651 for more information on the issue and potential workarounds.

Example

No code snippet is provided as the issue is more related to configuration and tuning rather than code changes.

Notes

The optimal value for max_soft_tokens may vary depending on the specific use case and model variant. It is recommended to experiment with different values to find the best trade-off between accuracy and performance.

Recommendation

Apply workaround: Increase the max_soft_tokens value to a higher value, such as 560, to improve accuracy in fine-grained visual tasks. This is recommended as the default value of 280 is insufficient for certain tasks, and increasing the value has been shown to improve accuracy in the provided example.