ollama - 💡(How to fix) Fix Gemma4:31b Structured Output inconsistent with Reasoning / Thinking [3 comments, 2 participants]

TL;DR

The issue may be resolved by passing the think= parameter to the chat function to ensure the model's reasoning is reflected in the final JSON output.

Guidance

• Review the documentation for the chat function to understand the purpose of the think= parameter and how it affects the model's output.
• Verify that the format= parameter is correctly passing the Decision structure to the model.
• Check the model's configuration and version to ensure it is compatible with the gemma4:31b model and the Ollama version 0.20.2.
• Test the model with different inputs and edge cases to see if the inconsistency is specific to certain scenarios.

Example

No code snippet is provided as the issue does not contain sufficient information to create a specific example.

Notes

The issue may be related to the model's internal decision-making process or the way the Ollama library handles the model's output. Further investigation is needed to determine the root cause of the inconsistency.

Recommendation

Apply a workaround by passing the think= parameter to the chat function to see if it resolves the issue, as the documentation suggests it may affect the model's output.