ollama - 💡(How to fix) Fix gemma4:e4b thinking failing with structured output [1 participants]

import ollama from pydantic import BaseModel, ValidationError

class TestModel(BaseModel): name: str age: int

def test_llm_client(): client = ollama.Client() system = "You are a helpful assistant that generates JSON objects with name and age." prompt = "Generate a JSON object with name and age." try: response = client.chat( model="gemma4:e4b", messages=[ {"role": "system", "content": system}, {"role": "user", "content": prompt}, ], format=TestModel.model_json_schema(), think="low", options={ "temperature": 1.0, "top_p": 0.95, "top_k": 64, }, ) content = response.message.content or "" print("LLM Response:", content) result = TestModel.model_validate_json(content) print("Parsed Result:", result) except ValidationError as ve: print("Validation Error:", ve) except Exception as e: print("Error:", e)

if name == "main": test_llm_client()

TL;DR

Setting thinking to False in the client.chat method may resolve the issue with JSON fences in the output.

Guidance

• Verify that the thinking parameter is set to False in the client.chat method to prevent JSON fences in the output.
• Check the ollama package documentation to ensure that the thinking parameter is supported in version 0.6.1.
• Review the TestModel.model_validate_json method to ensure it can handle the JSON output from the LLM client.
• Consider updating the ollama package to the latest version (0.20.3) to ensure compatibility and potential bug fixes.

Example

response = client.chat(
    model="gemma4:e4b",
    messages=[
            {"role": "system", "content": system},
            {"role": "user",   "content": prompt},
        ],
        format=TestModel.model_json_schema(),
        think=False,  # Set thinking to False
        options={
            "temperature": 1.0,
            "top_p": 0.95,
            "top_k": 64,
        },
    )

Notes

The issue may be specific to the ollama package version 0.6.1, and updating to the latest version (0.20.3) may resolve the issue. However, without further information, it is unclear if this is the root cause.

Recommendation

Apply workaround: Set thinking to False in the client.chat method, as this is the most likely cause of the issue with JSON fences in the output.