litellm - 💡(How to fix) Fix [Bug]: Ollama Gemma 4 Infinite Tool Loop: Role mismatch ("tool" vs "tool_responses")

# In transform_request loop:
role = cast(str, m.get("role"))

# Fix for Gemma 4 Tool Calling Loop: 
# Map 'tool' role to 'tool_responses' for Gemma models to prevent infinite loops.
if role == "tool" and "gemma4" in model.lower():
    role = "tool_responses"

ollama_message = OllamaChatCompletionMessage(
    role=role,
)

import litellm
import json

# 1. Setup - Use the exact model that failed
MODEL = "ollama_chat/gemma4:31b-cloud"

# 2. Define the tool
tools = [
    {
        "type": "function",
        "function": {
            "name": "add",
            "description": "Add two numbers together.",
            "parameters": {
                "type": "object",
                "properties": {
                    "a": {"type": "number", "description": "First number"},
                    "b": {"type": "number", "description": "Second number"},
                },
                "required": ["a", "b"],
                "strict": True,
            },
        },
    }
]


def add_numbers(a, b):
    return {"result": a + b}


# 3. The Conversation
messages = [{"role": "user", "content": "What is 2 + 2?"}]

print(f"Starting reproduction loop with model: {MODEL}\n")

# We limit to 5 turns. If the bug exists, it will loop 5 times.
# If fixed, it will stop after the first tool result.
for i in range(5):
    print(f"--- Turn {i+1} ---")

    response = litellm.completion(
        model=MODEL,
        messages=messages,
        tools=tools,
        tool_choice="auto",
        think=True,
    )

    message = response.choices[0].message

    # Check for tool calls
    if message.tool_calls:
        print(f"Model requested tool: {message.tool_calls[0].function.name}")

        # Add assistant message to history
        messages.append(message)

        # Execute and add tool result to history
        for tool in message.tool_calls:
            args = json.loads(tool.function.arguments)
            result = add_numbers(args["a"], args["b"])

            messages.append(
                {
                    "role": "tool",
                    "tool_call_id": tool.id,
                    "name": tool.function.name,
                    "content": json.dumps(result),
                }
            )
            print(f"Tool result provided: {result}")
    else:
        print(f"Final Answer: {message.content}")
        break  # Success! The loop stopped.

if i == 4:
    print("\n❌ BUG REPRODUCED: The model entered an infinite tool loop.")
else:
    print("\n✅ NO LOOP: The model responded correctly.")

Starting reproduction loop with model: ollama_chat/gemma4:31b-cloud

--- Turn 1 ---
Model requested tool: add
Tool result provided: {'result': 4}
--- Turn 2 ---
Model requested tool: add
Tool result provided: {'result': 4}
--- Turn 3 ---
Model requested tool: add
Tool result provided: {'result': 4}
--- Turn 4 ---
Model requested tool: add
Tool result provided: {'result': 4}
--- Turn 5 ---
Model requested tool: add
Tool result provided: {'result': 4}

❌ BUG REPRODUCED: The model entered an infinite tool loop.

*After applying the patch in the local `litellm\llms\ollama\chat\transformation.py`, within the `transform_request`*

Starting reproduction loop with model: ollama_chat/gemma4:31b-cloud

--- Turn 1 ---
Model requested tool: add
Tool result provided: {'result': 4}
--- Turn 2 ---
Final Answer: 2 + 2 is 4.

✅ NO LOOP: The model responded correctly.