litellm - 💡(How to fix) Fix Anthropic /v1/messages streaming drops tool_use.input for gemini/gemma-4-31b-it while non-streaming works [1 participants]

Describe the bug When using LiteLLM’s Anthropic-compatible /v1/messages endpoint with a Gemma 4 route backed by gemini/gemma-4-31b-it, tool calling works in non-streaming mode but fails in streaming mode.

With the exact same request payload:

non-streaming returns a correct tool_use block with populated input streaming returns a tool_use block with input: {}

This breaks streaming agent clients like Claude Code / Cline because the client receives an empty tool call and then complains that required tool parameters are missing.

LiteLLM model route

• model_name: gemma-4-31b litellm_params: model: gemini/gemma-4-31b-it api_key: ...

Request payload (non-streaming)

{ "model": "gemma-4-31b", "max_tokens": 256, "messages": [ { "role": "user", "content": "Use grep_tool with pattern exactly GetExerciseAnchors and path exactly lib/domain/repositories/workout_set_repository.dart. Do not answer normally." } ], "tools": [ { "name": "grep_tool", "description": "Search for a pattern in a file", "input_schema": { "type": "object", "properties": { "pattern": { "type": "string" }, "path": { "type": "string" } }, "required": ["pattern", "path"] } } ] }

Observed non-streaming response

{ "type": "tool_use", "id": "call_b37716fd8d224c66acf270ca40cb__thought__...", "name": "grep_tool", "input": { "path": "lib/domain/repositories/workout_set_repository.dart", "pattern": "GetExerciseAnchors" }, "provider_specific_fields": { "signature": "..." } }

This is correct.

Request payload (streaming) Same payload, but with:

"stream": true

Observed streaming SSE output

event: content_block_start data: {"type": "content_block_start", "index": 1, "content_block": {"type": "tool_use", "id": "call_f4e00a4fcaa045969b305568579e__thought__...", "name": "grep_tool", "input": {}}}

The tool name is correct, but input is empty.

Minimal simpler repro This also reproduces with a one-arg tool:

Tool schema:

{ "name": "echo_arg", "description": "Echo a single required string", "input_schema": { "type": "object", "properties": { "value": { "type": "string" } }, "required": ["value"] } }

Prompt:

Call the tool echo_arg with value exactly banana-42. Do not answer normally.

Non-streaming: returns input: {"value":"banana-42"} Streaming: returns input: {}

Expected behavior Streaming Anthropic /v1/messages should preserve tool-call arguments the same way non-streaming does.

Expected streamed tool_use:

{ "type": "tool_use", "name": "grep_tool", "input": { "pattern": "GetExerciseAnchors", "path": "lib/domain/repositories/workout_set_repository.dart" } }

Actual behavior Streaming Anthropic /v1/messages emits:

{ "type": "tool_use", "name": "grep_tool", "input": {} }

Why this matters This makes streaming-based agent clients unusable for Gemma on this route:

model clearly understands the tool and arguments non-streaming proves the arguments are generated correctly streaming drops the arguments clients then error with missing required tool params

Additional context I also previously hit a separate Gemini-path issue on Anthropic /v1/messages/count_tokens, where LiteLLM raised:

ModuleNotFoundError: No module named 'google.genai'

That was from the Gemini token-counting path in the LiteLLM environment. The relevant log snippet is below.

POST /v1/messages?beta=true HTTP/1.1" 200 OK POST /v1/messages/count_tokens?beta=true HTTP/1.1" 500 Internal Server Error ModuleNotFoundError: No module named 'google.genai'

That issue is separate from the current streaming tool-input loss, but both occurred on the Gemini/Gemma path.

Summary

Anthropic /v1/messages gemini/gemma-4-31b-it non-streaming tool calls work streaming tool calls lose input reproducible with both one-arg and two-arg tools

Happy to provide full raw request/response logs if needed.