Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix Streaming tool call input is empty for Gemini models via Anthropic-compatible API [1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24721•Fetched 2026-04-08 01:41:59
View on GitHub
Comments
1
Participants
1
Timeline
4
Reactions
0
Author
kowyokowyo
Participants
kowyokowyo
Timeline (top)
closed ×1commented ×1cross-referenced ×1labeled ×1

When using LiteLLM's Anthropic-compatible streaming API with Gemini models, tool call input is always {}. No input_json_delta events are emitted, so the arguments are completely lost and cannot be recovered client-side.

Root Cause

When using LiteLLM's Anthropic-compatible streaming API with Gemini models, tool call input is always {}. No input_json_delta events are emitted, so the arguments are completely lost and cannot be recovered client-side.

Fix Action

Workaround

Switch to non-streaming (client.messages.create), which returns tool inputs correctly.

Code Example

RawContentBlockStartEvent(content_block=ToolUseBlock(id='...', input={}, name='bash', type='tool_use'), index=1, type='content_block_start')
ParsedContentBlockStopEvent(index=1, type='content_block_stop', content_block=ToolUseBlock(id='...', input={}, name='bash', type='tool_use'))
RAW_BUFFERClick to expand / collapse

Description

When using LiteLLM's Anthropic-compatible streaming API with Gemini models, tool call input is always {}. No input_json_delta events are emitted, so the arguments are completely lost and cannot be recovered client-side.

Environment

  • LiteLLM (v1.82.3)
  • Model: gemini-3-flash-preview
  • Client: Anthropic Python SDK (anthropic>=0.86.0)
  • Streaming: client.messages.stream()

Steps to Reproduce

  1. Set ANTHROPIC_BASE_URL to your LiteLLM proxy URL
  2. Call client.messages.stream() with a tool defined and a prompt that triggers a tool call
  3. Iterate over stream events and inspect ToolUseBlock.input

Expected Behavior

The stream should emit input_json_delta events (or populate input in content_block_start) with the tool arguments, as the Anthropic API does natively.

Actual Behavior

The RawContentBlockStartEvent for the ToolUseBlock already has input={}, and no input_json_delta events follow. The final message also has input={}.

Stream events observed:

RawContentBlockStartEvent(content_block=ToolUseBlock(id='...', input={}, name='bash', type='tool_use'), index=1, type='content_block_start')
ParsedContentBlockStopEvent(index=1, type='content_block_stop', content_block=ToolUseBlock(id='...', input={}, name='bash', type='tool_use'))

Note: output_tokens=16 and stop_reason=end_turn, so this is not a truncation issue — the response completed normally with the arguments simply missing.

Workaround

Switch to non-streaming (client.messages.create), which returns tool inputs correctly.

extent analysis

Fix Plan

To fix the issue with missing tool arguments in LiteLLM's Anthropic-compatible streaming API, we need to modify the client-side code to handle the input_json_delta events correctly.

Here are the steps:

  • Update the client.messages.stream() call to handle input_json_delta events.
  • Modify the event handling logic to accumulate the input_json_delta events and update the ToolUseBlock.input accordingly.

Example Code

import json

# Initialize an empty dictionary to store the accumulated input
accumulated_input = {}

# Iterate over the stream events
for event in client.messages.stream():
    if event.type == 'content_block_start' and event.content_block.type == 'tool_use':
        # Initialize the input for this tool use block
        event.content_block.input = {}
        
    elif event.type == 'input_json_delta':
        # Accumulate the input_json_delta events
        accumulated_input.update(json.loads(event.delta))
        # Update the ToolUseBlock.input
        event.content_block.input = accumulated_input
        
    # Handle other event types as needed
    print(event)

Verification

To verify that the fix worked, you can check the ToolUseBlock.input in the content_block_start and content_block_stop events. The input should now be populated with the correct tool arguments.

Extra Tips

  • Make sure to handle any potential errors when parsing the input_json_delta events.
  • Consider adding logging or debugging statements to verify that the input_json_delta events are being accumulated and applied correctly.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#environment setup#docker error#permission error#memory optimization

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - 💡(How to fix) Fix Streaming tool call input is empty for Gemini models via Anthropic-compatible API [1 comments, 1 participants]