Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: Vertex AI Gemini fails to resolve tool calls when history contains text-only assistant messages [2 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26965•Fetched 2026-05-02 05:28:18
View on GitHub
Comments
2
Participants
2
Timeline
7
Reactions
0
Author
meenalgupta2306meenalgupta2306
Participants
meenalgupta2306meenalgupta2306smqd19smqd19
Timeline (top)
labeled ×3commented ×2cross-referenced ×2

Error Message

litellm.APIConnectionError: Missing corresponding tool call for tool response message. Received - message={'role': 'tool', 'content': 'Chunk ID: 629156 Wall time: 0.0000 seconds Process exited with code 1 Original token count: 12 Output: bwrap: setting up uid map: Permission denied ', 'tool_call_id': 'call_c9266589ae6b421380c5acee31ed'}, last_message_with_tool_calls={'role': 'assistant', 'content': [{'type': 'text', 'text': "Okay, I understand you want to add a card for audio transcription to the AI Solution Showcase. First, I'll explore the project structure to find the right place to add it."}]}

Fix Action

Fixed

PR fix notes

PR #26917: fix(vertex-ai): recover tool-call history when Responses API conversion drops tool_calls

Description (problem / solution / changelog)

Relevant issues

#26965

Problem

When routing Responses API requests (e.g. from Codex CLI) to Vertex AI Gemini, multi-turn tool use breaks .

Root cause: LiteLLM's conversion from Responses API format to Chat Completions messages can produce assistant messages with an empty tool_calls list. When Gemini's history builder walks those messages, last_message_with_tool_calls ends up None at the tool-response side — so it cannot recover the function name to include in the Gemini functionResponse part.

Fix

Changes in _gemini_convert_messages_with_history:

  1. Pre-build a lookup map. Before walking the message list, build a tool_call_id → assistant message index from all assistant messages that do have tool_calls. When a tool response is encountered, look up its tool_call_id in the map and restore last_message_with_tool_calls to the correct assistant message, even when that message has an empty `tool_calls list in the converted history.

Test plan

  • Tests added in tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformati on.py
  • All existing tests in tests/test_litellm/llms/vertex_ai/ pass

Screenshots / Proof of Fix

Before Fix

<img width="841" height="444" alt="image" src="https://github.com/user-attachments/assets/52091bc3-8ed6-4ddb-b0e4-8e025f42415c" /> <br>

After Fix (Working ✅)

<img width="851" height="376" alt="image" src="https://github.com/user-attachments/assets/4dedf944-77f5-4a83-9c9b-61539f7fbba5" />

Running the sample test script:

<img width="813" height="75" alt="image" src="https://github.com/user-attachments/assets/de6d01ac-d8e7-4e39-a57b-871cfc8b9b78" />

Changed files

  • litellm/llms/vertex_ai/gemini/transformation.py (modified, +18/-1)
  • tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py (modified, +58/-0)

PR #26995: fix(vertex): resolve Gemini tool results by call id

Description (problem / solution / changelog)

What changed

Fixes #26965 by preserving a lookup from OpenAI tool call ids to the assistant message that declared each tool call during Vertex/Gemini message conversion.

Previously the converter only kept a single last_message_with_tool_calls pointer. A text-only assistant message between a tool call and its tool result could overwrite that context, so the later tool result could no longer recover the original function name and the request failed before reaching Vertex AI.

This change keeps the existing fallback behavior but first resolves tool results by tool_call_id, so non-adjacent tool results still map to the right function call.

Validation

  • uv run ruff check litellm/llms/vertex_ai/gemini/transformation.py tests/litellm/llms/vertex_ai/gemini/test_transformation.py
  • uv run pytest tests/litellm/llms/vertex_ai/gemini/test_transformation.py::test_gemini_tool_result_resolves_after_text_only_assistant_message -q
  • uv run pytest tests/litellm/llms/vertex_ai/gemini/test_transformation.py -q

Changed files

  • litellm/llms/vertex_ai/gemini/transformation.py (modified, +22/-3)
  • tests/litellm/llms/vertex_ai/gemini/test_transformation.py (modified, +59/-2)

Code Example

litellm.APIConnectionError: Missing corresponding tool call for tool response message. Received - message={'role': 'tool', 'content': 'Chunk ID: 629156
Wall time: 0.0000 seconds
Process exited with code 1
Original token count: 12
Output:
bwrap: setting up uid map: Permission denied
', 'tool_call_id': 'call_c9266589ae6b421380c5acee31ed'}, last_message_with_tool_calls={'role': 'assistant', 'content': [{'type': 'text', 'text': "Okay, I understand you want to add a card for audio transcription to the AI Solution Showcase. First, I'll explore the project structure to find the right place to add it."}]}

---

import sys
import os

sys.path.insert(0, os.getcwd())

from litellm.llms.vertex_ai.gemini.transformation import _gemini_convert_messages_with_history

# This specific sequence caused the crash 
messages = [
    {
        "role": "assistant", 
        "tool_calls": [{"id": "call_123", "type": "function", "function": {"name": "get_weather", "arguments": "{}"}}]
    },
    {
        "role": "assistant", 
        "content": "Checking that for you..." 
    },
    {
        "role": "tool", 
        "tool_call_id": "call_123", 
        "content": "Sunny"
    }
]

print("Running Gemini transformation check...")
try:
    result = _gemini_convert_messages_with_history(messages, model="gemini-1.5-flash")
    print("\n✅ SUCCESS: The transformation worked!")
    print("The fix successfully recovered the tool call name 'get_weather' from the history.")
except Exception as e:
    print(f"\n❌ FAILED: {e}")

---

❌ FAILED: Missing corresponding tool call for tool response message. Received - message={'role': 'tool', 'tool_call_id': 'call_123', 'content': 'Sunny'}, last_message_with_tool_calls={'role': 'assistant', 'content': 'Checking that for you...'}
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using Vertex AI Gemini, if the conversation history contains an assistant message with tool calls followed by another assistant message that only contains text (common in some client flows like Codex or when a model "thinks" out loud before calling a tool), LiteLLM fails to correctly map the subsequent tool result back to the original tool call. This results in a crash with the following error:

litellm.APIConnectionError: Missing corresponding tool call for tool response message. Received - message={'role': 'tool', 'content': 'Chunk ID: 629156
Wall time: 0.0000 seconds
Process exited with code 1
Original token count: 12
Output:
bwrap: setting up uid map: Permission denied
', 'tool_call_id': 'call_c9266589ae6b421380c5acee31ed'}, last_message_with_tool_calls={'role': 'assistant', 'content': [{'type': 'text', 'text': "Okay, I understand you want to add a card for audio transcription to the AI Solution Showcase. First, I'll explore the project structure to find the right place to add it."}]}

Steps to Reproduce

You can reproduce this by running the following snippet using the internal transformation logic:

import sys
import os

sys.path.insert(0, os.getcwd())

from litellm.llms.vertex_ai.gemini.transformation import _gemini_convert_messages_with_history

# This specific sequence caused the crash 
messages = [
    {
        "role": "assistant", 
        "tool_calls": [{"id": "call_123", "type": "function", "function": {"name": "get_weather", "arguments": "{}"}}]
    },
    {
        "role": "assistant", 
        "content": "Checking that for you..." 
    },
    {
        "role": "tool", 
        "tool_call_id": "call_123", 
        "content": "Sunny"
    }
]

print("Running Gemini transformation check...")
try:
    result = _gemini_convert_messages_with_history(messages, model="gemini-1.5-flash")
    print("\n✅ SUCCESS: The transformation worked!")
    print("The fix successfully recovered the tool call name 'get_weather' from the history.")
except Exception as e:
    print(f"\n❌ FAILED: {e}")

Relevant log output

❌ FAILED: Missing corresponding tool call for tool response message. Received - message={'role': 'tool', 'tool_call_id': 'call_123', 'content': 'Sunny'}, last_message_with_tool_calls={'role': 'assistant', 'content': 'Checking that for you...'}

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on ?

v1.83.10

Twitter / LinkedIn details

No response

extent analysis

TL;DR

The issue can be fixed by modifying the _gemini_convert_messages_with_history function to correctly handle cases where an assistant message without tool calls precedes a tool response message.

Guidance

  • Review the _gemini_convert_messages_with_history function to ensure it properly tracks tool call IDs and their corresponding messages, even when an assistant message without tool calls intervenes.
  • Verify that the function correctly updates the last_message_with_tool_calls variable to point to the message containing the original tool call, rather than the most recent assistant message.
  • Consider adding a check to ensure that the tool_call_id in the tool response message matches a tool call ID in the conversation history.
  • Test the modified function with the provided example code to confirm that it correctly handles the specific sequence of messages that caused the crash.

Example

def _gemini_convert_messages_with_history(messages, model):
    # ... existing code ...
    for message in messages:
        if message['role'] == 'assistant' and 'tool_calls' in message:
            # Update last_message_with_tool_calls to point to this message
            last_message_with_tool_calls = message
        elif message['role'] == 'tool':
            # Check if the tool_call_id matches a tool call ID in the conversation history
            if message['tool_call_id'] not in [tool_call['id'] for tool_call in last_message_with_tool_calls.get('tool_calls', [])]:
                raise Exception('Missing corresponding tool call for tool response message')
    # ... existing code ...

Notes

The provided example code and log output suggest that the issue is specific to the _gemini_convert_messages_with_history function and its handling of tool call IDs. However, without the full implementation of this function, it is difficult to provide a complete solution.

Recommendation

Apply workaround: Modify the _gemini_convert_messages_with_history function to correctly handle cases where an assistant message

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#conversation history#tool integration#LLM response#prompt template

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: Vertex AI Gemini fails to resolve tool calls when history contains text-only assistant messages [2 pull requests, 2 comments, 2 participants]