litellm - 💡(How to fix) Fix [Bug]: Responses API bridged backend (Anthropic) multi-turn tool calls fail [1 participants]

== Turn 2 request body (client → LiteLLM) == { "input": [{ "type": "function_call_output", "call_id": "toolu_012GsmPDoQ3VrFJevHSvHNJ8", "output": "<tool result text>" }], "previous_response_id": "resp_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOmFudGhyb3BpYzttb2RlbF9pZDpjMjhi…", "model": "<alias>", "stream": true }

Decoded previous_response_id: litellm:custom_llm_provider:anthropic;model_id:c28b…;response_id:resp_1a8d95c2-0000-4381-0000-a67dde1347a4

== LiteLLM_SpendLogs row for turn 1 == request_id : chatcmpl-575206f9-0000-4833-0000-bf679ca50c5b session_id : 4be83faf-0000-4094-0000-c59fa4f21058 messages : {} - 2: empty response : {"id":"chatcmpl-575206f9-…","object":"chat.completion","model":"claude-haiku-4-5-20251001", "choices":[{"message":{"role":"assistant","tool_calls":[ {"id":"toolu_012GsmPDoQ3VrFJevHSvHNJ8","type":"function", "function":{"name":"…","arguments":"{…}"}}]}, "finish_reason":"tool_calls"}]} note: stored id namespace is chatcmpl-…, but client receives resp_… - 1

== LiteLLM_SpendLogs row for turn 2 (failed) == request_id : 82e99a0d-0000-4fd8-0000-8dbce574f671 session_id : 155b5dff-0000-4f84-0000-bccf543b4eed - different from turn 1 = 3 messages : {} response : {}

== 400 response to client == {"error":{"message":"litellm.BadRequestError: Unable to create messages for completion request. This can happen when: 1) Using previous_response_id without a session database, AND 2) Input contains only function_call_output with empty or invalid call_id. Please ensure function_call_output has a valid call_id from a previous response. Original request: previous_response_id=resp_1a8d95c2-0000-4381-0000-a67dde1347a4. Received Model Group=<alias>","type":"BadRequestError","code":"400"}}

TL;DR

The issue can be resolved by addressing the namespace mismatch between the request_id stored in the database and the previous_response_id returned to the client, and ensuring that the messages column is properly populated for aresponses traffic.

Guidance

• Investigate and fix the namespace mismatch between chatcmpl-<uuid> and resp_<uuid> to ensure consistent request_id handling.
• Modify the code to populate the messages column for aresponses traffic when store_prompts_in_spend_logs is true.
• Ensure that the session_id is stable across turns by properly deriving it from the previous_response_id chain.
• Verify the fixes by checking the LiteLLM_SpendLogs table for correct request_id, session_id, and messages values.

Example

No code snippet is provided as the issue requires changes to the existing codebase, which is not explicitly mentioned in the issue.

Notes

The issue is specific to the Proxy component of LiteLLM, and the fixes may require modifications to the database schema, the ResponsesSessionHandler, and the SpendLogs table. The provided information suggests that the issue is related to the handling of previous_response_id and session_id, but the exact changes required are not specified.

Recommendation

Apply a workaround to address the namespace mismatch and ensure proper population of the messages column, as the root cause of the issue is related to the inconsistent handling of request_id and session_id.