litellm - 💡(How to fix) Fix Responses API streaming bridge: multi-step Anthropic tool calls emit text-delta with unregistered chatcmpl- ID

Description

When using LiteLLM's Responses API (/v1/responses) to proxy Anthropic models, multi-step tool call flows produce:

text part chatcmpl-<id> not found

The AI SDK tracks text parts via text-start → text-delta → text-end by ID. During step 2+ (text after tool use), the text-delta event arrives with a chatcmpl- response-level ID that was never registered via text-start, causing the AI SDK to drop the text.

Single-step text-only responses work fine. Only multi-step tool call sequences break.

The Bug

In _transform_chat_completion_chunk_to_response_api_chunk:

  if self._cached_item_id is None and chunk.id:
      self._cached_item_id = chunk.id          # ← sets cache to "chatcmpl-aaa" on first chunk
  item_id = self._cached_item_id or chunk.id   # ← uses cached value for all events

Step 1 (tool call): First chunk arrives, cached_item_id is None, so it caches chunk.id = "chatcmpl-aaa". The ensure_output_item_for_chunk method then overwrites it to "msg{uuid}" for the output item. Text events use "msg{uuid}" — consistent. ✅

Step 2 (text after tool result): LiteLLM internally makes a new Chat Completion call, which returns chunks with a new chunk.id = "chatcmpl-bbb". But cached_item_id is already set from step 1 (either "chatcmpl-aaa" or "msg{uuid}"), so the if self._cached_item_id is None check never fires.

The problem is that sent_output_item_added_event and sent_content_part_added_event are not reset between steps. So step 2 skips _ensure_output_item_for_chunk (which would emit a new text-start with OutputItemAddedEvent + ContentPartAddedEvent), and jumps straight to emitting text-delta with the stale item_id.

Related Issues

• #14962 / PR #14965 — Fixed per-chunk ID inconsistency in Chat Completions streaming
• #19125 / PR #19390 — Fixed text-start/text-delta ID mismatch in Responses API streaming (single-step). Added ID caching at stream start.
• #26529 — Same symptom on the Chat Completions path (/v1/chat/completions), still open

This issue covers the Responses API path (/v1/responses), which was refactored into a shared format mapping in 1.83.14.

Environment

• LiteLLM version: 1.83.14-stable
• Model: claude-haiku-4-5 (Anthropic, via Responses API bridge)
• Client: @ai-sdk/openai (uses Responses API by default)
• AI SDK: 5.x / 6.x

Steps to Reproduce

1. Point @ai-sdk/openai at LiteLLM proxy's /v1 endpoint
2. Send a prompt to an Anthropic model that triggers tool use followed by a text response (multi-step)
3. Step 1 (tool call) streams correctly
4. Step 2 (text after tool result) emits text-delta with chatcmpl-<id> that has no matching text-start

Root Cause

PR #19390 added ID caching at stream start to ensure consistency within a single step. However, in multi-step tool call flows, each step creates a new internal completion with a new chatcmpl- ID. The cache doesn't reset cleanly between steps, so the new chatcmpl- ID leaks into text-delta events without a corresponding text-start registration.

Step 1 (tool call):  cached ID → text-start(id="0") → text-delta(id="0") ✅
    ↓ new internal completion, new chatcmpl-bbb generated
Step 2 (text):       text-start(id="0") but text-delta(id="chatcmpl-bbb") 💥

Evidence from Production Logs

{"level":"error","event":"bridge.session_error","error_msg":"\"text part chatcmpl-3b58c921-4f8f-4f85-90c2-e53c9f79750c not found\""}
{"level":"info","event":"prompt.run","model":"claude-haiku-4-5","reasoning_effort":"max","outcome":"error","duration_ms":21914}

Same session, later prompt (text-only, no tool calls):

{"level":"info","event":"prompt.run","model":"claude-haiku-4-5","reasoning_effort":"max","outcome":"success","duration_ms":8176}

Expected Behavior

All text-start, text-delta, and text-end events within a step should share the same part ID, including across step boundaries in multi-step tool call flows.