litellm - 💡(How to fix) Fix OpenAI-proxy streaming for Claude breaks AI SDK 6.x multi-step tool calls: "text part {id} not found" [1 participants]

Description

When using LiteLLM's OpenAI-compatible proxy (/v1/chat/completions) with a Claude model and the Vercel AI SDK 6.x ([email protected]), multi-step tool calls produce streaming errors of the form:

text part chatcmpl-<id> not found

This causes the AI SDK to silently drop text deltas for the affected steps — producing empty or truncated assistant responses.

Environment

• LiteLLM version: 1.x (confirmed on hosted proxy at llm.behaviorlabs.ai/v1)
• Model: claude-sonnet-4-6 (via LiteLLM Claude alias)
• Client: @ai-sdk/[email protected] pointing at LiteLLM proxy
• AI SDK: [email protected]
• Framework: Next.js App Router, streamText + stopWhen: stepCountIs(20) for multi-step tool use

Steps to Reproduce

1. Configure @ai-sdk/openai to point at LiteLLM proxy:

   const litellm = createOpenAI({ baseURL: 'https://<litellm>/v1', apiKey: '...' })

2. Call streamText with tools and stopWhen: stepCountIs(20) using a Claude model:

   const result = streamText({
     model: litellm('claude-sonnet-4-6'),
     tools: { searchEvidence: ..., webSearch: ... },
     stopWhen: stepCountIs(20),
   })

3. Ask a question that triggers tool use followed by a text response (multi-step)
4. Observe: the final text response is empty OR the stream emits an error event:

   { type: "error", errorText: "text part chatcmpl-<id> not found" }

Root Cause

AI SDK 6.x tracks streaming text parts by ID using a activeTextContent[part.id] map. The flow is:

• text-start { id: "0" } → registers the text part
• text-delta { id: "0", delta: "..." } → appends to registered part
• text-end { id: "0" } → cleans up

The @ai-sdk/openai provider uses id: "0" for all text events in a single step. However, when LiteLLM proxies a multi-step Claude response (where Claude uses tools and then continues with text), the translated stream appears to emit a text-delta with an ID matching the OpenAI chatcmpl-<id> response-level field, which was never registered via text-start. This causes the AI SDK to drop the delta and emit an error.

The error in AI SDK source (ai/dist/index.mjs):

if (part.type === "text-delta") {
  const activeText = activeTextContent[part.id];
  if (activeText == null) {
    controller.enqueue({ part: { type: "error", error: `text part ${part.id} not found` } });
    return; // delta is silently dropped
  }
}

Expected Behavior

The streaming translation from Anthropic's native SSE format to OpenAI's format should preserve consistent part IDs across all events within a step, so that text-start/text-delta/text-end events all carry the same ID.

Workaround

Use @ai-sdk/anthropic natively instead of routing Claude through the LiteLLM OpenAI proxy. This bypasses the translation layer entirely and the "text part not found" errors disappear.

import { createAnthropic } from '@ai-sdk/anthropic'
const anthropic = createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY })

// Route claude-* models directly to Anthropic, LiteLLM for everything else
function resolveModel(modelId: string) {
  if (modelId.startsWith('claude-')) return anthropic(modelId)
  return litellm(modelId)
}

Additional Notes

• The issue only manifests in multi-step tool call sequences (the first step usually works fine)
• Affects any AI SDK 6.x consumer that uses LiteLLM as an OpenAI-compatible proxy for Claude
• Non-fatal in scenarios where text still arrives despite the error, but causes empty responses in some cases