Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug] /v1/messages pass-through: Anthropic rejects tool_use ordering when context compaction merges assistant turns [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#22946•Fetched 2026-04-08 00:39:19
View on GitHub
Comments
0
Participants
1
Timeline
6
Reactions
0
Author
xykongxykong
Participants
xykongxykong
Timeline (top)
cross-referenced ×3closed ×1labeled ×1subscribed ×1

Error Message

messages.17: tool_use ids were found without tool_result blocks immediately after: toolu_014pWwdiQm6ear25PGFtxsuB

Root Cause

The /v1/messages pass-through handler (litellm/llms/anthropic/experimental_pass_through/messages/handler.py) forwards messages directly to Anthropic without sanitization. In contrast, the standard /chat/completions path goes through sanitize_messages_for_tool_calling() in factory.py, but that function only handles OpenAI-format messages and is never called for Anthropic-native format.

When a client with context compaction merges two assistant turns:

  • Turn 1: [text_A, tool_use_A]
  • Turn 2: [text_B, tool_use_B]

...into a single assistant message:

  • Merged: [text_A, tool_use_A, text_B, tool_use_B]

Anthropic's API rejects it because tool_use_A is not immediately followed by a tool_result.

Fix Action

Fix / Workaround

Simulate what context compaction produces:

Two assistant turns merged into one, creating invalid ordering

messages = [ { "role": "user", "content": "What is 2+2 and what is the weather?" }, { "role": "assistant", "content": [ {"type": "text", "text": "Let me calculate that for you."}, {"type": "tool_use", "id": "tool_1", "name": "calculator", "input": {"expr": "2+2"}}, # text block after tool_use — this is what context compaction produces {"type": "text", "text": "And let me check the weather."}, {"type": "tool_use", "id": "tool_2", "name": "weather", "input": {"city": "SF"}}, ] }, # ... tool_result blocks would follow, but the ordering above already causes the error ]

PR fix notes

PR #22947: fix(anthropic): sanitize /v1/messages pass-through for tool_use content ordering

Description (problem / solution / changelog)

Fixes #22946

Summary

The /v1/messages Anthropic-native pass-through endpoint forwards messages to Anthropic without sanitization. When an agentic client uses context compaction (merging multiple assistant turns into one), it can produce content blocks in an order that Anthropic rejects:

[text_A, tool_use_A, text_B, tool_use_B]

Error: messages.17: tool_use ids were found without tool_result blocks immediately after: toolu_014pWwdiQm6ear25PGFtxsuB

Root Cause

The standard /chat/completions path calls sanitize_messages_for_tool_calling() for OpenAI-format messages, but the /v1/messages pass-through path in anthropic_messages_handler() has no equivalent sanitization for Anthropic-native format.

Changes

litellm/litellm_core_utils/prompt_templates/factory.py

Adds sanitize_anthropic_native_messages_for_tool_calling(messages) that handles two cases:

  • Case A (orphaned tool_use): A tool_use block in an assistant message is not immediately followed by a tool_result in the next user message → inject a dummy tool_result to satisfy Anthropic's requirement
  • Case C (interleaved text/tool_use): Within an assistant message, text blocks appear after tool_use blocks (e.g., [text, tool_use, text, tool_use]) → reorder so all text blocks precede all tool_use blocks

Only runs when litellm.modify_params = True (same gate as existing sanitize_messages_for_tool_calling).

litellm/llms/anthropic/experimental_pass_through/messages/handler.py

Calls sanitize_anthropic_native_messages_for_tool_calling() at the entry of anthropic_messages_handler(), before forwarding to Anthropic.

Tests

tests/test_litellm/llms/anthropic/test_anthropic_native_message_sanitization.py — 12 unit tests covering:

  • No-op when modify_params=False
  • Normal messages pass through unchanged
  • Case A: orphaned tool_use gets dummy tool_result injected
  • Case A: multiple orphaned tool_use blocks
  • Case C: interleaved text/tool_use reordered correctly
  • Case C: multiple interleaved blocks in a single message
  • Combined Case A + C in the same conversation
  • Edge cases: empty content, non-assistant messages, mixed role sequences

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective
  • New and existing unit tests pass locally with my changes
  • modify_params gate preserved — no behavior change when modify_params=False

Changed files

  • litellm/litellm_core_utils/prompt_templates/factory.py (modified, +201/-7)
  • litellm/llms/anthropic/experimental_pass_through/messages/handler.py (modified, +6/-0)
  • tests/test_litellm/llms/anthropic/test_anthropic_native_message_sanitization.py (added, +299/-0)

PR #23104: fix(anthropic): deduplicate tool_result messages by tool_call_id

Description (problem / solution / changelog)

Related Issues

Related to #16711 — "each tool_use must have a single result. Found multiple tool_result blocks" with tool response handling Fixes #22946 — Anthropic rejects tool_use ordering when context compaction merges assistant turns Related to #22878 — Claude Code Bad Request errors in multi-turn tool calling

What this PR does

Adds "Case D" deduplication to sanitize_messages_for_tool_calling() — when multiple tool messages reference the same tool_call_id within a contiguous tool-result block, only the last occurrence is kept.

Anthropic requires exactly one tool_result per tool_use and rejects with:

each tool_use must have a single result. Found multiple tool_result blocks
with id: toolu_xxx

Root Cause

Session history replay (conversation resume, checkpointing) can produce duplicate tool_result messages for the same tool_call_id. The existing sanitize_messages_for_tool_calling() handles orphaned tool results (Cases A-C) but does NOT detect duplicates.

Note: This dedup already exists for Bedrock (_deduplicate_bedrock_content_blocks) but was missing for the general OpenAI-format message path used by Anthropic and Vertex AI.

Fix

After existing sanitization (Cases A-C), a second pass scans sanitized_messages for duplicate tool_call_id values within each contiguous block of tool results:

  • Track seen_in_block: Dict[str, int] mapping tool_call_id → message index
  • When a duplicate is found, mark the earlier occurrence for removal (last-wins strategy)
  • Reset tracking on any non-tool message (user/assistant/system) to scope dedup per conversation turn
  • Tool/function messages without a tool_call_id (malformed) do NOT reset the block — only real turn boundaries do
  • Log each dedup via verbose_logger.warning() for observability

The last-wins strategy differs from Bedrock's first-wins (_deduplicate_bedrock_content_blocks) because the duplicate here arises from session history replay where the last entry represents the final state, not provider-side content block duplication.

Tests

5 unit tests (no network calls):

  • test_sanitize_messages_deduplicates_tool_results — duplicate tool_call_id within one turn, keeps last
  • test_sanitize_messages_preserves_unique_tool_results — distinct IDs pass through unchanged with content verified
  • test_sanitize_messages_dedup_disabled_when_modify_params_false — no sanitization when flag is off
  • test_sanitize_messages_dedup_scoped_per_turn_preserves_cross_turn — same tool_call_id in two different turns, both preserved
  • test_sanitize_messages_combined_case_a_and_case_d — combined scenario: missing result (Case A dummy) + duplicate results (Case D dedup) in same assistant message, with ordering assertions

Files Changed

FileChange
litellm/litellm_core_utils/prompt_templates/factory.pyCase D dedup logic in sanitize_messages_for_tool_calling()
tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py5 new test functions

Changed files

  • litellm/litellm_core_utils/prompt_templates/factory.py (modified, +48/-0)
  • tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py (modified, +287/-1)

Code Example

[text_A, tool_use_A, text_B, tool_use_B]

---

messages.17: tool_use ids were found without tool_result blocks immediately after: toolu_014pWwdiQm6ear25PGFtxsuB

---

import litellm

# Simulate what context compaction produces:
# Two assistant turns merged into one, creating invalid ordering
messages = [
    {
        "role": "user",
        "content": "What is 2+2 and what is the weather?"
    },
    {
        "role": "assistant",
        "content": [
            {"type": "text", "text": "Let me calculate that for you."},
            {"type": "tool_use", "id": "tool_1", "name": "calculator", "input": {"expr": "2+2"}},
            # text block after tool_use — this is what context compaction produces
            {"type": "text", "text": "And let me check the weather."},
            {"type": "tool_use", "id": "tool_2", "name": "weather", "input": {"city": "SF"}},
        ]
    },
    # ... tool_result blocks would follow, but the ordering above already causes the error
]

# This fails with:
# "tool_use ids were found without tool_result blocks immediately after: tool_1"
response = litellm.acompletion(model="anthropic/claude-opus-4-5", messages=messages)
RAW_BUFFERClick to expand / collapse

Bug Description

When using LiteLLM's /v1/messages Anthropic-native pass-through endpoint with an agentic client that performs context compaction (e.g., merging multiple assistant turns into one), the resulting message content can have tool_use blocks interleaved with text blocks in an order that Anthropic's API rejects:

[text_A, tool_use_A, text_B, tool_use_B]

Anthropic requires that within an assistant message, all text blocks must appear before any tool_use blocks, and tool_use blocks must be immediately followed by tool_result blocks. The pass-through endpoint sends these messages verbatim without any sanitization, causing a 400 error.

Error Message

messages.17: tool_use ids were found without tool_result blocks immediately after: toolu_014pWwdiQm6ear25PGFtxsuB

Root Cause

The /v1/messages pass-through handler (litellm/llms/anthropic/experimental_pass_through/messages/handler.py) forwards messages directly to Anthropic without sanitization. In contrast, the standard /chat/completions path goes through sanitize_messages_for_tool_calling() in factory.py, but that function only handles OpenAI-format messages and is never called for Anthropic-native format.

When a client with context compaction merges two assistant turns:

  • Turn 1: [text_A, tool_use_A]
  • Turn 2: [text_B, tool_use_B]

...into a single assistant message:

  • Merged: [text_A, tool_use_A, text_B, tool_use_B]

Anthropic's API rejects it because tool_use_A is not immediately followed by a tool_result.

Steps to Reproduce

  1. Configure LiteLLM proxy with modify_params: true
  2. Use any Anthropic-native client (e.g., Anthropic SDK directly) that calls /v1/messages
  3. Run a multi-turn agentic workflow that uses tools
  4. Enable context compaction in the client (or manually construct the edge case)
  5. Observe 400 error from Anthropic

Minimal Reproducer (Python)

import litellm

# Simulate what context compaction produces:
# Two assistant turns merged into one, creating invalid ordering
messages = [
    {
        "role": "user",
        "content": "What is 2+2 and what is the weather?"
    },
    {
        "role": "assistant",
        "content": [
            {"type": "text", "text": "Let me calculate that for you."},
            {"type": "tool_use", "id": "tool_1", "name": "calculator", "input": {"expr": "2+2"}},
            # text block after tool_use — this is what context compaction produces
            {"type": "text", "text": "And let me check the weather."},
            {"type": "tool_use", "id": "tool_2", "name": "weather", "input": {"city": "SF"}},
        ]
    },
    # ... tool_result blocks would follow, but the ordering above already causes the error
]

# This fails with:
# "tool_use ids were found without tool_result blocks immediately after: tool_1"
response = litellm.acompletion(model="anthropic/claude-opus-4-5", messages=messages)

Expected Behavior

LiteLLM should sanitize Anthropic-native format messages in the /v1/messages pass-through path, just as it does for OpenAI-format messages in /chat/completions. Specifically:

  1. Reorder content blocks: Move all text blocks before tool_use blocks within assistant messages
  2. Handle orphaned tool_use blocks: Inject a dummy tool_result for any tool_use not followed by a tool_result, to prevent API rejection

This behavior should be gated on modify_params=True (already used by the existing sanitize_messages_for_tool_calling).

Environment

  • LiteLLM version: main branch (tested against commit bc23c08d03)
  • Model: Any anthropic/* model via /v1/messages pass-through
  • modify_params: true in config

Proposed Fix

A PR is available at: https://github.com/xykong/litellm/tree/fix/anthropic-messages-tool-use-ordering

The fix adds sanitize_anthropic_native_messages_for_tool_calling() to factory.py and calls it from anthropic_messages_handler().

extent analysis

Fix Plan

To fix the issue, we need to sanitize Anthropic-native format messages in the /v1/messages pass-through path. We will create a new function sanitize_anthropic_native_messages_for_tool_calling() in factory.py and call it from anthropic_messages_handler().

Here are the steps:

  • Create a new function sanitize_anthropic_native_messages_for_tool_calling() that:
    • Reorders content blocks to move all text blocks before tool_use blocks within assistant messages
    • Handles orphaned tool_use blocks by injecting a dummy tool_result for any tool_use not followed by a tool_result
  • Call sanitize_anthropic_native_messages_for_tool_calling() from anthropic_messages_handler() when modify_params=True

Example Code

def sanitize_anthropic_native_messages_for_tool_calling(messages):
    for message in messages:
        if message["role"] == "assistant":
            text_blocks = [block for block in message["content"] if block["type"] == "text"]
            tool_use_blocks = [block for block in message["content"] if block["type"] == "tool_use"]
            tool_result_blocks = [block for block in message["content"] if block["type"] == "tool_result"]
            
            # Reorder content blocks
            message["content"] = text_blocks + tool_use_blocks + tool_result_blocks
            
            # Handle orphaned tool_use blocks
            for i, block in enumerate(message["content"]):
                if block["type"] == "tool_use" and (i == len(message["content"]) - 1 or message["content"][i + 1]["type"] != "tool_result"):
                    message["content"].insert(i + 1, {"type": "tool_result", "id": block["id"], "result": ""})
    return messages

Verification

To verify the fix, you can use the minimal reproducer provided in the issue body. After applying the fix, the code should no longer raise a 400 error from Anthropic.

Extra Tips

  • Make sure to test the fix with different scenarios, including multiple tool uses and results.
  • Consider adding additional logging or error handling to ensure that the fix is working as expected.
  • Review the code changes to ensure that they are consistent with the existing codebase and do not introduce any new issues.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#request error#file not found#serialization error#model compatibility

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING