Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix Anthropic passthrough: server_tool_use parsed as dict instead of ServerToolUse in usage [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26749•Fetched 2026-04-30 06:20:26
View on GitHub
Comments
0
Participants
1
Timeline
1
Reactions
0
Author
gilparkgilpark
Participants
gilparkgilpark
Timeline (top)
labeled ×1

Error Message

AttributeError: 'dict' object has no attribute 'web_search_requests'

At litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py:343:

and usage.server_tool_use.web_search_requests is not None

Root Cause

The passthrough handler at anthropic_passthrough_logging_handler.py:66 constructs a ModelResponse via anthropic_config().transform_response(), but the usage.server_tool_use field remains as a raw dict instead of being validated into a ServerToolUse Pydantic object.

Non-standard Anthropic-compatible providers (e.g., z.ai) return server_tool_use as a JSON object in the response, and the passthrough path doesn't fully validate nested usage fields.

Code Example

AttributeError: 'dict' object has no attribute 'web_search_requests'

---

and usage.server_tool_use.web_search_requests is not None
RAW_BUFFERClick to expand / collapse

Bug Description

In the Anthropic passthrough handler, when a response includes usage.server_tool_use, it remains a raw dict instead of being cast to the ServerToolUse Pydantic model. This causes an AttributeError in tool_call_cost_tracking.py when completion_cost() tries to access usage.server_tool_use.web_search_requests.

Error

AttributeError: 'dict' object has no attribute 'web_search_requests'

At litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py:343:

and usage.server_tool_use.web_search_requests is not None

Root Cause

The passthrough handler at anthropic_passthrough_logging_handler.py:66 constructs a ModelResponse via anthropic_config().transform_response(), but the usage.server_tool_use field remains as a raw dict instead of being validated into a ServerToolUse Pydantic object.

Non-standard Anthropic-compatible providers (e.g., z.ai) return server_tool_use as a JSON object in the response, and the passthrough path doesn't fully validate nested usage fields.

Repro

  1. Set up LiteLLM proxy with an Anthropic-compatible provider that returns server_tool_use in usage
  2. Make a request through the passthrough endpoint
  3. completion_cost() crashes on dict attribute access

Expected

usage.server_tool_use should be a ServerToolUse instance, or tool_call_cost_tracking.py should handle the dict case with getattr() / isinstance().

Environment

  • LiteLLM version: 1.83.14
  • Python: 3.12.12
  • Provider: Anthropic-compatible passthrough (z.ai)

extent analysis

TL;DR

The issue can be fixed by validating the usage.server_tool_use field into a ServerToolUse Pydantic object in the passthrough handler.

Guidance

  • Modify the anthropic_passthrough_logging_handler.py to validate the usage.server_tool_use field using Pydantic's parse_obj method.
  • Update tool_call_cost_tracking.py to handle the case where usage.server_tool_use is a dict using getattr() or isinstance().
  • Verify that the ServerToolUse model is correctly defined and imported in the relevant files.
  • Test the changes with a request through the passthrough endpoint to ensure that completion_cost() no longer crashes.

Example

from pydantic import parse_obj
from models import ServerToolUse

# In anthropic_passthrough_logging_handler.py
usage.server_tool_use = parse_obj(ServerToolUse, usage.server_tool_use)

Notes

This fix assumes that the ServerToolUse model is correctly defined and can parse the server_tool_use JSON object returned by the provider.

Recommendation

Apply workaround: Update tool_call_cost_tracking.py to handle the dict case using getattr() or isinstance(), as this is a more straightforward change that can be made immediately, while also considering the longer-term fix of validating the usage.server_tool_use field in the passthrough handler.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#memory management#API rate limit#retriever error#indexing error#inference speed

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING