litellm - ✅(Solved) Fix [Bug]: `completion_cost()` raises `AttributeError` on streaming Anthropic responses with web_search in v1.83.10 [3 pull requests, 1 participants]

litellm2026-04-21 05:42:42

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

BerriAI/litellm#26153•Fetched 2026-04-22 07:46:18

View on GitHub

Comments

Participants

Timeline

Reactions

Author

jph00

Participants

jph00

Timeline (top)

cross-referenced ×3labeled ×3referenced ×1

Error Message

File .../litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py:343, in response_object_includes_web_search_call and usage.server_tool_use.web_search_requests is not None AttributeError: 'dict' object has no attribute 'web_search_requests'

Root Cause

The root cause: stream_chunk_builder reconstructs usage.server_tool_use as a dict rather than a ServerToolUse pydantic object. In v1.83.10, a new check was added in StandardBuiltInToolCostTracking.response_object_includes_web_search_call that accesses usage.server_tool_use.web_search_requests directly (see tool_call_cost_tracking.py#L340-L344). The same pattern also exists in get_cost_for_anthropic_web_search in anthropic/cost_calculation.py#L131.

Fix Action

Fixed

Fixed by PR: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder (https://github.com/BerriAI/litellm/pull/26174)
Fixed by PR: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder (https://github.com/BerriAI/litellm/pull/26178)
Fixed by PR: fix(cost_calculator.py): fix AttributeError in _get_usage_object for streaming responses (https://github.com/BerriAI/litellm/pull/26206)

PR fix notes

PR #26174: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder

Repository: BerriAI/litellm
Author: Anai-Guo
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/26174

Description (problem / solution / changelog)

Summary

Fixes #26153

When streaming Anthropic responses with web_search_options, stream_chunk_builder reconstructs usage.server_tool_use as a plain dict rather than a ServerToolUse pydantic object. In v1.83.10, a new check in StandardBuiltInToolCostTracking.response_object_includes_web_search_call (and get_cost_for_anthropic_web_search) accesses .web_search_requests as an attribute, causing:

AttributeError: 'dict' object has no attribute 'web_search_requests'

Root cause: In _calculate_usage_per_chunk, server_tool_use is assigned directly from usage_chunk.server_tool_use without checking whether it is already a ServerToolUse object or a raw dict.

Fix: Apply the same dict→object coercion pattern already used for prompt_tokens_details in the same method:

# Before:
server_tool_use = usage_chunk.server_tool_use

# After:
_stu = usage_chunk.server_tool_use
server_tool_use = ServerToolUse(**_stu) if isinstance(_stu, dict) else _stu

Reproduction

from litellm import completion, stream_chunk_builder, completion_cost

chunks = list(completion(
    'claude-sonnet-4-6',
    [{'role': 'user', 'content': 'Search the web for otters and tell me one fact'}],
    stream=True,
    web_search_options={'search_context_size': 'low'},
    stream_options={'include_usage': True},
))
r = stream_chunk_builder(chunks)
print(type(r.usage.server_tool_use))  # was <class 'dict'>, now ServerToolUse
completion_cost(completion_response=r)  # was AttributeError, now works

Test plan

Verify type(r.usage.server_tool_use) returns ServerToolUse after streaming Anthropic web search response
Verify completion_cost() no longer raises AttributeError for streaming Anthropic responses with web_search_options
Existing streaming tests continue to pass

🤖 Generated with Claude Code

Changed files

litellm/litellm_core_utils/streaming_chunk_builder_utils.py (modified, +4/-1)

PR #26178: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder

Repository: BerriAI/litellm
Author: VANDRANKI
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/26178

Description (problem / solution / changelog)

What

Fixes AttributeError: 'dict' object has no attribute 'web_search_requests' raised by completion_cost() on streaming Anthropic responses that used web_search_options.

Closes #26153.

Root cause

stream_chunk_builder accumulates server_tool_use from streaming chunks at:

server_tool_use = usage_chunk.server_tool_use

When the Anthropic streaming JSON is deserialized, usage_chunk.server_tool_use may be a plain dict rather than a ServerToolUse pydantic object (pydantic does not auto-coerce nested model fields on a SafeAttributeModel). The reconstructed Usage object then carries a dict for server_tool_use.

In v1.83.10, StandardBuiltInToolCostTracking.response_object_includes_web_search_call added a direct attribute access:

and usage.server_tool_use.web_search_requests is not None

This crashes when server_tool_use is a dict.

Fix

Coerce the value to ServerToolUse at the point of accumulation in streaming_chunk_builder_utils.py:

stu = usage_chunk.server_tool_use
server_tool_use = ServerToolUse(**stu) if isinstance(stu, dict) else stu

ServerToolUse is already imported in the file. The fix is one-liner and addresses the root cause rather than patching the two call sites that access .web_search_requests.

Test

Added test_stream_chunk_builder_coerces_server_tool_use_dict_to_object in tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py. The test injects a dict-typed server_tool_use into a usage chunk and asserts that the rebuilt Usage has a proper ServerToolUse instance with the correct web_search_requests value.

Changed files

litellm/litellm_core_utils/streaming_chunk_builder_utils.py (modified, +4/-1)
litellm/types/utils.py (modified, +2/-0)
tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py (modified, +55/-1)

PR #26206: fix(cost_calculator.py): fix AttributeError in _get_usage_object for streaming responses

Repository: BerriAI/litellm
Author: EaCognitive
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/26206

Description (problem / solution / changelog)

Fixes #26153

Problem

In LiteLLM's cost calculation and streaming aggregation logic, the server_tool_use field (used by Anthropic web_search) often remains as a raw dict after being assembled from streaming chunks. Downstream functions in cost_calculator.py and anthropic/cost_calculation.py expect a Pydantic model and attempt to access attributes like web_search_requests directly, resulting in an AttributeError.

Solution

This PR implements a robust, multi-layered fix:

Core Model Coercion: Updated Usage.__init__ in litellm/types/utils.py to automatically coerce server_tool_use dictionaries into ServerToolUse objects.
Streaming Aggregation: Updated ChunkProcessor in streaming_chunk_builder_utils.py to correctly handle dictionary-based usage chunks when extracting tool metadata.
Defensive Access: Updated call sites in tool_call_cost_tracking.py and anthropic/cost_calculation.py to use getattr when accessing tool-specific fields.

Changes

litellm/types/utils.py: Added type coercion for server_tool_use.
litellm/litellm_core_utils/streaming_chunk_builder_utils.py: Fixed usage extraction from dict chunks.
litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py: Added defensive attribute checks.
litellm/llms/anthropic/cost_calculation.py: Added defensive attribute checks.
litellm/tests/test_cost_calculator_attributeerror.py: Added a new regression test.

Verification

Verified with the included test case that completion_cost() no longer crashes when server_tool_use is a dictionary.

Changed files

.circleci/config.yml (modified, +9/-161)
.github/scripts/close_duplicate_issues.py (modified, +31/-9)
.github/scripts/scan_keywords.py (modified, +14/-6)
.github/workflows/guard-main-branch.yml (added, +42/-0)
.github/workflows/test-linting.yml (modified, +5/-1)
.github/workflows/test-litellm-ui-build.yml (modified, +5/-1)
.github/workflows/test-mcp.yml (modified, +5/-1)
.github/workflows/test-model-map.yaml (modified, +5/-1)
.github/workflows/test-unit-core-utils.yml (modified, +5/-1)
.github/workflows/test-unit-documentation.yml (modified, +5/-1)
.github/workflows/test-unit-enterprise-routing.yml (modified, +5/-1)
.github/workflows/test-unit-integrations.yml (modified, +5/-1)
.github/workflows/test-unit-llm-providers.yml (modified, +5/-1)
.github/workflows/test-unit-misc.yml (modified, +5/-1)
.github/workflows/test-unit-proxy-auth.yml (modified, +5/-1)
.github/workflows/test-unit-proxy-db.yml (modified, +1/-1)
.github/workflows/test-unit-proxy-endpoints.yml (modified, +5/-1)
.github/workflows/test-unit-proxy-infra.yml (modified, +5/-1)
.github/workflows/test-unit-proxy-legacy.yml (modified, +5/-1)
.github/workflows/test-unit-responses-caching-types.yml (modified, +5/-1)
.github/workflows/test-unit-security.yml (modified, +1/-1)
.github/workflows/test_server_root_path.yml (modified, +6/-2)
CLAUDE.md (modified, +1/-1)
ci_cd/run_migration.py (modified, +3/-1)
cookbook/anthropic_agent_sdk/agent_with_mcp.py (modified, +27/-23)
cookbook/anthropic_agent_sdk/common.py (modified, +33/-29)
cookbook/anthropic_agent_sdk/main.py (modified, +23/-19)
cookbook/litellm_proxy_server/batch_api/bedrock/bedrock.py (modified, +3/-3)
cookbook/litellm_proxy_server/cli_token_usage.py (modified, +10/-9)
cookbook/litellm_proxy_server/mcp/mcp_with_litellm_proxy.py (modified, +6/-5)
cookbook/litellm_proxy_server/secret_manager/my_secret_manager.py (modified, +4/-3)
cookbook/livekit_agent_sdk/main.py (modified, +42/-33)
cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md (modified, +26/-0)
cookbook/misc/test_responses_api.py (modified, +5/-9)
cookbook/nova_sonic_realtime.py (modified, +7/-4)
cookbook/veo_video_generation.py (modified, +87/-77)
deploy/charts/litellm-helm/templates/deployment.yaml (modified, +2/-2)
deploy/charts/litellm-helm/templates/migrations-job.yaml (modified, +2/-2)
deploy/charts/litellm-helm/tests/deployment_tests.yaml (modified, +58/-0)
deploy/charts/litellm-helm/tests/migrations-job_tests.yaml (modified, +66/-0)
docs/my-website/blog/claude_opus_4_7/index.md (added, +366/-0)
docs/my-website/docs/completion/prompt_compression.md (added, +123/-0)
docs/my-website/docs/index.md (modified, +4/-0)
docs/my-website/docs/providers/github_copilot.md (modified, +7/-0)
docs/my-website/docs/proxy/config_settings.md (modified, +11/-0)
docs/my-website/docs/proxy/cost_tracking.md (modified, +4/-0)
docs/my-website/docs/proxy/guardrails/hiddenlayer.md (modified, +1/-0)
docs/my-website/docs/proxy/guardrails/policy_flow_builder.md (modified, +99/-5)
docs/my-website/docs/proxy/health.md (modified, +25/-1)
docs/my-website/docs/proxy/ui_team_soft_budget_alerts.md (modified, +10/-0)
docs/my-website/docs/proxy/users.md (modified, +61/-0)
docs/my-website/docs/skills_gateway.md (added, +111/-0)
docs/my-website/docs/troubleshoot/cost_discrepancy.md (added, +205/-0)
docs/my-website/docs/tutorials/claude_code_byok.md (modified, +11/-0)
docs/my-website/docusaurus.config.js (modified, +27/-1)
docs/my-website/img/release_notes/guardrail_fallbacks.png (added, +0/-0)
docs/my-website/package-lock.json (modified, +403/-0)
docs/my-website/package.json (modified, +2/-0)
docs/my-website/release_notes/v1.83.3/index.md (modified, +337/-51)
docs/my-website/release_notes/v1.83.7.rc.1/index.md (added, +223/-0)
docs/my-website/sidebars.js (modified, +14/-0)
docs/my-website/src/pages/index.md (modified, +4/-0)
docs/my-website/static/img/cost-discrepancy-debug/date-range-picker.png (added, +0/-0)
docs/my-website/static/img/cost-discrepancy-debug/go-to-model-activity.png (added, +0/-0)
docs/my-website/static/img/cost-discrepancy-debug/scroll-to-model.png (added, +0/-0)
docs/my-website/static/img/cost-discrepancy-debug/token-categories.png (added, +0/-0)
enterprise/litellm_enterprise/enterprise_callbacks/send_emails/base_email.py (modified, +168/-28)
enterprise/pyproject.toml (modified, +2/-2)
litellm-proxy-extras/litellm_proxy_extras/_logging.py (modified, +5/-2)
litellm-proxy-extras/litellm_proxy_extras/migrations/20260401000000_add_budget_limits/migration.sql (added, +5/-0)
litellm-proxy-extras/litellm_proxy_extras/migrations/20260401000000_add_team_member_model_scope/migration.sql (added, +9/-0)
litellm-proxy-extras/litellm_proxy_extras/migrations/20260414140000_add_mcp_server_instructions/migration.sql (added, +2/-0)
litellm-proxy-extras/litellm_proxy_extras/migrations/20260415120000_health_check_latest_per_model_index/migration.sql (added, +12/-0)
litellm-proxy-extras/litellm_proxy_extras/schema.prisma (modified, +7/-1)
litellm-proxy-extras/litellm_proxy_extras/utils.py (modified, +76/-21)
litellm-proxy-extras/pyproject.toml (modified, +2/-2)
litellm/__init__.py (modified, +70/-54)
litellm/_internal_context.py (added, +13/-0)
litellm/_lazy_imports.py (modified, +1/-0)
litellm/_logging.py (modified, +2/-0)
litellm/a2a_protocol/main.py (modified, +3/-3)
litellm/a2a_protocol/providers/bedrock_agentcore/handler.py (modified, +1/-3)
litellm/a2a_protocol/streaming_iterator.py (modified, +3/-3)
litellm/anthropic_beta_headers_config.json (modified, +4/-4)
litellm/anthropic_interface/__init__.py (modified, +1/-0)
litellm/anthropic_interface/messages/__init__.py (modified, +2/-2)
litellm/caching/caching_handler.py (modified, +6/-4)
litellm/caching/gcs_cache.py (modified, +1/-0)
litellm/caching/in_memory_cache.py (modified, +4/-3)
litellm/completion_extras/litellm_responses_transformation/handler.py (modified, +1/-3)
litellm/completion_extras/litellm_responses_transformation/transformation.py (modified, +20/-16)
litellm/compression/__init__.py (added, +3/-0)
litellm/compression/compress.py (added, +257/-0)
litellm/compression/content_detection.py (added, +45/-0)
litellm/compression/message_stubbing.py (added, +120/-0)
litellm/compression/retrieval_tool.py (added, +35/-0)
litellm/compression/scoring/__init__.py (added, +4/-0)
litellm/compression/scoring/bm25.py (added, +123/-0)
litellm/compression/scoring/embedding_scorer.py (added, +95/-0)
litellm/constants.py (modified, +51/-2)

Code Example

from litellm import completion, stream_chunk_builder, completion_cost

chunks = list(completion(
    'claude-sonnet-4-6',
    [{'role':'user','content':'Search the web for otters and tell me one fact'}],
    stream=True,
    web_search_options={'search_context_size':'low'},
    stream_options={'include_usage':True},
))
r = stream_chunk_builder(chunks)
print(type(r.usage.server_tool_use))  # <class 'dict'>
completion_cost(completion_response=r)  # AttributeError

---

File .../litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py:343, in response_object_includes_web_search_call
    and usage.server_tool_use.web_search_requests is not None
AttributeError: 'dict' object has no attribute 'web_search_requests'

RAW_BUFFERClick to expand / collapse

Check for existing issues

I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

In v1.83.10, completion_cost() raises AttributeError: 'dict' object has no attribute 'web_search_requests' when called on a streaming Anthropic response that used web_search_options.

This was working in 1.83.0 because the new server_tool_use branch in the guard didn't exist, so _handle_web_search_cost() was never reached for streaming Anthropic responses.

Suggested fix

Either:

Coerce server_tool_use to ServerToolUse in stream_chunk_builder when reconstructing usage from streaming chunks (addresses root cause), or
Use getattr(..., 'web_search_requests', None) or nested_idx-style dict/attr tolerant access in the two call sites above.

Version

litellm==1.83.10 (works in 1.83.0) Python 3.12

Steps to Reproduce

from litellm import completion, stream_chunk_builder, completion_cost

chunks = list(completion(
    'claude-sonnet-4-6',
    [{'role':'user','content':'Search the web for otters and tell me one fact'}],
    stream=True,
    web_search_options={'search_context_size':'low'},
    stream_options={'include_usage':True},
))
r = stream_chunk_builder(chunks)
print(type(r.usage.server_tool_use))  # <class 'dict'>
completion_cost(completion_response=r)  # AttributeError

Expected

completion_cost() returns a float (as in 1.83.0).

Relevant log output

File .../litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py:343, in response_object_includes_web_search_call
    and usage.server_tool_use.web_search_requests is not None
AttributeError: 'dict' object has no attribute 'web_search_requests'

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on ?

1.83.10

Twitter / LinkedIn details

No response

extent analysis

TL;DR

The most likely fix is to coerce server_tool_use to ServerToolUse in stream_chunk_builder or use dict/attr tolerant access in the affected call sites.

Guidance

Verify that the issue is indeed caused by server_tool_use being a dict instead of a ServerToolUse object by checking the type of r.usage.server_tool_use in the provided reproduction code.
Consider coercing server_tool_use to ServerToolUse in stream_chunk_builder as a potential fix, addressing the root cause of the issue.
Alternatively, use getattr(..., 'web_search_requests', None) or nested_idx-style dict/attr tolerant access in the two call sites (tool_call_cost_tracking.py and anthropic/cost_calculation.py) to mitigate the error.
Test the suggested fixes using the provided reproduction code to ensure the issue is resolved.

Example

from pydantic import ServerToolUse

# Coerce server_tool_use to ServerToolUse in stream_chunk_builder
r.usage.server_tool_use = ServerToolUse(**r.usage.server_tool_use)

Notes

The provided reproduction code and error message suggest that the issue is specific to the litellm package version 1.83.10 and Python 3.12.

Recommendation

Apply the workaround by coercing server_tool_use to ServerToolUse in stream_chunk_builder or using dict/attr tolerant access, as this addresses the root cause of the issue and is a targeted fix.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#latency issue #model loading #dependency error #configuration error #environment variable

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

litellm - ✅(Solved) Fix [Bug]: `completion_cost()` raises `AttributeError` on streaming Anthropic responses with web_search in v1.83.10 [3 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Fix Action

Fixed

PR fix notes

PR #26174: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder

Description (problem / solution / changelog)

Summary

Reproduction

Test plan

Changed files

PR #26178: fix: coerce server_tool_use dict to ServerToolUse in stream_chunk_builder

Description (problem / solution / changelog)

What

Root cause

Fix

Test

Changed files

PR #26206: fix(cost_calculator.py): fix AttributeError in _get_usage_object for streaming responses

Description (problem / solution / changelog)

Problem

Solution

Changes

Verification

Changed files

Code Example

Check for existing issues

What happened?

Suggested fix

Version

Steps to Reproduce

Expected

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING