langchain - 💡(How to fix) Fix _to_protocol_usage drops input_token_details in v3 streaming path, causing cache token counts to always be 0

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Root Cause

In langchain_core/language_models/_compat_bridge.py, _to_protocol_usage only copies four keys and silently drops input_token_details and output_token_details:

# _compat_bridge.py line ~485
def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens"):
        if key in usage:
            result[key] = usage[key]
    # input_token_details and output_token_details are never forwarded
    return cast("UsageInfo", result) if result else None

_accumulate_usage (called in achunks_to_events) correctly accumulates input_token_details from each chunk. But _to_protocol_usage discards it when building the message-finish protocol event. _ChatModelStreamBase._finish sets _usage_value from that event, so the assembled AIMessage.usage_metadata never carries the cache breakdown.

Fix Action

Fix

def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    if usage is None:
        return None
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens",
                "input_token_details", "output_token_details"):
        if key in usage:
            result[key] = usage[key]
    return cast("UsageInfo", result) if result else None

Workaround

from langchain_core.language_models import _compat_bridge as _lcb
_orig = _lcb._to_protocol_usage
def _patched(usage):
    if not usage:
        return None
    result = {k: usage[k] for k in ("input_tokens", "output_tokens", "total_tokens",
              "cached_tokens", "input_token_details", "output_token_details") if k in usage}
    return result or None
_lcb._to_protocol_usage = _patched

Code Example

# _compat_bridge.py line ~485
def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens"):
        if key in usage:
            result[key] = usage[key]
    # input_token_details and output_token_details are never forwarded
    return cast("UsageInfo", result) if result else None

---

def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    if usage is None:
        return None
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens",
                "input_token_details", "output_token_details"):
        if key in usage:
            result[key] = usage[key]
    return cast("UsageInfo", result) if result else None

---

from langchain_core.language_models import _compat_bridge as _lcb
_orig = _lcb._to_protocol_usage
def _patched(usage):
    if not usage:
        return None
    result = {k: usage[k] for k in ("input_tokens", "output_tokens", "total_tokens",
              "cached_tokens", "input_token_details", "output_token_details") if k in usage}
    return result or None
_lcb._to_protocol_usage = _patched
RAW_BUFFERClick to expand / collapse

Bug

When using astream_events(version="v3"), AIMessage.usage_metadata never contains input_token_details (cache token breakdown), even when the provider returns cache tokens.

Root cause

In langchain_core/language_models/_compat_bridge.py, _to_protocol_usage only copies four keys and silently drops input_token_details and output_token_details:

# _compat_bridge.py line ~485
def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens"):
        if key in usage:
            result[key] = usage[key]
    # input_token_details and output_token_details are never forwarded
    return cast("UsageInfo", result) if result else None

_accumulate_usage (called in achunks_to_events) correctly accumulates input_token_details from each chunk. But _to_protocol_usage discards it when building the message-finish protocol event. _ChatModelStreamBase._finish sets _usage_value from that event, so the assembled AIMessage.usage_metadata never carries the cache breakdown.

Fix

def _to_protocol_usage(usage: dict[str, Any] | None) -> UsageInfo | None:
    if usage is None:
        return None
    result: dict[str, Any] = {}
    for key in ("input_tokens", "output_tokens", "total_tokens", "cached_tokens",
                "input_token_details", "output_token_details"):
        if key in usage:
            result[key] = usage[key]
    return cast("UsageInfo", result) if result else None

To reproduce

  1. Use ChatBedrockConverse with prompt caching enabled (or any provider that returns cache token details in usage_metadata.input_token_details)
  2. Run a LangGraph graph with astream_events(version="v3")
  3. Inspect AIMessage.usage_metadatainput_token_details is absent

Environment

  • langchain-core 1.4.0 (latest as of May 2026) — bug confirmed present
  • langchain-aws 1.5.0, provider: ChatBedrockConverse
  • Triggered by any framework that attaches _V2StreamingCallbackHandler (e.g. deepagents ≥ 0.6.3)

Workaround

from langchain_core.language_models import _compat_bridge as _lcb
_orig = _lcb._to_protocol_usage
def _patched(usage):
    if not usage:
        return None
    result = {k: usage[k] for k in ("input_tokens", "output_tokens", "total_tokens",
              "cached_tokens", "input_token_details", "output_token_details") if k in usage}
    return result or None
_lcb._to_protocol_usage = _patched

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING