litellm - 💡(How to fix) Fix [Feature]: Preserve Anthropic cache_creation TTL breakdown in Chat Completions usage [1 pull requests]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

LiteLLM should preserve Anthropic's prompt cache creation TTL breakdown in OpenAI-compatible Chat Completions usage responses.

Anthropic returns cache write token details under usage.cache_creation, for example:

{
  "cache_creation": {
    "ephemeral_5m_input_tokens": 0,
    "ephemeral_1h_input_tokens": 6179
  }
}

LiteLLM currently exposes the aggregate cache_creation_input_tokens, but downstream clients cannot reliably distinguish how many cache write tokens used the 5 minute TTL versus the 1 hour TTL in Chat Completions responses, especially for streaming responses where usage is aggregated before being returned to the client.

The requested behavior is to include the TTL-level breakdown in Chat Completions usage while keeping the existing aggregate fields:

{
  "usage": {
    "cache_creation_input_tokens": 6179,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 6179
    }
  }
}

Motivation, pitch

This is needed for accurate downstream billing and cost reconciliation.

Anthropic can charge prompt cache writes differently depending on TTL. If LiteLLM only returns cache_creation_input_tokens, clients can see the total cache write tokens, but cannot calculate the correct cost when a request uses 1 hour cache writes instead of 5 minute cache writes.

Preserving usage.cache_creation.ephemeral_5m_input_tokens and usage.cache_creation.ephemeral_1h_input_tokens would let clients calculate Anthropic prompt cache write costs from the same Chat Completions response they already consume, without needing to call the Anthropic Messages API directly or depend on internal LiteLLM cost calculation.

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response