Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: `cache_control` directive dropped from `file`-type blocks in `anthropic_messages_pt()` [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23873•Fetched 2026-04-08 00:54:07
View on GitHub
Comments
0
Participants
1
Timeline
7
Reactions
0
Author
kelvin-trankelvin-tran
Participants
kelvin-trankelvin-tran
Timeline (top)
labeled ×3cross-referenced ×2closed ×1referenced ×1

Fix Action

Fixed

PR fix notes

PR #23906: fix(anthropic): preserve cache directive on file-type content blocks

Description (problem / solution / changelog)

Relevant issues

Fixes #23873

Pre-Submission checklist

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

cache_control was dropped from file-type content blocks when translating to Anthropic's document params in anthropic_messages_pt(). The text and image_url block types both call add_cache_control_to_content() to preserve this field, but file blocks (handled by anthropic_process_openai_file_message()) did not.

This meant prompt caching for PDF/document inputs never took effect even when cache_control: {"type": "ephemeral"} was explicitly set.

Fix: Apply add_cache_control_to_content() to the result of anthropic_process_openai_file_message(), matching the pattern used for text and image_url blocks.

Changed files

  • litellm/litellm_core_utils/prompt_templates/factory.py (modified, +7/-4)
  • tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py (modified, +53/-0)

PR #23911: fix: cache_control directive dropped anthropic document/file blocks

Description (problem / solution / changelog)

Relevant issues

Fixes #23873: cache_control being silently dropped from file content blocks during Anthropic message conversion, preventing prompt caching from working for PDF/document inputs.

Pre-Submission checklist

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🐛 Bug Fix

Changes

Problem

When sending messages with file or document content blocks that include cache_control: {"type": "ephemeral"}, the cache_control metadata is silently dropped during the OpenAI → Anthropic message format conversion. This prevents Anthropic prompt caching from working for PDF and document inputs.

Root Cause

In anthropic_messages_pt() (litellm/litellm_core_utils/prompt_templates/factory.py), text and image_url content blocks correctly preserve cache_control via add_cache_control_to_content(). However:

  • file blocks are converted by anthropic_process_openai_file_message(), which creates a new AnthropicMessagesDocumentParam without copying cache_control from the original block.
  • document blocks are appended directly via cast() without calling add_cache_control_to_content(), so cache_control is also not preserved.

Fix

Call add_cache_control_to_content() on the converted result for both file and document blocks, matching the existing pattern used for text and image_url blocks.

Before

elif m.get("type", "") == "document":
    user_content.append(cast(AnthropicMessagesDocumentParam, m))
elif m.get("type", "") == "file":
    user_content.append(
        anthropic_process_openai_file_message(
            cast(ChatCompletionFileObject, m)
        )
    )

After

elif m.get("type", "") == "document":
    _content_element = add_cache_control_to_content(
        anthropic_content_element=cast(AnthropicMessagesDocumentParam, m),
        original_content_element=dict(m),
    )
    user_content.append(_content_element)
elif m.get("type", "") == "file":
    _content_element = add_cache_control_to_content(
        anthropic_content_element=anthropic_process_openai_file_message(
            cast(ChatCompletionFileObject, m)
        ),
        original_content_element=dict(m),
    )
    user_content.append(_content_element)

Changed files

  • litellm/litellm_core_utils/prompt_templates/factory.py (modified, +14/-2)
  • tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py (modified, +88/-1)

Code Example

import litellm
import base64

# 1. Create a message with a file block that includes cache_control
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "document.pdf",
                    "file_data": "data:application/pdf;base64," + base64.b64encode(b"%PDF-1.4 fake pdf content").decode(),
                },
                "cache_control": {"type": "ephemeral"},  # <-- This gets dropped
            },
            {
                "type": "text",
                "text": "Summarize this document.",
                "cache_control": {"type": "ephemeral"},  # <-- This is preserved
            },
        ],
    }
]

# 2. Call litellm targeting an Anthropic model
response = litellm.completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=messages,
)

# 3. Observe: the file block's cache_control is not sent to Anthropic,
#    so prompt caching for the PDF does not take effect.
#    You can verify by inspecting the translated request payload
#    (e.g., via litellm.set_verbose = True) and noting that the
#    document content block has no "cache_control" key.

---

from litellm.litellm_core_utils.prompt_templates.factory import anthropic_messages_pt

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "doc.pdf",
                    "file_data": "data:application/pdf;base64,JVBERi0xLjQ=",
                },
                "cache_control": {"type": "ephemeral"},
            },
        ],
    }
]

result = anthropic_messages_pt(messages, model="claude-sonnet-4-20250514", llm_provider="anthropic")

# Inspect the translated content block:
print(result[0]["content"][0])
# Expected: dict should contain "cache_control": {"type": "ephemeral"}
# Actual:   "cache_control" key is missing

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When invoking Anthropic models, cache_control is silently dropped from file-type content blocks (e.g., PDFs). This means prompt caching does not work for document/file inputs even when cache_control is explicitly set.

In anthropic_messages_pt() (in factory.py), text and image_url blocks both correctly preserve cache_control by calling add_cache_control_to_content(). However, file blocks are handled by anthropic_process_openai_file_message(), which constructs a new AnthropicMessagesDocumentParam dict without copying the cache_control field from the original content block.

Relevant code paths (factory.py, approximately lines 2349–2383):

Content block typeHandlercache_control preserved?
textadd_cache_control_to_content()Yes
image_urladd_cache_control_to_content()Yes
fileanthropic_process_openai_file_message()No — dropped

Expected behavior: cache_control on file blocks should be preserved and forwarded to the Anthropic API, just like it is for text and image_url blocks. This would allow prompt caching to work correctly for PDF and other document inputs.

Steps to Reproduce

import litellm
import base64

# 1. Create a message with a file block that includes cache_control
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "document.pdf",
                    "file_data": "data:application/pdf;base64," + base64.b64encode(b"%PDF-1.4 fake pdf content").decode(),
                },
                "cache_control": {"type": "ephemeral"},  # <-- This gets dropped
            },
            {
                "type": "text",
                "text": "Summarize this document.",
                "cache_control": {"type": "ephemeral"},  # <-- This is preserved
            },
        ],
    }
]

# 2. Call litellm targeting an Anthropic model
response = litellm.completion(
    model="anthropic/claude-sonnet-4-20250514",
    messages=messages,
)

# 3. Observe: the file block's cache_control is not sent to Anthropic,
#    so prompt caching for the PDF does not take effect.
#    You can verify by inspecting the translated request payload
#    (e.g., via litellm.set_verbose = True) and noting that the
#    document content block has no "cache_control" key.

To confirm the bug at the code level, you can also inspect the translation directly:

from litellm.litellm_core_utils.prompt_templates.factory import anthropic_messages_pt

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "doc.pdf",
                    "file_data": "data:application/pdf;base64,JVBERi0xLjQ=",
                },
                "cache_control": {"type": "ephemeral"},
            },
        ],
    }
]

result = anthropic_messages_pt(messages, model="claude-sonnet-4-20250514", llm_provider="anthropic")

# Inspect the translated content block:
print(result[0]["content"][0])
# Expected: dict should contain "cache_control": {"type": "ephemeral"}
# Actual:   "cache_control" key is missing

Suggested Fix

In anthropic_process_openai_file_message(), after constructing the AnthropicMessagesDocumentParam, copy cache_control from the original content block if present — mirroring what add_cache_control_to_content() does for text and image_url blocks.

Relevant log output

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on ?

1.81.13

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To fix the issue, we need to modify the anthropic_process_openai_file_message() function to preserve the cache_control field from the original content block.

Here are the steps:

  • Open the factory.py file and locate the anthropic_process_openai_file_message() function.
  • After constructing the AnthropicMessagesDocumentParam dict, add the following code to copy the cache_control field:
if "cache_control" in content_block:
    document_param["cache_control"] = content_block["cache_control"]
  • The updated function should look like this:
def anthropic_process_openai_file_message(content_block):
    # ... (existing code)
    document_param = {
        # ... (existing code)
    }
    if "cache_control" in content_block:
        document_param["cache_control"] = content_block["cache_control"]
    return document_param

Verification

To verify the fix, run the following code:

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "doc.pdf",
                    "file_data": "data:application/pdf;base64,JVBERi0xLjQ=",
                },
                "cache_control": {"type": "ephemeral"},
            },
        ],
    }
]

result = anthropic_messages_pt(messages, model="claude-sonnet-4-20250514", llm_provider="anthropic")

# Inspect the translated content block:
print(result[0]["content"][0])
# Expected: dict should contain "cache_control": {"type": "ephemeral"}

The output should now include the cache_control field with the expected value.

Extra Tips

  • Make sure to test the fix with different types of files and cache control settings to ensure it works as expected.
  • Consider adding a test case to the LiteLLM test suite to prevent regressions.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model download#tokenizer error#prompt formatting#chain error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING