litellm - 💡(How to fix) Fix [Bug]: vertex_ai.compact-2026-01-12 set to null appears to block compact_20260112 over /v1/messages [1 participants]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

We hit HTTP 400 from Vertex when sending compact_20260112 edits via /v1/messages to a Vertex AI Anthropic model. Tracing through it, the behavior seems consistent with the compact-2026-01-12 beta header being stripped before the request reaches Vertex:

• experimental_pass_through/transformation.py:78-101 adds compact-2026-01-12 to anthropic-beta when a compact_20260112 edit is present
• update_headers_with_filtered_beta (called from llm_http_handler.py:1980) then filters per anthropic_beta_headers_config.json, where vertex_ai's entry maps compact-2026-01-12 to null

To check whether Vertex itself supports the beta, we tried calling :rawPredict directly with anthropic-beta: compact-2026-01-12 set — that returned HTTP 200 with content blocks: ['compaction', 'text'] and usage.iterations[] showing the compaction step ran. So at least on our project (europe-west4 / claude-sonnet-4-6@default), Vertex appears to handle it correctly when the header is forwarded.

Would the following toggle make sense? It mirrors what was done in #21867 for context-1m-2025-08-07:

   "vertex_ai": {
     ...
-    "compact-2026-01-12": null,
+    "compact-2026-01-12": "compact-2026-01-12",
     ...
   }

(Locally we worked around it by setting LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True and patching the bundled config, but that has the side effect of freezing the rest of the allowlist to whatever's in the image, so it's not ideal as a long-term fix.)

Happy to put up a PR if it'd be helpful — wanted to file the issue first to check whether there's a reason it's currently null that I'm missing.

Steps to Reproduce

1. Use a LiteLLM proxy ≥ v1.82.1 with a Vertex AI Anthropic model (e.g. vertex_ai/claude-sonnet-4-6@default).

2. POST to /v1/messages with a compact_20260112 edit:

   import httpx
   r = httpx.post(
       f"{LITELLM_BASE}/v1/messages",
       headers={
           "Authorization": f"Bearer {KEY}",
           "anthropic-version": "2023-06-01",
           "anthropic-beta": "compact-2026-01-12",
       },
       json={
           "model": "vertex_ai/claude-sonnet-4-6@default",
           "max_tokens": 100,
           "messages": [
               {"role": "user", "content": "Hello."},
               {"role": "assistant", "content": "Sure. " + "padding " * 5000},
               {"role": "user", "content": "Summarize briefly."},
           ],
           "context_management": {
               "edits": [{
                   "type": "compact_20260112",
                   "trigger": {"type": "input_tokens", "value": 50000},
               }]
           },
       },
       timeout=120,
   )
   print(r.status_code, r.text[:600])

3. Observe HTTP 400 from Vertex with: context_management.edits.0: Input tag 'compact_20260112' found using 'type' does not match any of the expected tags: 'clear_thinking_20251015', 'clear_tool_uses_20250919'

Relevant log output

{"type":"error","error":{"type":"invalid_request_error","message":"context_management.edits.0: Input tag 'compact_20260112' found using 'type' does not match any of the expected tags: 'clear_thinking_20251015', 'clear_tool_uses_20250919'"},"request_id":"req_vrtx_..."}

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.1