Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: VertexAI (Gemini) + LiteLLM Request Caching is broken [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#22970•Fetched 2026-04-08 00:39:08
View on GitHub
Comments
0
Participants
1
Timeline
6
Reactions
0
Author
andresC98andresC98
Participants
andresC98andresC98
Timeline (top)
labeled ×3cross-referenced ×2closed ×1

Error Message

{ "error": { "message": "litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "description": "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."\n }\n ]\n }\n ]\n }\n}\n", "type": null, "param": null, "code": "400" } }

Fix Action

Fixed

PR fix notes

PR #22991: fix(vertex): drop extra_body.cache from Gemini request payload

Description (problem / solution / changelog)

relevant issues

Fixes : #22970

🐛 Bug Fix

✅ Test

Changes

  • Stop forwarding extra_body.cache to Vertex/Gemini payloads.
  • Keep all valid extra_body fields unchanged.
  • Add regression tests for Vertex and Gemini paths.

Changed files

  • litellm/llms/vertex_ai/gemini/transformation.py (modified, +1/-0)
  • tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py (modified, +48/-0)

PR #23131: fix(vertex_ai): strip LiteLLM-internal keys from extra_body before merging to Gemini request

Description (problem / solution / changelog)

Problem

After upgrading from litellm proxy 1.79.3 to 1.81.12, Vertex AI Gemini models fail when using proxy cache with extra_body={"cache": {"use-cache": True, "ttl": 86400}}:

Invalid JSON payload received. Unknown name "cache": Cannot find field.

Fixes #22970

Root Cause

PR #20950 added extra_body forwarding to the Vertex AI Gemini completion transformation. Before that, extra_body was silently dropped—so cache never reached Vertex AI. After the change, all keys from extra_body are merged into the request body. Vertex AI enforces a strict JSON schema and rejects unknown fields like cache and tags.

Solution

  • Add _LITELLM_INTERNAL_EXTRA_BODY_KEYS frozenset (cache, tags)
  • Skip these keys in _pop_and_merge_extra_body before merging into the Vertex AI request
  • cache is consumed by LiteLLM's proxy response caching layer (caching.md)
  • tags is consumed by LiteLLM's logging/tracking (litellm_proxy.md)

Testing

  • test_extra_body_cache_not_forwarded_to_vertex_ai — verifies cache is stripped
  • test_extra_body_tags_not_forwarded_to_vertex_ai — verifies tags is stripped
  • Both tests ensure legitimate extra_body keys still pass through

Made with Cursor

Changed files

  • litellm/llms/vertex_ai/gemini/transformation.py (modified, +6/-0)
  • litellm/model_prices_and_context_window_backup.json (modified, +36/-0)
  • tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py (modified, +69/-0)

Code Example

{
  "error": {
    "message": "litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n",
    "type": null,
    "param": null,
    "code": "400"
  }
}

---

messages = [{"role": "user", "content": llm_request.message}]
        tools = [{"googleSearch": {}}] if llm_config.use_ground_search else []
        extra_body = {"cache": {"use-cache": True, "ttl": 86400}} if config.use_pod_level_cache else {}

        completion = litellm.completion(
            model=self.MODEL_PREFIX + llm_config.model_name,
            messages=messages,
            temperature=llm_config.temperature,
            metadata=llm_request.metadata or {},
            response_format=llm_config.response_model,
            tools=tools or [],
            max_tokens=llm_config.max_tokens or config.max_tokens,
            extra_body=extra_body,
        )

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using litellm proxy on 1.79.3, using gemini-2.5 models plus litellm proxy cache worked just fine However after upgrading litellm proxy to 1.81.12 stable, users find this error

{
  "error": {
    "message": "litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-pro\nAvailable Model Group Fallbacks=['gemini-2.5-flash']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n. Received Model Group=gemini-2.5-flash\nAvailable Model Group Fallbacks=['gemini-2.5-pro']\nError doing the fallback: litellm.BadRequestError: Vertex_aiException BadRequestError - {\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"cache\\\": Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n",
    "type": null,
    "param": null,
    "code": "400"
  }
}

Caching with other providers (vllm) works fine.

Steps to Reproduce

messages = [{"role": "user", "content": llm_request.message}]
        tools = [{"googleSearch": {}}] if llm_config.use_ground_search else []
        extra_body = {"cache": {"use-cache": True, "ttl": 86400}} if config.use_pod_level_cache else {}

        completion = litellm.completion(
            model=self.MODEL_PREFIX + llm_config.model_name,
            messages=messages,
            temperature=llm_config.temperature,
            metadata=llm_request.metadata or {},
            response_format=llm_config.response_model,
            tools=tools or [],
            max_tokens=llm_config.max_tokens or config.max_tokens,
            extra_body=extra_body,
        )

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

1.81.12

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

The fix involves removing the "cache" field from the extra_body dictionary in the litellm.completion call, as it is no longer supported in the upgraded version of LiteLLM.

  • Remove the "cache" field from the extra_body dictionary:
extra_body =

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#retrieval issue#search optimization#API routing#API middleware

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: VertexAI (Gemini) + LiteLLM Request Caching is broken [2 pull requests, 1 participants]