litellm - 💡(How to fix) Fix [Bug]: Gemini grounding silently disabled when web_grounding tools are mixed with function tools (3 compounding failures)

What happened?

When a Gemini-3+ chat completion request mixes a hosted search tool (google_search / web_search_preview) with function-shape tools, LiteLLM silently drops the search tool entry and the model answers from training data only. Three independent code paths compound — each alone is sufficient to break grounding — so the user-visible "fix" of passing include_server_side_tool_invocations: true doesn't actually take effect even when the request body contains it.

The only signal in the proxy logs is one warning per request:

Vertex AI does not support mixing function declarations with search tools (googleSearch, enterpriseWebSearch, urlContext, googleSearchRetrieval) in the same request. Dropping search tools and keeping function declarations.

The model's response looks plausible — it's not an error. So callers don't notice unless they specifically validate the answer was grounded. We hit this in production: a query about a real recently-released product (Deepgram Flux) returned a confident "this product doesn't exist" reply, and a "what are the dates of upcoming SAASTR 2026" query returned a fabricated date.

This is the same surface symptom flagged in #16962 (closed as "not planned"). Filing this as a focused, reproducible report with the three causes traced.

Relevant log output

LiteLLM:WARNING vertex_and_google_ai_studio_gemini.py:525
  Vertex AI does not support mixing function declarations with search tools
  (googleSearch, enterpriseWebSearch, urlContext, googleSearchRetrieval) in
  the same request. Dropping search tools and keeping function declarations.
  To use search tools, send a request without function calling tools.

Are you a ML Ops Team?

No

What LiteLLM version are you on?

v1.83.10-stable (also reproducible on current main as of 2026-05-08).

Reproduction

Request body (snake_case via OpenAI-compatible chat-completions endpoint):

{
  "model": "gemini-3.1-pro-preview",
  "messages": [{"role": "user", "content": "What are the dates of upcoming SAASTR 2026?"}],
  "tools": [
    {"google_search": {}},
    {"type": "function", "function": {
      "name": "send_message",
      "description": "Send a message back",
      "parameters": {"type": "object", "properties": {"message": {"type": "string"}}, "required": ["message"]}
    }}
  ],
  "include_server_side_tool_invocations": true,
  "tool_choice": "required"
}

Expected: googleSearch reaches Vertex, model uses search results, returns grounded answer.

Actual: googleSearch is silently stripped, model answers from training data, returns plausible-but-fabricated reply.

The same reproduces with web_search_preview (the OpenAI-shape entry that LiteLLM's gemini transform converts to googleSearch internally).

Root-cause trace

The flag has to traverse three filters in order. Any one of them silently drops it.

Failure 1 — DEFAULT_CHAT_COMPLETION_PARAM_VALUES (constants.py)

include_server_side_tool_invocations is not in the dict at litellm/constants.py (the dict ends at "context_management": None).

get_non_default_params (utils.py:4838 and base_pre_process_non_default_params at utils.py:3709) keeps only request-body keys present in this dict. The flag gets stripped here, before any provider-specific code runs.

The Vertex provider's map_openai_params has a handler at line ~1208 (elif param == "include_server_side_tool_invocations" and value is True: optional_params["include_server_side_tool_invocations"] = True) that would correctly thread the flag through, but that handler never fires because the flag never reaches non_default_params.

Failure 2 — GoogleAIStudioGeminiConfig.get_supported_openai_params

Even if the flag survives Failure 1, the chat path's _check_valid_arg (utils.py:~4030) compares non_default_params keys against the provider's supported_params list and pops any not in it (when litellm.drop_params=True).

VertexGeminiConfig.get_supported_openai_params (litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py:320) does list the flag. But the resolved config for models routed via gemini/... (Google AI Studio) is GoogleAIStudioGeminiConfig (litellm/llms/gemini/chat/transformation.py:75), a subclass that overrides the parent's method with its own hardcoded list and omits the flag:

def get_supported_openai_params(self, model: str) -> List[str]:
    supported_params = [
        "temperature", "top_p", "max_tokens", "max_completion_tokens",
        "stream", "tools", "tool_choice", "functions", "response_format",
        "n", "stop", "logprobs", "frequency_penalty", "presence_penalty",
        "modalities", "parallel_tool_calls", "web_search_options",
        "service_tier",
    ]   # <-- "include_server_side_tool_invocations" missing

With drop_params=True set, the flag silently gets popped from non_default_params before reaching map_openai_params.

Failure 3 — Iteration order in map_openai_params

If 1 and 2 are both fixed, the flag finally lands in non_default_params for VertexGeminiConfig.map_openai_params (the gemini subclass inherits this method).

map_openai_params walks non_default_params in dict insertion order. Python's completion() function signature in litellm/main.py defines tools early and include_server_side_tool_invocations late, so non_default_params has tools first.

When the loop reaches tools, it calls _map_function, which calls _resolve_search_tool_conflict (line 484). That function reads:

server_side_tool_invocations = optional_params.get(
    "include_server_side_tool_invocations", False
)

But the flag's handler at line ~1208 hasn't run yet (iteration is currently at tools, hasn't reached the flag). optional_params doesn't have the flag. The conflict resolver fires the drop warning and strips googleSearch before iteration ever reaches the flag's handler.

Verification

Isolated reproduction (in a Python shell against the running proxy's venv):

from litellm.llms.gemini.chat.transformation import GoogleAIStudioGeminiConfig
cfg = GoogleAIStudioGeminiConfig()

# non_default_params with the flag LAST (matches the order completion() emits)
result = cfg.map_openai_params(
    non_default_params={
        "tools": [
            {"google_search": {}},
            {"type": "function", "function": {"name": "f", "description": "x",
                                              "parameters": {"type": "object", "properties": {}}}},
        ],
        "include_server_side_tool_invocations": True,
    },
    optional_params={},
    model="gemini-3.1-pro-preview",
    drop_params=False,
)
# Pre-fix:  result["tools"] = [{"function_declarations": [...]}]   <- googleSearch dropped, drop warning fires
# Post-fix: result["tools"] = [{"function_declarations": [...]},
#                              {"googleSearch": {}}]                <- preserved, no warning

The same dict with include_server_side_tool_invocations first preserves googleSearch even on unpatched LiteLLM — that's the smoking gun for Failure 3.

Suggested fixes

Three small, independent patches. Each fixes one failure point.

Fix 1 — litellm/constants.py:

@@ DEFAULT_CHAT_COMPLETION_PARAM_VALUES = {
     "context_management": None,
+    "include_server_side_tool_invocations": None,
 }

Fix 2 — litellm/llms/gemini/chat/transformation.py (GoogleAIStudioGeminiConfig.get_supported_openai_params):

@@ supported_params = [
             "service_tier",
+            "include_server_side_tool_invocations",
         ]

Fix 3 — litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py (VertexGeminiConfig.map_openai_params, immediately after the signature):

     ) -> Dict:
+        # Set include_server_side_tool_invocations in optional_params BEFORE
+        # iterating, so the tools handler (_resolve_search_tool_conflict) sees
+        # it regardless of where the param sits in non_default_params'
+        # iteration order.
+        if non_default_params.get("include_server_side_tool_invocations") is True:
+            optional_params["include_server_side_tool_invocations"] = True
         for param, value in non_default_params.items():

Happy to open a PR with all three if maintainers are receptive (we already wrote and tested all three as monkey-patches in our LiteLLM image; field-verified that the trio together restores grounding end-to-end on gemini-3.1-pro-preview, and any one removed re-introduces the silent drop).

Are you willing to submit a PR?

Yes — pending maintainer ack on this issue.