litellm - 💡(How to fix) Fix [Bug]: `file_search` tool results are missing from response for vertex_ai models, but returned correctly for bedrock models [1 participants]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Description When calling /chat/completions with the file_search tool enabled, the response for vertex_ai or azure models, for example,vertex_ai/gemini-2.0-flash-lite-001 does not include the expected vector search results in provider_specific_fields.search_results.

The same request works as expected with bedrock models, e.g. bedrock/global.anthropic.claude-opus-4-6-v1, where the search results are returned correctly.

Steps to Reproduce Send the following request:

curl $LITELLM_BASE_URL/chat/completions -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d @- << EOF
{
  "model": "vertex_ai/gemini-2.0-flash-lite-001",
  "max_tokens": 1000,
  "messages": [{"role": "user", "content": "xxx"}],
  "stream": true,
  "tools": [{
    "type": "file_search",
    "vector_store_ids": ["my-vector-store"]
  }]
}
EOF

Expected Behavior The response should include the vector search results, for example under:

{
  "provider_specific_fields": {
    "search_results": [
      {
        "object": "vector_store.search_results.page",
        "search_query": "xxx",
        "data": [
          {
            "score": 0.6659582853317261,
            "content": ...
          }
        ]
      }
    ]
  }
}

Actual Behavior For vertex_ai/gemini-2.0-flash-lite-001, the response does not include provider_specific_fields.search_results.

Comparison / Notes

• vertex_ai/gemini-2.0-flash-lite-001: search results are missing
• bedrock/global.anthropic.claude-opus-4-6-v1: search results are returned correctly with the same file_search tool setup

Impact This causes inconsistent behavior across models/providers and makes it difficult to rely on file_search tool output in a provider-agnostic way.

Possible Root Cause The file_search tool output may not be correctly surfaced or mapped into the response for Vertex AI Gemini models, even though the same flow works for Bedrock Claude models.

Suggested Fix Ensure that file_search tool results are consistently included in the response payload across providers, including Vertex AI models, ideally under the same response field structure as other supported providers.

Steps to Reproduce

Steps to Reproduce Send the following request:

curl $LITELLM_BASE_URL/chat/completions -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d @- << EOF
{
  "model": "vertex_ai/gemini-2.0-flash-lite-001",
  "max_tokens": 1000,
  "messages": [{"role": "user", "content": "xxx"}],
  "stream": true,
  "tools": [{
    "type": "file_search",
    "vector_store_ids": ["my-vector-store"]
  }]
}
EOF

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.14

Twitter / LinkedIn details

No response