litellm - 💡(How to fix) Fix [Bug]: Gemini(Vertex AI) context caching error [1 comments, 1 participants]

When passing both cache_control and tool_choice together in a Gemini request, the following error is returned: error

TL;DR

Remove the tool_choice parameter from the Gemini request when using cached content to resolve the error.

Guidance

• Verify that the tool_choice parameter is not necessary when using cached content, as Gemini does not allow it.
• Check the LiteLLM Python package documentation to see if there are any specific guidelines for handling cached content and tool configuration.
• Consider modifying the request to exclude tool_choice when cached content is being used, to prevent the error from occurring.
• Review the cached content creation request to ensure that tool_choice (or toolConfig in Gemini API) is included, so it is not passed in the subsequent completion request.

Example

No code snippet is provided as the issue does not contain sufficient information about the code.

Notes

The solution assumes that the tool_choice parameter is not required when using cached content. If this parameter is necessary, further investigation into the Gemini API and LiteLLM Python package may be required.

Recommendation

Apply workaround: Remove the tool_choice parameter from the Gemini request when using cached content, as this is the most straightforward way to resolve the error based on the provided information.