File ".../litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py", line 484, in async_check_and_create_cache token, url = self._get_token_and_url_context_caching( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py", line 84, in _get_token_and_url_context_caching return self._check_custom_proxy( ^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../litellm/llms/vertex_ai/vertex_llm_base.py", line 393, in _check_custom_proxy raise ValueError( ValueError: Model parameter is required for Gemini custom API base URLs

Fix Plan

To resolve the ValueError caused by the missing model parameter when using Gemini models with Context Caching enabled and a custom api_base, you need to modify the _get_token_and_url_context_caching method in vertex_ai_context_caching.py to pass the model name to the _check_custom_proxy method.

Here are the steps:

• Open litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py
• Modify the _get_token_and_url_context_caching method to pass the model name:

def _get_token_and_url_context_caching(self, ...):
    # ...
    model = self.model  # Assuming self.model has the model name
    return self._check_custom_proxy(model, ...)

• Alternatively, if self.model is not available, you can pass the model name as an argument to the _get_token_and_url_context_caching method.

Verification

To verify that the fix worked:

• Trigger a request that involves context caching with a Gemini model and a custom api_base.
• Check the logs for any ValueError exceptions related to the missing model parameter.
• If the fix is successful, the request should complete without raising a ValueError.

Extra Tips

• Make sure to update the litellm package to the latest version if available, as this issue might be fixed in a later version.
• If you are using a custom implementation of the _check_custom_proxy method, ensure that it handles the model parameter correctly.