Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix [Bug]: Unable to use Gemini Files on responses API [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24776•Fetched 2026-04-08 01:54:07
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
rodricirurodriciru
Participants
rodricirurodriciru
Timeline (top)
labeled ×3closed ×1renamed ×1

Error Message

09:12:38 - LiteLLM Proxy:DEBUG: common_request_processing.py:1508 - An error occurred: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history _part = _process_gemini_media( image_url=passed_file, ...<3 lines>... video_metadata=video_metadata, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media image = convert_to_anthropic_image_obj(image_url, format=format) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj openai_image_url = convert_url_to_base64(url=openai_image_url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64 return _process_image_response(response, url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response raise litellm.ImageFetchError( f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}" ) litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion response = await init_response ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion request_body = await async_transform_request_body( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<4 lines>... ) # type: ignore ^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body return _transform_request_body( messages=messages, ...<4 lines>... optional_params=optional_params, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body content = litellm.GoogleAIStudioGeminiConfig()._transform_messages( messages=messages, model=model ) File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages return _gemini_convert_messages_with_history(messages=messages, model=model) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history raise Exception( ...<3 lines>... ) Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5586, in async_function_with_retries response = await self.make_call(original_function, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5754, in make_call response = await response ^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3891, in _ageneric_api_call_with_fallbacks_helper raise e File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3877, in _ageneric_api_call_with_fallbacks_helper response = await response # type: ignore ^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async raise e File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async result = await original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 580, in aresponses raise litellm.exception_type( ...<5 lines>... ) File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 558, in aresponses response = await init_response ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/responses/litellm_completion_transformation/handler.py", line 114, in async_response_api_handler ] = await litellm.acompletion( ^^^^^^^^^^^^^^^^^^^^^^^^^^ **acompletion_args, ^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async raise e File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async result = await original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/main.py", line 639, in acompletion raise exception_type( ~~~~~~~~~~~~~~^ model=model, ^^^^^^^^^^^^ ...<3 lines>... extra_kwargs=kwargs, ^^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type raise e File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2432, in exception_type raise APIConnectionError( ...<8 lines>... ) INFO: 172.28.0.1:51004 - "POST /responses HTTP/1.1" 500 Internal Server Error litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history _part = _process_gemini_media( image_url=passed_file, ...<3 lines>... video_metadata=video_metadata, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media image = convert_to_anthropic_image_obj(image_url, format=format) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj openai_image_url = convert_url_to_base64(url=openai_image_url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64 return _process_image_response(response, url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response raise litellm.ImageFetchError( f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}" ) litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion response = await init_response ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion request_body = await async_transform_request_body( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<4 lines>... ) # type: ignore ^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body return _transform_request_body( messages=messages, ...<4 lines>... optional_params=optional_params, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body content = litellm.GoogleAIStudioGeminiConfig()._transform_messages( messages=messages, model=model ) File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages return _gemini_convert_messages_with_history(messages=messages, model=model) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history raise Exception( ...<3 lines>... ) Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history _part = _process_gemini_media( image_url=passed_file, ...<3 lines>... video_metadata=video_metadata, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media image = convert_to_anthropic_image_obj(image_url, format=format) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj openai_image_url = convert_url_to_base64(url=openai_image_url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64 return _process_image_response(response, url) File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response raise litellm.ImageFetchError( f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}" ) litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion response = await init_response ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion request_body = await async_transform_request_body( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<4 lines>... ) # type: ignore ^ File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body return _transform_request_body( messages=messages, ...<4 lines>... optional_params=optional_params, ) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body content = litellm.GoogleAIStudioGeminiConfig()._transform_messages( messages=messages, model=model ) File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages return _gemini_convert_messages_with_history(messages=messages, model=model) File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history raise e File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history raise Exception( ...<3 lines>... ) Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format . Received Model Group=gemini/gemini-2.5-flash-Promia Available Model Group Fallbacks=None LiteLLM Retried: 2 times, LiteLLM Max Retries: 2 None

Code Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",
    api_key="sk-1234"
)

file = client.files.create(
    file=open("det-1.jpg", "rb"),
    purpose="user_data",
    extra_body={
        "target_model_names": "gemini/gemini-2.5-flash" 
    }
)

print(f"file: {file}")

assert file is not None

response = client.responses.create(
    model="gemini/gemini-2.5-flash",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_file",
                    "file_id": file.id
                },
                {
                    "type": "input_text",
                    "text": "What is the file?",
                },
            ]
        }
    ]
)

print(response)

---

09:12:38 - LiteLLM Proxy:DEBUG: common_request_processing.py:1508 - An error occurred: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3891, in _ageneric_api_call_with_fallbacks_helper
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3877, in _ageneric_api_call_with_fallbacks_helper
    response = await response  # type: ignore
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 580, in aresponses
    raise litellm.exception_type(
    ...<5 lines>...
    )
  File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 558, in aresponses
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/responses/litellm_completion_transformation/handler.py", line 114, in async_response_api_handler
    ] = await litellm.acompletion(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        **acompletion_args,
        ^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 639, in acompletion
    raise exception_type(
          ~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<3 lines>...
        extra_kwargs=kwargs,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2432, in exception_type
    raise APIConnectionError(
    ...<8 lines>...
    )
INFO:     172.28.0.1:51004 - "POST /responses HTTP/1.1" 500 Internal Server Error
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
. Received Model Group=gemini/gemini-2.5-flash-Promia
Available Model Group Fallbacks=None LiteLLM Retried: 2 times, LiteLLM Max Retries: 2 None
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",
    api_key="sk-1234"
)

file = client.files.create(
    file=open("det-1.jpg", "rb"),
    purpose="user_data",
    extra_body={
        "target_model_names": "gemini/gemini-2.5-flash" 
    }
)

print(f"file: {file}")

assert file is not None

response = client.responses.create(
    model="gemini/gemini-2.5-flash",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_file",
                    "file_id": file.id
                },
                {
                    "type": "input_text",
                    "text": "What is the file?",
                },
            ]
        }
    ]
)

print(response)

Error from Docker image:

09:12:38 - LiteLLM Proxy:DEBUG: common_request_processing.py:1508 - An error occurred: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3891, in _ageneric_api_call_with_fallbacks_helper
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/router.py", line 3877, in _ageneric_api_call_with_fallbacks_helper
    response = await response  # type: ignore
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 580, in aresponses
    raise litellm.exception_type(
    ...<5 lines>...
    )
  File "/usr/lib/python3.13/site-packages/litellm/responses/main.py", line 558, in aresponses
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/responses/litellm_completion_transformation/handler.py", line 114, in async_response_api_handler
    ] = await litellm.acompletion(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        **acompletion_args,
        ^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 2094, in wrapper_async
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/utils.py", line 1893, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 639, in acompletion
    raise exception_type(
          ~~~~~~~~~~~~~~^
        model=model,
        ^^^^^^^^^^^^
    ...<3 lines>...
        extra_kwargs=kwargs,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2432, in exception_type
    raise APIConnectionError(
    ...<8 lines>...
    )
INFO:     172.28.0.1:51004 - "POST /responses HTTP/1.1" 500 Internal Server Error
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 394, in _gemini_convert_messages_with_history
    _part = _process_gemini_media(
        image_url=passed_file,
    ...<3 lines>...
        video_metadata=video_metadata,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 230, in _process_gemini_media
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 222, in _process_gemini_media
    image = convert_to_anthropic_image_obj(image_url, format=format)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 892, in convert_to_anthropic_image_obj
    openai_image_url = convert_url_to_base64(url=openai_image_url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 113, in convert_url_to_base64
    return _process_image_response(response, url)
  File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/image_handling.py", line 21, in _process_image_response
    raise litellm.ImageFetchError(
        f"Error: Unable to fetch image from URL. Status code: {response.status_code}, url={url}"
    )
litellm.exceptions.ImageFetchError: litellm.BadRequestError: Error: Unable to fetch image from URL. Status code: 403, url=https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/litellm/main.py", line 620, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2738, in async_completion
    request_body = await async_transform_request_body(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )  # type: ignore
    ^
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 868, in async_transform_request_body
    return _transform_request_body(
        messages=messages,
    ...<4 lines>...
        optional_params=optional_params,
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 771, in _transform_request_body
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 695, in _transform_request_body
    content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
        messages=messages, model=model
    )
  File "/usr/lib/python3.13/site-packages/litellm/llms/gemini/chat/transformation.py", line 153, in _transform_messages
    return _gemini_convert_messages_with_history(messages=messages, model=model)
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 628, in _gemini_convert_messages_with_history
    raise e
  File "/usr/lib/python3.13/site-packages/litellm/llms/vertex_ai/gemini/transformation.py", line 403, in _gemini_convert_messages_with_history
    raise Exception(
    ...<3 lines>...
    )
Exception: Unable to determine mime type for file_id: https://generativelanguage.googleapis.com/v1beta/files/bc3uc0goe6xe, set this explicitly using message[0].content[0].file.format
. Received Model Group=gemini/gemini-2.5-flash-Promia
Available Model Group Fallbacks=None LiteLLM Retried: 2 times, LiteLLM Max Retries: 2 None

Trying to use a just uploaded file on Gemini by responses API, breaks as it is unable to fetch image.... Same ID on completions works perfectly

The issue its it tries to get this image without the API key, so Gemini servers reject it.

When using completions API it enters in this else: https://github.com/BerriAI/litellm/blob/58120537af4f3975550206a88bdf133e31832de8/litellm/llms/vertex_ai/gemini/transformation.py#L211

but when using responses API enters in that other: https://github.com/BerriAI/litellm/blob/58120537af4f3975550206a88bdf133e31832de8/litellm/llms/vertex_ai/gemini/transformation.py#L221

[!NOTE]
For finding the true error I use https://github.com/BerriAI/litellm/issues/24193, but the log output pasted its whitout it

Steps to Reproduce

  1. Copy the code
  2. Execute

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

1.82.6

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To fix the issue of being unable to fetch an image without an API key when using the responses API, you need to explicitly set the MIME type and ensure the API key is used for authentication.

Here are the steps to follow:

  • Set the format parameter when creating the input file content:

response = client.responses.create( model="gemini/gemini-2.5-flash", input=[ { "role": "user", "content": [ { "type": "input_file", "file_id": file.id, "format": "image/jpeg" # Set the format explicitly }, { "type": "input_text", "text": "What is the file?", }, ] } ] )

*   Ensure the API key is used for authentication when fetching the image. This might involve modifying the `transformation.py` file in the LiteLLM library to handle the responses API in the same way as the completions API.

### Verification
To verify that the fix worked, you can try the following:

*   Run the modified code and check if the image is fetched successfully.
*   Verify that the API key is being used for authentication when fetching the image.
*   Test the code with different image formats to ensure the `format` parameter is being set correctly.

### Extra Tips
*   Make sure to handle any exceptions that may occur when fetching the image, such as network errors or invalid API keys.
*   Consider adding logging statements to track the API requests and responses, which can help with debugging any issues that may arise.
*   If you're using a library like LiteLLM, it's a good idea to check the library's documentation and issue tracker for any known issues or workarounds related to authentication and file handling.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ISR setup#authentication setup#request error#file not found

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING