litellm - 💡(How to fix) Fix [Bug]: Unable to add an image file_id on Gemini responses API [1 comments, 2 participants]

TL;DR

Modify the LiteLLM completion transformation to handle file_id for input_image type by creating a file instead of an object when file_id is present.

Guidance

• Check if the file_id is being passed correctly when uploading an image to Gemini and trying to access it through the LiteLLM API.
• Verify that the modification to the transformation.py file, as suggested in the issue, resolves the Invalid image received error.
• Consider testing this fix with different models and image uploads to ensure it works consistently.
• Review the LiteLLM documentation and code to understand how input_image and file_id are handled, and if there are any other potential issues or edge cases.

Example

elif item.get("type") == "input_image":
    if item.get("file_id"):
        content_list.append(
            LiteLLMCompletionResponsesConfig._transform_input_file_item_to_file_item(
                item
            )
        )
    else:
        content_list.append(
            dict(
                LiteLLMCompletionResponsesConfig._transform_input_image_item_to_image_item(
                    item
                )
            )
        )

Notes

This fix assumes that the issue is specific to the handling of file_id for input_image type in the LiteLLM completion transformation. However, without further testing and review, it's uncertain if this fix will work for all cases or introduce other issues.

Recommendation

Apply the workaround by modifying the transformation.py file as suggested, as it has been reported to work for the user's specific use case with OpenAI and Gemini models.