litellm - 💡(How to fix) Fix [Bug]: response_format Ignored When Calling the Image Generation Endpoint [2 comments, 1 participants]

curl -sS -X POST "https://my-litellm/images/generations" -H "Content-Type: application/json" -H "Authorization: Bearer sk-s3cr3t" -d '{ "prompt": "un chat", "response_format": "b64_json", "model": "z-image-turbo" }'

{"error":{"message":"litellm.BadRequestError: OpenAIException - Error code: 400 - {'detail': "response_format='url' requires cloud storage to be configured."} ....

Fix Plan

To fix the issue, we need to modify the LiteLLM proxy to forward the response_format parameter to the OpenAI-compatible image generation model.

Here are the steps:

• Update the LiteLLM proxy configuration to include the response_format parameter in the request to the image generation model.
• Modify the health check request to allow specifying the response_format parameter.

Example Code

# Update the LiteLLM proxy configuration
def generate_image(prompt, model, response_format):
    # Forward the response_format parameter to the image generation model
    params = {
        "prompt": prompt,
        "response_format": response_format
    }
    response = requests.post(f"{api_base}/images/generations", 
                             headers={"Authorization": f"Bearer {api_key}"},
                             json=params)
    return response

# Modify the health check request
def health_check(model, response_format):
    # Allow specifying the response_format parameter in the health check request
    params = {
        "response_format": response_format
    }
    response = requests.get(f"{api_base}/models/{model}/health", 
                            headers={"Authorization": f"Bearer {api_key}"},
                            params=params)
    return response

Verification

To verify that the fix worked, call the image generation endpoint with the response_format parameter set to "b64_json" and check that the response is in the expected format. Also, test the model's health check with the response_format parameter specified.

Extra Tips

• Make sure to update the LiteLLM proxy configuration to include the response_format parameter in the request to the image generation model.
• Verify that the response_format parameter is correctly forwarded to the image generation model by checking the request logs.
• Test the health check with different response_format values to ensure that it works as expected.