Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: /v1/messages with image url source returns "URL sources are not supported" but /v1/chat/completions works fine [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23016•Fetched 2026-04-08 00:38:52
View on GitHub
Comments
1
Participants
2
Timeline
7
Reactions
0
Author
an9xyzan9xyz
Participants
an9xyzan9xyzJerry-XinJerry-Xin
Timeline (top)
labeled ×3cross-referenced ×2commented ×1referenced ×1

Error Message

When sending an image via POST /v1/messages using the native Anthropic message format with "type": "url" source, LiteLLM fails with a 400 error. However, the same image URL sent via POST /v1/chat/completions using OpenAI-compatible format works correctly. Error response: "error": { "message": "{"type":"error","error":{"type":"invalid_request_error","message":"messages.0.content.0.image.source.base64.data: URL sources are not supported"}}. Received Model Group=claude-sonnet-4-5\nAvailable Model Group Fallbacks=None", POST /v1/messages forwards the URL source as-is to Vertex AI Anthropic, which does not support URL sources, resulting in a 400 error.

Fix Action

Fixed

PR fix notes

PR #23026: fix(vertex_ai): convert image URLs to base64 for /v1/messages endpoint

Description (problem / solution / changelog)

Summary

Fixes #23016

Problem

The /v1/messages endpoint with Vertex AI Anthropic returns URL sources are not supported error when sending images with URL sources. However, /v1/chat/completions works correctly because PR #18497 added URL-to-base64 conversion for that path.

Root Cause

The /v1/messages endpoint uses the experimental_pass_through code path which forwards requests directly to Vertex AI without converting image URLs to base64. Since Vertex AI Anthropic doesn't support URL sources for images, this causes the error.

Solution

Add the same URL-to-base64 conversion logic to the Vertex AI pass-through transformation layer:

  1. Add _convert_image_urls_to_base64() method to VertexAIPartnerModelsAnthropicMessagesConfig
  2. Detect Anthropic native image format: {"type": "image", "source": {"type": "url", ...}}
  3. Convert to base64 format: {"type": "image", "source": {"type": "base64", "media_type": "...", "data": "..."}}
  4. Call conversion in transform_anthropic_messages_request() before forwarding

Changes

  • litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py: Add image URL to base64 conversion
  • tests/test_litellm/.../test_vertex_ai_anthropic_image_url_handling.py: Add 6 new test cases

Testing

All tests pass:

15 passed, 2 warnings in 1.10s

New test cases cover:

  • Single image URL conversion
  • Multiple images conversion
  • Preserving existing base64 images
  • Preserving cache_control attributes
  • Handling text-only messages
  • Handling string content

Changed files

  • litellm/litellm_core_utils/prompt_templates/image_handling.py (modified, +59/-20)
  • litellm/llms/base_llm/anthropic_messages/transformation.py (modified, +22/-0)
  • litellm/llms/custom_httpx/llm_http_handler.py (modified, +2/-2)
  • litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py (modified, +220/-14)
  • tests/test_litellm/litellm_core_utils/test_image_handling.py (modified, +247/-12)
  • tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_anthropic_image_url_handling.py (modified, +482/-10)

Code Example

POST /v1/messages
Content-Type: application/json

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "url",
            "url": "https://example.com/image.jpg"
          }
        },
        {
          "type": "text",
          "text": "Describe the image."
        }
      ]
    }
  ]
}

---

{
  "error": {
    "message": "{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.0.content.0.image.source.base64.data: URL sources are not supported\"}}. Received Model Group=claude-sonnet-4-5\nAvailable Model Group Fallbacks=None",
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

---

POST /v1/chat/completions
Content-Type: application/json

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg"
          }
        },
        {
          "type": "text",
          "text": "Describe the image."
        }
      ]
    }
  ]
}

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Bug Description

When sending an image via POST /v1/messages using the native Anthropic message format with "type": "url" source, LiteLLM fails with a 400 error. However, the same image URL sent via POST /v1/chat/completions using OpenAI-compatible format works correctly.

This inconsistency suggests that the URL-to-base64 conversion is only applied in the OpenAI-compatible path, but not in the /v1/messages path when proxying to Vertex AI Anthropic.

Related

  • PR #18497 added URL-to-base64 conversion for Vertex AI in the anthropic_messages_pt path (used by /v1/chat/completions).
  • The /v1/messages pass-through path appears to lack equivalent handling.

Steps to Reproduce

❌ Fails — /v1/messages with native Anthropic format:

POST /v1/messages
Content-Type: application/json

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "url",
            "url": "https://example.com/image.jpg"
          }
        },
        {
          "type": "text",
          "text": "Describe the image."
        }
      ]
    }
  ]
}

Error response:

{
  "error": {
    "message": "{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.0.content.0.image.source.base64.data: URL sources are not supported\"}}. Received Model Group=claude-sonnet-4-5\nAvailable Model Group Fallbacks=None",
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

✅ Works — /v1/chat/completions with OpenAI-compatible format:

POST /v1/chat/completions
Content-Type: application/json

{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg"
          }
        },
        {
          "type": "text",
          "text": "Describe the image."
        }
      ]
    }
  ]
}

Returns a valid response successfully. ✅

Expected Behavior

POST /v1/messages should behave consistently with POST /v1/chat/completions. LiteLLM should detect that the backend is Vertex AI Anthropic and automatically convert "type": "url" image sources to base64 before forwarding the request.

Actual Behavior

POST /v1/messages forwards the URL source as-is to Vertex AI Anthropic, which does not support URL sources, resulting in a 400 error.

Environment

  • Backend provider: Vertex AI Anthropic (claude-sonnet-4-5)
  • Endpoint: /v1/messages

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.12

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To fix the inconsistency between POST /v1/messages and POST /v1/chat/completions, we need to apply the URL-to-base64 conversion in the /v1/messages path when proxying to Vertex AI Anthropic.

Here are the steps:

  • Modify the /v1/messages endpoint to detect if the backend is Vertex AI Anthropic.
  • If it is, check for image sources with "type": "url" and convert them to base64.
  • Update the request payload with the converted base64 image sources before forwarding it to Vertex AI Anthropic.

Example code snippet in Python:

import base64
import requests

def convert_url_to_base64(url):
    response = requests.get(url)
    return base64.b64encode(response.content).decode('utf-8')

def fix_image_sources(messages):
    for message in messages:
        for content in message['content']:
            if content['type'] == 'image' and content['source']['type'] == 'url':
                url = content['source']['url']
                base64_data = convert_url_to_base64(url)
                content['source']['type'] = 'base64'
                content['source']['base64'] = base64_data
    return messages

# Example usage:
messages = [
    {
        'role': 'user',
        'content': [
            {
                'type': 'image',
                'source': {
                    'type': 'url',
                    'url': 'https://example.com/image.jpg'
                }
            }
        ]
    }
]

fixed_messages = fix_image_sources(messages)
print(fixed_messages)

Verification

To verify that the fix worked, send a POST /v1/messages request with an image URL source and check if the response is successful. You can use the same example request payload provided in the issue body.

Extra Tips

Make sure to handle any exceptions that may occur during the URL-to-base64 conversion, such as network errors or invalid image URLs. Additionally, consider adding logging to track any issues that may arise during the conversion process.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#environment variable#network issue#logging issue#authentication issue#prompt issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING