Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

In a GKE Autopilot production deployment (16 replicas), only 1 pod remained available. The root cause is that convert_url_to_base64() uses a synchronous httpx client (litellm.module_level_client) to fetch image/file URLs. When the request contains an unreachable internal URL (e.g., http://10.254.3.71/uploads/...), the TCP connect hangs for ~2 minutes, blocking the entire asyncio event loop. This prevents FastAPI from responding to health checks, nginx records 499, kubelet marks the pod as unhealthy, and restarts it — creating a cascading failure across all replicas.

This is the same class of bug as #20268 (sync next() blocking event loop), but occurring in the image/file URL handling path.

Root Cause

File: litellm/litellm_core_utils/prompt_templates/image_handling.py:98-121

convert_url_to_base64() uses litellm.module_level_client (sync httpx) with no explicit connect timeout. It also retries 3 times on failure, meaning a single unreachable URL can block the event loop for ~6 minutes (3 retries × ~2 min TCP timeout).

The sync version is called from multiple async code paths:

While async variants (async_convert_url_to_base64) exist and are used in some paths (e.g., gpt_transformation.py:247, vertex_ai/ocr/transformation.py:157), many code paths still use the sync version directly within async request handlers.

Steps to Reproduce

1. Deploy LiteLLM proxy with multiple replicas behind a load balancer with health checks
2. Send a chat completion request containing an image URL pointing to an unreachable internal IP (e.g., http://10.x.x.x/image.png) to a provider that triggers convert_url_to_base64 (Anthropic on Vertex/Bedrock, Gemini, Ollama, etc.)
3. Observe:
   • The request thread blocks for ~2 minutes per retry (up to ~6 min total)
   • During this time, the pod cannot respond to health checks
   • Health check failures trigger pod restarts
   • Under sustained traffic with such URLs, most/all pods become unhealthy

Suggested Fixes

1. Add connect timeout (quick fix / stop the bleeding)

Set a short connect_timeout (e.g., 5s) on module_level_client in _lazy_imports.py:433:

sync_client = HTTPHandler(timeout=httpx.Timeout(timeout, connect=5.0))

2. Replace sync calls with async versions (proper fix)

Replace convert_url_to_base64() with async_convert_url_to_base64() in all async code paths. For call sites where the function signature is sync, use asyncio.to_thread() as an interim measure:

base64_data = await asyncio.to_thread(convert_url_to_base64, url)

3. Reject private/unreachable URLs at the entry point (defense in depth)

Add an optional check in convert_url_to_base64() to reject RFC 1918 private IPs (10.x.x.x, 172.16-31.x.x, 192.168.x.x) or make it configurable.

Related Issues

• #20268 — Same class of bug: sync next() blocking event loop in streaming handler
• #24193 — ImageFetchError masked by misleading APIConnectionError in the same code path
• #19921 — Performance regression in 1.81.x (possibly related to event loop blocking)

Relevant log output

# nginx access log showing 499 (client closed connection = health check timeout)
10.x.x.x - - [30/Mar/2026:xx:xx:xx +0000] "GET /health HTTP/1.1" 499 0

# kubelet events
Warning  Unhealthy  pod/litellm-xxx  Liveness probe failed: HTTP probe failed with statuscode: 499
Warning  Unhealthy  pod/litellm-xxx  Readiness probe failed: HTTP probe failed with statuscode: 499
Normal   Killing    pod/litellm-xxx  Container litellm failed liveness probe, will be restarted

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.65.3+

Twitter / LinkedIn details

No response