Summary When calling litellm.completion(..., stream=True) against a Vertex AI / Google AI Studio Gemini model, the per-request timeout argument is silently discarded. Streams that stall before the first token hang for the full litellm.request_timeout (default 6000s = 100 minutes) instead of failing at the configured timeout. The non-streaming branch of the same handler propagates timeout correctly — only the streaming branch is broken.

Version litellm == 1.83.0 (also reproduces on main as of filing) Python 3.10 Provider: vertex_ai and gemini (both routes through the same handler) Repro import litellm import time

Set a small request_timeout so the bug doesn't take 100 minutes to surface.

In practice, the bug means the per-call timeout is ignored regardless.

litellm.request_timeout = 30

start = time.monotonic() try: response = litellm.completion( model="vertex_ai/gemini-2.5-flash", messages=[{"role": "user", "content": "hello"}], stream=True, timeout=2, # expect to fail in ~2s on a stalled stream ) for chunk in response: pass except Exception as e: print(f"failed after {time.monotonic() - start:.1f}s with: {e!s}") When the upstream stream stalls (we observe this regularly under load against vertex_ai/gemini-2.5-flash), the call hangs ~30s (the litellm.request_timeout), not ~2s (the per-call timeout). The exception text is also misleading:

litellm.Timeout: Connection timed out after None seconds. — note the literal None, despite a numeric timeout=2 having been passed.

Root cause In litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py, VertexLLM.completion() has two branches. The non-streaming branch correctly bakes the request timeout into the httpx client:

non-streaming branch (~line 2971)

if client is None or isinstance(client, AsyncHTTPHandler): _params = {} if timeout is not None: if isinstance(timeout, float) or isinstance(timeout, int): timeout = httpx.Timeout(timeout) _params["timeout"] = timeout client = _get_httpx_client(params=_params) The streaming branch (~line 2944) constructs the make_sync_call partial without any timeout argument:

SYNC STREAMING CALL

if stream is True: request_data_str = json.dumps(data) streaming_response = CustomStreamWrapper( completion_stream=None, make_call=partial( make_sync_call, gemini_client=( client if client is not None and isinstance(client, HTTPHandler) else None ), api_base=url, data=request_data_str, model=model, messages=messages, logging_obj=logging_obj, headers=headers, ), ... ) return streaming_response make_sync_call itself does not accept a timeout parameter, and its client.post(...) invocation does not pass one:

def make_sync_call( client, gemini_client, api_base, headers, data, model, messages, logging_obj, ): if gemini_client is not None: client = gemini_client if client is None: client = HTTPHandler() # uses _DEFAULT_TIMEOUT = httpx.Timeout(5.0, connect=5.0) response = client.post( api_base, headers=headers, data=data, stream=True, logging_obj=logging_obj, ) Net result: the user-supplied timeout is dropped on the floor. Whatever httpx.Client instance is reused/created by make_sync_call is what governs the streaming read timeout.

Why "Connection timed out after None seconds" In litellm/llms/custom_httpx/http_handler.py:1031, the sync HTTPHandler.post() reports timeouts using its local timeout parameter:

except httpx.TimeoutException: raise litellm.Timeout( message=f"Connection timed out after {timeout} seconds.", model="default-model-name", llm_provider="litellm-httpx-handler", ) Because make_sync_call calls client.post(...) without a timeout argument, the local timeout is None here even though the actual httpx.Client instance has a real timeout configured. The exception message thus reports None. (The async sibling at line 488 was patched to log Timeout passed={timeout}, time taken={time_delta} — the sync handler was missed.)

Expected behavior The streaming branch should mirror the non-streaming branch: when timeout is provided to completion(), propagate it down to make_sync_call so that client.post(..., timeout=timeout) enforces the user's configured timeout.

Suggested fix Two small changes:

Make make_sync_call accept and forward timeout: def make_sync_call( client, gemini_client, api_base, headers, data, model, messages, logging_obj, timeout: Optional[Union[float, httpx.Timeout]] = None, # NEW ): if gemini_client is not None: client = gemini_client if client is None: client = HTTPHandler(timeout=timeout) # NEW: bake into fresh client

response = client.post(
    api_base,
    headers=headers,
    data=data,
    stream=True,
    timeout=timeout,                                     # NEW
    logging_obj=logging_obj,
)
...

Pass timeout through the streaming partial(...) in VertexLLM.completion(): make_call=partial( make_sync_call, gemini_client=( client if client is not None and isinstance(client, HTTPHandler) else None ), api_base=url, data=request_data_str, model=model, messages=messages, logging_obj=logging_obj, headers=headers, timeout=timeout, # NEW ), The async streaming sibling (async_streaming at ~line 2575) already receives timeout via its signature; only the sync streaming path is broken.

Impact For any sync streaming Gemini call where the upstream stalls before the first chunk:

The user's timeout is ignored. The call hangs for the full litellm.request_timeout (default 6000s). The error message reports Connection timed out after None seconds, hiding the actual configured timeout. In production we observed individual calls hanging 6,008–6,017 seconds — exact matches for litellm.request_timeout=6000 plus connect overhead.

Workaround Until this is fixed, users of streaming Gemini must rely on application-level timeout managers (e.g., a thread that raises TimeoutError into the main thread) or explicitly construct a short-timeout HTTPHandler and pass it as the client kwarg.