litellm - 💡(How to fix) Fix Bug: vLLM pass-through endpoint ignores api_key from DB-stored model deployments (401 error) [4 comments, 2 participants]

What happened?

When a vLLM model is configured via the database (e.g., through the UI or management API) with a per-deployment api_key, the vLLM pass-through endpoint (/vllm/{endpoint}) does not send the API key to the upstream vLLM server, resulting in a 401 Unauthorized error.

Models configured in YAML with api_key in litellm_params are also affected by the same code path.

Steps to reproduce

1. Add a vLLM model via DB/UI with an api_key:

   model_name: my-vllm-model
   litellm_params:
     model: hosted_vllm/my-model
     api_base: https://my-vllm-server.com
     api_key: sk-my-vllm-key

2. Send a pass-through request:

   curl -X POST 'http://localhost:4000/vllm/v1/chat/completions' \
     -H 'Authorization: Bearer <litellm-virtual-key>' \
     -H 'Content-Type: application/json' \
     -d '{"model": "my-vllm-model", "messages": [{"role": "user", "content": "hello"}]}'

3. The upstream vLLM server returns 401 because no x-api-key header is sent.

Root cause

In litellm/llms/vllm/common_utils.py, VLLMModelInfo.get_api_key() unconditionally returns None, discarding the api_key passed in from the router deployment:

# litellm/llms/vllm/common_utils.py:56-57
@staticmethod
def get_api_key(api_key: Optional[str] = None) -> Optional[str]:
    return None  # <-- always returns None, ignoring the input

This is called in the pass-through flow at litellm/passthrough/main.py:259:

provider_api_key = provider_config.get_api_key(api_key)  # returns None

auth_headers = provider_config.validate_environment(
    ...,
    api_key=provider_api_key,  # None → no x-api-key header set
)

Even though the Router correctly extracts api_key from the deployment's litellm_params and passes it through allm_passthrough_route → llm_passthrough_route, get_api_key() throws it away.

Expected behavior

The api_key from the deployment configuration should be forwarded to the upstream vLLM server as the x-api-key header.

Suggested fix

# litellm/llms/vllm/common_utils.py
@staticmethod
def get_api_key(api_key: Optional[str] = None) -> Optional[str]:
    return api_key or get_secret_str("VLLM_API_KEY")

This preserves backward compatibility (falls back to env var) while correctly using per-deployment keys.

LiteLLM version

Latest main branch (commit e37efc4218)

Relevant code path

1. litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py:328 — vllm_proxy_route calls router
2. litellm/router.py:3716-3730 — Router spreads deployment["litellm_params"] (including api_key) into kwargs
3. litellm/passthrough/main.py:259 — calls provider_config.get_api_key(api_key)
4. litellm/llms/vllm/common_utils.py:56-57 — get_api_key() returns None ← bug is here
5. litellm/llms/vllm/common_utils.py:42-43 — validate_environment() skips setting header since api_key is None