litellm - 💡(How to fix) Fix [Bug]: Health check fails for vLLM classify models — always reports unhealthy [1 comments, 2 participants]

Description

When a vLLM classification model is registered in model_list using the hosted_vllm/ prefix, the LiteLLM health check always reports it as unhealthy, even though the model works correctly in production via the /vllm/classify endpoint.

The health check sends a request to /chat/completions but classification models only expose a /classify endpoint — causing a 404 Not Found error that incorrectly marks the deployment as unhealthy.

LiteLLM Version

v1.82.3-stable

Config

model_list:
  - model_name: model/xlm-roberta-base-mention-classifier
    litellm_params:
      model: hosted_vllm/model/xlm-roberta-base-mention-classifier
      api_base: http://localhost:8095

Steps to Reproduce

1. Register a vLLM classification model in model_list (e.g. XLM-RoBERTa)
2. Call GET /health — the model appears in unhealthy_endpoints
3. Call POST /vllm/classify with the same model — it works correctly

Expected Behavior

Health check should detect that the model's mode is classify and use the /classify endpoint instead of /chat/completions, OR skip the health check for classify models entirely.

Actual Behavior

Health check always calls /chat/completions, gets a 404 Not Found from vLLM, and marks the model as unhealthy:

litellm.NotFoundError: Hosted_vllmException - {"detail":"Not Found"}
httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://localhost:8095/chat/completions'

Workaround

Setting disable_health_check: true in general_settings does not resolve the issue in v1.82.3.

The model works correctly in production — this is a false negative in the health check only.

Related Issues

• #11205 — Feature request for vLLM classify endpoint support (implemented in production via /vllm/classify, but health check not updated accordingly)