Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix [Bug]: litellm.BadRequestError when using llm_as_a_judge guardrail with self-hosted vllm model

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

INFO: 127.0.0.1:53786 - "POST /chat/completions HTTP/1.1" 200 OK 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 19:56:35 - LiteLLM:WARNING: init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers

Code Example

INFO:     127.0.0.1:53786 - "POST /chat/completions HTTP/1.1" 200 OK
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

I am running a self-hosted model using vLLM (google/gemma-4-31B-it). I have LiteLLM installed via helm chart in Kubernetes with the model added using the vllm provider. I have also tried the OpenAI compatible endpoint provider. In both cases, the model itself works fine in LiteLLM. I can query it programmatically and from the playground.

However, when I try to add a LLM as a Judge guardrail backed by the model, the guardrail execution fails with init.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it.

Steps to Reproduce

  1. Install v1.83.14
  2. Deploy self-hosted model using vllm
  3. Models + Endpoints -> LLM Credentials -> Add Credential (use provider vllm)
  4. Models + Endpoints -> Add Model (provider vllm, add name of model, choose existing creds from previous step)
  5. Guardrails -> Add New Guardrail -> Add Provider Guardrail -> Guardrail Provider LiteLLM LLM as a Judge (select judge model from previous step)
  6. Playground -> select model and guardrails and enter example query
  7. Query succeeds but observe guardrail failure in logs

Relevant log output

INFO:     127.0.0.1:53786 - "POST /chat/completions HTTP/1.1" 200 OK
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
19:56:35 - LiteLLM:WARNING: __init__.py:177 - llm_as_a_judge guardrail: judge call failed, failing open. Error: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=google/gemma-4-31B-it
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.83.14

Twitter / LinkedIn details

No response

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#dependency conflict#environment setup#docker error#permission error#memory optimization

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING