Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: misleading documentation: Trying to use Llm GuardYou must be a LiteLLM Enterprise user to use this feature. [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#27072•Fetched 2026-05-04 04:59:16
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
brokedbabrokedba
Participants
brokedbabrokedba
Timeline (top)
labeled ×3cross-referenced ×1

Error Message

File "/app/litellm/proxy/common_utils/callback_utils.py", line 180, in initialize_callbacks_on_proxy │ │ raise Exception( │ │ ...<2 lines>... │ │ )
Exception: Trying to use Llm GuardYou must be a LiteLLM Enterprise user to use this feature. If you have a license please set LITELLM_LICENSE in your env. Get a 7 day trial key here: https://w │ │ Pricing: https://www.litellm.ai/#pricing │ │ │ │ ERROR: Application startup failed. Exiting. │ │ stream closed EOF for inference/litellm-74b8899fcd-9tqd6 (litellm)

Fix Action

Fixed

PR fix notes

PR #69: docs(enterprise): clarify which legacy guardrail callbacks are Enterprise-only

Description (problem / solution / changelog)

Closes BerriAI/litellm#27072

What

The OSS-vs-Enterprise comparison row for Guardrails in docs/enterprise.md previously read:

| Guardrails | Always-on / request-based | Key and team scoped guardrails |

This is misleading: while the guardrail framework itself is OSS, several of the built-in callback integrations advertised in the older guardrails docs (old_guardrails.md) actually require an Enterprise license — users who try to enable them on the OSS build are greeted with You must be a LiteLLM Enterprise user to use this feature.

Source-of-truth check

Looked at litellm/proxy/common_utils/callback_utils.py (callback name → implementation switch). The callbacks that import from litellm_enterprise.* and raise CommonProxyErrors.missing_enterprise_package are:

  • llmguard_moderations
  • llamaguard_moderations
  • hide_secrets
  • openai_moderations
  • google_text_moderation
  • lakera_prompt_injection
  • aporia_prompt_injection

presidio (PII masking) and detect_prompt_injection are OSS.

Change

Adds a footnote to the OSS column of the Guardrails row that lists the callback names that require Enterprise. Keeps the table layout intact and does not move any other docs.

Why this matters

Per the bug, users implement features against the comparison table believing them to be OSS, then only discover the licensing requirement at deploy time. The footnote turns the failure mode into a docs lookup.

🤖 Generated with Claude Code

Changed files

  • docs/enterprise.md (modified, +4/-1)

Code Example

# templatefile — injects api_base + vllm_api_key
model_list:
  - model_name: tinyllama
    litellm_params:
      model: openai//data/models/tinyllama
      api_base: "${internal_api_base}"
      api_key: "internal"

general_settings:
  custom_auth: custom_auth.user_api_key_auth
  always_include_stream_usage: true

litellm_settings:
  callbacks:
    - custom_callbacks.proxy_handler_instance    
    - llmguard_moderations

---

File "/app/litellm/proxy/common_utils/callback_utils.py", line 180, in initialize_callbacks_on_proxy                                                                                             │
│     raise Exception(                                                                                                                                                                               │
│     ...<2 lines>...                                                                                                                                                                                │
│     )  
Exception: Trying to use Llm GuardYou must be a LiteLLM Enterprise user to use this feature. 
If you have a license please set `LITELLM_LICENSE` in your env. Get a 7 day trial key here: https://w │
│ Pricing: https://www.litellm.ai/#pricing                                                                                                                                                           │
│                                                                                                                                                                                                    │
│ ERROR:    Application startup failed. Exiting.                                                                                                                                                     │
│ stream closed EOF for inference/litellm-74b8899fcd-9tqd6 (litellm)

---

elif isinstance(callback, str) and callback == "llmguard_moderations":  
    try:  
        from litellm_enterprise.enterprise_callbacks.llm_guard import (  
            _ENTERPRISE_LLMGuard,  
        )  
    except ImportError:  
        raise Exception(  
            "Trying to use Llm Guard"  
            + CommonProxyErrors.missing_enterprise_package.value  
        )  
  
    if premium_user is not True:  
        raise Exception(  
            "Trying to use Llm Guard"  
            + CommonProxyErrors.not_premium_user.value  
        )
RAW_BUFFERClick to expand / collapse

What happened?

A bug happened!

Documentation incorrectly claims llmguard_moderations is open source.

The LiteLLM documentation states that "Guardrails: Always-on / request-based" is an open source feature, but llmguard_moderations specifically requires an enterprise license. see link

<img width="2704" height="1262" alt="Image" src="https://github.com/user-attachments/assets/8dcaec3b-1ecb-4fbc-82b1-0c21a61f87d8" />

Expected behavior

Documentation should clearly distinguish between:

  • Open source guardrail framework (OSS)
  • Specific guardrail implementations (some enterprise-only)

Steps to Reproduce

  1. Read the Enterprise comparison table showing "Guardrails: Always-on / request-based" as OSS
  2. Try to configure llmguard_moderations in OSS version
 # templatefile — injects api_base + vllm_api_key
model_list:
  - model_name: tinyllama
    litellm_params:
      model: openai//data/models/tinyllama
      api_base: "${internal_api_base}"
      api_key: "internal"

general_settings:
  custom_auth: custom_auth.user_api_key_auth
  always_include_stream_usage: true

litellm_settings:
  callbacks:
    - custom_callbacks.proxy_handler_instance    
    - llmguard_moderations
  1. Get error: "You must be a LiteLLM Enterprise user to use this feature"

Impact

Users waste time implementing features they believe are open source, only to discover they require enterprise licensing during deployment.

Suggested fix

Make up you mind. Update the documentation to clarify either of the below :

  • The guardrail framework is OSS
  • Specific implementations like llmguard_moderations are enterprise-only
  • List which guardrails are available in OSS vs Enterprise

Component

Docs

Relevant log output

File "/app/litellm/proxy/common_utils/callback_utils.py", line 180, in initialize_callbacks_on_proxy                                                                                             │
│     raise Exception(                                                                                                                                                                               │
│     ...<2 lines>...                                                                                                                                                                                │
│     )  
Exception: Trying to use Llm GuardYou must be a LiteLLM Enterprise user to use this feature. 
If you have a license please set `LITELLM_LICENSE` in your env. Get a 7 day trial key here: https://w │
│ Pricing: https://www.litellm.ai/#pricing                                                                                                                                                           │
│                                                                                                                                                                                                    │
│ ERROR:    Application startup failed. Exiting.                                                                                                                                                     │
│ stream closed EOF for inference/litellm-74b8899fcd-9tqd6 (litellm)

Code evidence

The llmguard_moderations callback explicitly checks for enterprise license : https://github.com/BerriAI/litellm/blob/62920a0c/litellm/proxy/common_utils/callback_utils.py#L64-L88

elif isinstance(callback, str) and callback == "llmguard_moderations":  
    try:  
        from litellm_enterprise.enterprise_callbacks.llm_guard import (  
            _ENTERPRISE_LLMGuard,  
        )  
    except ImportError:  
        raise Exception(  
            "Trying to use Llm Guard"  
            + CommonProxyErrors.missing_enterprise_package.value  
        )  
  
    if premium_user is not True:  
        raise Exception(  
            "Trying to use Llm Guard"  
            + CommonProxyErrors.not_premium_user.value  
        )

What part of LiteLLM is this about?

Docs

What LiteLLM version are you on ?

v1.83.3

Twitter / LinkedIn details

@clouddude_

extent analysis

TL;DR

Update the LiteLLM documentation to clearly distinguish between open source and enterprise-only features, specifically marking llmguard_moderations as an enterprise-only implementation.

Guidance

  • Review the documentation for accuracy, ensuring that features are correctly labeled as open source or enterprise-only.
  • Verify that the documentation update includes a clear list of available guardrails for both OSS and Enterprise versions.
  • Check the code evidence in callback_utils.py to understand how the enterprise license check is implemented for llmguard_moderations.
  • Consider adding a note to the documentation about the specific error message users may encounter when trying to use an enterprise-only feature in the OSS version.

Example

No code changes are necessary for this issue, but the documentation update could include a table or list similar to:

### Guardrail Availability

| Guardrail | OSS | Enterprise |
| --- | --- | --- |
| llmguard_moderations | | |

Notes

This fix only addresses the documentation issue and does not change the underlying functionality of llmguard_moderations. Users will still need an enterprise license to use this feature.

Recommendation

Apply workaround: Update the documentation to reflect the correct availability of features, as this will help prevent user confusion and save time in the long run.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Documentation should clearly distinguish between:

  • Open source guardrail framework (OSS)
  • Specific guardrail implementations (some enterprise-only)
#api#ssr#retriever error#indexing error#inference speed

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: misleading documentation: Trying to use Llm GuardYou must be a LiteLLM Enterprise user to use this feature. [1 pull requests, 1 participants]