Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix [Bug]: "Failure metrics missing/incorrect labels for dynamic rate limits" [1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23772•Fetched 2026-04-08 00:49:11
View on GitHub
Comments
1
Participants
2
Timeline
5
Reactions
0
Author
shivamrawat1shivamrawat1
Participants
FocusMode-coderFocusMode-codershivamrawat1shivamrawat1
Timeline (top)
labeled ×3commented ×1referenced ×1

Root Cause

We are getting metrics , but we are not able to properly display because of metric labels.

Code Example

- model_name: aws/anthropic/claude-opus-4-5
          litellm_params:
            model: bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0
            aws_region_name: us-west-2
            tpm: 17000000
            rpm: 4000
          model_info:
            id: aws/anthropic/claude-opus-4-5
            mode: completion
            access_groups:
            - default-models

---

curl -X POST http://localhost:4000/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-repro" \
  -d '{
    "model": "aws/anthropic/claude-opus-4-5",
    "messages": [{"role": "user", "content": "hi"}]
  }'
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

We are getting metrics , but we are not able to properly display because of metric labels.

requested_model_name is empty litellm_model_name has value which is actually requested_model_name litellm_model_name should have been global.anthropic.claude-opus-4-5-20251101-v1:0 or empty if you haven't figured this out. Corresponding model definition

- model_name: aws/anthropic/claude-opus-4-5
          litellm_params:
            model: bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0
            aws_region_name: us-west-2
            tpm: 17000000
            rpm: 4000
          model_info:
            id: aws/anthropic/claude-opus-4-5
            mode: completion
            access_groups:
            - default-models

Basically our ask is, the metrics should have requested_mode and litellm_model_name lables there everywhere. Since they are not present every where , we are foced touse mode_id which is also not consistent and which has issues with weighted routing.  If the labels are resolved then majority of issues will be resolved.  Can you please look into this. litellm_deployment_failure_responses_total{api_base="",api_key_alias="seerai-prod-iguy-fc451b11",api_provider="None",client_ip="10.3.156.11",exception_class="HTTPException",exception_status="429",hashed_api_key="f7e5320693384c60e531eb8393bce90ecb6f74e084220f80233f2d5e4870f89d",litellm_model_name="aws/anthropic/claude-opus-4-5",model_id="",requested_model="",team="13125321-faac-4b54-b01b-bf3b00235afe",team_alias="None",user_agent="OpenAI/Python 2.16.0"} 1.0

Steps to Reproduce

  1. Config: model_list:
  • model_name: aws/anthropic/claude-opus-4-5 litellm_params: model: bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0 aws_region_name: us-west-2 #other creds model_info: id: aws/anthropic/claude-opus-4-5 mode: completion

general_settings: master_key: sk-repro

litellm_settings: callbacks: ["prometheus"]

  1. Run the proxy
  2. Trigger a failure
curl -X POST http://localhost:4000/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-repro" \
  -d '{
    "model": "aws/anthropic/claude-opus-4-5",
    "messages": [{"role": "user", "content": "hi"}]
  }'
  1. Inspect litellm_deployment_failure_responses_total metric on /metrics

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.0

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To resolve the issue with missing requested_model_name and incorrect litellm_model_name labels in metrics, we need to update the metric labeling logic.

Here are the steps:

  • Update the litellm_deployment_failure_responses_total metric to include requested_model_name and litellm_model_name labels.
  • Modify the litellm_params to correctly set the litellm_model_name label.

Example code changes:

# Update metric labeling logic
def update_metric_labels(metric, requested_model_name, litellm_model_name):
    metric.labels(
        requested_model_name=requested_model_name,
        litellm_model_name=litellm_model_name,
        # other labels...
    )

# Modify litellm_params to correctly set litellm_model_name label
litellm_params = {
    'model': 'bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0',
    # other params...
}
litellm_model_name = litellm_params['model']
  • Update the model_list configuration to include the correct litellm_model_name:
model_list:
  - model_name: aws/anthropic/claude-opus-4-5
    litellm_params:
      model: bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0
      # other params...
    model_info:
      id: aws/anthropic/claude-opus-4-5
      mode: completion
    litellm_model_name: bedrock/global.anthropic.claude-opus-4-5-20251101-v1:0

Verification

To verify the fix, trigger a failure and inspect the litellm_deployment_failure_responses_total metric on /metrics. The metric should now include the correct requested_model_name and litellm_model_name labels.

Extra Tips

  • Ensure that the litellm_params are correctly configured for each model in the model_list.
  • Verify that the litellm_model_name label is correctly set for each model.
  • Consider adding additional logging or monitoring to detect any issues with metric labeling.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#integration issue#index setup#retrieval issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING