Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug] Bad Error code for batch creation (OpenAI) Rate Limits [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23901•Fetched 2026-04-08 00:53:58
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
shivamrawat1shivamrawat1
Participants
shivamrawat1shivamrawat1
Timeline (top)
labeled ×3cross-referenced ×1

Root Cause

This is bad for us because sometimes our alerts triggers saying we have a lot of 500 errors in the App but they are just 429. I would suggest this priorization after you finish the "to-do" tasks:

Fix Action

Fixed

PR fix notes

PR #24703: fix: 429s from batch creation being converted to 500

Description (problem / solution / changelog)

Relevant issues

Fixes #23901

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

<!-- Select the type of Pull Request --> <!-- Keep only the necessary ones -->

🐛 Bug Fix

Changes

handle_exception_on_proxy in litellm/proxy/utils.py was hardcoding HTTP_500 as the fallback status code for any exception that wasn't an HTTPException or ProxyException. LiteLLM's own exceptions (like RateLimitError) have a status_code attribute — we just weren't using it. One line fix: getattr(e, "status_code", 500) instead of the hardcoded 500.

Changed files

  • litellm/proxy/utils.py (modified, +3/-2)
  • tests/proxy_unit_tests/test_proxy_utils.py (modified, +47/-0)
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

OpenAI has a new rate limit about batch creation (2k per hour). We got some 429 from OpenAI but the LiteLLM convert this to 500.

We think this translations happens here:

https://github.com/BerriAI/litellm/blob/main/litellm/router.py#L4478

https://github.com/BerriAI/litellm/blob/main/litellm/proxy/utils.py#L5122

This is bad for us because sometimes our alerts triggers saying we have a lot of 500 errors in the App but they are just 429. I would suggest this priorization after you finish the "to-do" tasks:

Note:

OpenAI Batch Rate Limiting

Steps to Reproduce

NA

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.0

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To address the issue of LiteLLM converting 429 errors from OpenAI to 500 errors, we need to modify the error handling in the router.py and utils.py files.

Step-by-Step Solution

  1. Update error handling in router.py: Modify the error handling code to correctly identify and handle 429 errors from OpenAI.

In litellm/router.py

if response.status_code == 429: # Handle 429 error specifically, e.g., by retrying the request or returning a custom error message return jsonify({"error": "Rate limit exceeded"}), 429


2. **Update error handling in `utils.py`**:
   Ensure that the `utils.py` file also handles 429 errors correctly, potentially by passing the original status code through to the caller.
   ```python
# In litellm/proxy/utils.py
if response.status_code == 429:
    # Pass the 429 status code through to the caller
    return response.json(), 429
  1. Implement retry logic for 429 errors: Consider implementing a retry mechanism with exponential backoff for 429 errors to handle rate limiting more robustly.

Verification

To verify that the fix worked:

  • Test the API endpoint that previously returned 500 errors for 429 rate limit errors from OpenAI.
  • Confirm that the endpoint now correctly returns a 429 status code or a custom error message indicating rate limit exceeded.

Extra Tips

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model save/load#optimization#mixed precision#training loop

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug] Bad Error code for batch creation (OpenAI) Rate Limits [1 pull requests, 1 participants]