Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Feature]: Enable Rate limiting per team models [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23269•Fetched 2026-04-08 00:37:52
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
saisurya237saisurya237
Participants
saisurya237saisurya237
Timeline (top)
labeled ×3cross-referenced ×1

Fix Action

Fixed

PR fix notes

PR #23274: Litellm permodel per team ratelimiting

Description (problem / solution / changelog)

Relevant issues

"Fixes #23269 "

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

<!-- Select the type of Pull Request --> <!-- Keep only the necessary ones -->

🆕 New Feature 🐛 Bug Fix 🧹 Refactoring 📖 Documentation 🚄 Infrastructure ✅ Test

Changes

Changed files

  • docs/my-website/docs/proxy/model_management.md (modified, +36/-1)
  • docs/my-website/docs/proxy/team_model_add.md (modified, +84/-0)
  • docs/my-website/docs/proxy/users.md (modified, +39/-0)
  • docs/my-website/release_notes/v1.80.8-stable/index.md (modified, +1/-0)
  • litellm/proxy/_experimental/out/api-reference/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/guardrails.html (removed, +0/-1)
  • litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.guardrails.__PAGE__.txt (removed, +0/-9)
  • litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.guardrails.txt (removed, +0/-4)
  • litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.txt (removed, +0/-7)
  • litellm/proxy/_experimental/out/guardrails/__next._full.txt (removed, +0/-28)
  • litellm/proxy/_experimental/out/guardrails/__next._head.txt (removed, +0/-6)
  • litellm/proxy/_experimental/out/guardrails/__next._index.txt (removed, +0/-8)
  • litellm/proxy/_experimental/out/guardrails/__next._tree.txt (removed, +0/-4)
  • litellm/proxy/_experimental/out/login/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/logs/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/model-hub/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/model_hub_table/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/models-and-endpoints/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/onboarding.html (removed, +0/-1)
  • litellm/proxy/_experimental/out/onboarding/__next._full.txt (removed, +0/-22)
  • litellm/proxy/_experimental/out/onboarding/__next._head.txt (removed, +0/-6)
  • litellm/proxy/_experimental/out/onboarding/__next._index.txt (removed, +0/-8)
  • litellm/proxy/_experimental/out/onboarding/__next._tree.txt (removed, +0/-4)
  • litellm/proxy/_experimental/out/onboarding/__next.onboarding.__PAGE__.txt (removed, +0/-9)
  • litellm/proxy/_experimental/out/onboarding/__next.onboarding.txt (removed, +0/-4)
  • litellm/proxy/_experimental/out/organizations/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/playground/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/teams/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/test-key/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/usage/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/users/index.html (renamed, +0/-0)
  • litellm/proxy/_experimental/out/virtual-keys/index.html (renamed, +0/-0)
  • litellm/proxy/common_utils/http_parsing_utils.py (modified, +13/-1)
  • litellm/proxy/management_endpoints/model_management_endpoints.py (modified, +233/-8)
  • litellm/proxy/management_endpoints/ui_sso.py (modified, +28/-12)
  • litellm/types/router.py (modified, +4/-0)
  • tests/test_litellm/proxy/management_endpoints/test_model_management_endpoints.py (modified, +96/-1)
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Current Rate limiting does no support setting rate limits for team models specifically. https://litellm.vercel.app/docs/proxy/users#set-rate-limits

Motivation, pitch

IT is requirement for our use case when leveraging bedrock pay as you go models

What part of LiteLLM is this about?

Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

https://www.linkedin.com/in/thatode-sai-surya-abhishek/

extent analysis

Fix Plan

To add rate limiting for team models, we need to modify the existing rate limiting logic to include team-specific limits.

Steps to Implement

  • Update the rate limiting configuration to include team IDs
  • Modify the rate limiting middleware to check for team-specific limits
  • Add a new endpoint to set rate limits for team models

Example Code

# Update rate limiting configuration
rate_limits = {
    'global': 100,  # global rate limit
    'teams': {  # team-specific rate limits
        'team1': 50,
        'team2': 200
    }
}

# Modify rate limiting middleware
def rate_limit_middleware(team_id):
    def decorator(func):
        def wrapper(*args, **kwargs):
            # Check for team-specific rate limit
            if team_id in rate_limits['teams']:
                team_limit = rate_limits['teams'][team_id]
                # Check if team limit is exceeded
                if exceed_rate_limit(team_limit):
                    return 'Rate limit exceeded for team'
            # Check global rate limit
            if exceed_rate_limit(rate_limits['global']):
                return 'Global rate limit exceeded'
            return func(*args, **kwargs)
        return wrapper
    return decorator

# Add new endpoint to set rate limits for team models
@app.route('/set-team-rate-limit', methods=['POST'])
def set_team_rate_limit():
    team_id = request.json['team_id']
    limit = request.json['limit']
    rate_limits['teams'][team_id] = limit
    return 'Team rate limit set successfully'

Verification

To verify the fix, test the new endpoint to set rate limits for team models and check that the rate limiting middleware is correctly enforcing the team-specific limits.

Extra Tips

  • Make sure to handle cases where team IDs are not provided or are invalid.
  • Consider adding logging to track rate limit exceedances for teams.
  • Review the existing rate limiting logic to ensure it is compatible with the new team-specific rate limits.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model download#tokenizer error#prompt formatting#chain error#conversation history

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Feature]: Enable Rate limiting per team models [1 pull requests, 1 participants]