litellm - 💡(How to fix) Fix [Bug]: AllowedFailsPolicy.InternalServerErrorAllowedFails is silently ignored in get_allowed_fails_from_policy [2 pull requests]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

AllowedFailsPolicy defines InternalServerErrorAllowedFails as a field in litellm/types/router.py, but get_allowed_fails_from_policy() in litellm/router.py (line ~11652) does not have a branch for InternalServerError.

The method handles 5 error types (AuthenticationError, Timeout, RateLimitError, ContentPolicyViolationError, BadRequestError) but is missing InternalServerError. As a result, the setting is accepted in config but silently ignored at runtime, falling back to the global allowed_fails value.

Expected behavior: When InternalServerErrorAllowedFails: 2 is configured, a deployment should be cooled down after 2 InternalServerError failures.

Actual behavior: The InternalServerErrorAllowedFails value is never read. get_allowed_fails_from_policy returns None for InternalServerError, so the router falls back to the global allowed_fails (e.g. 10000), effectively disabling cooldown for this error type.

Expected fix: Add the missing branch in get_allowed_fails_from_policy:

if isinstance(exception, litellm.InternalServerError) and allowed_fails_policy.InternalServerErrorAllowedFails is not None: return allowed_fails_policy.InternalServerErrorAllowedFails

Related: PR #25644 fixed the same class of bug for RetryPolicy.InternalServerErrorRetries in get_num_retries_from_retry_policy(). The PR description stated "AllowedFailsPolicy has no corresponding InternalServerErrorAllowedFails field", but the field does exist in the Pydantic model — so the parallel fix for get_allowed_fails_from_policy was missed.

Steps to Reproduce

1. Configure proxy with allowed_fails_policy: router_settings: allowed_fails: 10000 allowed_fails_policy: InternalServerErrorAllowedFails: 2 cooldown_time: 60 fallbacks:
   • model-a:
     • modeal-b
2. Send requests that trigger InternalServerError on model-a
3. After 2 failures, expect cooldown + fallback to model-b
4. Observe: cooldown does not trigger because the router uses allowed_fails: 10000 instead of InternalServerErrorAllowedFails: 2

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.85.2

Twitter / LinkedIn details

No response