Bug Report

Environment

• LiteLLM version: latest (main-latest Docker image)
• Python: 3.13
• Router strategy: usage-based-routing-v2

Description

When using an OpenAI-like provider registered via providers.json (e.g. any custom provider with a base_url and api_key_env), 429 rate limit errors are incorrectly wrapped as APIConnectionError instead of RateLimitError. This prevents the router from cooling down the failing deployment and routing to healthy alternatives in the same model group.

Root Cause

Two issues combine to cause this:

1. Providers from providers.json are not added to litellm._openai_like_providers

In exception_mapping_utils.py, the status-code-aware exception mapping for OpenAI-like providers only runs when custom_llm_provider in litellm._openai_like_providers (line ~857). Providers loaded from providers.json are not added to this list, so their exceptions fall through to the catch-all handler at line ~2406, which wraps everything as APIConnectionError regardless of HTTP status code.

2. Cooldown handler blanket-ignores APIConnectionError

In router_utils/cooldown_handlers.py, _is_cooldown_required() returns False for any exception string containing "APIConnectionError":

ignored_strings = ["APIConnectionError"]
if exception_str is not None:
    for ignored_string in ignored_strings:
        if ignored_string in exception_str:
            return False  # ← never cools down, even for wrapped 429s

Result

When a providers.json provider returns HTTP 429:

1. Exception mapping falls through to catch-all → APIConnectionError
2. Cooldown handler sees "APIConnectionError" in the string → skips cooldown
3. Router retries the same deployment (up to num_retries times), all fail with 429
4. Error surfaces to caller: Available Model Group Fallbacks=None
5. Other healthy deployments in the same model group are never tried

Reproduction

1. Register a provider in providers.json with two deployments under the same model_name
2. Hit rate limits on the first deployment (429 response)
3. Observe: second deployment never receives traffic; all retries go to the rate-limited deployment

Expected Behavior

429 from any provider should:

• Be mapped to RateLimitError (not APIConnectionError)
• Trigger deployment cooldown so the router picks a healthy alternative

Suggested Fix

Option A (comprehensive): Register providers.json providers in _openai_like_providers so they enter the status-code-aware exception mapping path.

Option B (targeted): In _is_cooldown_required(), do not skip cooldown for APIConnectionError when the exception string also indicates a rate limit (e.g. contains "429" or "Rate limit"):

ignored_strings = ["APIConnectionError"]
if exception_str is not None:
    for ignored_string in ignored_strings:
        if ignored_string in exception_str:
            if "429" in exception_str or "Rate limit" in exception_str:
                pass  # still cooldown for rate limits
            else:
                return False

Option A is the proper fix; Option B is a safety net.

Logs

litellm.exceptions.APIConnectionError: litellm.APIConnectionError: <Provider>Exception - Error code: 429 - {'error': {'message': 'Rate limit exceeded. Retry in 6s.', 'type': 'rate_limit'}}. Received Model Group=<model-group>
Available Model Group Fallbacks=None LiteLLM Retried: 3 times, LiteLLM Max Retries: 3