litellm - 💡(How to fix) Fix Cost map reload doesn't fix provider routing for new bedrock models (e.g. claude-opus-4-7) [4 comments, 3 participants]

Bug

/reload/model_cost_map successfully loads new model data (e.g. us.anthropic.claude-opus-4-7 with litellm_provider: bedrock_converse, mode: chat), but requests still fail because provider routing ignores the cost map and relies on hardcoded model lists.

Reproduction

1. Configure litellm proxy with a model:

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: bedrock/us.anthropic.claude-opus-4-7
      aws_region_name: us-west-2

1. Call POST /reload/model_cost_map — returns "2670 models updated" successfully

2. Verify the cost map has the model:

   • us.anthropic.claude-opus-4-7 exists with litellm_provider: bedrock_converse, mode: chat

3. Send a request:

curl -X POST /v1/chat/completions \
  -d '{"model": "claude-opus-4-7", "messages": [{"role": "user", "content": "hello"}], "max_tokens": 50}'

1. Get error: max_tokens_to_sample: Extra inputs are not permitted

The request is routed through the legacy bedrock InvokeModel handler instead of bedrock_converse (Messages API).

Root cause

get_llm_provider in litellm_core_utils/get_llm_provider_logic.py determines the provider by splitting the model string on / and checking if the prefix is in litellm.provider_list. It never consults litellm.model_cost for the litellm_provider field.

For known models like claude-opus-4-6, a separate hardcoded auto-upgrade path in the bedrock handler switches from bedrock → bedrock_converse. New models like claude-opus-4-7 aren't in that hardcoded list, so they fall through to the legacy API.

Similarly, /model/info returns mode: None and max_tokens: None for the new model even after the reload, suggesting the cost map data isn't being used to populate model info for config-defined models.

Expected behavior

After a successful /reload/model_cost_map, the litellm_provider and mode fields from the cost map should be used for provider routing and model info. This is the entire value proposition of the reload endpoint and "day-0 model support" — new models should work without a code upgrade.

Workaround

Explicitly set bedrock_converse as the provider in the config:

model_list:
  - model_name: claude-opus-4-7
    litellm_params:
      model: bedrock_converse/us.anthropic.claude-opus-4-7

Environment

• LiteLLM proxy (self-hosted, version unknown — latest as of 2026-04-16)
• Model cost map reloaded from GitHub after merge of #25867 (day-0 opus 4.7 support)