litellm_settings:
  turn_off_message_logging: true

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  store_model_in_db: True
  database_url: os.environ/DATABASE_URL

model_list:

  - model_name: "gpt-5"
    litellm_params:
      model: openai/gpt-5
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      id: gpt-5-custom-pricing
      mode: "chat"
      input_cost_per_token: 0.000125        # 100x standard ($1.25/1M = $0.00000125)
      output_cost_per_token: 0.001         # 100x standard ($10.00/1M = $0.00001)

  - model_name: "claude-sonnet-4-20250514"
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY
    model_info:
      id: claude-sonnet-4-custom-pricing
      input_cost_per_token: 0.0003         # 100x standard ($0.000003)
      output_cost_per_token: 0.0015         # 100x standard ($0.000015)

---

Request: POST /v1/messages
  {
    "model": "claude-sonnet-4-20250514",
    "messages": [
      {
        "role": "user",
        "content": "Say exactly: hello world (ref:18599-9699)"
      }
    ],
    "max_tokens": 20,
    "stream": true
  }

---

Request: POST /v1/responses
  {
    "model": "gpt-5",
    "input": "Say exactly: hello world (ref:5494-4215)",
    "stream": true
  }

---

Agent's code analysis (not validated or attempted to fix):

Regression commit: 1ee43b11de — PR #20679: "[Fix] prevent shared backend model key from being polluted by per-deployment custom pricing"

LiteLLM registers each model deployment in litellm.model_cost under TWO keys:
  - Deployment-specific ID (e.g., gpt-5.2) — with full _model_info including custom pricing
  - Shared base model name (e.g., openai/gpt-5.2 → resolves to gpt-5.2) — used as a fallback lookup

  `v1.81.0 (router.py:5651)`: the shared key got the full _model_info including custom pricing:
  litellm.register_model(model_cost={_model_name: _model_info})

  `v1.81.14 (router.py:6217-6229)`: PR #20679 strips all pricing fields from the shared key to prevent cross-deployment pollution:
  _custom_pricing_fields = CustomPricingLiteLLMParams.model_fields.keys()
  _shared_model_info = {
      k: v for k, v in _model_info.items() if k not in _custom_pricing_fields
  }
  litellm.register_model(model_cost={_model_name: _shared_model_info})

  This is correct in principle (two deployments of the same model with different pricing shouldn't overwrite each other's shared entry), but it breaks all code paths that look up cost by base model name — which includes:

  1. /spend/calculate endpoint — Chat App calls this with completion_response.model = "gpt-5.2", which looks up the now-stripped shared entry
  2. /responses API internal cost tracking — the router puts model_info into kwargs["litellm_metadata"] (via _get_router_metadata_variable_name("generic_api_call") → "litellm_metadata"),but responses/main.py reads metadata (set by proxy with user API key info, always truthy) and never reaches litellm_metadata. So custom_pricing is not detected, and cost calculation falls back to the base model name lookup — which no longer has pricing.
  3. Anthropic /messages API — same _generic_api_call_with_fallbacks → litellm_metadata path, same issue in llm_http_handler.py:1886 where kwargs.get("metadata", {}) gets proxy metadata without model_info

The /v1/chat/completions path works because it uses get_litellm_params(**kwargs) which explicitly passes input_cost_per_token etc. into the logging object's litellm_params, and the router's _acompletion uses _update_kwargs_with_deployment(deployment, kwargs) with no function_name — so model_info goes into kwargs["metadata"] (not litellm_metadata), making custom_pricing detectable.