litellm - 💡(How to fix) Fix [Bug]: Internal-user max_budget blocks zero-cost models — _PROXY_MaxBudgetLimiter ignores skip_budget_checks [2 pull requests]

  429 "Max budget limit reached."

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-stable   # reproduces on v1.87+; not on v1.81.12
    ports: ["4000:4000"]
    volumes: ["./config.yaml:/app/config.yaml"]
    command: ["--config=/app/config.yaml", "--port", "4000"]
    environment:
      DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
      STORE_MODEL_IN_DB: "True"
    depends_on: [db]
  db:
    image: postgres:16 
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: dbpassword9090
    ports: ["5433:5432"]

2. config.yaml — one free (cost 0) and one paid model:
model_list:
  - model_name: free-model
    litellm_params:
      model: openai/x
      api_base: http://127.0.0.1:9/v1   # unreachable on purpose
      api_key: x
      input_cost_per_token: 0
      output_cost_per_token: 0
  - model_name: paid-model
    litellm_params:
      model: openai/y
      api_base: http://127.0.0.1:9/v1
      api_key: y
      input_cost_per_token: 0.00001
      output_cost_per_token: 0.00001
general_settings:
  master_key: sk-1234
  
3. Start it
docker compose up -d

4. Create an internal user with a small budget, and grab its key
curl -s -X POST localhost:4000/user/new \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"user_id":"u@test","user_role":"internal_user","max_budget":0.01,"budget_duration":"30d"}'
# -> copy the "key" (sk-...) from the response

5. Push that user OVER budget, then clear the cache
docker compose exec db psql -U llmproxy -d litellm \
  -c "UPDATE \"LiteLLM_UserTable\" SET spend=1 WHERE user_id='u@test';"
docker compose restart litellm   # clears the in-memory user cache

6. Call the FREE model with the user's key
curl -s localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer <USER_KEY_FROM_STEP_4>" -H "Content-Type: application/json" \
  -d '{"model":"free-model","messages":[{"role":"user","content":"hi"}]}'

Observed: 429 {"error":{"message":"Max budget limit reached."}} — the zero-cost model is blocked.
Expected: the free model is allowed (it adds nothing to spend); only paid-model should return a budget error.

  # FREE (zero-cost) model — over-budget internal user  ->  WRONGLY BLOCKED
  HTTP 429
  {"error":{"message":"Max budget limit reached.","type":"rate_limit_error","code":"429"}}

  # PAID model — over-budget internal user  ->  correctly blocked (different message/path)
  HTTP 429
  {"error":{"message":"ExceededBudget: User=u@test over budget. Spend=1.0, Budget=0.01","type":"budget_exceeded"}}