PR #28248: fix(proxy): make spend counter reseed idempotent across pods via SET NX

• Repository: BerriAI/litellm
• Author: silencedoctor
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/28248

Description (problem / solution / changelog)

Relevant issues

Fixes #28247

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

Problem

SpendCounterReseed.coalesced() re-seeds the Redis spend counter from DB using async_increment() (Redis INCRBYFLOAT), which is additive, not idempotent. The per-process asyncio.Lock added in #26459 collapses duplicate reseeds within a single pod, but provides no cross-pod coordination. With N proxy pods racing the same cold or expired counter:

Pod A: Redis miss → read DB (spend = X) → INCRBYFLOAT +X → Redis = X
Pod B: Redis miss → read DB (spend = X) → INCRBYFLOAT +X → Redis = 2X
Pod C: Redis miss → read DB (spend = X) → INCRBYFLOAT +X → Redis = 3X
...

Subsequent get_current_spend() calls in user_api_key_auth → common_checks read the inflated Redis value and raise BudgetExceededError, falsely rejecting requests well under the configured limit.

Fix

Switch the reseed write to async_set_cache(..., nx=True) and, on a lost race, read back the winner's value — mirroring the existing pattern already used by coalesced_window() in the very same file (litellm/proxy/db/spend_counter_reseed.py). After this PR, both reseed paths use consistent, idempotent semantics.

Why this fix should be easy to merge

• Minimal: ~15-line change in one function.
• Reuses the existing pattern from the same file: RedisCache.async_set_cache(..., nx=True) already exists, and coalesced_window() already uses it. This PR brings coalesced() in line with its sibling.
• No new public APIs, config flags, or dependencies.
• Single-pod behavior is unchanged: INCRBYFLOAT from absent ≡ SET, so this PR has no observable effect on single-pod deployments. It only fixes the broken multi-pod path.
• Scope is isolated: one function, one concern, no incidental refactors.

Tests

Added test_coalesced_reseed_idempotent_under_concurrent_multi_pod_reseed in tests/test_litellm/proxy/test_proxy_server.py:

• Simulates N pods by replacing SpendCounterReseed._get_lock so each concurrent caller receives an independent asyncio.Lock (mirroring the real multi-pod condition where the in-process lock provides no cross-pod coordination).
• Backs spend_counter_cache.redis_cache with a small in-memory fake that implements async_set_cache(nx=...), async_get_cache, async_increment, and serializes operations with an asyncio.Lock (single shared Redis instance).
• Asserts that after N = 5 concurrent coalesced() calls, redis.get(counter_key) == db_spend (not N × db_spend), and that all callers observe the same value.

The new test fails on main (final value 500.0) and passes after this patch (final value 100.0).

Existing reseed tests (test_init_and_increment_spend_counter_reseeds_from_db_on_counter_miss, test_init_spend_counter_redis_clean_miss_skips_stale_in_memory, test_get_current_spend_reseeds_from_db_when_counter_missing, test_get_current_spend_coalesces_concurrent_reseeds, test_concurrent_read_and_write_paths_share_one_db_query, test_reseed_warms_cache_even_on_zero_db_spend) had their fake-Redis stubs extended with the async_set_cache(nx=True) side-effect so they continue to model the new (idempotent) reseed path correctly. No assertion semantics were weakened.

Screenshots / Proof of Fix

$ uv run pytest tests/test_litellm/proxy/test_proxy_server.py -v \
    -k "reseed or spend_counter or current_spend"
...
27 passed

Including the new regression test:

tests/test_litellm/proxy/test_proxy_server.py::test_coalesced_reseed_idempotent_under_concurrent_multi_pod_reseed PASSED

Changed files

• litellm/proxy/db/spend_counter_reseed.py (modified, +26/-6)
• tests/test_litellm/proxy/test_proxy_server.py (modified, +203/-12)