litellm - 💡(How to fix) Fix [Bug] AdaptiveRouter reload crashes with "gammavariate: alpha and beta must be > 0.0" after restart

What happened?

After a proxy restart, requests to an auto_router/adaptive_router model return HTTP 500:

ValueError: gammavariate: alpha and beta must be > 0.0

Root cause

AdaptiveRouter.load_state_from_db reloads persisted state by overwriting each cold-start prior with the raw DB row:

self._cells[(rt, row.model_name)] = BanditCell(alpha=row.alpha, beta=row.beta)

But AdaptiveRouterUpdateQueue.flush_state_to_db persists accumulated deltas, not absolute posteriors (Prisma {"increment": ...}, so multiple pods don't clobber each other), and _compute_bandit_delta maps satisfaction to +alpha only. So:

• a satisfaction-only cell persists beta = 0
• a negative-only cell persists alpha = 0

On reload these become BanditCell(alpha, 0) / BanditCell(0, beta) — the cold-start prior (which guarantees beta >= 0.5 via initial_cell) is discarded. Then thompson_sample → random.betavariate(alpha, beta) raises whenever alpha <= 0 or beta <= 0. Because pick_best samples every pool model's cell for the classified request_type, one poisoned cell 500s all requests of that request type until the state table is cleared.

It's easy to hit: a clean, satisfied multi-turn conversation produces exactly +1 alpha, 0 beta (the happy path), persisted within the ~10s flush interval.

Secondary issue: even when it doesn't crash, the overwrite discards the tier-seeded prior (a cell with one satisfaction + one failure reloads as Beta(1,1) instead of prior + (1,1)), corrupting learned routing across restarts.

Reproduce

With an auto_router/adaptive_router configured, simulate a satisfaction-only learned cell (identical to what the flusher writes), restart, and send a matching request:

INSERT INTO "LiteLLM_AdaptiveRouterState"(router_name,request_type,model_name,alpha,beta,total_samples)
VALUES ('<router>','code_generation','<a tier-3 model>',5.0,0.0,5);

→ HTTP 500 gammavariate: alpha and beta must be > 0.0 for code_generation; other request types still succeed.

Minimal trigger:

import random; random.betavariate(2.0, 0.0)   # ValueError: gammavariate: alpha and beta must be > 0.0

Version

litellm v1.86.2 (and unchanged on main — load_state_from_db, bandit.py: thompson_sample).

Fix

Merge persisted deltas onto the cold-start prior in load_state_from_db instead of overwriting it (keeps beta >= 0.5 and preserves the tier prior). PR incoming.