PR #25652: fix(proxy): prevent LiteLLM_HealthCheckTable unbounded growth

• Repository: BerriAI/litellm
• Author: balgaly
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25652

Description (problem / solution / changelog)

Summary

Fixes #25623

LiteLLM_HealthCheckTable grows without bound when external monitoring tools (e.g. Uptime Kuma) ping health endpoints continuously. With 500k+ rows, the Model Dashboard query loads every row into Python memory, causing 8GB+ container memory and 400% CPU spikes.

Root cause

get_all_latest_health_checks ran a full-table find_many with no where filter and no take limit, then deduplicated in Python. On a large deployment this means hundreds of thousands of rows materialized in memory on every dashboard load.

Changes

litellm/proxy/utils.py

get_all_latest_health_checks — add a checked_at >= (now - TTL) WHERE clause so only recent rows are fetched. The DB does the filtering; Python only sees O(unique models) rows.

cleanup_old_health_checks — new method that deletes rows older than TTL via a single delete_many. Called from save_health_check_result at most once per hour (tracked via _health_check_last_cleanup_ts) to keep the table bounded without adding per-insert overhead.

TTL defaults to 7 days and is configurable via HEALTH_CHECK_TTL_DAYS env var.

Tests

Added 5 new tests in tests/test_litellm/proxy/test_health_check_functions.py:

• test_get_all_latest_health_checks_applies_ttl_filter — verifies the WHERE filter is passed to the DB
• test_get_all_latest_health_checks_ttl_env_override — verifies HEALTH_CHECK_TTL_DAYS controls the cutoff
• test_cleanup_old_health_checks_calls_delete_many — verifies delete_many is called with a lt cutoff
• test_save_health_check_result_triggers_cleanup_after_one_hour — cleanup runs when last run was > 1 hour ago
• test_save_health_check_result_skips_cleanup_within_one_hour — cleanup is skipped when last run was < 1 hour ago

All 27 tests in the file pass.

Checklist

• Tests added (5 new unit tests, all mocked)
• make test-unit passes for this test file
• Black formatting applied
• Scope isolated to this single bug
• Uses Prisma model methods (find_many, delete_many) — no raw SQL

Changed files

• docs/my-website/docs/proxy/config_settings.md (modified, +1/-0)
• litellm/proxy/utils.py (modified, +76/-34)
• tests/test_litellm/proxy/test_health_check_functions.py (modified, +248/-81)