PR #25952: fix(proxy): honor client-supplied x-litellm-call-id in spend log request_id

• Repository: BerriAI/litellm
• Author: dkindlund
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25952

Description (problem / solution / changelog)

Summary

When a client sends x-litellm-call-id: <my-uuid> explicitly, today the proxy silently discards it from the spend log: the spend record exists, but with an unrelated provider response id as request_id, so /spend/logs/v2?request_id=<my-uuid> returns empty.

Root cause

get_spend_logs_id (litellm/proxy/spend_tracking/spend_tracking_utils.py:166) does:

id = response_obj.get("id") or kwargs.get("litellm_call_id")

response_obj.id is the provider's own response id (e.g. Gemini's "QanhadapC_val7oP15PyuQM", OpenAI's "chatcmpl-...", Anthropic's "msg_..."). It's always present, so it always wins — litellm_call_id is never used to populate request_id in the spend log.

This is most painful on the Google-native :generateContent route, where the response also lacks x-litellm-* headers (see #25500), so clients can't even read back the resolved id from the response. But the underlying defect is cross-route.

Fix

Track whether litellm_call_id was supplied by the client via x-litellm-call-id (vs. an auto-generated UUID). When the client supplied it, prefer it over response.id for the spend log request_id. When the call_id is auto-generated, fall back to existing behavior (response.id first, then the auto-uuid).

This preserves backward compatibility for callers that don't set the header (existing tests at tests/test_keys.py:519, tests/test_spend_logs.py:115/181, etc., still assert request_id == response["id"]).

Files changed

• litellm/proxy/common_request_processing.py — set data["litellm_call_id_from_client"] = True when the header is present (covers /v1/chat/completions, /v1/messages, etc.)
• litellm/proxy/google_endpoints/endpoints.py — same flag for both :generateContent and :streamGenerateContent
• litellm/proxy/spend_tracking/spend_tracking_utils.py — get_spend_logs_id honors the new flag
• tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py — 6 new tests (TestGetSpendLogsId) covering both default and flagged precedence, plus the aretrieve_batch carve-out

Manual testing

# Without the fix
curl -X POST "https://<proxy>/v1beta/models/gemini-2.5-pro:generateContent" \
  -H "x-goog-api-key: sk-..." \
  -H "x-litellm-call-id: my-uuid-123" \
  -d '{"contents": [{"parts": [{"text": "hi"}], "role": "user"}]}'

curl "https://<proxy>/spend/logs/v2?request_id=my-uuid-123"
# → data: []  (BAD)

# With the fix
# → data: [{"request_id": "my-uuid-123", ...}]  (GOOD)

Tests

tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_default_prefers_response_id PASSED
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_default_falls_back_to_litellm_call_id PASSED
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_client_supplied_flag_prefers_litellm_call_id PASSED
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_client_supplied_flag_falls_back_to_response_id PASSED
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_client_supplied_flag_false_uses_default_precedence PASSED
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py::TestGetSpendLogsId::test_aretrieve_batch_unchanged PASSED

Test plan:

• Add unit tests for new behavior
• Verify existing tests asserting request_id == response["id"] are unaffected (default path unchanged)
• Manual end-to-end on Google-native :generateContent route

Related

Part of a larger cluster of bugs in the Google-native /v1beta/... route (separate detailed issue forthcoming). Companion fixes:

• #25500 (in flight) — outbound x-litellm-* response headers on Google-native routes
• #24097 — streaming success_callback silently skipped on Google-native streaming
• Forthcoming PR — metadata / user propagation into spend logs on Google-native non-streaming

Tracking issue

#25956 — full root-cause writeup for the Google-native correlation cluster (this PR is Fix #1 of 4 defects)

Changed files

• litellm/proxy/common_request_processing.py (modified, +10/-3)
• litellm/proxy/google_endpoints/endpoints.py (modified, +20/-6)
• litellm/proxy/spend_tracking/spend_tracking_utils.py (modified, +6/-0)
• tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py (modified, +93/-0)

PR #25955: fix(google_genai): propagate user from kwargs to logging obj in agenerate_content

• Repository: BerriAI/litellm
• Author: dkindlund
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25955

Description (problem / solution / changelog)

Summary

The Google-native /v1beta/models/{model}:generateContent and /v1beta/models/{model}:streamGenerateContent routes silently drop the user field from the spend log. Spend records are written with the correct cost and tokens, but user: "" and end_user: "", blocking per-end-user attribution on this route.

Root cause

The Google-native handler in litellm/proxy/google_endpoints/endpoints.py reads user from the request body via add_litellm_data_to_request, which correctly populates data["user"]. That value flows through function_setup and then through llm_router.agenerate_content(**data) into litellm.agenerate_content, where kwargs["user"] is set.

But in setup_generate_content_call (litellm/google_genai/main.py:185), the call to litellm_logging_obj.update_from_kwargs doesn't pass user:

litellm_logging_obj.update_from_kwargs(
    kwargs=kwargs,
    model=model,
    optional_params=dict(generate_content_config_dict),
    litellm_params={
        "litellm_call_id": litellm_call_id,
    },
    custom_llm_provider=custom_llm_provider,
)

Without user=..., update_from_kwargs defers to its default user=None, so logging_obj.user and model_call_details["user"] are None, and the spend log row shows an empty user field.

OpenAI-compat /v1/chat/completions works because litellm.completion calls logging.update_environment_variables(user=user, ...) with the resolved user value (litellm/main.py:1597, 4806, 6325, 6496, 6746).

Fix

Pass user=kwargs.get("user") to update_from_kwargs so the logging object reflects what the client sent.

Files changed

• litellm/google_genai/main.py — pass user through (one-line change)
• tests/test_litellm/google_genai/test_google_genai_main.py — regression test using a real Logging instance with stubbed update_from_kwargs to verify user is propagated

Tests

Added test_setup_generate_content_call_propagates_user_to_logging_obj which:

1. Constructs a real Logging instance (required by Pydantic on GenerateContentSetupResult)
2. Stubs update_from_kwargs
3. Calls setup_generate_content_call with user="my-end-user-uuid-456"
4. Asserts update_from_kwargs.call_args.kwargs["user"] == "my-end-user-uuid-456"

Verified the test fails on main and passes with this commit.

tests/test_litellm/google_genai/test_google_genai_main.py::test_setup_generate_content_call_propagates_user_to_logging_obj PASSED

Manual testing

# Without the fix
curl -X POST "https://<proxy>/v1beta/models/gemini-2.5-pro:generateContent" \
  -H "x-goog-api-key: sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [{"parts": [{"text": "hi"}], "role": "user"}],
    "user": "my-end-user-uuid-456"
  }'

curl "https://<proxy>/spend/logs/v2?api_key=<hashed>"
# → {... "user": "", "end_user": ""}  (BAD)

# With the fix
# → {... "user": "my-end-user-uuid-456", "end_user": "my-end-user-uuid-456"}  (GOOD)

Related

Part of a larger cluster of bugs in the Google-native /v1beta/... route (separate detailed issue forthcoming). Companion fixes:

• #25500 (in flight) — outbound x-litellm-* response headers on Google-native routes
• #25952 — x-litellm-call-id precedence in spend log request_id
• #24097 — streaming success_callback silently skipped on Google-native streaming

Note: tags propagation

The developer's repro report also noted request_tags: [] (empty) when sending x-litellm-tags. From code inspection, add_litellm_data_to_request and update_from_kwargs should propagate metadata["tags"] correctly through the agenerate_content path, so tags may already work — or there may be a separate downstream gap in how StandardLoggingPayload._get_request_tags reads from litellm_params.metadata for the agenerate_content call_type. That investigation is out of scope for this PR. The user fix here is independent and unblocks per-end-user attribution today; tags can be addressed in a follow-up.

Tracking issue

#25956 — full root-cause writeup for the Google-native correlation cluster (this PR is Fix #3 of 4 defects)

Changed files

• litellm/google_genai/main.py (modified, +1/-0)
• tests/test_litellm/google_genai/test_google_genai_main.py (modified, +65/-8)

PR #25960: fix(google_genai): route streaming chunks to GeminiPassthroughLoggingHandler so success_callbacks fire (closes #24097, supersedes #24114)

• Repository: BerriAI/litellm
• Author: dkindlund
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25960

Description (problem / solution / changelog)

Summary

Fixes #24097.

Supersedes #24114 (closed by author without merging on 2026-03-19). This revival preserves the structural fix and incorporates Greptile's review feedback that was outstanding when the original was closed.

What was broken

For the Google-native streaming endpoints /models/{model}:streamGenerateContent and /v1beta/models/{model}:streamGenerateContent, the streaming iterator was tagging collected chunks as EndpointType.VERTEX_AI. The downstream _route_streaming_logging_to_handler had no VERTEX_AI branch that knew how to parse Google GenAI native chunks, so async_complete_streaming_response was never set and every function-based and CustomLogger success callback was silently skipped on stream end.

Sync callers were doubly broken: __next__ re-raised StopIteration without ever invoking the logging route at all.

The fix

1. Add EndpointType.GOOGLE_GENAI = "google-genai" to the enum.
2. In the async iterator, tag chunks as GOOGLE_GENAI, pass the real /models/{model}:streamGenerateContent URL, and forward the explicit model kwarg so downstream handlers do not have to fall back to URL parsing.
3. In the sync iterator, mirror the async path on StopIteration so sync callers also receive callbacks. (This was the gap PR #24114 left open.)
4. Add a GOOGLE_GENAI routing branch in _route_streaming_logging_to_handler that dispatches to GeminiPassthroughLoggingHandler._handle_logging_gemini_collected_chunks.

Tests

tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_google_genai_streaming_callbacks.py adds 7 regression tests, all runtime-behavior based:

Verified locally: 6 of 7 fail on main without this commit, all 7 pass with it.

============================== 7 passed in 0.54s ===============================

Differences from PR #24114 (Greptile feedback addressed)

Also dropped unrelated formatting-only changes that were carried in the original PR (team_endpoints.py, litellm_logging.py) to keep this PR's scope focused.

Files changed

• litellm/types/passthrough_endpoints/pass_through_endpoints.py (+1 enum value)
• litellm/google_genai/streaming_iterator.py (refactored async logging into _build_logging_kwargs; new sync logging path; switched to GOOGLE_GENAI)
• litellm/proxy/pass_through_endpoints/streaming_handler.py (added GOOGLE_GENAI routing branch + import)
• tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_google_genai_streaming_callbacks.py (new file, 7 tests)

Credits

Original structural approach by @awais786 in #24114. Sync iterator fix, test rewrite, and end-to-end callback-fires test added in this revival.

Tracking

Part of the Google-native correlation cluster — see #25956 for the unified writeup. Companion PRs:

• #25500 — outbound x-litellm-* response headers (in flight)
• #25952 — x-litellm-call-id precedence in spend log request_id (this issue's #25956 Fix #1)
• #25955 — user propagation in agenerate_content logging (this issue's #25956 Fix #3)

Changed files

• litellm/google_genai/streaming_iterator.py (modified, +58/-12)
• litellm/proxy/pass_through_endpoints/streaming_handler.py (modified, +20/-0)
• litellm/types/passthrough_endpoints/pass_through_endpoints.py (modified, +1/-0)
• tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_google_genai_streaming_callbacks.py (added, +397/-0)