Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: litellm_metadata from request body is overridden on /v1/messages endpoint [2 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24945•Fetched 2026-04-08 02:23:35
View on GitHub
Comments
1
Participants
2
Timeline
9
Reactions
2
Author
nepshshshnepshshsh
Participants
nepshshshnepshshshsamarth9008samarth9008
Timeline (top)
labeled ×3cross-referenced ×2subscribed ×2commented ×1

PR fix notes

PR #25243: fix: body litellm_metadata values overwritten by header on /v1/messages

Description (problem / solution / changelog)

Relevant issues

Fixes #24945

Pre-Submission checklist

  • I have Added testing in tests/test_litellm/proxy/test_litellm_pre_call_utils.py under TestAddLitellmMetadataFromRequestHeaders
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

On the /v1/messages (Anthropic-compatible) endpoint, user-supplied values inside litellm_metadata were being silently overwritten by proxy-injected metadata derived from request headers.

Root cause: /v1/messages uses litellm_metadata as both the user-facing and proxy-internal metadata container (unlike /chat/completions which separates them into metadata vs litellm_metadata). The proxy was calling .update() to inject header-derived values (e.g. trace_id from x-litellm-trace-id), overwriting any values the user had already set in the request body.

Fix: Changed .update() to a conditional merge that only injects a header-derived value if the key isn't already set by the user. Body values now take priority; headers act as a fallback.

Changed files

  • litellm/proxy/litellm_pre_call_utils.py (modified, +24/-3)
  • tests/test_litellm/proxy/test_litellm_pre_call_utils.py (modified, +121/-0)

PR #25253: litellm24945_Bug_litellm_metadata_from_request_body_is_overridden_on_/v1/messages_endpoint

Description (problem / solution / changelog)

Relevant issues

Fixes - #24945

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix ✅ Test

Changes

litellm/proxy/litellm_pre_call_utils.py

  • add_litellm_metadata_from_request_headers (line 669-672): Changed blind .update() to a non-overwriting loop — header values now only fill keys the user did not supply in the request body. Headers act as defaults, not overrides.

  • Merge guard (line 1067-1072): Added an is not identity check so the litellm_metadata → _metadata_variable_name merge loop is skipped when both sides are the same dict object (the /v1/messages case). Makes the self-reference explicit instead of a silent no-op.
    tests/test_litellm/proxy/test_litellm_pre_call_utils.py

  • test_litellm_metadata_not_overridden_by_headers_on_messages_endpoint — verifies body values (trace_id, tags) survive header injection.

  • test_header_metadata_added_when_not_in_body_on_messages_endpoint — verifies header values are still injected when the body does not supply them.

Changed files

  • litellm/proxy/litellm_pre_call_utils.py (modified, +8/-3)
  • tests/test_litellm/proxy/test_litellm_pre_call_utils.py (modified, +58/-0)

Code Example

curl -X POST http://localhost:4000/v1/messages \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-trace-id: from-header" \
  -d '{
    "model": "mock-gpt-thinking-anthropic",
    "max_tokens": 10,
    "messages": [{"role": "user", "content": "hi"}],
    "litellm_metadata": { "trace_id": "from-body" }
  }'

---

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-trace-id: from-header" \
  -d '{
    "model": "mock-gpt-thinking-anthropic",
    "max_tokens": 10,
    "messages": [{"role": "user", "content": "hi"}],
    "metadata": { "trace_id": "from-body" }
  }'

---

{"message": "Request received by LiteLLM:\n{\n    \"model\": \"mock-gpt-thinking-anthropic\",\n    \"max_tokens\": 5,\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"hi\"\n        }\n    ],\n    \"litellm_metadata\": {\n        \"trace_id\": \"from-body\",\n        \"spend_logs_metadata\": {\n            \"session\": \"from-body\"\n        }\n    }\n}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898205", "model": "mock-gpt-thinking-anthropic", "max_tokens": 5, "messages": [{"role": "user", "content": "hi"}], "litellm_metadata": {"trace_id": "from-body", "spend_logs_metadata": {"session": "from-body"}}}
{"message": "Request Headers: {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898609", "host": "localhost:4000", "user-agent": "curl/8.7.1", "accept": "*/*", "content-type": "application/json", "x-litellm-trace-id": "from-header", "content-length": "224"}
{"message": "Extracted chain_id from header (trace-id/session-id): from-header", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898768"}
{"message": "receiving data: {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}}, 'proxy_server_request': {'url': 'http://localhost:4000/v1/messages', 'method': 'POST', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}, 'body': {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}}}, 'arrival_time': 1775075665.8987367}, 'litellm_session_id': 'from-header', 'litellm_trace_id': 'from-header', 'secret_fields': {'raw_headers': RedactedDict(REDACTED)}}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898821"}
{"message": "[PROXY] returned data from litellm_pre_call_utils: {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}, 'user_api_key_hash': '88dc28d0f030c55ed4ab77ed8faf098196cb1c05df778539800c9f1243fe6b4b', 'user_api_key_alias': None, 'user_api_key_spend': 0.0, 'user_api_key_max_budget': None, 'user_api_key_team_id': None, 'user_api_key_project_id': None, 'user_api_key_user_id': 'default_user_id', 'user_api_key_org_id': None, 'user_api_key_team_alias': None, 'user_api_key_end_user_id': None, 'user_api_key_user_email': None, 'user_api_key_request_route': '/v1/messages', 'user_api_key_budget_reset_at': None, 'user_api_key_auth_metadata': {}, 'agent_id': None, 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.82.3', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': None, 'user_api_key_model_max_budget': {}, 'user_api_key_end_user_model_max_budget': None, 'user_api_key_user_spend': None, 'user_api_key_user_max_budget': None, 'user_api_key_metadata': {}, 'user_api_key_team_metadata': None, 'user_api_key_object_permission_id': None, 'user_api_key_team_object_permission_id': None, 'endpoint': 'http://localhost:4000/v1/messages', 'litellm_parent_otel_span': None, 'requester_ip_address': '172.18.0.1', 'user_agent': 'curl/8.7.1'}}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.900498"}
{"message": "getting payload for SpendLogs, available keys in metadata: ['trace_id', 'spend_logs_metadata', 'session_id', 'headers', 'user_api_key_hash', 'user_api_key_alias', 'user_api_key_spend', 'user_api_key_max_budget', 'user_api_key_team_id', 'user_api_key_project_id', 'user_api_key_user_id', 'user_api_key_org_id', 'user_api_key_team_alias', 'user_api_key_end_user_id', 'user_api_key_user_email', 'user_api_key_request_route', 'user_api_key_budget_reset_at', 'user_api_key_auth_metadata', 'user_api_key', 'agent_id', 'user_api_end_user_max_budget', 'user_api_key_auth', 'litellm_api_version', 'global_max_parallel_requests', 'user_api_key_team_max_budget', 'user_api_key_team_spend', 'user_api_key_model_max_budget', 'user_api_key_end_user_model_max_budget', 'user_api_key_user_spend', 'user_api_key_user_max_budget', 'user_api_key_metadata', 'user_api_key_team_metadata', 'user_api_key_object_permission_id', 'user_api_key_team_object_permission_id', 'endpoint', 'litellm_parent_otel_span', 'requester_ip_address', 'user_agent', 'queue_time_seconds', 'model_group', 'model_group_alias', 'model_group_size', 'attempted_retries', 'max_retries', 'deployment', 'model_info', 'api_base', 'deployment_model_name', 'caching_groups']", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.956610"}
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

On POST /v1/messages, user-supplied values inside litellm_metadata are silently overwritten by proxy-injected metadata. This affects spend log tracking, guardrails, and pipeline behavior — any field passed in litellm_metadata (e.g. trace_id, tags, guardrails, _guardrail_pipelines) can be clobbered before the request reaches the LLM.

Why

/v1/messages uses "litellm_metadata" as its primary internal metadata container (see litellm_pre_call_utils.py:72-74). Proxy internals write into it via .update() at lines 654 and 698, overwriting user values. The key-protection merge at lines 1051–1054 that should restore user priority is inert for this route — both sides of the loop reference the same dict, so the condition if key not in data[_metadata_variable_name] is always False.

On all other endpoints (e.g. /v1/chat/completions), "metadata" and "litellm_metadata" are separate dicts, so the merge correctly preserves user-supplied values. On /v1/messages there is no such separation and no recovery path.

Steps to Reproduce

curl -X POST http://localhost:4000/v1/messages \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-trace-id: from-header" \
  -d '{
    "model": "mock-gpt-thinking-anthropic",
    "max_tokens": 10,
    "messages": [{"role": "user", "content": "hi"}],
    "litellm_metadata": { "trace_id": "from-body" }
  }'

Check spend logs — trace_id = "from-header" (body value lost) Confirm the bug is specific to /v1/messages by repeating with /v1/chat/completions and metadata instead of litellm_metadata:

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-trace-id: from-header" \
  -d '{
    "model": "mock-gpt-thinking-anthropic",
    "max_tokens": 10,
    "messages": [{"role": "user", "content": "hi"}],
    "metadata": { "trace_id": "from-body" }
  }'

Check spend logs — trace_id = "from-body" (body value preserved) Expected: step 3 should also show "from-body".

Relevant log output

{"message": "Request received by LiteLLM:\n{\n    \"model\": \"mock-gpt-thinking-anthropic\",\n    \"max_tokens\": 5,\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"hi\"\n        }\n    ],\n    \"litellm_metadata\": {\n        \"trace_id\": \"from-body\",\n        \"spend_logs_metadata\": {\n            \"session\": \"from-body\"\n        }\n    }\n}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898205", "model": "mock-gpt-thinking-anthropic", "max_tokens": 5, "messages": [{"role": "user", "content": "hi"}], "litellm_metadata": {"trace_id": "from-body", "spend_logs_metadata": {"session": "from-body"}}}
{"message": "Request Headers: {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898609", "host": "localhost:4000", "user-agent": "curl/8.7.1", "accept": "*/*", "content-type": "application/json", "x-litellm-trace-id": "from-header", "content-length": "224"}
{"message": "Extracted chain_id from header (trace-id/session-id): from-header", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898768"}
{"message": "receiving data: {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}}, 'proxy_server_request': {'url': 'http://localhost:4000/v1/messages', 'method': 'POST', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}, 'body': {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}}}, 'arrival_time': 1775075665.8987367}, 'litellm_session_id': 'from-header', 'litellm_trace_id': 'from-header', 'secret_fields': {'raw_headers': RedactedDict(REDACTED)}}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.898821"}
{"message": "[PROXY] returned data from litellm_pre_call_utils: {'model': 'mock-gpt-thinking-anthropic', 'max_tokens': 5, 'messages': [{'role': 'user', 'content': 'hi'}], 'litellm_metadata': {'trace_id': 'from-header', 'spend_logs_metadata': {'session': 'from-body'}, 'session_id': 'from-header', 'headers': {'host': 'localhost:4000', 'user-agent': 'curl/8.7.1', 'accept': '*/*', 'content-type': 'application/json', 'x-litellm-trace-id': 'from-header', 'content-length': '224'}, 'user_api_key_hash': '88dc28d0f030c55ed4ab77ed8faf098196cb1c05df778539800c9f1243fe6b4b', 'user_api_key_alias': None, 'user_api_key_spend': 0.0, 'user_api_key_max_budget': None, 'user_api_key_team_id': None, 'user_api_key_project_id': None, 'user_api_key_user_id': 'default_user_id', 'user_api_key_org_id': None, 'user_api_key_team_alias': None, 'user_api_key_end_user_id': None, 'user_api_key_user_email': None, 'user_api_key_request_route': '/v1/messages', 'user_api_key_budget_reset_at': None, 'user_api_key_auth_metadata': {}, 'agent_id': None, 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.82.3', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': None, 'user_api_key_model_max_budget': {}, 'user_api_key_end_user_model_max_budget': None, 'user_api_key_user_spend': None, 'user_api_key_user_max_budget': None, 'user_api_key_metadata': {}, 'user_api_key_team_metadata': None, 'user_api_key_object_permission_id': None, 'user_api_key_team_object_permission_id': None, 'endpoint': 'http://localhost:4000/v1/messages', 'litellm_parent_otel_span': None, 'requester_ip_address': '172.18.0.1', 'user_agent': 'curl/8.7.1'}}", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.900498"}
{"message": "getting payload for SpendLogs, available keys in metadata: ['trace_id', 'spend_logs_metadata', 'session_id', 'headers', 'user_api_key_hash', 'user_api_key_alias', 'user_api_key_spend', 'user_api_key_max_budget', 'user_api_key_team_id', 'user_api_key_project_id', 'user_api_key_user_id', 'user_api_key_org_id', 'user_api_key_team_alias', 'user_api_key_end_user_id', 'user_api_key_user_email', 'user_api_key_request_route', 'user_api_key_budget_reset_at', 'user_api_key_auth_metadata', 'user_api_key', 'agent_id', 'user_api_end_user_max_budget', 'user_api_key_auth', 'litellm_api_version', 'global_max_parallel_requests', 'user_api_key_team_max_budget', 'user_api_key_team_spend', 'user_api_key_model_max_budget', 'user_api_key_end_user_model_max_budget', 'user_api_key_user_spend', 'user_api_key_user_max_budget', 'user_api_key_metadata', 'user_api_key_team_metadata', 'user_api_key_object_permission_id', 'user_api_key_team_object_permission_id', 'endpoint', 'litellm_parent_otel_span', 'requester_ip_address', 'user_agent', 'queue_time_seconds', 'model_group', 'model_group_alias', 'model_group_size', 'attempted_retries', 'max_retries', 'deployment', 'model_info', 'api_base', 'deployment_model_name', 'caching_groups']", "level": "DEBUG", "timestamp": "2026-04-01T20:34:25.956610"}

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.3

Twitter / LinkedIn details

No response

extent analysis

TL;DR

The issue can be fixed by modifying the litellm_pre_call_utils.py file to properly merge user-supplied metadata with proxy-injected metadata for the /v1/messages endpoint.

Guidance

  • Review the litellm_pre_call_utils.py file, specifically lines 654, 698, and 1051-1054, to understand how metadata is being merged and overwritten.
  • Modify the code to ensure that user-supplied metadata is properly preserved and merged with proxy-injected metadata for the /v1/messages endpoint.
  • Test the changes using the provided curl commands to verify that user-supplied metadata is no longer being overwritten.
  • Consider adding additional logging or debugging statements to help identify and resolve any further issues.

Example

No specific code example is provided, as the fix will depend on the specific requirements and implementation details of the litellm_pre_call_utils.py file.

Notes

The issue appears to be specific to the /v1/messages endpoint and is caused by the way metadata is being merged and overwritten. The fix will require modifying the litellm_pre_call_utils.py file to properly handle user-supplied metadata.

Recommendation

Apply a workaround by modifying the litellm_pre_call_utils.py file to properly merge user-supplied metadata with proxy-injected metadata for the /v1/messages endpoint. This will ensure that user-supplied metadata is properly preserved and merged with proxy-injected metadata.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#inference speed#output truncation#response parsing#generation error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING