Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Bug]: prompt is missing in Received request only params not prompts [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38537•Fetched 2026-04-08 01:53:28
View on GitHub
Comments
2
Participants
2
Timeline
6
Reactions
0
Author
tongcutongcu
Participants
JosephAhn23JosephAhn23tongcutongcu
Timeline (top)
commented ×2referenced ×2cross-referenced ×1labeled ×1

Fix Action

Fixed

PR fix notes

PR #38583: [Bugfix] Opt-in INFO prompt summaries for request logging (--enable-log-request-prompts)

Description (problem / solution / changelog)

Summary

Addresses #38537 while respecting security posture (see maintainer feedback): truncated prompt previews at INFO are not logged by default. They require an explicit second flag alongside --enable-log-requests.

Behavior

  • --enable-log-requests alone: INFO logs request id, params, LoRA (same as today); prompt content remains DEBUG only.
  • --enable-log-requests + --enable-log-request-prompts: INFO logs also include bounded prompt summary (string, token ids, or embed shape), with the same caps as before when --max-log-len is unset.

CLI

  • New: --enable-log-request-prompts / --no-enable-log-request-prompts (default off).
  • Validation: prompts flag requires --enable-log-requests.

Code notes

  • RequestLogger(..., log_prompts_at_info=...)
  • Gemini review: wording Prompt details; prompt_embeds.shape without redundant tuple(...).

Testing

Please run:

python -m pytest tests/test_logger.py tests/entrypoints/openai/test_cli_args.py -k "log_request_prompt or log_outputs_integration or prompt_at_info" -v

(or full files if preferred).

AI assistance

Developed with AI assistance (Claude). Submitter should review and run tests.

Changed files

  • docs/configuration/serve_args.md (modified, +5/-0)
  • docs/usage/security.md (modified, +14/-0)
  • docs/usage/troubleshooting.md (modified, +2/-0)
  • tests/entrypoints/openai/test_cli_args.py (modified, +13/-0)
  • tests/test_logger.py (modified, +24/-0)
  • vllm/engine/arg_utils.py (modified, +3/-2)
  • vllm/entrypoints/logger.py (modified, +54/-2)
  • vllm/entrypoints/openai/api_server.py (modified, +8/-2)
  • vllm/entrypoints/openai/cli_args.py (modified, +14/-0)
RAW_BUFFERClick to expand / collapse

Your current environment

<details>

prompt is missing in Received request only params not prompts in 0.17.0 (APIServer pid=1) INFO 03-30 17:55:29 [logger.py:63] Received request chatcmpl-9b07611b4cc561b0: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.3, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=15000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.

in v0.10.0 something like following (APIServer pid=1) INFO 03-30 18:23:14 [logger.py:40] Received request chatcmpl-474f89bd31b4402ea1122918eb218bf3: prompt: '<|im_start|>system\nxxxxx。\n\n\nIn order to complete the objective that the user asks of you, you have access to a number of standard tools.\n\n## write_todos\n\nYou have access to the write_todos tool to help you manage and plan

the prompt part is missing?

<summary>The output of <code>python collect_env.py</code></summary>

v0.17.0

</details>

🐛 Describe the bug

prompt is missing in Received request only params not prompts in 0.17.0 (APIServer pid=1) INFO 03-30 17:55:29 [logger.py:63] Received request chatcmpl-9b07611b4cc561b0: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.3, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=15000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None), lora_request: None.

in v0.10.0 something like following (APIServer pid=1) INFO 03-30 18:23:14 [logger.py:40] Received request chatcmpl-474f89bd31b4402ea1122918eb218bf3: prompt: '<|im_start|>system\nxxxxx。\n\n\nIn order to complete the objective that the user asks of you, you have access to a number of standard tools.\n\n## write_todos\n\nYou have access to the write_todos tool to help you manage and plan

the prompt part is missing?

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To fix the missing prompt issue in version 0.17.0, we need to modify the logging functionality to include the prompt in the received request log message.

Steps:

  • Modify the logger.py file to include the prompt in the log message.
  • Update the Received request log message to include the prompt.

Example Code:

# logger.py
def log_received_request(request_id, params, prompt):
    # Log the received request with prompt
    logging.info(f"Received request {request_id}: prompt: {prompt}, params: {params}")

# Example usage:
prompt = '<|im_start|>system\nxxxxx。\n\n\nIn order to complete the objective that the user asks of you, you have access to a number of standard tools.\n\n## `write_todos`\n\nYou have access to the `write_todos` tool to help you manage and plan'
params = SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.3, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=15000, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None)
log_received_request("chatcmpl-9b07611b4cc561b0", params, prompt)

Verification

To verify that the fix worked, check the log messages for received requests and ensure that the prompt is included.

Extra Tips

  • Make sure to update the logging functionality to handle cases where the prompt is missing or empty.
  • Consider adding additional logging for other important request parameters to improve debugging capabilities.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#environment setup#docker error#permission error#memory optimization

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING