vllm - ✅(Solved) Fix [Bug]: "stop" parameter will stop the reasoning_content [1 pull requests, 2 comments, 3 participants]

vllm2026-03-30 03:37:04

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#38499•Fetched 2026-04-08 01:48:58

View on GitHub

Comments

Participants

Timeline

Reactions

Author

miaolanxinlisa-netizen

Participants

chaunceyjiang

jeqer666

miaolanxinlisa-netizen

Timeline (top)

commented ×2cross-referenced ×1labeled ×1

Fix Action

Fixed

Fixed by PR: [Frontend] Skip stop in reasoning content (https://github.com/vllm-project/vllm/pull/38533)

PR fix notes

PR #38533: [Frontend] Skip stop in reasoning content

Repository: vllm-project/vllm
Author: chaunceyjiang
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/38533

Description (problem / solution / changelog)

Purpose

FIX #38499

Test Plan

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is lower?"}]
response = client.chat.completions.create(
    model=model,
    messages=messages,
    stop="9.8",
)

reasoning = response.choices[0].message.reasoning
content = response.choices[0].message.content

print("reasoning for Round 1:", reasoning)
print("-" * 80)
print("content for Round 1:", content)

Test Result

reasoning for Round 1: We need to answer the user query: "9.11 and 9.8, which is lower?" The user is asking which is lower: 9.11 or 9.8. That is straightforward: 9.8 is lower (or is 9.11? Let's check: 9.8 is 9.80, 9.11 is 9.11, which is less than 9.80, so 9.11 is lower. Wait we need to see the numbers: 9.11 vs 9.8. Interpreted as decimal numbers, 9.8 = 9.80, 9.11 = 9.11. The lower one is 9.11. However sometimes people might think about "9.11" as 9 and 11/100 = 9.11. Yes. So 9.11 < 9.8. So answer: 9.11 is lower.

But maybe they think about something else? The question "9.11 and 9.8, which is lower?" Might be ambiguous: It's just a simple numeric comparison.

Thus answer: 9.11 is lower (less than 9.8).

Now, in case they want to compare them: The lower number is 9.11.

We should answer simply: 9.11 is lower than 9.8. Or we can explain: 9.8 is higher.

But we could also add a small explanation: 9.8 = 9.80; 9.11 is less. So 9.11 is lower. Provide answer.

Make it clear and concise.

Will answer accordingly.

--------------------------------------------------------------------------------
content for Round 1: 

9.11 is lower than

<details> <summary> Essential Elements of an Effective PR Description Checklist </summary>

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

</details>

Changed files

tests/entrypoints/openai/chat_completion/test_chat_with_tool_reasoning.py (modified, +24/-0)
tests/tokenizers_/test_detokenize.py (modified, +7/-3)
tests/v1/engine/test_fast_incdec_prefix_err.py (modified, +5/-2)
tests/v1/engine/test_output_processor.py (modified, +30/-8)
vllm/v1/engine/async_llm.py (modified, +1/-0)
vllm/v1/engine/detokenizer.py (modified, +86/-43)
vllm/v1/engine/llm_engine.py (modified, +2/-1)
vllm/v1/engine/output_processor.py (modified, +6/-0)

RAW_BUFFERClick to expand / collapse

Your current environment

the input request: { "model": "Kimi", "messages": [ { "role": "user", "content": "元曲四大家有谁" } ], "ignore_eos": false, "max_completion_tokens": 1000, "stop": "四", "temperature": 0, "stream": true, "stream_options": { "include_usage": true } }

the output response: data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}}], "service_tier": "default", "first_token_return_time": 1774598361.9974504, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": " 用户"}}], "service_tier": "default", "first_token_return_time": 1774598361.9976811, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": "询问"}}], "service_tier": "default", "first_token_return_time": 1774598362.0438664, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": "的是"}}], "service_tier": "default", "first_token_return_time": 1774598362.0907245, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": "“"}}], "service_tier": "default", "first_token_return_time": 1774598362.1391256, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": "元"}}], "service_tier": "default", "first_token_return_time": 1774598362.1867049, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": "曲"}}], "service_tier": "default", "first_token_return_time": 1774598362.233036, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "", "reasoning_content": ""}, "finish_reason": "stop"}], "service_tier": "default", "first_token_return_time": 1774598362.2797952, "usage": null} data: {"id": "1dfa1784e6b64e09ade35bedf15798ed", "object": "chat.completion.chunk", "created": 1774598361, "model": "Kimi", "choices": [], "usage": {"prompt_tokens": 31, "total_tokens": 38, "completion_tokens": 7, "prompt_tokens_details": {"cached_tokens": 30}, "completion_tokens_details": {"reasoning_tokens": 0}}, "service_tier": "default", "first_token_return_time": 1774598362.2799811} data: [DONE]

🐛 Describe the bug

The stop parameter will stop the reasoning_content. Why does it stop the reasoning_content instead of only stopping the content? Is there any solution to set the stop parameter to stop only the "content" and not stop the "reasoning_content"?

Before submitting a new issue...

Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To address the issue where the stop parameter stops the reasoning_content instead of only stopping the content, you can modify the code that handles the stop parameter.

Here are the steps:

Check if the stop parameter is set and if the current token is equal to the stop token.
If true, set a flag to indicate that the content should be stopped.
Continue generating the reasoning_content until a different stop condition is met.

Example code snippet in Python:

def generate_content(model, input_text, stop_token, max_tokens):
    # Initialize flags and variables
    stop_content = False
    content = []
    reasoning_content = []

    # Generate content and reasoning content
    for token in model.generate(input_text, max_tokens):
        if token == stop_token and not stop_content:
            stop_content = True
        if not stop_content:
            content.append(token)
        reasoning_content.append(token)

        # Stop generating content if max tokens are reached
        if len(content) >= max_tokens:
            break

    return content, reasoning_content

# Example usage
model = "Kimi"
input_text = "元曲四大家有谁"
stop_token =四max_tokens =

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#model download #tokenizer error #prompt formatting #chain error #conversation history

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [Bug]: "stop" parameter will stop the reasoning_content [1 pull requests, 2 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Fix Action

Fixed

PR fix notes

PR #38533: [Frontend] Skip stop in reasoning content

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [Bug]: "stop" parameter will stop the reasoning_content [1 pull requests, 2 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Fix Action

Fixed

PR fix notes

PR #38533: [Frontend] Skip stop in reasoning content

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Still need to ship something?

RELATED_DISCOVERY

TRENDING