vllm - ✅(Solved) Fix [Bug]: Missing logprobs for `<tool_call>` in streaming chat completions [1 pull requests, 1 participants]

vllm2026-03-21 06:44:36

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#37737•Fetched 2026-04-08 01:08:27

View on GitHub

Comments

Participants

Timeline

Reactions

Author

sdpkjc

Participants

sdpkjc

Timeline (top)

cross-referenced ×2labeled ×1referenced ×1subscribed ×1

Root Cause

This happens because some streaming tool parsers suppress control-token chunks by returning None, and the chat streaming path skips the whole chunk.

Fix Action

Fixed

Fixed by PR: Fix: preserve streaming logprobs (https://github.com/vllm-project/vllm/pull/37738)

PR fix notes

PR #37738: Fix: preserve streaming logprobs

Repository: vllm-project/vllm
Author: sdpkjc
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/37738

Description (problem / solution / changelog)

Purpose

Fixes #37737.

When a streaming tool parser suppresses a control-token chunk by returning None, the chat streaming path can skip the whole chunk even if it still carries logprobs. This change keeps emitting an empty delta chunk when per-token metadata must be preserved.

Test Plan

Run the streaming chat completion flow with logprobs=true and reproduce the tool-call path with Qwen/Qwen3-VL-8B-Thinking. Confirm that <tool_call> logprobs are preserved even when the parser suppresses the visible delta.

Test Result

After the fix, repeated requests all returned logprobs for the <tool_call> token.

<details> <summary> Essential Elements of an Effective PR Description Checklist </summary>

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

</details>

Changed files

vllm/entrypoints/openai/chat_completion/serving.py (modified, +15/-7)

Code Example

BASE_URL=http://127.0.0.1:8000

curl -sN "$BASE_URL/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-8B-Thinking",
    "stream": true,
    "logprobs": true,
    "tool_choice": "auto",
    "messages": [
      {"role": "system", "content": "If a tool is useful, call it immediately."},
      {"role": "user", "content": "Please call get_weather for Beijing. Do not explain."}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather by city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

---

14e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1ab

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"The","tool_calls":null},"logprobs":{"content":[{"token":"The","bytes":[84,104,101],"logprob":-0.0009746531140990555,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2df

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" user is asking me","tool_calls":null},"logprobs":{"content":[{"token":" user","bytes":[32,117,115,101,114],"logprob":-0.000022053474822314456,"top_logprobs":[]},{"token":" is","bytes":[32,105,115],"logprob":-0.2891930043697357,"top_logprobs":[]},{"token":" asking","bytes":[32,97,115,107,105,110,103],"logprob":-0.012495091184973717,"top_logprobs":[]},{"token":" me","bytes":[32,109,101],"logprob":-0.000013351351299206726,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2d0

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" to call the get","tool_calls":null},"logprobs":{"content":[{"token":" to","bytes":[32,116,111],"logprob":-4.6491513785440475e-6,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-0.001095529063604772,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-0.014845846220850945,"top_logprobs":[]},{"token":" get","bytes":[32,103,101,116],"logprob":-0.0011074369540438056,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2a6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"_weather function for","tool_calls":null},"logprobs":{"content":[{"token":"_weather","bytes":[95,119,101,97,116,104,101,114],"logprob":-0.00011359999916749075,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-4.529942543740617e-6,"top_logprobs":[]},{"token":" for","bytes":[32,102,111,114],"logprob":-0.011603484861552715,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

212

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" Beijing.","tool_calls":null},"logprobs":{"content":[{"token":" Beijing","bytes":[32,66,101,105,106,105,110,103],"logprob":-0.011142883449792862,"top_logprobs":[]},{"token":".","bytes":[46],"logprob":-0.505378246307373,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

246

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" They specifically","tool_calls":null},"logprobs":{"content":[{"token":" They","bytes":[32,84,104,101,121],"logprob":-0.036309339106082916,"top_logprobs":[]},{"token":" specifically","bytes":[32,115,112,101,99,105,102,105,99,97,108,108,121],"logprob":-0.2774711847305298,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2b6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" say \"Do not","tool_calls":null},"logprobs":{"content":[{"token":" say","bytes":[32,115,97,121],"logprob":-0.9159097671508789,"top_logprobs":[]},{"token":" \"","bytes":[32,34],"logprob":-0.006729202810674906,"top_logprobs":[]},{"token":"Do","bytes":[68,111],"logprob":-0.000603493710514158,"top_logprobs":[]},{"token":" not","bytes":[32,110,111,116],"logprob":-5.960462772236497e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c1

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" explain\" so I","tool_calls":null},"logprobs":{"content":[{"token":" explain","bytes":[32,101,120,112,108,97,105,110],"logprob":-6.437280717364047e-6,"top_logprobs":[]},{"token":"\"","bytes":[34],"logprob":-0.024111026898026466,"top_logprobs":[]},{"token":" so","bytes":[32,115,111],"logprob":-0.5177842378616333,"top_logprobs":[]},{"token":" I","bytes":[32,73],"logprob":-0.03217101842164993,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

28e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" should just make","tool_calls":null},"logprobs":{"content":[{"token":" should","bytes":[32,115,104,111,117,108,100],"logprob":-0.00022718709078617394,"top_logprobs":[]},{"token":" just","bytes":[32,106,117,115,116],"logprob":-0.008903564885258675,"top_logprobs":[]},{"token":" make","bytes":[32,109,97,107,101],"logprob":-0.04116453230381012,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

30d

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" the function call without","tool_calls":null},"logprobs":{"content":[{"token":" the","bytes":[32,116,104,101],"logprob":-2.0265558760002023e-6,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.00006770858453819528,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-4.6491513785440475e-6,"top_logprobs":[]},{"token":" without","bytes":[32,119,105,116,104,111,117,116],"logprob":-0.010122864507138729,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

234

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" any additional","tool_calls":null},"logprobs":{"content":[{"token":" any","bytes":[32,97,110,121],"logprob":-0.07130363583564758,"top_logprobs":[]},{"token":" additional","bytes":[32,97,100,100,105,116,105,111,110,97,108],"logprob":-0.1052059456706047,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1d6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" commentary","tool_calls":null},"logprobs":{"content":[{"token":" commentary","bytes":[32,99,111,109,109,101,110,116,97,114,121],"logprob":-1.7721552848815918,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

21b

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":".\n\nLooking","tool_calls":null},"logprobs":{"content":[{"token":".\n\n","bytes":[46,10,10],"logprob":-0.1671697348356247,"top_logprobs":[]},{"token":"Looking","bytes":[76,111,111,107,105,110,103],"logprob":-0.41703465580940247,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

311

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" at the function definition","tool_calls":null},"logprobs":{"content":[{"token":" at","bytes":[32,97,116],"logprob":-1.1920928244535389e-7,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-1.1920928244535389e-7,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.0059434822760522366,"top_logprobs":[]},{"token":" definition","bytes":[32,100,101,102,105,110,105,116,105,111,110],"logprob":-1.1973540782928467,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

270

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":":\n- Function","tool_calls":null},"logprobs":{"content":[{"token":":\n","bytes":[58,10],"logprob":-0.0005900310934521258,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-2.145764938177308e-6,"top_logprobs":[]},{"token":" Function","bytes":[32,70,117,110,99,116,105,111,110],"logprob":-0.014584832824766636,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2cb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" name: get_weather","tool_calls":null},"logprobs":{"content":[{"token":" name","bytes":[32,110,97,109,101],"logprob":-0.006715822499245405,"top_logprobs":[]},{"token":":","bytes":[58],"logprob":0.0,"top_logprobs":[]},{"token":" get","bytes":[32,103,101,116],"logprob":-0.0012191252317279577,"top_logprobs":[]},{"token":"_weather","bytes":[95,119,101,97,116,104,101,114],"logprob":-0.00006210611172718927,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2eb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"\n- Required parameter","tool_calls":null},"logprobs":{"content":[{"token":"\n","bytes":[10],"logprob":-0.001348182326182723,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-9.059865078597795e-6,"top_logprobs":[]},{"token":" Required","bytes":[32,82,101,113,117,105,114,101,100],"logprob":-0.05764629691839218,"top_logprobs":[]},{"token":" parameter","bytes":[32,112,97,114,97,109,101,116,101,114],"logprob":-0.11494840681552887,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c5

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":": city (string","tool_calls":null},"logprobs":{"content":[{"token":":","bytes":[58],"logprob":-3.576278118089249e-7,"top_logprobs":[]},{"token":" city","bytes":[32,99,105,116,121],"logprob":-0.00008034383063204587,"top_logprobs":[]},{"token":" (","bytes":[32,40],"logprob":-0.003986626863479614,"top_logprobs":[]},{"token":"string","bytes":[115,116,114,105,110,103],"logprob":-0.028655385598540306,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2bb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":")\n- The user","tool_calls":null},"logprobs":{"content":[{"token":")\n","bytes":[41,10],"logprob":-0.0036669413093477488,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-3.6954811548639555e-6,"top_logprobs":[]},{"token":" The","bytes":[32,84,104,101],"logprob":-0.030071474611759186,"top_logprobs":[]},{"token":" user","bytes":[32,117,115,101,114],"logprob":-0.0005220004240982234,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

275

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" provided \"Be","tool_calls":null},"logprobs":{"content":[{"token":" provided","bytes":[32,112,114,111,118,105,100,101,100],"logprob":-0.7353255152702332,"top_logprobs":[]},{"token":" \"","bytes":[32,34],"logprob":-0.014822943136096,"top_logprobs":[]},{"token":"Be","bytes":[66,101],"logprob":-0.000030397906812140718,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"ijing\" as the","tool_calls":null},"logprobs":{"content":[{"token":"ijing","bytes":[105,106,105,110,103],"logprob":-7.867782187531702e-6,"top_logprobs":[]},{"token":"\"","bytes":[34],"logprob":-0.00002276871418871451,"top_logprobs":[]},{"token":" as","bytes":[32,97,115],"logprob":-0.000049470632802695036,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-2.3841855067985307e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2bf

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" city\n\nI have","tool_calls":null},"logprobs":{"content":[{"token":" city","bytes":[32,99,105,116,121],"logprob":-0.0008453133050352335,"top_logprobs":[]},{"token":"\n\n","bytes":[10,10],"logprob":-0.04928658530116081,"top_logprobs":[]},{"token":"I","bytes":[73],"logprob":-0.015265769325196743,"top_logprobs":[]},{"token":" have","bytes":[32,104,97,118,101],"logprob":-0.012821147218346596,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

314

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" all the required parameters","tool_calls":null},"logprobs":{"content":[{"token":" all","bytes":[32,97,108,108],"logprob":-0.0007233387441374362,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-0.0006307758158072829,"top_logprobs":[]},{"token":" required","bytes":[32,114,101,113,117,105,114,101,100],"logprob":-0.00518774101510644,"top_logprobs":[]},{"token":" parameters","bytes":[32,112,97,114,97,109,101,116,101,114,115],"logprob":-0.6931779384613037,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

296

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":", so I can","tool_calls":null},"logprobs":{"content":[{"token":",","bytes":[44],"logprob":-0.0372004471719265,"top_logprobs":[]},{"token":" so","bytes":[32,115,111],"logprob":-2.3841855067985307e-7,"top_logprobs":[]},{"token":" I","bytes":[32,73],"logprob":0.0,"top_logprobs":[]},{"token":" can","bytes":[32,99,97,110],"logprob":-0.005181930027902126,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1b2

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" make","tool_calls":null},"logprobs":{"content":[{"token":" make","bytes":[32,109,97,107,101],"logprob":-1.553441047668457,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2e0

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" the function call.","tool_calls":null},"logprobs":{"content":[{"token":" the","bytes":[32,116,104,101],"logprob":-2.3841855067985307e-7,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.0012218636693432927,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-4.768370445162873e-7,"top_logprobs":[]},{"token":".","bytes":[46],"logprob":-0.13067495822906494,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

37e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"call_d7c04620e8f14225b715c800","index":0,"type":"function","function":{"name":"get_weather","arguments":"{\"city\": \"Beijing\"}"}}]},"logprobs":{"content":[{"token":"Be","bytes":[66,101],"logprob":-0.000016331539882230572,"top_logprobs":[]},{"token":"ijing","bytes":[105,106,105,110,103],"logprob":0.0,"top_logprobs":[]},{"token":"</arg_value>","bytes":[60,47,97,114,103,95,118,97,108,117,101,62],"logprob":-4.768370445162873e-7,"top_logprobs":[]},{"token":"</tool_call>","bytes":[60,47,116,111,111,108,95,99,97,108,108,62],"logprob":-1.1920928244535389e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

153

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":"tool_calls","matched_stop":151338}],"usage":null,"metadata":null}

e

data: [DONE]

0

RAW_BUFFERClick to expand / collapse

Your current environment

vLLM version: v0.15.0, v0.17.1
Serving mode: OpenAI-compatible API server
Model: Qwen/Qwen3-VL-8B-Thinking
Request mode: streaming chat completions with logprobs=true

🐛 Describe the bug

We reproduced this with Qwen/Qwen3-VL-8B-Thinking.

The tool call is parsed successfully, but the <tool_call> token is missing from streamed logprobs.

We observed the same issue on both vLLM v0.15.0 and v0.17.1.

This is intermittent. With the same request, it reproduces roughly once every 3 to 5 runs.

This depends on streaming chunk boundaries. vLLM may return multiple tokens in one streamed chunk, and that grouping can vary across runs.

If <tool_call> is grouped with visible reasoning or content tokens, the chunk is emitted and its logprob is preserved. If <tool_call> falls into a chunk that only contains tool-call control tokens, the parser may suppress that chunk, and the <tool_call> logprob is dropped even though tool parsing still succeeds.

Expected:

the stream includes the tool call delta
the corresponding <tool_call> token also appears in choices[0].logprobs.content

Actual:

the stream includes delta.tool_calls
<tool_call> is missing from choices[0].logprobs.content
when it reproduces, the next chunk may already contain tool-call arguments or closing tags, which shows that tool parsing succeeded even though the <tool_call> logprob was dropped

This happens because some streaming tool parsers suppress control-token chunks by returning None, and the chat streaming path skips the whole chunk.

Minimal repro:

BASE_URL=http://127.0.0.1:8000

curl -sN "$BASE_URL/v1/chat/completions" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-VL-8B-Thinking",
    "stream": true,
    "logprobs": true,
    "tool_choice": "auto",
    "messages": [
      {"role": "system", "content": "If a tool is useful, call it immediately."},
      {"role": "user", "content": "Please call get_weather for Beijing. Do not explain."}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather by city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

Chunk list:

14e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1ab

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"The","tool_calls":null},"logprobs":{"content":[{"token":"The","bytes":[84,104,101],"logprob":-0.0009746531140990555,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2df

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" user is asking me","tool_calls":null},"logprobs":{"content":[{"token":" user","bytes":[32,117,115,101,114],"logprob":-0.000022053474822314456,"top_logprobs":[]},{"token":" is","bytes":[32,105,115],"logprob":-0.2891930043697357,"top_logprobs":[]},{"token":" asking","bytes":[32,97,115,107,105,110,103],"logprob":-0.012495091184973717,"top_logprobs":[]},{"token":" me","bytes":[32,109,101],"logprob":-0.000013351351299206726,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2d0

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" to call the get","tool_calls":null},"logprobs":{"content":[{"token":" to","bytes":[32,116,111],"logprob":-4.6491513785440475e-6,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-0.001095529063604772,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-0.014845846220850945,"top_logprobs":[]},{"token":" get","bytes":[32,103,101,116],"logprob":-0.0011074369540438056,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2a6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"_weather function for","tool_calls":null},"logprobs":{"content":[{"token":"_weather","bytes":[95,119,101,97,116,104,101,114],"logprob":-0.00011359999916749075,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-4.529942543740617e-6,"top_logprobs":[]},{"token":" for","bytes":[32,102,111,114],"logprob":-0.011603484861552715,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

212

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" Beijing.","tool_calls":null},"logprobs":{"content":[{"token":" Beijing","bytes":[32,66,101,105,106,105,110,103],"logprob":-0.011142883449792862,"top_logprobs":[]},{"token":".","bytes":[46],"logprob":-0.505378246307373,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

246

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" They specifically","tool_calls":null},"logprobs":{"content":[{"token":" They","bytes":[32,84,104,101,121],"logprob":-0.036309339106082916,"top_logprobs":[]},{"token":" specifically","bytes":[32,115,112,101,99,105,102,105,99,97,108,108,121],"logprob":-0.2774711847305298,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2b6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" say \"Do not","tool_calls":null},"logprobs":{"content":[{"token":" say","bytes":[32,115,97,121],"logprob":-0.9159097671508789,"top_logprobs":[]},{"token":" \"","bytes":[32,34],"logprob":-0.006729202810674906,"top_logprobs":[]},{"token":"Do","bytes":[68,111],"logprob":-0.000603493710514158,"top_logprobs":[]},{"token":" not","bytes":[32,110,111,116],"logprob":-5.960462772236497e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c1

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" explain\" so I","tool_calls":null},"logprobs":{"content":[{"token":" explain","bytes":[32,101,120,112,108,97,105,110],"logprob":-6.437280717364047e-6,"top_logprobs":[]},{"token":"\"","bytes":[34],"logprob":-0.024111026898026466,"top_logprobs":[]},{"token":" so","bytes":[32,115,111],"logprob":-0.5177842378616333,"top_logprobs":[]},{"token":" I","bytes":[32,73],"logprob":-0.03217101842164993,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

28e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" should just make","tool_calls":null},"logprobs":{"content":[{"token":" should","bytes":[32,115,104,111,117,108,100],"logprob":-0.00022718709078617394,"top_logprobs":[]},{"token":" just","bytes":[32,106,117,115,116],"logprob":-0.008903564885258675,"top_logprobs":[]},{"token":" make","bytes":[32,109,97,107,101],"logprob":-0.04116453230381012,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

30d

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" the function call without","tool_calls":null},"logprobs":{"content":[{"token":" the","bytes":[32,116,104,101],"logprob":-2.0265558760002023e-6,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.00006770858453819528,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-4.6491513785440475e-6,"top_logprobs":[]},{"token":" without","bytes":[32,119,105,116,104,111,117,116],"logprob":-0.010122864507138729,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

234

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" any additional","tool_calls":null},"logprobs":{"content":[{"token":" any","bytes":[32,97,110,121],"logprob":-0.07130363583564758,"top_logprobs":[]},{"token":" additional","bytes":[32,97,100,100,105,116,105,111,110,97,108],"logprob":-0.1052059456706047,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1d6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" commentary","tool_calls":null},"logprobs":{"content":[{"token":" commentary","bytes":[32,99,111,109,109,101,110,116,97,114,121],"logprob":-1.7721552848815918,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

21b

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":".\n\nLooking","tool_calls":null},"logprobs":{"content":[{"token":".\n\n","bytes":[46,10,10],"logprob":-0.1671697348356247,"top_logprobs":[]},{"token":"Looking","bytes":[76,111,111,107,105,110,103],"logprob":-0.41703465580940247,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

311

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" at the function definition","tool_calls":null},"logprobs":{"content":[{"token":" at","bytes":[32,97,116],"logprob":-1.1920928244535389e-7,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-1.1920928244535389e-7,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.0059434822760522366,"top_logprobs":[]},{"token":" definition","bytes":[32,100,101,102,105,110,105,116,105,111,110],"logprob":-1.1973540782928467,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

270

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":":\n- Function","tool_calls":null},"logprobs":{"content":[{"token":":\n","bytes":[58,10],"logprob":-0.0005900310934521258,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-2.145764938177308e-6,"top_logprobs":[]},{"token":" Function","bytes":[32,70,117,110,99,116,105,111,110],"logprob":-0.014584832824766636,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2cb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" name: get_weather","tool_calls":null},"logprobs":{"content":[{"token":" name","bytes":[32,110,97,109,101],"logprob":-0.006715822499245405,"top_logprobs":[]},{"token":":","bytes":[58],"logprob":0.0,"top_logprobs":[]},{"token":" get","bytes":[32,103,101,116],"logprob":-0.0012191252317279577,"top_logprobs":[]},{"token":"_weather","bytes":[95,119,101,97,116,104,101,114],"logprob":-0.00006210611172718927,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2eb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"\n- Required parameter","tool_calls":null},"logprobs":{"content":[{"token":"\n","bytes":[10],"logprob":-0.001348182326182723,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-9.059865078597795e-6,"top_logprobs":[]},{"token":" Required","bytes":[32,82,101,113,117,105,114,101,100],"logprob":-0.05764629691839218,"top_logprobs":[]},{"token":" parameter","bytes":[32,112,97,114,97,109,101,116,101,114],"logprob":-0.11494840681552887,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c5

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":": city (string","tool_calls":null},"logprobs":{"content":[{"token":":","bytes":[58],"logprob":-3.576278118089249e-7,"top_logprobs":[]},{"token":" city","bytes":[32,99,105,116,121],"logprob":-0.00008034383063204587,"top_logprobs":[]},{"token":" (","bytes":[32,40],"logprob":-0.003986626863479614,"top_logprobs":[]},{"token":"string","bytes":[115,116,114,105,110,103],"logprob":-0.028655385598540306,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2bb

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":")\n- The user","tool_calls":null},"logprobs":{"content":[{"token":")\n","bytes":[41,10],"logprob":-0.0036669413093477488,"top_logprobs":[]},{"token":"-","bytes":[45],"logprob":-3.6954811548639555e-6,"top_logprobs":[]},{"token":" The","bytes":[32,84,104,101],"logprob":-0.030071474611759186,"top_logprobs":[]},{"token":" user","bytes":[32,117,115,101,114],"logprob":-0.0005220004240982234,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

275

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" provided \"Be","tool_calls":null},"logprobs":{"content":[{"token":" provided","bytes":[32,112,114,111,118,105,100,101,100],"logprob":-0.7353255152702332,"top_logprobs":[]},{"token":" \"","bytes":[32,34],"logprob":-0.014822943136096,"top_logprobs":[]},{"token":"Be","bytes":[66,101],"logprob":-0.000030397906812140718,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2c6

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":"ijing\" as the","tool_calls":null},"logprobs":{"content":[{"token":"ijing","bytes":[105,106,105,110,103],"logprob":-7.867782187531702e-6,"top_logprobs":[]},{"token":"\"","bytes":[34],"logprob":-0.00002276871418871451,"top_logprobs":[]},{"token":" as","bytes":[32,97,115],"logprob":-0.000049470632802695036,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-2.3841855067985307e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2bf

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" city\n\nI have","tool_calls":null},"logprobs":{"content":[{"token":" city","bytes":[32,99,105,116,121],"logprob":-0.0008453133050352335,"top_logprobs":[]},{"token":"\n\n","bytes":[10,10],"logprob":-0.04928658530116081,"top_logprobs":[]},{"token":"I","bytes":[73],"logprob":-0.015265769325196743,"top_logprobs":[]},{"token":" have","bytes":[32,104,97,118,101],"logprob":-0.012821147218346596,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

314

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" all the required parameters","tool_calls":null},"logprobs":{"content":[{"token":" all","bytes":[32,97,108,108],"logprob":-0.0007233387441374362,"top_logprobs":[]},{"token":" the","bytes":[32,116,104,101],"logprob":-0.0006307758158072829,"top_logprobs":[]},{"token":" required","bytes":[32,114,101,113,117,105,114,101,100],"logprob":-0.00518774101510644,"top_logprobs":[]},{"token":" parameters","bytes":[32,112,97,114,97,109,101,116,101,114,115],"logprob":-0.6931779384613037,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

296

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":", so I can","tool_calls":null},"logprobs":{"content":[{"token":",","bytes":[44],"logprob":-0.0372004471719265,"top_logprobs":[]},{"token":" so","bytes":[32,115,111],"logprob":-2.3841855067985307e-7,"top_logprobs":[]},{"token":" I","bytes":[32,73],"logprob":0.0,"top_logprobs":[]},{"token":" can","bytes":[32,99,97,110],"logprob":-0.005181930027902126,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

1b2

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" make","tool_calls":null},"logprobs":{"content":[{"token":" make","bytes":[32,109,97,107,101],"logprob":-1.553441047668457,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

2e0

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":" the function call.","tool_calls":null},"logprobs":{"content":[{"token":" the","bytes":[32,116,104,101],"logprob":-2.3841855067985307e-7,"top_logprobs":[]},{"token":" function","bytes":[32,102,117,110,99,116,105,111,110],"logprob":-0.0012218636693432927,"top_logprobs":[]},{"token":" call","bytes":[32,99,97,108,108],"logprob":-4.768370445162873e-7,"top_logprobs":[]},{"token":".","bytes":[46],"logprob":-0.13067495822906494,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

37e

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"call_d7c04620e8f14225b715c800","index":0,"type":"function","function":{"name":"get_weather","arguments":"{\"city\": \"Beijing\"}"}}]},"logprobs":{"content":[{"token":"Be","bytes":[66,101],"logprob":-0.000016331539882230572,"top_logprobs":[]},{"token":"ijing","bytes":[105,106,105,110,103],"logprob":0.0,"top_logprobs":[]},{"token":"</arg_value>","bytes":[60,47,97,114,103,95,118,97,108,117,101,62],"logprob":-4.768370445162873e-7,"top_logprobs":[]},{"token":"</tool_call>","bytes":[60,47,116,111,111,108,95,99,97,108,108,62],"logprob":-1.1920928244535389e-7,"top_logprobs":[]}]},"finish_reason":null,"matched_stop":null}],"usage":null,"metadata":null}

153

data: {"id":"841e4a9a29a7402a82f537ae3808a916","object":"chat.completion.chunk","created":1774074676,"model":"Qwen/Qwen3-VL-8B-Thinking","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":"tool_calls","matched_stop":151338}],"usage":null,"metadata":null}

e

data: [DONE]

0

</details>

This is not specific to one parser. Any streaming tool parser that returns None for control tokens can trigger the same behavior.

Before submitting a new issue...

Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To fix the issue of missing <tool_call> tokens in streamed logprobs, we need to modify the chat streaming path to handle chunks with control tokens correctly. Here are the steps:

Modify the streaming_tool_parser function to return an empty list [] instead of None when encountering control tokens. This will ensure that the chunk is not skipped.
Update the chat_streaming_path function to handle chunks with control tokens by adding a conditional statement to check if the tool_calls field is not empty. If it's not empty, add the <tool_call> token to the logprobs content.

Example code:

def streaming_tool_parser(tokens):
    # ...
    if is_control_token(token):
        return []  # Return an empty list instead of None
    # ...

def chat_streaming_path(chunk):
    # ...
    if chunk['delta']['tool_calls']:
        logprobs_content = chunk['logprobs']['content']
        logprobs_content.append({'token': '<tool_call>', 'logprob': 0.0})
    # ...

Additionally, we can add a check in the chat_streaming_path function to ensure that the <tool_call> token is only added once per chunk.

Verification

To verify that the fix worked, we can test the chat streaming API with the same request that reproduced the issue. We should see the <tool_call> token included in the logprobs content.

Extra Tips

Make sure to test the fix with different streaming tool parsers to ensure that the issue is resolved for all cases.
Consider adding a test case to the chat streaming API to cover this scenario and prevent regressions in the future.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #inference speed #output truncation #response parsing #generation error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [Bug]: Missing logprobs for `<tool_call>` in streaming chat completions [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #37738: Fix: preserve streaming logprobs

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [Bug]: Missing logprobs for `<tool_call>` in streaming chat completions [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #37738: Fix: preserve streaming logprobs

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING