Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Bug]: DeepSeek V4 DSML tool parser mishandles typed and wrapped arguments [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#41240•Fetched 2026-04-30 06:19:23
View on GitHub
Comments
1
Participants
2
Timeline
7
Reactions
0
Author
QwertyJackQwertyJack
Participants
QwertyJackQwertyJackrobertgshaw2-redhatrobertgshaw2-redhat
Timeline (top)
mentioned ×2subscribed ×2commented ×1cross-referenced ×1

Root Cause

  • typed DSML parameters such as string="false", where numbers, booleans, arrays, and objects should be preserved as JSON values rather than returned as strings
  • model-emitted single arguments or input wrapper parameters when those names are not actual fields in the requested tool schema
  • real tool schema parameters named arguments, which can be confused with OpenAI tool-call wrapper semantics
  • streaming responses that end while plain text is being held because it looks like the start of a DSML marker, for example 2 <

Fix Action

Fixed

PR fix notes

PR #41241: Fix DeepSeek V4 DSML tool argument parsing

Description (problem / solution / changelog)

Related to #41240

Summary

  • fix the existing deepseek_v4 tool parser to preserve DSML parameter types from string="false" values instead of returning every parameter as a string
  • unwrap single arguments / input wrapper parameters when the wrapper is not part of the requested tool schema and the wrapped object matches the schema fields
  • escape real tool parameters named arguments while rendering DeepSeek V4 tool schemas/history, then unescape them when parsing model output
  • flush held plain text at the end of streaming when it only looked like the start of a DSML marker, such as 2 <
  • add focused parser, tokenizer, and CLI validation coverage for the DeepSeek V4 serving flag combination

This PR does not add the deepseek_v4 parser from scratch; upstream already has the parser registered. It narrows the existing implementation so DeepSeek V4 DSML tool calls round-trip correctly in edge cases seen with typed parameters and wrapper fields.

This PR also does not add top-level thinking={...} request support. Per review feedback, DeepSeek V4 thinking toggles should continue to use chat_template_kwargs, matching the vLLM DeepSeek V4 recipe.

Duplicate-work check

  • Reviewed issue #41240 with gh issue view 41240 --repo vllm-project/vllm --comments.
  • Checked open PRs with gh pr list --repo vllm-project/vllm --state open --search "41240 in:body". No existing PR references the issue.
  • Checked open PRs with gh pr list --repo vllm-project/vllm --state open --search "deepseek_v4 DSML tool parser". The only other match was #40991, which is DSv4 Nvidia/model enablement and does not address this parser behavior.
  • Related issue #41132 covers structured output routed into reasoning when thinking is enabled; this PR is narrower and only targets DeepSeek V4 DSML tool-call parsing/tokenizer round-tripping.

Tests

  • .venv/bin/python -m pytest tests/entrypoints/openai/chat_completion/test_chat.py::test_chat_completion_request_accepts_model_specific_reasoning_effort tests/tool_parsers/test_deepseekv4_tool_parser.py tests/tokenizers_/test_deepseek_v4.py tests/reasoning/test_deepseekv3_reasoning_parser.py::test_deepseek_v4_reasoning_parser_alias tests/entrypoints/openai/test_cli_args.py::test_deepseek_v4_agentic_flags_pass_validation -q
  • .venv/bin/ruff check vllm/entrypoints/openai/chat_completion/protocol.py tests/entrypoints/openai/chat_completion/test_chat.py vllm/tool_parsers/deepseekv4_tool_parser.py vllm/tokenizers/deepseek_v4_encoding.py tests/tool_parsers/test_deepseekv4_tool_parser.py tests/tokenizers_/test_deepseek_v4.py tests/entrypoints/openai/test_cli_args.py
  • pre-commit hooks run during git commit --amend and passed

AI assistance

AI assistance was used to develop and validate this change. I reviewed the changed lines and test results.

Changed files

  • tests/entrypoints/openai/test_cli_args.py (modified, +26/-0)
  • tests/tokenizers_/test_deepseek_v4.py (modified, +53/-0)
  • tests/tool_parsers/test_deepseekv4_tool_parser.py (modified, +136/-1)
  • vllm/tokenizers/deepseek_v4_encoding.py (modified, +74/-7)
  • vllm/tool_parsers/deepseekv4_tool_parser.py (modified, +257/-0)

Code Example

--tokenizer-mode deepseek_v4 \
--tool-call-parser deepseek_v4 \
--enable-auto-tool-choice \
--reasoning-parser deepseek_v4
RAW_BUFFERClick to expand / collapse

Current behavior

vLLM already has a registered deepseek_v4 tokenizer mode, tool-call parser, and reasoning parser. However, the current DeepSeek V4 tool parser is mostly a thin wrapper around the DeepSeek V3.2 parser with different DSML tool-call boundary tokens, and it misses several DeepSeek V4 DSML argument edge cases.

With the documented serving flag combination:

--tokenizer-mode deepseek_v4 \
--tool-call-parser deepseek_v4 \
--enable-auto-tool-choice \
--reasoning-parser deepseek_v4

DeepSeek V4 DSML tool calls should round-trip OpenAI-compatible tool arguments correctly.

Problems

The existing parser/tokenizer behavior can mishandle:

  • typed DSML parameters such as string="false", where numbers, booleans, arrays, and objects should be preserved as JSON values rather than returned as strings
  • model-emitted single arguments or input wrapper parameters when those names are not actual fields in the requested tool schema
  • real tool schema parameters named arguments, which can be confused with OpenAI tool-call wrapper semantics
  • streaming responses that end while plain text is being held because it looks like the start of a DSML marker, for example 2 <

Expected behavior

The DeepSeek V4 tokenizer/parser path should:

  • parse string="false" parameter values using the requested schema where available, falling back to JSON parsing
  • unwrap single arguments / input wrapper objects only when the wrapper is not a real schema field and the wrapped object matches the schema fields
  • safely render and parse tool schemas/history containing a real parameter named arguments
  • flush held plain text at the end of a stream when no DSML tool call actually follows

Notes

This issue is about DeepSeek V4 DSML tool-call parser/tokenizer correctness. It does not require adding the deepseek_v4 parser from scratch, since that parser is already present upstream.

DeepSeek's official top-level thinking={"type": "enabled"|"disabled"} request shape is a related API-compatibility topic, but per review feedback it is not part of the current fix. vLLM users should continue to use chat_template_kwargs for DeepSeek V4 thinking toggles, matching the vLLM DeepSeek V4 recipe.

Related PR

Proposed fix: #41241

extent analysis

TL;DR

The DeepSeek V4 DSML tool-call parser/tokenizer needs to be updated to correctly handle typed parameters, unwrap single wrapper objects, and safely render tool schemas with a parameter named arguments.

Guidance

  • Review the proposed fix in PR #41241 to address the parser/tokenizer correctness issues.
  • Verify that the updated parser/tokenizer correctly handles typed DSML parameters, such as string="false", by preserving numbers, booleans, arrays, and objects as JSON values.
  • Test the unwrap logic for single arguments / input wrapper objects to ensure it only unwraps when the wrapper is not a real schema field and the wrapped object matches the schema fields.
  • Check that the updated parser/tokenizer safely renders and parses tool schemas/history containing a real parameter named arguments.

Example

No code snippet is provided as the issue does not contain sufficient code context.

Notes

The fix should focus on updating the existing DeepSeek V4 parser/tokenizer to address the mentioned correctness issues, without adding new functionality or modifying unrelated components.

Recommendation

Apply the workaround by reviewing and applying the proposed fix in PR #41241, as it addresses the specific parser/tokenizer correctness issues mentioned in the issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

The DeepSeek V4 tokenizer/parser path should:

  • parse string="false" parameter values using the requested schema where available, falling back to JSON parsing
  • unwrap single arguments / input wrapper objects only when the wrapper is not a real schema field and the wrapped object matches the schema fields
  • safely render and parse tool schemas/history containing a real parameter named arguments
  • flush held plain text at the end of a stream when no DSML tool call actually follows
#api#API routing#API middleware#SSR setup#ISR setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - ✅(Solved) Fix [Bug]: DeepSeek V4 DSML tool parser mishandles typed and wrapped arguments [1 pull requests, 1 comments, 2 participants]