PR #38443: [Bugfix][Tool Parser] Fix Kimi-K2 streaming regex to handle leading newline before tool call ID

• Repository: vllm-project/vllm
• Author: saifmb0
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/38443

Description (problem / solution / changelog)

Summary

Fixes #38441.

The model (Kimi K2 / K2.5) occasionally emits a stray \n between <|tool_call_begin|> and the function name during streaming (observed on long-context inference with tool_choice: auto, without constrained decoding):

# What the model sometimes produces:
<|tool_call_begin|>
functions.edit:15<|tool_call_argument_begin|>{"path": "..."}

# Instead of:
<|tool_call_begin|>functions.edit:15<|tool_call_argument_begin|>{"path": "..."}

Because Python regex's . does not match \n by default, both stream_tool_call_portion_regex and stream_tool_call_name_regex silently failed to match, causing the tool call to be entirely dropped during streaming.

Root cause

# Before (broken on leading \n)
self.stream_tool_call_portion_regex = re.compile(
    r"(?P<tool_call_id>.+:\d+)\s*<\|tool_call_argument_begin\|>\s*(?P<function_arguments>.*)"
)
self.stream_tool_call_name_regex = re.compile(r"(?P<tool_call_id>.+:\d+)\s*")

Fix

# After
self.stream_tool_call_portion_regex = re.compile(
    r"\s*(?P<tool_call_id>.+:\d+)\s*"
    r"<\|tool_call_argument_begin\|>\s*(?P<function_arguments>.*)",
    re.DOTALL,
)
self.stream_tool_call_name_regex = re.compile(
    r"\s*(?P<tool_call_id>.+:\d+)\s*", re.DOTALL
)

Two changes per regex:

1. Leading \s* — consumes any leading whitespace/newlines before the function name.
2. re.DOTALL — makes . match \n so the tool_call_id capture group spans newlines.

Why this is not a duplicate

Checked open PRs: #37384, #37445, #32504, #24847, #26918, #36891.

• PR #37384 adds re.DOTALL only to stream_tool_call_portion_regex (for multi-line arguments), but does not add the leading \s* that handles a newline before the tool_call_id, and does not fix stream_tool_call_name_regex at all. The \s* prefix is the critical fix for this issue.
• No other open PR addresses stream_tool_call_name_regex.

Test plan

Two new tests added to tests/tool_parsers/test_kimi_k2_tool_parser.py:

• test_stream_tool_call_portion_regex_handles_leading_newline — unit test: both regexes must match with/without a leading \n, and correctly extract tool_call_id and function_arguments.
• test_streaming_tool_call_with_newline_after_begin_token — end-to-end streaming simulation of the exact failure scenario from the issue; asserts at least one tool-call delta is emitted (i.e., the tool call is not silently dropped).

Existing tests: pre-commit run ruff-format and pre-commit run ruff-check pass clean.

AI assistance disclosure

This fix was investigated, reproduced, and implemented with AI assistance (GitHub Copilot / Claude Sonnet 4.6). Every changed line has been reviewed. The reviewer (submitter) understands the change end-to-end.

Changed files

• tests/tool_parsers/test_kimi_k2_tool_parser.py (modified, +102/-0)
• vllm/tool_parsers/kimi_k2_tool_parser.py (modified, +8/-4)