PR #33248: Kimi K2.5 model generates "(no content)" placeholder in tool call responses

• Repository: vllm-project/vllm
• Author: shivamashtikar
• State: closed | merged: False
• Link: https://github.com/vllm-project/vllm/pull/33248

Description (problem / solution / changelog)

Purpose

Fixes the issue where Kimi K2.5 model generates "(no content)" placeholder text in responses when making tool calls without accompanying text content. This placeholder leaks through to clients in both non-streaming and streaming modes.

Key Changes:

1. kimi_k2_reasoning_parser.py:
   • Add _clean_content() method to strip "(no content)" placeholder
   • Filter placeholder from extract_reasoning() output
   • Filter placeholder from extract_reasoning_streaming() DeltaMessage results
   • Return None when content becomes empty after cleaning
   • Preserve reasoning and tool_calls when content is filtered
2. kimi_k2_tool_parser.py:
   • Add _clean_content() method to strip "(no content)" placeholder
   • Add _strip_all_tool_markers() to prevent marker leakage into content
   • Filter placeholder from all extraction methods (non-tool, tool call, error paths)
   • Increase buffer_max_size from 1024 to 4096 to handle larger tool arguments
   • Add regex pattern to strip thinking blocks that may leak into content
   • Improve streaming content handling:
     • Extract and return content before tool section markers
     • Return None instead of empty DeltaMessage for cleaned content
     • Strip all markers from post-section content
   • Fix IndexError by initializing prev_tool_call_arr before returning
   • Strip whitespace from tool_call_portion for better regex matching
   • Fix array bounds checking in tool call state management
   • Add deferred_section_exit handling in exception handler

Before this fix:

{ choices: [{ message: { content: (no content), tool_calls: [...] } }] }

After this fix:

{ choices: [{ message: { content: null, tool_calls: [...] } }] }

Test Plan

To test this fix, run the following tests with Kimi K2/K2.5 model using tool calls:

Test 1: Non-streaming tool call without content

python -c "
from vllm import LLM, SamplingParams
from vllm.schemas import ChatMessage, Tool
llm = LLM(model='moonshot-v1-8k')
messages = [
    ChatMessage(role='user', content='What is the weather in Tokyo?'),
]
tools = [
    Tool(type='function', function={
        'name': 'get_weather',
        'description': 'Get weather for a location',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {'type': 'string'}
            },
            'required': ['location']
        }
    })
]
outputs = llm.chat(messages, tools=tools)
print(outputs[0].outputs[0].text)
# Should NOT contain '(no content)' in the text
"

Test 2: Streaming tool call without content

python -c "
from vllm import LLM, SamplingParams
from vllm.schemas import ChatMessage, Tool
llm = LLM(model='moonshot-v1-8k')
messages = [
    ChatMessage(role='user', content='What is the weather in Tokyo?'),
]
tools = [
    Tool(type='function', function={
        'name': 'get_weather',
        'description': 'Get weather for a location',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {'type': 'string'}
            },
            'required': ['location']
        }
    })
]
sampling_params = SamplingParams(temperature=0.0, max_tokens=256)
stream = llm.chat(messages, tools=tools, sampling_params=sampling_params, stream=True)
for output in stream:
    for delta in output.outputs[0].delta:
        # Verify no chunk contains '(no content)'
        assert '(no content)' not in str(delta.content), f'Found placeholder in delta: {delta}'
        print(delta)
"

Test 3: Existing tests should still pass

pytest tests/tool_parsers/test_kimi_k2_tool_parser.py -v pytest tests/reasoning/test_kimi_k2_reasoning_parser.py -v Test Result Expected Results:

1. Non-streaming test:
   • Response content should be null or empty string (not "(no content)")
   • Tool calls should be properly extracted
   • No placeholder text appears in the response
2. Streaming test:
   • No delta chunk contains "(no content)" in content
   • Empty content chunks are skipped (return None)
   • Tool calls are properly parsed and returned
   • Content before tool markers is correctly extracted
3. Existing tests:
   • All existing parser tests should continue to pass
   • No regression in functionality Manual Testing Observations:

• Verified that "(no content)" placeholder is filtered from all response paths
• Confirmed tool markers and thinking blocks do not leak into content
• Streaming responses no longer emit empty chunks
• Parser handles edge cases without errors

Changed files

• vllm/reasoning/kimi_k2_reasoning_parser.py (modified, +47/-2)
• vllm/tool_parsers/kimi_k2_tool_parser.py (modified, +132/-27)