vllm - 💡(How to fix) Fix [Bug]: [MiniMax] Function call content is not streamed incrementally; buffered until generation completes [1 comments, 2 participants]

Your current environment

Environment

vLLM version: 0.18.0 Model: MiniMax-M2.5

🐛 Describe the bug

Description

When deploying the MiniMax model via vLLM with streaming enabled (stream: true), the function call (or tool call) payload is not returned token-by-token. Instead, vLLM appears to buffer the entire function call content and only emits it after a noticeable delay, typically right after the generation finishes. This breaks streaming consumers that expect incremental tool_calls.function.arguments chunks.

Steps to Reproduce

1. Serve the MiniMax model using vLLM: vllm serve <model_path_or_name> --enable-auto-tool-choice ...
2. Send a streaming request via the OpenAI-compatible API with a prompt that triggers a function/tool call.

{
  "model": "<model_name>",
  "messages": [{"role": "user", "content": "Please call the weather function for Beijing"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}}],
  "stream": true
}

1. Observe the SSE/stream output.

Expected Behavior

Function/tool call tokens (e.g., JSON arguments) should be streamed incrementally as they are generated, consistent with the OpenAI streaming specification and vLLM’s behavior for other supported models (e.g., Qwen, Llama).

Actual Behavior

Normal text streams normally, but once the model begins generating the function call, the stream pauses/buffers. After generation completes (or after a delay), the entire function call payload is returned in a single chunk (or very few chunks), rather than incrementally.

Additional Context

This issue appears specific to MiniMax. Other tool-calling models served via the same vLLM version stream function calls correctly.

We observed that happends both on GPU and NPU.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.