vllm - 💡(How to fix) Fix [Bug]: reasoning_effort passed to MistralCommonTokenizer.apply_chat_template breaks Mistral Small 4 chat completions on vLLM 0.18.0 [2 comments, 2 participants]

Your current environment

Environment vLLM: 0.18.0 (e.g. Docker image vllm/vllm-openai or internal vllm-audio:v0.18.0) Model: mistralai/Mistral-Small-4-119B-2603 (or equivalent weights served with Mistral tokenizer path) Hardware / stack: (fill in: GPU type, CUDA, --tensor-parallel-size, etc.)

🐛 Describe the bug

Problem POST /v1/chat/completions fails with 400 and: ValueError: Kwargs ['reasoning_effort'] are not supported by MistralCommonTokenizer.apply_chat_template. The failure occurs in vLLM’s Mistral chat rendering path, e.g.: File ".../vllm/entrypoints/openai/chat_completion/serving.py", line 209, in render_chat_request return await self.openai_serving_render.render_chat(request) ... File ".../vllm/renderers/mistral.py", line 125, in render_messages_async prompt_raw = await self._apply_chat_template_async( ... File ".../vllm/renderers/mistral.py", line 34, in safe_apply_chat_template return tokenizer.apply_chat_template(messages, **kwargs) ... File ".../transformers/tokenization_mistral_common.py", line 1432, in apply_chat_template raise ValueError( ValueError: Kwargs ['reasoning_effort'] are not supported by MistralCommonTokenizer.apply_chat_template.

Reproduction Important: The request body does not need to include reasoning_effort. A minimal call is enough to trigger the error (LiteLLM is not required—we reproduced with direct HTTP to vLLM). Example:

curl -sS "http://<vllm-host>:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<served-model-id>",
    "messages": [{"role": "user", "content": "hi"}],
    "max_tokens": 16,
    "temperature": 0.0
  }'

Actual: HTTP 400, error message as above. Expected: Request accepted and completion generated (or a clear validation error only if the client sends unsupported fields). Context The Hugging Face chat template for Mistral Small 4 uses reasoning_effort in Jinja for [MODEL_SETTINGS], but MistralCommonTokenizer.apply_chat_template in Transformers rejects reasoning_effort as an unsupported kwarg when vLLM forwards it into that API. We confirmed the same failure when bypassing LiteLLM, so this is not proxy-specific. What would help vLLM should not pass reasoning_effort (and any other unsupported kwargs) into MistralCommonTokenizer.apply_chat_template, or align with Transformers / tokenizer behavior for Mistral 4–style templates. If there is an intended flag or server-side default for Mistral Small 4 + OpenAI chat API, documenting it would help.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.