vllm - 💡(How to fix) Fix [Bug]: Using vLLM to deploy Minimax m2.5, the thinking/reasoning cannot be disable. [3 comments, 3 participants]

Fix Plan

To disable thinking/reasoning in vLLM, you can try the following steps:

• Set enable_thinking to false in the model configuration file.
• Use the --no-reasoning flag when deploying the model.
• Update the chat_template_kwargs to include "enable_thinking": false and ensure it is being passed to the model correctly.

Example code snippet:

model_config = {
    "enable_thinking": False
}

chat_template_kwargs = {
    "enable_thinking": False
}

# Deploy the model with the updated configuration
vllm.deploy(model_config, chat_template_kwargs=chat_template_kwargs)

Alternatively, you can try using the --no-reasoning flag when deploying the model:

vllm deploy --no-reasoning --model-config model_config.json

Verification

To verify that the fix worked, you can test the model by sending a query and checking if the thinking/reasoning mode is disabled. You can do this by checking the model's output for any indicators of thinking/reasoning, such as a "thinking" or "reasoning" prompt.

Extra Tips

• Make sure to update the correct configuration file and pass the correct arguments when deploying the model.
• If you are still experiencing issues, try checking the vLLM documentation and FAQs for any known issues or workarounds.