litellm - 💡(How to fix) Fix [Bug]: reasoning with recent vllm backends does not work [1 participants]

TL;DR

Update the LiteLLM version or modify the model to not rely on the deprecated reasoning_content tag, as VLLM now returns the reasoning tag instead.

Guidance

• Verify that the issue persists with the latest version of LiteLLM, as the problem may have been addressed in a more recent release.
• Check the documentation for VLLM (0.19.0) to ensure that the reasoning tag is correctly handled in streaming mode.
• Consider modifying the model to use the new reasoning tag instead of the deprecated reasoning_content tag.
• Test the model with a different provider to determine if the issue is specific to VLLM.

Notes

The issue may be specific to the combination of LiteLLM (v1.81.12) and VLLM (0.19.0), and updating either component may resolve the problem.

Recommendation

Apply workaround: Modify the model to use the new reasoning tag, as updating to a fixed version is not explicitly mentioned in the issue.