vllm - 💡(How to fix) Fix [Bug]: Tool call leaks into content [3 comments, 2 participants]

TL;DR

The issue can be fixed by properly handling the "current_text" variable to prevent it from containing parts of the tool call token when transitioning from reasoning to tool call.

Guidance

• Review the code at the specified line in abstract_parser.py to understand how "current_text" is being updated and ensure it is correctly reset or cleaned up before making a tool call.
• Verify that the tool call recognition logic is correctly implemented and that the presence of "<|" in "current_text" is the root cause of the issue.
• Consider adding a check or a cleanup step before making a tool call to remove any unwanted characters from "current_text".
• Investigate if there are any existing functions or methods in the abstract_parser.py file that can be used to sanitize or normalize the "current_text" variable.

Notes

The provided information does not include the exact code changes or the desired behavior of the tool call recognition logic, so a precise fix cannot be provided.

Recommendation

Apply workaround: Implement a cleanup step to remove any unwanted characters from "current_text" before making a tool call, as this seems to be the most direct way to address the issue based on the provided information.