vllm - 💡(How to fix) Fix [Bug]: Kwargs passed to `processor.__call__` have to be in `processor_kwargs` dict, not in `**kwargs` [1 comments, 2 participants]

TL;DR

Downgrade the transformers library to a version compatible with vllm, such as 5.0, to resolve the issue with passing kwargs to processor.__call__.

Guidance

• Verify the version of transformers currently installed using pip show transformers and check if it's indeed 5.3, which is reportedly causing the issue.
• Consider downgrading transformers to version 5.0 using pip install transformers==5.0 to align with vllm's supported version.
• Review the documentation for vllm and transformers to understand how to properly pass SamplingParams and other kwargs to processor.__call__ in a version-compatible manner.
• If downgrading is not feasible, explore alternative methods for passing necessary parameters that are compatible with both vllm and the current version of transformers.

Example

No specific code example can be provided without more context on how processor.__call__ and SamplingParams are being used. However, the focus should be on ensuring that kwargs are passed according to the specifications of the version of transformers being used.

Notes

The solution assumes that the issue is indeed caused by the version mismatch between vllm's supported transformers version (5.0) and the currently installed version (5.3). Other factors could potentially contribute to the problem, but based on the information provided, version compatibility is the most likely cause.

Recommendation

Apply workaround: Downgrade transformers to version 5.0. This is recommended because vllm explicitly states that it supports up to version 5.0 of transformers, and the error message suggests a version-related compatibility issue.