vllm - 💡(How to fix) Fix [Feature]: Support responses api in tokenize endpoint [1 comments, 2 participants]

TL;DR

The /tokenize API should be updated to support responses API input in addition to chat completions input.

Guidance

• Review the documentation for the /tokenize endpoint to ensure it accurately reflects the supported input formats.
• Consider updating the /tokenize endpoint to accept responses API input, allowing for more flexible tokenization.
• Verify that the updated endpoint can handle both chat completions and responses API input correctly.
• Check the discussion thread https://discuss.vllm.ai/t/tell-me-about-the-current-status-of-the-tokenize-endpoint-in-vllm/2089 for any additional context or insights.

Notes

The issue lacks technical details about the current implementation of the /tokenize endpoint, so a more detailed solution cannot be provided.

Recommendation

Apply workaround: Update the /tokenize endpoint to support responses API input, as this would provide more flexibility and consistency in tokenization.