vllm - 💡(How to fix) Fix [Bug]: Mismatch between batch queue size and max-num-seqs [1 participants]

TL;DR

Set the batch queue size and max-num-sequences to the same value as recommended by the official documentation to potentially resolve the issue.

Guidance

• Review the official documentation images provided to understand the recommended configuration for batch queue size and max-num-sequences.
• Compare the current configuration with the recommended settings and adjust them accordingly.
• Verify that the issue persists after updating the configuration to ensure the fix is effective.
• If the issue remains, consider seeking further assistance from the vllm community or support channels.

Notes

The provided images and documentation suggest a specific configuration, but without direct access to the images, it's challenging to provide a precise fix. The guidance is based on the assumption that the images contain relevant configuration recommendations.

Recommendation

Apply workaround: Update the batch queue size and max-num-sequences to match the recommended values, as this is the most direct action implied by the provided documentation and issue description.