vllm - 💡(How to fix) Fix [Feature]: Parity with CUDA: vLLM router should have ROCm CI [4 comments, 4 participants]

TL;DR

Add AMD GPU tests to the vLLM router to ensure compatibility and functionality.

Guidance

• Review the existing CUDA GPU tests in the vLLM router and adapt them for AMD GPU support.
• Implement tests for RIXL in the vLLM router, as AMD already supports this feature.
• Consider adding tests for MoRI kvcache transfer in the vLLM router once this feature is implemented.
• Examine the pipeline configuration in .buildkite/pipeline.yml to ensure it accommodates the new tests.

Notes

The lack of AMD GPU tests in the vLLM router may limit its functionality and compatibility with certain systems.

Recommendation

Apply workaround: Add AMD GPU tests to the vLLM router, as this will improve compatibility and ensure the feature works as expected on different hardware configurations.