vllm - 💡(How to fix) Fix [CI Failure]: mi355_2: NixlConnector PD + Spec Decode acceptance (2 GPUs) [2 comments, 1 participants]

TL;DR

The most likely fix involves ensuring that all kv cache tensors have the same number of blocks, potentially by updating the spec_decode_acceptance_test.sh script or the code that generates these tensors.

Guidance

• Verify the test failure by running the spec_decode_acceptance_test.sh script locally and checking the output for any errors or inconsistencies in the kv cache tensors.
• Investigate the code that generates the kv cache tensors to ensure they are being created with the same number of blocks.
• Check the requirements/kv_connectors_rocm.txt file to see if any dependencies related to tensor creation or caching need to be updated.
• Review the test history and hardware status to determine if the issue is specific to certain hardware configurations.

Example

No specific code snippet can be provided without more information about the spec_decode_acceptance_test.sh script or the code that generates the kv cache tensors.

Notes

The fix may depend on the specific implementation of the spec_decode_acceptance_test.sh script and the code that generates the kv cache tensors, which is not provided in the issue.

Recommendation

Apply a workaround by modifying the spec_decode_acceptance_test.sh script to handle kv cache tensors with different numbers of blocks, or update the code that generates these tensors to ensure consistency. This is recommended because the issue seems to be related to a specific test case and hardware configuration, and a workaround may be a quicker solution than a full fix.