vllm - 💡(How to fix) Fix [CI Failure]: mi325_2: Distributed Tests (2 GPUs)(H100-MI325) [3 comments, 2 participants]

TL;DR

Investigate the mi325_2: Distributed Tests (2 GPUs)(H100-MI325) test group to identify the cause of the failing tests on AMD hardware.

Guidance

• Review the test logs and output from the latest nightly build (#7791) to identify the specific tests that are failing and any error messages.
• Check the hardware status of mi325_2 to ensure that it is functioning correctly and that there are no issues with the GPUs or other components.
• Investigate any recent changes to the test code or environment that may be contributing to the failures.
• Verify that the sync_ready_tickets.py script is correctly configured and functioning as expected.

Notes

The issue appears to be specific to the mi325_2 test group on AMD hardware, and the cause of the failures is not immediately clear. Further investigation is needed to determine the root cause of the issue.

Recommendation

Apply workaround: Investigate and address the root cause of the test failures, as the issue is likely specific to the test environment or code rather than a versioning problem.