vllm - 💡(How to fix) Fix [BUG]: Detect presence of p2p enable gpus/driver, not just nvlink, to enable direct connection [1 participants]

TL;DR

Modify the is_fully_connected function to check for direct GPU communication using nvidia-smi topo -p2p r to potentially improve performance on multi-GPU setups.

Guidance

• Investigate the nvidia-smi topo -p2p r command to understand its output and how it can be used to verify direct GPU communication.
• Review the patched driver from the provided GitHub repository to understand the changes made to enable better performance on multi-GPU setups.
• Consider modifying the is_fully_connected function to incorporate the nvidia-smi topo -p2p r check, potentially using the subprocess module to execute the command and parse its output.
• Test the modified function with the patched driver to verify the performance improvement.

Example

import subprocess

def is_fully_connected(cls, physical_device_ids: list[int]) -> bool:
    # ... existing code ...
    # Execute nvidia-smi topo -p2p r and parse output to verify direct GPU communication
    output = subprocess.check_output(["nvidia-smi", "topo", "-p2p", "r"]).decode("utf-8")
    # ... parse output and update function logic ...

Notes

The provided information suggests that the current implementation of is_fully_connected does not account for direct GPU communication, which can impact performance on multi-GPU setups. However, the exact modifications required to the function are not specified and may require further investigation.

Recommendation

Apply workaround by modifying the is_fully_connected function to incorporate the nvidia-smi topo -p2p r check, as this may provide a performance improvement on multi-GPU setups.