vllm - 💡(How to fix) Fix [Usage]: Using RIXL Connector on AMD GPU [2 comments, 3 participants]

Fix Plan

To resolve the issue with the Nixl connector on ROCm, we need to ensure that the RIXL library is installed in the Docker image.

Steps to Fix

• Option 1: Use a custom Dockerfile
  1. Create a new Dockerfile based on the official vllm/vllm-openai-rocm:latest image.
  2. Install RIXL using pip: RUN pip install rixl
  3. Build a new Docker image using the custom Dockerfile.
• Option 2: Modify the existing Docker image
  1. Pull the official vllm/vllm-openai-rocm:latest image.
  2. Run a container from the image and install RIXL using pip: docker exec -it <container_id> pip install rixl
  3. Commit the changes to create a new image.

Example Code

# Custom Dockerfile
FROM vllm/vllm-openai-rocm:latest
RUN pip install rixl

Verification

• Run a container from the new image and verify that RIXL is installed: docker run -it <new_image_id> python -c "import rixl"

Extra Tips

• For cross-vendor communication between AMD and NVIDIA GPUs, you may need to use a different connector or library that supports both vendors.
• Ensure that the Docker image is built and tested in an environment similar to the production environment to avoid compatibility issues.