vllm - 💡(How to fix) Fix [Bug]: qwen 3.5 crash under dp 8 [1 participants]

error log

Fix Plan

The fix involves updating the CUDA version and modifying the code to handle CUDA events properly.

• Update CUDA to the latest version to ensure compatibility with the current environment.
• Modify the code to check for CUDA events before destroying them to prevent illegal memory access.

Code Changes

import torch

# Check if CUDA is available
if torch.cuda.is_available():
    # Get the current CUDA device
    device = torch.cuda.current_device()
    # Check if the event is valid before destroying it
    if torch.cuda.event_query(device):
        torch.cuda.event_destroy(device)

Alternatively, you can use a try-except block to catch the CUDA warning and handle it accordingly:

try:
    torch.cuda.event_destroy(device)
except RuntimeError as e:
    print(f"CUDA warning: {e}")

Verification

To verify that the fix worked, run the vllm serve and vllm bench serve commands again with the updated code and CUDA version. Check the error log for any CUDA warnings or errors.

Extra Tips

• Make sure to update the CUDA version to the latest one compatible with your environment.
• Use the torch.cuda.event_query function to check if the event is valid before destroying it.
• Consider using a try-except block to catch and handle CUDA warnings and errors.