vllm - 💡(How to fix) Fix [Bug]: Does vllm support deploying glm-5 on A800 or A100, or are there any plans to support it? [1 participants]

Fix Plan

To support deploying GLM-5 on A800 or A100, we need to modify the VLLM configuration.

Steps to Fix

• Update the vllm configuration file to include support for GLM-5 on A800 and A100.
• Add the following code snippet to the vllm/config.py file:

SUPPORTED_MODELS = {
    # ... existing models ...
    'glm-5': ['A800', 'A100']
}

• Modify the vllm/deploy.py file to handle the new model and hardware combinations:

if model_name == 'glm-5' and hardware in ['A800', 'A100']:
    # deploy glm-5 on A800 or A100
    deploy_glm_5_on_a800_a100(model_name, hardware)

• Implement the deploy_glm_5_on_a800_a100 function to handle the deployment logic.

Verification

To verify the fix, run the vllm deployment script with the glm-5 model and A800 or A100 hardware. Check the deployment logs for success or failure messages.

Extra Tips

• Make sure to update the documentation to reflect the new supported models and hardware combinations.
• Test the deployment with different models and hardware combinations to ensure compatibility.