vllm - 💡(How to fix) Fix [Usage]: How to run Qwen3.5 models on V100 given the conflicting requirements of transformers version and vLLM architecture support

Your current environment

(The environment collection script cannot be executed because this machine is deployed in an air-gapped/intranet environment. However, the core hardware is NVIDIA V100 GPU, running CUDA 12.4 and Python 3.10.)

How would you like to use vllm

Description

I want to deploy and run inference for the Qwen3.5 series models (e.g., Qwen3.5-9B) on an NVIDIA V100 GPU using vLLM.

The Problem / Dilemma

I am currently facing a dependency and compatibility catch-22:

1. Model Requirement: The Qwen3.5 models require transformers>=5.x (or specific latest versions) to be correctly tokenized and loaded, which is only integrated into recent vLLM releases.
2. Hardware Constraint: Newer versions of vLLM have dropped full support or optimized kernels (like FlashAttention) for the Volta architecture (V100, CC 7.0), causing compilation errors or runtime failures when trying to launch the service.

If I downgrade vLLM to an older version that natively supports V100, it cannot load Qwen3.5 due to the outdated transformers and structural differences. If I upgrade vLLM, it fails on V100.

Questions

1. Is there a recommended/stable vLLM version or a specific commit that balances both V100 compatibility and newer model (Qwen3.5) support?
2. Can I bypass the flash-attention or hardware checks on V100 in newer vLLM versions (e.g., by forcing --vllm-attention-backend=XFORMERS or --vllm-attention-backend=ROCM alternative flags)?
3. Are there any community workarounds, such as building a specific branch from source with custom build flags for NVCC_APPEND_FLAGS="-gencode arch=compute_70,code=sm_70"?

Any guidance or temporary workarounds would be highly appreciated!

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.