INFO 03-10 11:30:12 [vllm.py:957] Cudagraph is disabled under eager mode <frozen importlib._bootstrap_external>:1296: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead. <frozen importlib._bootstrap_external>:1296: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead. Traceback (most recent call last): File "/ssd4/workspace/1_test/webserver_new2.py", line 1136, in <module> infer = QwenInfer(args) ^^^^^^^^^^^^^^^ File "/ssd4/workspace/1_test/qwen_infer_vllm.py", line 79, in init self.model = LLMEngine.from_engine_args(model_engine_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py", line 177, in from_engine_args return cls( ^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py", line 92, in init self.renderer = renderer = renderer_from_config(self.vllm_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/renderers/registry.py", line 89, in renderer_from_config return RENDERER_REGISTRY.load_renderer( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/renderers/registry.py", line 63, in load_renderer return renderer_cls.from_config(config, tokenizer_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/renderers/hf.py", line 607, in from_config return cls(config, tokenizer) ^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/renderers/hf.py", line 614, in init super().init(config, tokenizer) File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/renderers/base.py", line 93, in init self.mm_processor = mm_registry.create_processor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/multimodal/registry.py", line 214, in create_processor return factories.build_processor(ctx, cache=cache) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/multimodal/registry.py", line 95, in build_processor return self.processor(info, dummy_inputs_builder, cache=cache) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py", line 997, in init self.data_parser = self.info.get_data_parser() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_vl.py", line 643, in get_data_parser self.get_hf_config().vision_config.spatial_merge_size, ^^^^^^^^^^^^^^^^^^^^ File "/ssd4//.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_5.py", line 114, in get_hf_config return self.ctx.get_hf_config(Qwen3_5Config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/ssd4/.conda/envs/vllm0170_py312/lib/python3.12/site-packages/vllm/multimodal/processing/context.py", line 139, in get_hf_config raise TypeError( TypeError: Invalid type of HuggingFace config. Expected type: <class 'vllm.transformers_utils.configs.qwen3_5.Qwen3_5Config'>, but found type: <class 'transformers.models.qwen3_5.configuration_qwen3_5.Qwen3_5TextConfig'>

Fix Plan

To solve the issue, we need to ensure compatibility between the vllm library and the transformers library. The error occurs because the transformers library does not recognize the qwen3_5_text architecture.

Here are the steps to fix the issue:

• Downgrade the transformers library to version 4.57.6 to match the version used in the vllm library.
• Update the vllm library to use the correct Qwen3_5Config class.

Code Changes

# Downgrade transformers library
pip install transformers==4.57.6

# Update vllm library to use correct Qwen3_5Config class
# In vllm/model_executor/models/qwen3_5.py
from vllm.transformers_utils.configs.qwen3_5 import Qwen3_5Config

class Qwen3_5Model:
    def get_hf_config(self):
        return self.ctx.get_hf_config(Qwen3_5Config)

Verification

After making the changes, verify that the model can be loaded and used without any errors. You can do this by running the following code:

from vllm.model_executor.models.qwen3_5 import Qwen3_5Model

model = Qwen3_5Model()
model.get_hf_config()

If the code runs without any errors, it means that the fix was successful.

Extra Tips

• Make sure to check the compatibility of the vllm library with the transformers library before making any changes.
• If you encounter any issues, try downgrading or upgrading the transformers library to a version that is compatible with the vllm library.
• Always verify that the changes you make do not introduce any new errors or issues.