vllm - 💡(How to fix) Fix [Usage]: ValueError: mismatch of LoRA layer names for Gemma4 E2B trained with unsloth [3 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#41702Fetched 2026-05-06 06:15:21
View on GitHub
Comments
3
Participants
2
Timeline
9
Reactions
0
Assignees
Timeline (top)
commented ×3mentioned ×2subscribed ×2assigned ×1

Error Message

(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Invocation of add_lora method failed
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Traceback (most recent call last):
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1327, in _invoke_util ity_method
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] result = get_result()
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1288, in <lambda>
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] and method(*self._convert_msgspec_args(method, args))
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 735, in add_lora
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] return self.model_executor.add_lora(lora_request)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 298, in add_lor a
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] return all(self.collective_rpc("add_lora", args=(lora_request,)))
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/uniproc_executor.py", line 80, in collective_rpc
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] result = run_method(self.driver_worker, method, args, kwargs)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/serial_utils.py", line 510, in run_method
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] return func(*args, **kwargs)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 904, in add_lor a
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] return self.model_runner.add_lora(lora_request)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/lora_model_runner_mixin.py", line 27 6, in add_lora
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] return self.lora_manager.add_adapter(lora_request)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 314, in add_adap ter (EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] lora = self._load_adapter(lora_request)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 187, in _load_ad apter
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] raise e
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 136, in _load_ad apter
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] lora = self.lora_model_cls.from_local_checkpoint(
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 220, in from_local_c heckpoint
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] check_unexpected_modules(f)
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 188, in check_unexpe cted_modules
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] raise ValueError(
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ValueError: While loading /work/home/user1469/lora/model/gemma4e2b_unsloth/final, expected target modules in {'embedding_pro jection', 'down_proj', 'gate_proj', 'per_layer_input_gate', 'up_proj', 'k_proj', 'q_proj', 'per_layer_model_projection', 'per_layer_projection', 'v_proj', 'o_proj'} but received ['visio n_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.gate_proj.linear', 'vision_tower.encoder.layers. 0.mlp.gate_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.self_attn.k_proj.linear', 'vision_tower.encoder.layers.0.self_attn.k_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower .encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.v_proj.linear', 'vision_tower.encoder.layer s.0.self_attn.v_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.gate_proj.li near', 'vision_tower.encoder.layers.1.mlp.gate_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encode r.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.o_proj.linear', 'vision_tower.encoder.layers.1.sel f_attn.o_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.108:15:24 [134/369$ linear', 'vision_tower.encoder.layers.1.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_t ower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.up_proj.linear', 'vision_tower.encoder.layers.10 .mlp.up_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.o_pro j.linear', 'vision_tower.encoder.layers.10.self_attn.o_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.mlp.down_proj.linear', 'vision_tower. encoder.layers.11.mlp.down_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.ml p.up_proj.linear', 'vision_tower.encoder.layers.11.mlp.up_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear ', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.q_proj.linear', 'vision
tower.encoder.layers.11.self_attn.q_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encod er.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.gate_proj.linear', 'vision_tower.encoder.layers.12.mlp.gat e_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.self_attn.k_proj.linear', 'visi on_tower.encoder.layers.12.self_attn.k_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.en coder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.v_proj.linear', 'vision_tower.encoder.layer s.12.self_attn.v_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.gate_pro j.linear', 'vision_tower.encoder.layers.13.mlp.gate_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower .encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.o_proj.linear', 'vision_tower.encoder.la yers.13.self_attn.o_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.sel f_attn.v_proj.linear', 'vision_tower.encoder.layers.13.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.lin ear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.up_proj.linear', 'vision_tower.en coder.layers.14.mlp.up_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14. self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q _proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.mlp.down_proj.linear', 'vision_tower.encoder.layers.15.mlp.down_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encod er.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.self_attn.k_proj.linear', 'vision_tower.encoder.layers.15.self_att n.k_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.q_proj.li near', 'vision_tower.encoder.layers.15.self_attn.q_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vis ion_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.gate_proj.linear', 'vision_tower.encoder.layer s.2.mlp.gate_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear ', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tow er.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.v_proj.linear', 'vision_tower.encoder.lay ers.2.self_attn.v_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.gate_proj. linear', 'vision_tower.encoder.layers.3.mlp.gate_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.enco der.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.o_proj.linear', 'vision_tower.encoder.layers.3.s elf_attn.o_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.v_pro j.linear', 'vision_tower.encoder.layers.3.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_t ower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.up_proj.linear', 'vision_tower.encoder.layers.4.mlp .up_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.o_proj.linea r', 'vision_tower.encoder.layers.4.self_attn.o_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_to wer.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.mlp.down_proj.linear', 'vision_tower.encoder.layer s.5.mlp.down_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.up_proj.linear' , 'vision_tower.encoder.layers.5.mlp.up_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.enc oder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.q_proj.linear', 'vision_tower.encoder.layers.5. self_attn.q_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.mlp.down_proj. linear', 'vision_tower.encoder.layers.6.mlp.down_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower. encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.self_attn.k_proj.linear', 'vision_tower.encoder.layers.6.self_at tn.k_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.q_proj.line ar', 'vision_tower.encoder.layers.6.self_attn.q_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_t ower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.gate_proj.linear', 'vision_tower.encoder.layers.7.m lp.gate_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.self_attn.08:15:27 [110/394] ision_tower.encoder.layers.7.self_attn.k_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.en coder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.v_proj.linear', 'vision_tower.encoder.layers.7 .self_attn.v_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.gate_proj.linea r', 'vision_tower.encoder.layers.8.mlp.gate_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.l ayers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.o_proj.linear', 'vision_tower.encoder.layers.8.self_a ttn.o_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.v_proj.lin ear', 'vision_tower.encoder.layers.8.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower. encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_p roj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.o_proj.linear', ' vision_tower.encoder.layers.9.self_attn.o_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.e ncoder.layers.9.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.self_attn.v_proj.linear']. Please verify that the loaded LoRA module is correct
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1238] Shutdown initiated (timeout=0)
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1261] Shutdown complete

Code Example

Collecting environment information...                                                                                                                                                    
uv is set                                                                                                                                                                                
==============================                                                                                                                                                           
        System Info                                                                                                                                                                      
==============================                                                                                                                                                           
OS                           : Ubuntu 24.04.4 LTS (x86_64)                                                                                                                               
GCC version                  : (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0                                                                                                                   
Clang version                : Could not collect                                                                                                                                         
CMake version                : version 3.28.3                                                                                                                                            
Libc version                 : glibc-2.39                                                                                                                                                
                                                                                                                                                                                         
==============================                                                                                                                                                           
       PyTorch Info                                                                                                                                                                      
==============================                                                                                                                                                           
PyTorch version              : 2.11.0+cu129                                                                                                                                              
Is debug build               : False                                                                                                                                                     
CUDA used to build PyTorch   : 12.9                                                                                                                                                      
ROCM used to build PyTorch   : N/A                                                                                                                                                       
XPU used to build PyTorch    : N/A                                                                                                                                                       
                                                                                                                                                                                         
==============================                                                                                                                                                           
      Python Environment                                                                                                                                                                 
==============================                                                                                                                                                           
Python version               : 3.12.3 (main, Mar 23 2026, 19:04:32) [GCC 13.3.0] (64-bit runtime)                                                                                        
Python platform              : Linux-6.8.0-107-generic-x86_64-with-glibc2.39  
Is CUDA available            : True                                                                                                                                     08:29:20 [90/140]
CUDA runtime version         : 13.2.51                                                                                                                                                   
CUDA_MODULE_LOADING set to   :                                                                                                                                                           
GPU models and configuration :                                                                                                                                                           
GPU 0: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 1: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 2: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 3: NVIDIA GeForce RTX 4090                                                                                                                                                           
                                                                                                                                                                                         
Nvidia driver version        : 575.51.03                                                                                                                                                 
cuDNN version                : Probably one of the following:                                                                                                                            
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.7                                                                                                                                              
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.7                                                                                                                                    
HIP runtime version          : N/A                                                                                                                                                       
MIOpen runtime version       : N/A                                                                                                                                                       
Is XNNPACK available         : True                                                                                                                                                      
                                                                                                                                                                                         
==============================                                                                                                                                                           
          CPU Info                                                                                                                                                                       
==============================                                                                                                                                                           
REDACTED, This is an institutional server, I would rather not post its hardware here.

                                                                                                                                                                                  
==============================                                                                                                                                                           
Versions of relevant libraries                                                                                                                                                           
==============================                                                                                                                                                           
[pip3] flashinfer-python==0.6.8.post1                                                                                                                                                    
[pip3] numpy==2.3.5                                                                                                                                                                      
[pip3] nvidia-cublas-cu12==12.9.1.4                                                                                                                                                      
[pip3] nvidia-cuda-cupti-cu12==12.9.79                                                                                                                                                   
[pip3] nvidia-cuda-nvrtc-cu12==12.9.86                                                                                                                                                   
[pip3] nvidia-cuda-runtime-cu12==12.9.79                                                                                                                                                 
[pip3] nvidia-cudnn-cu12==9.17.1.4                                                                                                                                                       
[pip3] nvidia-cudnn-frontend==1.18.0                                                                                                                                                     
[pip3] nvidia-cufft-cu12==11.4.1.4                                                                                                                                                       
[pip3] nvidia-cufile-cu12==1.14.1.1                                                                                                                                                      
[pip3] nvidia-curand-cu12==10.3.10.19                                                                                                                                                    
[pip3] nvidia-cusolver-cu12==11.7.5.82                                                                                                                                                   
[pip3] nvidia-cusparse-cu12==12.5.10.65                                                                                                                                                  
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-cutlass-dsl==4.4.2
[pip3] nvidia-cutlass-dsl-libs-base==4.4.2
[pip3] nvidia-ml-py==13.595.45
[pip3] nvidia-nccl-cu12==2.28.9
[pip3] nvidia-nvjitlink-cu12==12.9.86
[pip3] nvidia-nvshmem-cu12==3.4.5
[pip3] nvidia-nvtx-cu12==12.9.79
[pip3] pyzmq==27.1.0
[pip3] torch==2.11.0+cu129
[pip3] torch-c-dlpack-ext==0.1.5
[pip3] torchaudio==2.11.0
[pip3] torchvision==0.26.0+cu129
[pip3] transformers==5.7.0
[pip3] triton==3.6.0
[conda] Could not collect

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.20.1
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled; XPU: Disabled
GPU Topology:
        GPU0    GPU1    GPU2    GPU3    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      NODE    NODE    NODE    0-63    0               N/A
GPU1    NODE     X      NODE    NODE    0-63    0               N/A
GPU2    NODE    NODE     X      NODE    0-63    0               N/A
GPU3    NODE    NODE    NODE     X      0-63    0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

==============================
     Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
TORCHINDUCTOR_CACHE_DIR=/tmp/torchinductor_usama

---

(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Invocation of add_lora method failed                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Traceback (most recent call last):                                                                                          
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1327, in _invoke_util
ity_method                                                                                                                                                                               
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     result = get_result()                                                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]              ^^^^^^^^^^^^                                                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1288, in <lambda>    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     and method(*self._convert_msgspec_args(method, args))                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 735, in add_lora     
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.model_executor.add_lora(lora_request)                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 298, in add_lor
a                                                                                                                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return all(self.collective_rpc("add_lora", args=(lora_request,)))                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/uniproc_executor.py", line 80, in 
collective_rpc                                                                                                                                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     result = run_method(self.driver_worker, method, args, kwargs)                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/serial_utils.py", line 510, in run_method  
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return func(*args, **kwargs)                                                                                            
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^                                                                                            
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 904, in add_lor
a                                                                                                                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.model_runner.add_lora(lora_request)                                                                         
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                         
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/lora_model_runner_mixin.py", line 27
6, in add_lora                                                                                                                                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.lora_manager.add_adapter(lora_request)                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 314, in add_adap
ter 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     lora = self._load_adapter(lora_request)                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 187, in _load_ad
apter                                                                                                                                                                                    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     raise e                                                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 136, in _load_ad
apter                                                                                                                                                                                    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     lora = self._lora_model_cls.from_local_checkpoint(                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 220, in from_local_c
heckpoint                                                                                                                                                                                
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     check_unexpected_modules(f)                                                                                             
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 188, in check_unexpe
cted_modules                                                                                                                                                                             
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     raise ValueError(                                                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ValueError: While loading /work/home/user1469/lora/model/gemma4e2b_unsloth/final, expected target modules in {'embedding_pro
jection', 'down_proj', 'gate_proj', 'per_layer_input_gate', 'up_proj', 'k_proj', 'q_proj', 'per_layer_model_projection', 'per_layer_projection', 'v_proj', 'o_proj'} but received ['visio
n_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.gate_proj.linear', 'vision_tower.encoder.layers.
0.mlp.gate_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.self_attn.k_proj.linear',
 'vision_tower.encoder.layers.0.self_attn.k_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower
.encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.v_proj.linear', 'vision_tower.encoder.layer
s.0.self_attn.v_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.gate_proj.li
near', 'vision_tower.encoder.layers.1.mlp.gate_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encode
r.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.o_proj.linear', 'vision_tower.encoder.layers.1.sel
f_attn.o_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.108:15:24 [134/369$
linear', 'vision_tower.encoder.layers.1.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_t
ower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.up_proj.linear', 'vision_tower.encoder.layers.10
.mlp.up_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.o_pro
j.linear', 'vision_tower.encoder.layers.10.self_attn.o_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 
'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.mlp.down_proj.linear', 'vision_tower.
encoder.layers.11.mlp.down_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.ml
p.up_proj.linear', 'vision_tower.encoder.layers.11.mlp.up_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear
', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.q_proj.linear', 'vision_
tower.encoder.layers.11.self_attn.q_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encod
er.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.gate_proj.linear', 'vision_tower.encoder.layers.12.mlp.gat
e_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.self_attn.k_proj.linear', 'visi
on_tower.encoder.layers.12.self_attn.k_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.en
coder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.v_proj.linear', 'vision_tower.encoder.layer
s.12.self_attn.v_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.gate_pro
j.linear', 'vision_tower.encoder.layers.13.mlp.gate_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower
.encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.o_proj.linear', 'vision_tower.encoder.la
yers.13.self_attn.o_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.sel
f_attn.v_proj.linear', 'vision_tower.encoder.layers.13.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.lin
ear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.up_proj.linear', 'vision_tower.en
coder.layers.14.mlp.up_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14.
self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q
_proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.mlp.down_proj.linear',
 'vision_tower.encoder.layers.15.mlp.down_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encod
er.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.self_attn.k_proj.linear', 'vision_tower.encoder.layers.15.self_att
n.k_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.q_proj.li
near', 'vision_tower.encoder.layers.15.self_attn.q_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vis
ion_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.gate_proj.linear', 'vision_tower.encoder.layer
s.2.mlp.gate_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear
', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tow
er.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.v_proj.linear', 'vision_tower.encoder.lay
ers.2.self_attn.v_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.gate_proj.
linear', 'vision_tower.encoder.layers.3.mlp.gate_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.enco
der.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.o_proj.linear', 'vision_tower.encoder.layers.3.s
elf_attn.o_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.v_pro
j.linear', 'vision_tower.encoder.layers.3.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_t
ower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.up_proj.linear', 'vision_tower.encoder.layers.4.mlp
.up_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.o_proj.linea
r', 'vision_tower.encoder.layers.4.self_attn.o_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_to
wer.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.mlp.down_proj.linear', 'vision_tower.encoder.layer
s.5.mlp.down_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.up_proj.linear'
, 'vision_tower.encoder.layers.5.mlp.up_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.enc
oder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.q_proj.linear', 'vision_tower.encoder.layers.5.
self_attn.q_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.mlp.down_proj.
linear', 'vision_tower.encoder.layers.6.mlp.down_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower.
encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.self_attn.k_proj.linear', 'vision_tower.encoder.layers.6.self_at
tn.k_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.q_proj.line
ar', 'vision_tower.encoder.layers.6.self_attn.q_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_t
ower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.gate_proj.linear', 'vision_tower.encoder.layers.7.m
lp.gate_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.self_attn.08:15:27 [110/394]
ision_tower.encoder.layers.7.self_attn.k_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.en
coder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.v_proj.linear', 'vision_tower.encoder.layers.7
.self_attn.v_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.gate_proj.linea
r', 'vision_tower.encoder.layers.8.mlp.gate_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.l
ayers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.o_proj.linear', 'vision_tower.encoder.layers.8.self_a
ttn.o_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.v_proj.lin
ear', 'vision_tower.encoder.layers.8.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower.
encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_p
roj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.o_proj.linear', '
vision_tower.encoder.layers.9.self_attn.o_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.e
ncoder.layers.9.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.self_attn.v_proj.linear']. Please verify that the loaded LoRA module is correct                                 
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1238] Shutdown initiated (timeout=0)                                                                                               
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1261] Shutdown complete
RAW_BUFFERClick to expand / collapse

Your current environment

Collecting environment information...                                                                                                                                                    
uv is set                                                                                                                                                                                
==============================                                                                                                                                                           
        System Info                                                                                                                                                                      
==============================                                                                                                                                                           
OS                           : Ubuntu 24.04.4 LTS (x86_64)                                                                                                                               
GCC version                  : (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0                                                                                                                   
Clang version                : Could not collect                                                                                                                                         
CMake version                : version 3.28.3                                                                                                                                            
Libc version                 : glibc-2.39                                                                                                                                                
                                                                                                                                                                                         
==============================                                                                                                                                                           
       PyTorch Info                                                                                                                                                                      
==============================                                                                                                                                                           
PyTorch version              : 2.11.0+cu129                                                                                                                                              
Is debug build               : False                                                                                                                                                     
CUDA used to build PyTorch   : 12.9                                                                                                                                                      
ROCM used to build PyTorch   : N/A                                                                                                                                                       
XPU used to build PyTorch    : N/A                                                                                                                                                       
                                                                                                                                                                                         
==============================                                                                                                                                                           
      Python Environment                                                                                                                                                                 
==============================                                                                                                                                                           
Python version               : 3.12.3 (main, Mar 23 2026, 19:04:32) [GCC 13.3.0] (64-bit runtime)                                                                                        
Python platform              : Linux-6.8.0-107-generic-x86_64-with-glibc2.39  
Is CUDA available            : True                                                                                                                                     08:29:20 [90/140]
CUDA runtime version         : 13.2.51                                                                                                                                                   
CUDA_MODULE_LOADING set to   :                                                                                                                                                           
GPU models and configuration :                                                                                                                                                           
GPU 0: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 1: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 2: NVIDIA GeForce RTX 4090                                                                                                                                                           
GPU 3: NVIDIA GeForce RTX 4090                                                                                                                                                           
                                                                                                                                                                                         
Nvidia driver version        : 575.51.03                                                                                                                                                 
cuDNN version                : Probably one of the following:                                                                                                                            
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.7                                                                                                                                              
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.7                                                                                                                                    
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.7                                                                                                                                    
HIP runtime version          : N/A                                                                                                                                                       
MIOpen runtime version       : N/A                                                                                                                                                       
Is XNNPACK available         : True                                                                                                                                                      
                                                                                                                                                                                         
==============================                                                                                                                                                           
          CPU Info                                                                                                                                                                       
==============================                                                                                                                                                           
REDACTED, This is an institutional server, I would rather not post its hardware here.

                                                                                                                                                                                  
==============================                                                                                                                                                           
Versions of relevant libraries                                                                                                                                                           
==============================                                                                                                                                                           
[pip3] flashinfer-python==0.6.8.post1                                                                                                                                                    
[pip3] numpy==2.3.5                                                                                                                                                                      
[pip3] nvidia-cublas-cu12==12.9.1.4                                                                                                                                                      
[pip3] nvidia-cuda-cupti-cu12==12.9.79                                                                                                                                                   
[pip3] nvidia-cuda-nvrtc-cu12==12.9.86                                                                                                                                                   
[pip3] nvidia-cuda-runtime-cu12==12.9.79                                                                                                                                                 
[pip3] nvidia-cudnn-cu12==9.17.1.4                                                                                                                                                       
[pip3] nvidia-cudnn-frontend==1.18.0                                                                                                                                                     
[pip3] nvidia-cufft-cu12==11.4.1.4                                                                                                                                                       
[pip3] nvidia-cufile-cu12==1.14.1.1                                                                                                                                                      
[pip3] nvidia-curand-cu12==10.3.10.19                                                                                                                                                    
[pip3] nvidia-cusolver-cu12==11.7.5.82                                                                                                                                                   
[pip3] nvidia-cusparse-cu12==12.5.10.65                                                                                                                                                  
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-cutlass-dsl==4.4.2
[pip3] nvidia-cutlass-dsl-libs-base==4.4.2
[pip3] nvidia-ml-py==13.595.45
[pip3] nvidia-nccl-cu12==2.28.9
[pip3] nvidia-nvjitlink-cu12==12.9.86
[pip3] nvidia-nvshmem-cu12==3.4.5
[pip3] nvidia-nvtx-cu12==12.9.79
[pip3] pyzmq==27.1.0
[pip3] torch==2.11.0+cu129
[pip3] torch-c-dlpack-ext==0.1.5
[pip3] torchaudio==2.11.0
[pip3] torchvision==0.26.0+cu129
[pip3] transformers==5.7.0
[pip3] triton==3.6.0
[conda] Could not collect

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.20.1
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled; XPU: Disabled
GPU Topology:
        GPU0    GPU1    GPU2    GPU3    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      NODE    NODE    NODE    0-63    0               N/A
GPU1    NODE     X      NODE    NODE    0-63    0               N/A
GPU2    NODE    NODE     X      NODE    0-63    0               N/A
GPU3    NODE    NODE    NODE     X      0-63    0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

==============================
     Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
TORCHINDUCTOR_CACHE_DIR=/tmp/torchinductor_usama

How would you like to use vllm

I have fine tuned a Gemma4 E2B model with unsloth and tried loading the LoRA in vllm.

I am getting this "mismatch" of layer names, and I am not sure how to deal with it.

(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Invocation of add_lora method failed                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] Traceback (most recent call last):                                                                                          
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1327, in _invoke_util
ity_method                                                                                                                                                                               
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     result = get_result()                                                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]              ^^^^^^^^^^^^                                                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1288, in <lambda>    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     and method(*self._convert_msgspec_args(method, args))                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                   
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 735, in add_lora     
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.model_executor.add_lora(lora_request)                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 298, in add_lor
a                                                                                                                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return all(self.collective_rpc("add_lora", args=(lora_request,)))                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/executor/uniproc_executor.py", line 80, in 
collective_rpc                                                                                                                                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     result = run_method(self.driver_worker, method, args, kwargs)                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/serial_utils.py", line 510, in run_method  
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return func(*args, **kwargs)                                                                                            
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^                                                                                            
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 904, in add_lor
a                                                                                                                                                                                        
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.model_runner.add_lora(lora_request)                                                                         
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                         
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/v1/worker/lora_model_runner_mixin.py", line 27
6, in add_lora                                                                                                                                                                           
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     return self.lora_manager.add_adapter(lora_request)                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 314, in add_adap
ter 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     lora = self._load_adapter(lora_request)                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 187, in _load_ad
apter                                                                                                                                                                                    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     raise e                                                                                                                 
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/worker_manager.py", line 136, in _load_ad
apter                                                                                                                                                                                    
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     lora = self._lora_model_cls.from_local_checkpoint(                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                      
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 220, in from_local_c
heckpoint                                                                                                                                                                                
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     check_unexpected_modules(f)                                                                                             
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]   File "/work/home/user1469/vllm_only/.venv/lib/python3.12/site-packages/vllm/lora/lora_model.py", line 188, in check_unexpe
cted_modules                                                                                                                                                                             
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337]     raise ValueError(                                                                                                       
(EngineCore pid=3352787) ERROR 05-05 08:15:24 [core.py:1337] ValueError: While loading /work/home/user1469/lora/model/gemma4e2b_unsloth/final, expected target modules in {'embedding_pro
jection', 'down_proj', 'gate_proj', 'per_layer_input_gate', 'up_proj', 'k_proj', 'q_proj', 'per_layer_model_projection', 'per_layer_projection', 'v_proj', 'o_proj'} but received ['visio
n_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.down_proj.linear', 'vision_tower.encoder.layers.0.mlp.gate_proj.linear', 'vision_tower.encoder.layers.
0.mlp.gate_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.mlp.up_proj.linear', 'vision_tower.encoder.layers.0.self_attn.k_proj.linear',
 'vision_tower.encoder.layers.0.self_attn.k_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower.encoder.layers.0.self_attn.o_proj.linear', 'vision_tower
.encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.q_proj.linear', 'vision_tower.encoder.layers.0.self_attn.v_proj.linear', 'vision_tower.encoder.layer
s.0.self_attn.v_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.down_proj.linear', 'vision_tower.encoder.layers.1.mlp.gate_proj.li
near', 'vision_tower.encoder.layers.1.mlp.gate_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encoder.layers.1.mlp.up_proj.linear', 'vision_tower.encode
r.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.k_proj.linear', 'vision_tower.encoder.layers.1.self_attn.o_proj.linear', 'vision_tower.encoder.layers.1.sel
f_attn.o_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.1.self_attn.q_proj.linear', 'vision_tower.encoder.layers.108:15:24 [134/369$
linear', 'vision_tower.encoder.layers.1.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_tower.encoder.layers.10.mlp.down_proj.linear', 'vision_t
ower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.gate_proj.linear', 'vision_tower.encoder.layers.10.mlp.up_proj.linear', 'vision_tower.encoder.layers.10
.mlp.up_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.k_proj.linear', 'vision_tower.encoder.layers.10.self_attn.o_pro
j.linear', 'vision_tower.encoder.layers.10.self_attn.o_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 'vision_tower.encoder.layers.10.self_attn.q_proj.linear', 
'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.10.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.mlp.down_proj.linear', 'vision_tower.
encoder.layers.11.mlp.down_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.mlp.gate_proj.linear', 'vision_tower.encoder.layers.11.ml
p.up_proj.linear', 'vision_tower.encoder.layers.11.mlp.up_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear', 'vision_tower.encoder.layers.11.self_attn.k_proj.linear
', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.o_proj.linear', 'vision_tower.encoder.layers.11.self_attn.q_proj.linear', 'vision_
tower.encoder.layers.11.self_attn.q_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encoder.layers.11.self_attn.v_proj.linear', 'vision_tower.encod
er.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.down_proj.linear', 'vision_tower.encoder.layers.12.mlp.gate_proj.linear', 'vision_tower.encoder.layers.12.mlp.gat
e_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.mlp.up_proj.linear', 'vision_tower.encoder.layers.12.self_attn.k_proj.linear', 'visi
on_tower.encoder.layers.12.self_attn.k_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.encoder.layers.12.self_attn.o_proj.linear', 'vision_tower.en
coder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.q_proj.linear', 'vision_tower.encoder.layers.12.self_attn.v_proj.linear', 'vision_tower.encoder.layer
s.12.self_attn.v_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.down_proj.linear', 'vision_tower.encoder.layers.13.mlp.gate_pro
j.linear', 'vision_tower.encoder.layers.13.mlp.gate_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower.encoder.layers.13.mlp.up_proj.linear', 'vision_tower
.encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.k_proj.linear', 'vision_tower.encoder.layers.13.self_attn.o_proj.linear', 'vision_tower.encoder.la
yers.13.self_attn.o_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.self_attn.q_proj.linear', 'vision_tower.encoder.layers.13.sel
f_attn.v_proj.linear', 'vision_tower.encoder.layers.13.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.linear', 'vision_tower.encoder.layers.14.mlp.down_proj.lin
ear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.gate_proj.linear', 'vision_tower.encoder.layers.14.mlp.up_proj.linear', 'vision_tower.en
coder.layers.14.mlp.up_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14.self_attn.k_proj.linear', 'vision_tower.encoder.layers.14.
self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.o_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q_proj.linear', 'vision_tower.encoder.layers.14.self_attn.q
_proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.14.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.mlp.down_proj.linear',
 'vision_tower.encoder.layers.15.mlp.down_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encoder.layers.15.mlp.gate_proj.linear', 'vision_tower.encod
er.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.mlp.up_proj.linear', 'vision_tower.encoder.layers.15.self_attn.k_proj.linear', 'vision_tower.encoder.layers.15.self_att
n.k_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.o_proj.linear', 'vision_tower.encoder.layers.15.self_attn.q_proj.li
near', 'vision_tower.encoder.layers.15.self_attn.q_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vision_tower.encoder.layers.15.self_attn.v_proj.linear', 'vis
ion_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.down_proj.linear', 'vision_tower.encoder.layers.2.mlp.gate_proj.linear', 'vision_tower.encoder.layer
s.2.mlp.gate_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.mlp.up_proj.linear', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear
', 'vision_tower.encoder.layers.2.self_attn.k_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tower.encoder.layers.2.self_attn.o_proj.linear', 'vision_tow
er.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.q_proj.linear', 'vision_tower.encoder.layers.2.self_attn.v_proj.linear', 'vision_tower.encoder.lay
ers.2.self_attn.v_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.down_proj.linear', 'vision_tower.encoder.layers.3.mlp.gate_proj.
linear', 'vision_tower.encoder.layers.3.mlp.gate_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.encoder.layers.3.mlp.up_proj.linear', 'vision_tower.enco
der.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.k_proj.linear', 'vision_tower.encoder.layers.3.self_attn.o_proj.linear', 'vision_tower.encoder.layers.3.s
elf_attn.o_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.q_proj.linear', 'vision_tower.encoder.layers.3.self_attn.v_pro
j.linear', 'vision_tower.encoder.layers.3.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_tower.encoder.layers.4.mlp.down_proj.linear', 'vision_t
ower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.gate_proj.linear', 'vision_tower.encoder.layers.4.mlp.up_proj.linear', 'vision_tower.encoder.layers.4.mlp
.up_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.k_proj.linear', 'vision_tower.encoder.layers.4.self_attn.o_proj.linea
r', 'vision_tower.encoder.layers.4.self_attn.o_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_tower.encoder.layers.4.self_attn.q_proj.linear', 'vision_to
wer.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.4.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.mlp.down_proj.linear', 'vision_tower.encoder.layer
s.5.mlp.down_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.gate_proj.linear', 'vision_tower.encoder.layers.5.mlp.up_proj.linear'
, 'vision_tower.encoder.layers.5.mlp.up_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.encoder.layers.5.self_attn.k_proj.linear', 'vision_tower.enc
oder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.o_proj.linear', 'vision_tower.encoder.layers.5.self_attn.q_proj.linear', 'vision_tower.encoder.layers.5.
self_attn.q_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.5.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.mlp.down_proj.
linear', 'vision_tower.encoder.layers.6.mlp.down_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower.encoder.layers.6.mlp.gate_proj.linear', 'vision_tower.
encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.mlp.up_proj.linear', 'vision_tower.encoder.layers.6.self_attn.k_proj.linear', 'vision_tower.encoder.layers.6.self_at
tn.k_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.o_proj.linear', 'vision_tower.encoder.layers.6.self_attn.q_proj.line
ar', 'vision_tower.encoder.layers.6.self_attn.q_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_tower.encoder.layers.6.self_attn.v_proj.linear', 'vision_t
ower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.down_proj.linear', 'vision_tower.encoder.layers.7.mlp.gate_proj.linear', 'vision_tower.encoder.layers.7.m
lp.gate_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.mlp.up_proj.linear', 'vision_tower.encoder.layers.7.self_attn.08:15:27 [110/394]
ision_tower.encoder.layers.7.self_attn.k_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.encoder.layers.7.self_attn.o_proj.linear', 'vision_tower.en
coder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.q_proj.linear', 'vision_tower.encoder.layers.7.self_attn.v_proj.linear', 'vision_tower.encoder.layers.7
.self_attn.v_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.down_proj.linear', 'vision_tower.encoder.layers.8.mlp.gate_proj.linea
r', 'vision_tower.encoder.layers.8.mlp.gate_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.layers.8.mlp.up_proj.linear', 'vision_tower.encoder.l
ayers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.k_proj.linear', 'vision_tower.encoder.layers.8.self_attn.o_proj.linear', 'vision_tower.encoder.layers.8.self_a
ttn.o_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.q_proj.linear', 'vision_tower.encoder.layers.8.self_attn.v_proj.lin
ear', 'vision_tower.encoder.layers.8.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower.encoder.layers.9.mlp.down_proj.linear', 'vision_tower.
encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.gate_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_proj.linear', 'vision_tower.encoder.layers.9.mlp.up_p
roj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.k_proj.linear', 'vision_tower.encoder.layers.9.self_attn.o_proj.linear', '
vision_tower.encoder.layers.9.self_attn.o_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.encoder.layers.9.self_attn.q_proj.linear', 'vision_tower.e
ncoder.layers.9.self_attn.v_proj.linear', 'vision_tower.encoder.layers.9.self_attn.v_proj.linear']. Please verify that the loaded LoRA module is correct                                 
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1238] Shutdown initiated (timeout=0)                                                                                               
(EngineCore pid=3352787) INFO 05-05 08:15:24 [core.py:1261] Shutdown complete

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The issue is likely due to a mismatch between the expected and actual layer names in the LoRA module, which can be resolved by verifying the correctness of the loaded LoRA module.

Guidance

  • Verify that the loaded LoRA module is correct and matches the expected layer names.
  • Check the documentation for vllm to ensure that the LoRA module is being loaded correctly.
  • Review the code that loads the LoRA module to ensure that it is correctly specifying the layer names.
  • If the issue persists, try to reproduce the error with a minimal example to help identify the root cause.

Example

No code example is provided as the issue is related to a specific library (vllm) and its usage.

Notes

The error message indicates a mismatch between the expected and actual layer names in the LoRA module. This suggests that the issue is related to the loading of the LoRA module rather than a problem with the vllm library itself.

Recommendation

Apply a workaround by verifying the correctness of the loaded LoRA module and ensuring that it matches the expected layer names. If the issue persists, try to reproduce the error with a minimal example to help identify the root cause.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING