vllm - ✅(Solved) Fix [Bug]: LoRA loading fails for Qwen 3.5 MoE (35b-A3b) due to expert module name mismatch [1 pull requests, 2 comments, 2 participants]

uYu · 2026-03-30T07:09:57Z

[vllm] PR 38522: LoRA fix MoE expert module name parsing for models with numeric expert indices Qwen 3.5 MoE - Repository: vllm-project/vllm - Author: NIK-TIGE… # PR #38522: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE) - Repository: vllm-project/vllm - Author: NIK-TIGER-BILL - State: open | merged: False - Link: https://github.com/vllm-project/vllm/pull/38522 ## Description (problem / solution / changelog) ## Purpose Fix LoRA loading failure for Qwen 3.5 MoE (and any other model with numeric expert indices in module paths). Closes #38520 When loading a LoRA adapter on a model whose expert weight names contain a numeric expert index (e.g. `language_model.model.layers.0.mlp.experts.0.down_proj`), `check_unexpected_modules()` raised: ``` ValueError: While loading ..., expected target modules in {...} but received ['language_model.model.layers.0.mlp.experts.0.down_proj', ...] ``` **Root cause:** The old logic extracted the substring following `.experts`: ```python expert_idx = module_name.find(".experts") expert_suffix = module_name[expert_idx + 1 :] # → "experts.0.down_proj" ``` This works when the path ends at `.experts` (suffix = `"experts"` which is in the expected set), but fails when a numeric index follows: suffix = `"experts.0.down_proj"` is never in the expected set even though `"down_proj"` is. **Fix:** Extract only the last dotted component of the module path: ```python expert_suffix = module_name.split(".")[-1] # → "down_proj" ``` This is correct for both forms: - `"...experts"` → last component = `"experts"` ✓ - `"...experts.0.down_proj"` → last component = `"down_proj"` ✓ ## Test Plan Unit test verifying that both naming styles are accepted: ```python # standard MoE style (unchanged behaviour) assert check("model.layer.mlp.experts") is None # "experts" in expected set # Qwen 3.5 MoE style (was broken) assert check("language_model.layers.0.mlp.experts.0.down_proj") is None # "down_proj" in expected set ``` Also tested end-to-end loading a LoRA adapter on `Qwen/Qwen3.5-MoE-35B-A3B` (local reproducer matching the issue). ## Test Result Previously produced `ValueError`; now loads without error. ## Changed files - `tests/lora/test_utils.py` (modified, +51/-0) - `vllm/lora/lora_model.py` (modified, +10/-3) ## Fixed - Fixed by PR: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE) (https://github.com/vllm-project/vllm/pull/38522) When starting the engine with **Qwen 3.5-35b-a3b** and LoRA enabled, the system throws a `ValueError`. It appears that the current expert module parsing logic in `lora_model.py` is incompatible with the naming convention used by Qwen 3.5. **Error Log:** ```text (EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] raise ValueError(^M (EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] ValueError: While loading /models/model_deploy/models/xxxxx_xxxxx/, expected target modules in {'in_proj_z', 'v_proj', 'in_proj_qkv', 'linear_fc2', 'q_proj', 'k_proj', 'gate', 'down_proj', 'gate_proj', 'experts', 'shared_expert_gate', 'conv1d', 'proj', 'qkv', 'o_proj', 'linear_fc1', 'in_proj_b', 'in_proj_a', 'out_proj', 'up_proj'} but received ['language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', ...] ``` **Current Implementation Issue:** The current logic in `/lora_model.py` incorrectly parses the expert module name, leading to a mismatch with `expected_lora_modules`: ```python module_name, _ = parse_fine_tuned_lora_name(lora_module, weights_mapper) # Case for expert lora weights if ".experts" in module_name: expert_idx = module_name.find(".experts") expert_suffix = module_name[expert_idx + 1 :] if expert_suffix not in expected_lora_modules: unexpected_modules.append(module_name) elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules: unexpected_modules.append(module_name) ``` **Suggested Fix:** Modify the logic to correctly extract the base module name (e.g., `down_proj`) by splitting from the last dot, specifically for MoE layers: ```python if ".experts" in module_name: # Fix: Correctly get the last component of the module path expert_suffix = module_name.split('.')[-1] if expert_suffix not in expected_lora_modules: unexpected_modules.append(module_name) elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules: unexpected_modules.append(module_name) ``` --- ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.

vllm2026-03-30 07:09:57

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#38520•Fetched 2026-04-08 01:53:37

View on GitHub

Comments

Participants

Timeline

Reactions

Author

uYu

Participants

jeejeelee

uYu

Timeline (top)

commented ×2cross-referenced ×1labeled ×1referenced ×1

Error Message

(EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] raise ValueError(^M (EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] ValueError: While loading /models/model_deploy/models/xxxxx_xxxxx/, expected target modules in {'in_proj_z', 'v_proj', 'in_proj_qkv', 'linear_fc2', 'q_proj', 'k_proj', 'gate', 'down_proj', 'gate_proj', 'experts', 'shared_expert_gate', 'conv1d', 'proj', 'qkv', 'o_proj', 'linear_fc1', 'in_proj_b', 'in_proj_a', 'out_proj', 'up_proj'} but received ['language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', ...]

Fix Action

Fixed

Fixed by PR: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE) (https://github.com/vllm-project/vllm/pull/38522)

PR fix notes

PR #38522: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE)

Repository: vllm-project/vllm
Author: NIK-TIGER-BILL
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/38522

Description (problem / solution / changelog)

Purpose

Fix LoRA loading failure for Qwen 3.5 MoE (and any other model with numeric expert indices in module paths).

Closes #38520

When loading a LoRA adapter on a model whose expert weight names contain a numeric expert index (e.g. language_model.model.layers.0.mlp.experts.0.down_proj), check_unexpected_modules() raised:

ValueError: While loading ..., expected target modules in {...} but
received ['language_model.model.layers.0.mlp.experts.0.down_proj', ...]

Root cause: The old logic extracted the substring following .experts:

expert_idx = module_name.find(".experts")
expert_suffix = module_name[expert_idx + 1 :]   # → "experts.0.down_proj"

This works when the path ends at .experts (suffix = "experts" which is in the expected set), but fails when a numeric index follows: suffix = "experts.0.down_proj" is never in the expected set even though "down_proj" is.

Fix: Extract only the last dotted component of the module path:

expert_suffix = module_name.split(".")[-1]   # → "down_proj"

This is correct for both forms:

"...experts" → last component = "experts" ✓
"...experts.0.down_proj" → last component = "down_proj" ✓

Test Plan

Unit test verifying that both naming styles are accepted:

# standard MoE style (unchanged behaviour)
assert check("model.layer.mlp.experts") is None   # "experts" in expected set
# Qwen 3.5 MoE style (was broken)
assert check("language_model.layers.0.mlp.experts.0.down_proj") is None  # "down_proj" in expected set

Also tested end-to-end loading a LoRA adapter on Qwen/Qwen3.5-MoE-35B-A3B (local reproducer matching the issue).

Test Result

Previously produced ValueError; now loads without error.

Changed files

tests/lora/test_utils.py (modified, +51/-0)
vllm/lora/lora_model.py (modified, +10/-3)

Code Example

(EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] raise ValueError(^M
(EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] ValueError: While loading /models/model_deploy/models/xxxxx_xxxxx/, expected target modules in {'in_proj_z', 'v_proj', 'in_proj_qkv', 'linear_fc2', 'q_proj', 'k_proj', 'gate', 'down_proj', 'gate_proj', 'experts', 'shared_expert_gate', 'conv1d', 'proj', 'qkv', 'o_proj', 'linear_fc1', 'in_proj_b', 'in_proj_a', 'out_proj', 'up_proj'} but received ['language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', ...]

---

module_name, _ = parse_fine_tuned_lora_name(lora_module, weights_mapper)
# Case for expert lora weights
if ".experts" in module_name:
    expert_idx = module_name.find(".experts")
    expert_suffix = module_name[expert_idx + 1 :]
    if expert_suffix not in expected_lora_modules:
        unexpected_modules.append(module_name)
elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules:
    unexpected_modules.append(module_name)

---

if ".experts" in module_name:
    # Fix: Correctly get the last component of the module path
    expert_suffix = module_name.split('.')[-1]
    if expert_suffix not in expected_lora_modules:
        unexpected_modules.append(module_name)
elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules:
    unexpected_modules.append(module_name)

RAW_BUFFERClick to expand / collapse

When starting the engine with Qwen 3.5-35b-a3b and LoRA enabled, the system throws a ValueError. It appears that the current expert module parsing logic in lora_model.py is incompatible with the naming convention used by Qwen 3.5.

Error Log:

(EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] raise ValueError(^M
(EngineCore pid=1114) ERROR 03-30 05:34:48 [core.py:1308] ValueError: While loading /models/model_deploy/models/xxxxx_xxxxx/, expected target modules in {'in_proj_z', 'v_proj', 'in_proj_qkv', 'linear_fc2', 'q_proj', 'k_proj', 'gate', 'down_proj', 'gate_proj', 'experts', 'shared_expert_gate', 'conv1d', 'proj', 'qkv', 'o_proj', 'linear_fc1', 'in_proj_b', 'in_proj_a', 'out_proj', 'up_proj'} but received ['language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.down_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', 'language_model.model.layers.0.mlp.experts.0.gate_proj', ...]

Current Implementation Issue: The current logic in /lora_model.py incorrectly parses the expert module name, leading to a mismatch with expected_lora_modules:

module_name, _ = parse_fine_tuned_lora_name(lora_module, weights_mapper)
# Case for expert lora weights
if ".experts" in module_name:
    expert_idx = module_name.find(".experts")
    expert_suffix = module_name[expert_idx + 1 :]
    if expert_suffix not in expected_lora_modules:
        unexpected_modules.append(module_name)
elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules:
    unexpected_modules.append(module_name)

Suggested Fix: Modify the logic to correctly extract the base module name (e.g., down_proj) by splitting from the last dot, specifically for MoE layers:

if ".experts" in module_name:
    # Fix: Correctly get the last component of the module path
    expert_suffix = module_name.split('.')[-1]
    if expert_suffix not in expected_lora_modules:
        unexpected_modules.append(module_name)
elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules:
    unexpected_modules.append(module_name)

Before submitting a new issue...

Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To resolve the ValueError caused by incompatible expert module parsing logic, follow these steps:

Modify the lora_model.py file to correctly extract the base module name for MoE layers.
Update the logic to split the module name from the last dot when encountering .experts.

Code Changes

Replace the existing logic in lora_model.py with the following code:

if ".experts" in module_name:
    # Fix: Correctly get the last component of the module path
    expert_suffix = module_name.split('.')[-1]
    if expert_suffix not in expected_lora_modules:
        unexpected_modules.append(module_name)
elif module_name.rsplit(".", 1)[-1] not in expected_lora_modules:
    unexpected_modules.append(module_name)

Verification

After applying the fix, restart the engine with Qwen 3.5-35b-a3b and LoRA enabled. Verify that the ValueError is no longer thrown and the system functions as expected.

Extra Tips

Ensure that the expected_lora_modules list is up-to-date and includes all required module names.
Test the updated logic with different module names to ensure correct parsing and handling.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#latency issue #model loading #dependency error #configuration error #environment variable

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [Bug]: LoRA loading fails for Qwen 3.5 MoE (35b-A3b) due to expert module name mismatch [1 pull requests, 2 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #38522: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE)

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Before submitting a new issue...

extent analysis

Fix Plan

Code Changes

Verification

Extra Tips

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [Bug]: LoRA loading fails for Qwen 3.5 MoE (35b-A3b) due to expert module name mismatch [1 pull requests, 2 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #38522: [LoRA] fix MoE expert module name parsing for models with numeric expert indices (Qwen 3.5 MoE)

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Before submitting a new issue...

extent analysis

Fix Plan

Code Changes

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING