Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [Feature]: Qwen3.5-Moe LoRA Support (experts) [5 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#40005•Fetched 2026-04-17 08:27:41
View on GitHub
Comments
5
Participants
3
Timeline
9
Reactions
1
Author
Jintao-HuangJintao-Huang
Participants
hjh0119hjh0119jeejeeleejeejeeleeJintao-HuangJintao-Huang
Assignees
jeejeeleejeejeelee
Timeline (top)
commented ×5assigned ×1labeled ×1mentioned ×1

Code Example

00000 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_a.lora_A.weight'
00001 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_a.lora_B.weight'
00002 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_b.lora_A.weight'
00003 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_b.lora_B.weight'
00004 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_qkv.lora_A.weight'
00005 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_qkv.lora_B.weight'
00006 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_z.lora_A.weight'
00007 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_z.lora_B.weight'
00008 =
'base_model.model.model.language_model.layers.0.linear_attn.out_proj.lora_A.weight'
00009 =
'base_model.model.model.language_model.layers.0.linear_attn.out_proj.lora_B.weight'
00010 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.down_proj.lora_A.weight'
00011 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.down_proj.lora_B.weight'
00012 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.gate_proj.lora_A.weight'
00013 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.gate_proj.lora_B.weight'
00014 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.up_proj.lora_A.weight'
00015 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.up_proj.lora_B.weight'
00016 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.down_proj.lora_A.weight'
00017 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.down_proj.lora_B.weight'
00018 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.gate_proj.lora_A.weight'
00019 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.gate_proj.lora_B.weight'
00020 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.up_proj.lora_A.weight'
00021 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.up_proj.lora_B.weight'
...
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

Hello!

The LoRA weights trained with megatron-swift for Qwen3.5-MoE cannot be loaded with vLLM, while Qwen3-MoE and Qwen3.5-Dense both work fine.

The structure of the experts' LoRA between these two models is the same (except for shared_experts).

We have provided weight examples here: https://modelscope.cn/models/huangjintao/test_vllm_qwen3_5_moe_lora

Could the vLLM team please add support for this?

00000 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_a.lora_A.weight'
00001 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_a.lora_B.weight'
00002 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_b.lora_A.weight'
00003 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_b.lora_B.weight'
00004 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_qkv.lora_A.weight'
00005 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_qkv.lora_B.weight'
00006 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_z.lora_A.weight'
00007 =
'base_model.model.model.language_model.layers.0.linear_attn.in_proj_z.lora_B.weight'
00008 =
'base_model.model.model.language_model.layers.0.linear_attn.out_proj.lora_A.weight'
00009 =
'base_model.model.model.language_model.layers.0.linear_attn.out_proj.lora_B.weight'
00010 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.down_proj.lora_A.weight'
00011 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.down_proj.lora_B.weight'
00012 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.gate_proj.lora_A.weight'
00013 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.gate_proj.lora_B.weight'
00014 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.up_proj.lora_A.weight'
00015 =
'base_model.model.model.language_model.layers.0.mlp.experts.0.up_proj.lora_B.weight'
00016 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.down_proj.lora_A.weight'
00017 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.down_proj.lora_B.weight'
00018 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.gate_proj.lora_A.weight'
00019 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.gate_proj.lora_B.weight'
00020 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.up_proj.lora_A.weight'
00021 =
'base_model.model.model.language_model.layers.0.mlp.experts.1.up_proj.lora_B.weight'
...

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The vLLM team needs to add support for loading LoRA weights trained with megatron-swift for Qwen3.5-MoE.

Guidance

  • The issue seems to be specific to the Qwen3.5-MoE model, as Qwen3-MoE and Qwen3.5-Dense work fine, suggesting a model-specific compatibility problem.
  • The provided weight examples may help the vLLM team identify and address the issue.
  • The structure of the experts' LoRA between Qwen3.5-MoE and other models is the same, except for shared_experts, which could be a key factor in resolving the issue.
  • The vLLM team should review the weight loading process for Qwen3.5-MoE and compare it with the working models to identify any discrepancies.

Example

No code snippet is provided as the issue seems to be related to model compatibility rather than a specific code error.

Notes

The solution to this issue relies on the vLLM team's ability to add support for the Qwen3.5-MoE model, which may require updates to their model loading code or compatibility fixes.

Recommendation

Apply workaround: The vLLM team should investigate and add support for loading LoRA weights trained with megatron-swift for Qwen3.5-MoE, as the current implementation does not support this specific model.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#GPU compatibility#latency issue#model loading#dependency error#configuration error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - 💡(How to fix) Fix [Feature]: Qwen3.5-Moe LoRA Support (experts) [5 comments, 3 participants]