PR #38882: [Bugfix] Fix GGUF parameter mapping for Transformers v5 fused MoE experts

• Repository: vllm-project/vllm
• Author: Lidang-Jiang
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/38882

Description (problem / solution / changelog)

Summary

Transformers v5 changed MoE model parameter naming from per-expert format (model.layers.{i}.mlp.experts.{j}.(gate|up|down)_proj.weight) to fused format (model.layers.{i}.mlp.experts.(gate_up_proj|down_proj)). This causes RuntimeError: failed to map GGUF parameters (18288) when loading Qwen3-30B-A3B or similar MoE GGUF models.

The fix adds sideload regex patterns for the v5 fused expert format alongside the existing v4 per-expert patterns in gguf_loader.py, for both deepseek_v2/v3 and qwen2_moe/qwen3_moe model types. The original v4 patterns and manual GGUF→HF name mappings are unchanged for backward compatibility.

• Not a duplicate: No open PR addresses this issue. Verified with gh pr list --search "38716".
• AI assistance was used (Claude) for code analysis and test generation. All changes reviewed by human submitter.

Fixes #38716

vllm serve /mnt/models/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf --tokenizer /mnt/models/Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf

RuntimeError: failed to map GGUF parameters (18288)

Root cause: Transformers v5 state_dict() returns fused expert parameter names like model.layers.0.mlp.experts.gate_up_proj which don't match the existing sideload regex model\.layers\.0\.mlp\.experts\.[0-9]+\.(gate|up|down)_proj\.weight, so they end up in unmapped_params and trigger the RuntimeError.

$ python -m pytest tests/model_executor/model_loader/test_gguf_moe_mapping.py -v
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestQwenMoESideloadPatterns::test_v4_per_expert_format[gate_proj-0-0] PASSED
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestQwenMoESideloadPatterns::test_v4_per_expert_format[gate_proj-0-1] PASSED
...
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestQwenMoESideloadPatterns::test_v5_fused_format[gate_up_proj-0] PASSED
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestQwenMoESideloadPatterns::test_v5_fused_format[down_proj-0] PASSED
...
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestDeepseekMoESideloadPatterns::test_v5_fused_format[gate_up_proj-0] PASSED
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestDeepseekMoESideloadPatterns::test_v5_fused_format[down_proj-0] PASSED
...
tests/model_executor/model_loader/test_gguf_moe_mapping.py::TestManualMappingIntegrity::test_v5_fused_params_filtered_by_sideload PASSED

======================= 83 passed in 19.70s ========================

Test plan

• 83 unit tests covering v4 per-expert and v5 fused regex matching for both Qwen MoE and DeepSeek MoE
• Edge cases: out-of-range layer index, unrelated params, no partial match
• End-to-end: v5 fused param names fully filtered by sideload regex
• ruff check and ruff format pass
• All pre-commit hooks pass
• End-to-end GGUF loading test (requires GGUF model file — verified by issue reporter)

Changed files

• tests/model_executor/model_loader/test_gguf_moe_mapping.py (added, +229/-0)
• vllm/model_executor/model_loader/gguf_loader.py (modified, +20/-0)