PR #40888: [Bugfix][Model] Qwen3-VL-MoE NVFP4 (ModelOpt) per-expert weight loading

• Repository: vllm-project/vllm
• Author: Code4me2
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/40888

Description (problem / solution / changelog)

Purpose

Fix checkpoint loading for Qwen3-VL-MoE models quantized by NVIDIA TensorRT-Model-Optimizer (nvidia-modelopt >= 0.43, quant_algo: NVFP4).

ModelOpt un-BMMs Qwen3-VL-MoE's fused expert tensors into per-expert nn.ModuleLists, producing names like experts.gate_proj.0.weight_packed instead of vLLM's expected experts.0.gate_proj.weight_packed. The current loader has three issues with this:

1. is_fused_expert uses substring matching ("experts.down_proj" in name) which false-positives on experts.down_proj.0.weight_scale_2. The loader then calls transpose(-1, -2) on a 0-D scale tensor and crashes with IndexError: Dimension out of range.
2. Even with the crash fixed, names need remapping from experts.<proj>.<N>.<suffix> to experts.<N>.<proj>.<suffix> so they flow through make_expert_params_mapping. Without it, scales are silently dropped and the model loads but outputs garbage.
3. ignore_suffixes does not list NVFP4 scale suffixes (.weight_scale_2, .weight_global_scale, .input_global_scale, .pre_quant_scale).

This PR adds, scoped to qwen3_vl_moe.py only:

1. A precise regex-based is_fused_expert check that targets the truly-fused BMM format and excludes per-expert un-BMM'd names.
2. An up-front name remap for the ModelOpt per-expert form.
3. The four NVFP4 scale suffixes added to ignore_suffixes.

The change is additive — Llama4-style BMM-fused NVFP4 checkpoints continue to load unchanged because the new regex matches them precisely. The recently added LoRA base_layer. adapter handling in fused_expert_params_mapping (#37114) is preserved.

Intended as a near-term stop-gap until RFC #40182 ("Unified ModelOpt Quantization in vLLM") lands; the per-model name-remap shim can be removed once the unified ModelOpt loader path is in place. Mirrors the per-model pattern established by #39045 (Gemma 4 quantized MoE, MERGED).

Fixes: #40885

AI assistance disclosure

Per AGENTS.md, this PR was produced with non-trivial assistance from an AI coding assistant (Anthropic Claude, model claude-opus-4-7). The submitter reviewed every changed line, ran pre-commit run --all-files (passes), executed the regex unit tests added under tests/model_executor/, and ran the integration test against a real ModelOpt NVFP4 Qwen3-VL-MoE checkpoint (Code4me2/bu-30b-a3b-preview-NVFP4). The BF16 vs NVFP4 accuracy comparison below confirms the scales are actually applied. The commit uses the Co-authored-by: Claude trailer per the AGENTS.md example.

This PR is not a duplicate: the only adjacent open work is RFC #40182 (long-term unified ModelOpt path) and the per-model fix series #39045/#39256/#39084/#39406 (Gemma 4 only). No open PR addresses Qwen3-VL-MoE specifically.

Test Plan

Unit tests (added in tests/model_executor/test_qwen3_vl_moe_loader.py): cover the two regex patterns. Verify _TRULY_FUSED_EXPERT_RE matches the fused BMM format (BF16 bare Parameter, Linear-wrapped, NVFP4 fused) but does NOT match the un-BMM'd per-expert form, and that _MODELOPT_PEREXP_RE correctly remaps experts.<proj>.<N>.<suffix> → experts.<N>.<proj>.<suffix>.

Integration test (manual, against a published reference checkpoint):

vllm serve Code4me2/bu-30b-a3b-preview-NVFP4 \
    --max-model-len 32768 --dtype bfloat16 \
    --gpu-memory-utilization 0.85 --kv-cache-dtype fp8 --max-num-seqs 2

Confirms the loader no longer crashes and the served model produces correct outputs.

Regression check: Llama4-style BMM-fused NVFP4 checkpoints still load (the new fused-detection regex matches them unchanged because their .weight / .weight_packed suffix follows the proj name with no intermediate numeric index).

Test Result

Before the patch (against Code4me2/bu-30b-a3b-preview-NVFP4):

ERROR [core.py:1104]   File ".../vllm/model_executor/models/qwen3_vl_moe.py", line 245, in load_weights
ERROR [core.py:1104]     loaded_weight = loaded_weight.transpose(-1, -2)
ERROR [core.py:1104] IndexError: Dimension out of range (expected to be in range of [-1, 0], but got -2)
RuntimeError: Engine core initialization failed.

After the patch: server starts, generation produces correct outputs ("What is 7 times 8?" → "56").

Mini-eval, BF16 baseline vs NVFP4 (this patch), same vLLM build on Blackwell sm_120:

Accuracy deltas are within typical NVFP4 ranges for an MoE; the meaningful confirmation is that the deltas are small, which means the scales are actually being applied (a broken loader would produce a much larger gap).

Checklist

• Purpose described (and linked to issue #40885)
• Test plan provided (unit tests + manual integration commands)
• Test result provided (MMLU/GSM8K comparison above)
• No documentation update needed (supported_models.md already lists Qwen3-VL-MoE)
• DCO sign-off present

Changed files

• tests/model_executor/test_qwen3_vl_moe_loader.py (added, +110/-0)
• vllm/model_executor/models/qwen3_vl_moe.py (modified, +35/-4)