Your current environment

vLLM: 0.18.0 (image: vllm/vllm-audio:v0.18.0 or equivalent) Model: (full main snapshot) GPU: NVIDIA H200 NVL, tensor_parallel_size=4 Quantization: ModelOpt NVFP4 (as detected by vLLM: Detected ModelOpt NVFP4 checkpoint)

🐛 Describe the bug

Summary Loading nvidia/MiniMax-M2.5-NVFP4 from a full Hugging Face–style directory (29 safetensors shards + model.safetensors.index.json) fails during load_weights with: KeyError: 'layers.0.self_attn.qkv_proj.k_scale' The index file does not contain any qkv_proj tensors; attention weights use separate q_proj, k_proj, and v_proj (e.g. model.layers.0.self_attn.k_proj.k_scale). So this looks like a loader / checkpoint layout mismatch, not a missing or corrupted file.

python3 -m vllm.entrypoints.openai.api_server
--model /path/to/MiniMax-M2.5-NVFP4
--trust-remote-code
--tensor-parallel-size 4
--kv-cache-dtype fp8
--tool-call-parser minimax_m2
--enable-auto-tool-choice

Actual behavior Engine startup fails while workers load weights. Stack trace includes:

File ".../vllm/model_executor/models/minimax_m2.py", line 442, in load_weights
    param = params_dict[name]
KeyError: 'layers.0.self_attn.qkv_proj.k_scale'

Expected behavior Weights load successfully for the official nvidia/MiniMax-M2.5-NVFP4 checkpoint, or documentation clearly states a minimum vLLM commit / image and any required conversion if the public HF layout is intentionally “unfused” q/k/v. Evidence: checkpoint key layout In model.safetensors.index.json from the HF repo: There are no keys containing qkv_proj (e.g. grep -c qkv_proj → 0). Layer 0 attention includes entries such as: model.layers.0.self_attn.k_proj.k_scale model.layers.0.self_attn.q_proj.weight model.layers.0.self_attn.v_proj.v_scale (and related weights) So the loader expecting layers.0.self_attn.qkv_proj.k_scale does not match the published safetensors index for this model.

Questions for maintainers Should MiniMaxM2ForCausalLM + ModelOpt NVFP4 use fused qkv_proj in vLLM while the NVIDIA HF checkpoint ships unfused q_proj / k_proj / v_proj? If so, is there a planned weight remapping or fusion step? Is there a known-good vLLM version / nightly for nvidia/MiniMax-M2.5-NVFP4 that we should use instead of 0.18.0? Additional context Full HF file set present locally (29 shards, index, config.json, hf_quant_config.json, tokenizer files, modeling_minimax_m2.py, etc.) — ~130 GiB, consistent with the model’s file list. NVIDIA model card references TensorRT-LLM for the primary NVFP4 recipe; we are specifically trying vLLM and hitting this key mismatch. Thank you.

serving, bug, model: MiniMax, quantization

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.