Your current environment

• vLLM version: main (commit approx. 2025-04-05)
• Model: google/gemma-4-27b-it (or any Gemma4 multimodal model)
• Python: 3.12
• CUDA: 13.x
• Transformers: installed via pip in .vllm venv

🐛 Describe the bug

Starting vLLM with a Gemma4 multimodal model (e.g. google/gemma-4-27b-it) fails during engine core initialization with:

ValueError: Expected hidden_size to be 5376, but found: 72

Full traceback (abbreviated):

File ".../vllm/v1/worker/gpu_model_runner.py", line 5761, in profile_run
    dummy_encoder_outputs = self.model.embed_multimodal(...)
File ".../vllm/model_executor/models/transformers/multimodal.py", line 350, in embed_multimodal
    vision_embeddings = self.model.get_image_features(...)
File ".../transformers/models/gemma4/modeling_gemma4.py", line 905, in forward
    value_states = self.v_norm(value_states)
File ".../vllm/model_executor/layers/layernorm.py", line 241, in forward_static
    raise ValueError(
ValueError: Expected hidden_size to be 5376, but found: 72

Root Cause

When vLLM's TransformersModelBase._recursive_replace walks the model graph and replaces norm modules, it calls:

# base.py:442
elif child_module.__class__.__name__.endswith("RMSNorm"):
    new_module = replace_rms_norm_class(
        child_module, self.text_config.hidden_size  # ← always 5376 (LM hidden size)
    )

Inside replace_rms_norm_class (transformers/utils.py), the hidden_size argument is only overridden when the norm module has a weight parameter:

weight_meta = getattr(rms_norm, "weight", None)
if weight_meta is not None:
    kwargs["hidden_size"] = weight_meta.size(0)

Gemma4's vision encoder v_norm is Gemma4RMSNorm(head_dim=72, with_scale=False). With with_scale=False, no weight is registered, so hidden_size stays at 5376. The resulting RMSNorm(hidden_size=5376, has_weight=False) then validates x.shape[-1] == 5376 for every forward call, but the vision value states have shape [..., 72].

Fix

Fix 1 (primary) — vllm/model_executor/layers/layernorm.py: Only validate hidden_size when weight is not None. A weightless RMSNorm has no constraint on the input dimension.

# Before
if x.shape[-1] != hidden_size:
    raise ValueError(...)

# After
if weight is not None and x.shape[-1] != hidden_size:
    raise ValueError(...)

Fix 2 (defensive) — vllm/model_executor/models/transformers/utils.py: In replace_rms_norm_class, for weightless norms, try to infer the correct dimension from module attributes (dim, hidden_size, normalized_shape) before falling back to the text model's hidden_size.

else:
    # No weight: try to infer the norm's dimension from other attributes
    inferred = getattr_iter(
        rms_norm, ("dim", "hidden_size", "normalized_shape"), None
    )
    if inferred is not None:
        if isinstance(inferred, (list, tuple)):
            inferred = inferred[-1]
        kwargs["hidden_size"] = int(inferred)

Note: Fix 2 alone does not fix the Gemma4 case because Gemma4RMSNorm with with_scale=False does not store dim as an attribute. Fix 1 is necessary.

How to Reproduce

vllm serve google/gemma-4-27b-it --trust-remote-code

Engine crashes before serving any requests.

Expected behavior

Engine should start successfully and serve Gemma4 multimodal requests.