vllm - 💡(How to fix) Fix [Bug]: Gemma 4 E4B weight loading fails `Gemma4ClippableLinear` parameter `input_max` not recognized [1 participants]

(EngineCore pid=322) INFO 04-03 03:39:22 [base.py:115] Using Transformers modeling backend. (EngineCore pid=322) WARNING 04-03 03:38:21 [utils.py:187] TransformersMultiModalForCausalLM has no vLLM implementation, falling back to Transformers implementation.

(EngineCore pid=322) ValueError: There is no module or parameter named 'model.audio_tower.layers.0.feed_forward1.ffw_layer_1.input_max' in TransformersMultiModalForCausalLM. The available parameters belonging to model.audio_tower.layers.0.feed_forward1.ffw_layer_1 (Gemma4ClippableLinear) are: {'model.audio_tower.layers.0.feed_forward1.ffw_layer_1.linear.weight'}

TL;DR

The most likely fix is to update the weight mapper in the fallback path to recognize the input_max parameter in the Gemma4ClippableLinear layer.

Guidance

• Verify that the issue is specific to the google/gemma-4-e4b-it model and the vLLM version 0.18.0 by trying to load other models.
• Check the documentation for TransformersMultiModalForCausalLM to see if there are any known issues or limitations with loading weights for the Gemma4ClippableLinear layer.
• Consider updating the vLLM version or the transformers library to see if the issue is resolved in a later version.
• If possible, try to modify the weight mapper to recognize the input_max parameter, or create a custom loader for the Gemma4ClippableLinear layer.

Notes

The issue seems to be specific to the Gemma4ClippableLinear layer and the TransformersMultiModalForCausalLM model, and may require a custom solution or an update to the vLLM or transformers libraries.

Recommendation

Apply a workaround by modifying the weight mapper or creating a custom loader for the Gemma4ClippableLinear layer, as updating to a fixed version is not clearly implied in the issue.