System Info

transformers 5.10.1 (latest release)

generate() pre-builds attention masks for compilable caches via create_masks_for_generate, which maps each config.layer_types entry through LAYER_PATTERN_TO_MASK_FUNCTION_MAPPING.

Hybrid linear-attention models (Qwen3.5, Qwen3-Next, MiniMax, …) have linear_attention layer types, which have no entry in LAYER_PATTERN_TO_MASK_FUNCTION_MAPPING (they build their own per-layer masks in the forward — create_causal_mask for attention layers, a separate 2D/None mask for linear ones). So requesting a static/compilable cache makes generate() fail with a cryptic KeyError: 'linear_attention', before reaching the model.

This affects any compilable cache (cache_implementation="static", or a pre-built StaticCache passed via past_key_values=).

Note: _can_compile_fullgraph = False on these models does not guard this at runtime — it is read only by test skip-guards (if not model_class._can_compile_fullgraph: skipTest(...)), never by _valid_auto_compile_criteria. So the path is user-reachable and the breakage is untested.

Reproduction

Self-contained (no checkpoint download), small hybrid Qwen3.5:

import torch
from transformers import Qwen3_5ForCausalLM
from transformers.models.qwen3_5.configuration_qwen3_5 import Qwen3_5TextConfig

cfg = Qwen3_5TextConfig(
    vocab_size=2048, hidden_size=256, intermediate_size=256, num_hidden_layers=8,
    num_attention_heads=4, num_key_value_heads=2, head_dim=64,
    linear_conv_kernel_dim=4, linear_key_head_dim=32, linear_value_head_dim=32,
    linear_num_key_heads=2, linear_num_value_heads=4, pad_token_id=0, bos_token_id=1, eos_token_id=2,
)
model = Qwen3_5ForCausalLM(cfg).to("cuda").eval()
ids = torch.randint(10, 2000, (1, 16), device="cuda")
model.generate(ids, max_new_tokens=4, cache_implementation="static", disable_compile=True)
# KeyError: 'linear_attention'  (in create_masks_for_generate)

A dynamic cache (the default) works fine; only the static/compilable path fails.

Proposed fix

Make create_masks_for_generate defer to the model — return the raw attention mask so the forward builds its own per-layer masks — whenever a layer type has no registered mask function, instead of raising.

Expected behavior

generate(cache_implementation="static") on a hybrid linear-attention model should work (defer mask building to the model), not raise a cryptic KeyError.

Who can help?

@ArthurZucker @Cyrilvallez