transformers - 💡(How to fix) Fix [Inquiry] What is the current status and roadmap for supporting packed sequences (packing) on Qwen3.5/hybrid linear attention models?

Context

With the release of Qwen3.5, we see a powerful hybrid architecture that heavily relies on GatedDeltaNet linear attention (powered by flash-linear-attention / FLA) and causal convolutions, alongside traditional self-attention layers.

In standard LLM training (like Llama/Mistral), Packing (padding-free training via TRL or custom trainers) is a vital optimization to eliminate padding waste and boost throughput, especially during SFT and RLHF.

Our Question

We would like to learn about the official stance, current support status, or future roadmap regarding packed sequence training specifically for Qwen3.5 (and similar linear attention hybrid models) within the transformers ecosystem.

Specifically, we want to clarify:

Current Compatibility: Does the current implementation of Qwen3.5 in transformers safely support packing=True out-of-the-box when specialized kernels like FLA or causal-conv1d are enabled?

Boundary Handling: If it is supported or planned, how does the framework ensure that sample boundaries within a packed sequence are properly isolated (e.g., preventing cross-sample information leakage in the convolution states and linear attention blocks via seq_idx or cu_seqlens)?

Community Efforts: Are there any ongoing PRs, refactoring branches, or recommended best practices/workarounds that the community should follow if we want to leverage packing for Qwen3.5?

We are keen to understand the best path forward for scaling Qwen3.5 training using Hugging Face tooling, and we would be more than happy to help test any experimental branches or contribute to discussions if this is actively being worked on.

Thanks for the amazing work on supporting these new architectures!