vllm - 💡(How to fix) Fix [Feature]: Integrate FlashQLA for fast linear attention [1 participants]

🚀 The feature, motivation and pitch

https://github.com/QwenLM/FlashQLA was published by the Qwen team recently which promises a fast implementation of chunk_gated_delta_rule used in Qwen3.5 / Qwen3.6. It might be useful to integrate it into vllm to speedup models using linear attention.

Alternatives

Currently vllm uses either a customised version of https://github.com/fla-org/flash-linear-attention/ or https://github.com/flashinfer-ai/flashinfer for GDN. Their benchmarks promise to outperform these alternatives.

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.