vllm - 💡(How to fix) Fix [Feature]: Support DFlash for Kimi K2.5 and Qwen3.5-27B for AMD [2 comments, 3 participants]

TL;DR

Enable support for non-causal attention in AMD backends to allow diffusion-based speculative decoders like DFlash to run on AMD hardware.

Guidance

• Investigate the current implementation of attention mechanisms in the AMD backend to identify potential modifications for supporting non-causal attention.
• Research the specific requirements of DFlash and similar diffusion-based speculative decoders to ensure compatibility with the proposed feature.
• Consider collaborating with developers of DFlash or similar models to ensure a comprehensive understanding of the necessary changes.
• Evaluate the feasibility of implementing non-causal attention in the AMD backend without compromising performance or introducing significant complexity.

Notes

The solution may require significant modifications to the existing AMD backend implementation, and careful consideration of performance and complexity trade-offs is necessary.

Recommendation

Apply a workaround by collaborating with the DFlash developers to implement a model-specific solution that can run on AMD hardware, as a full feature implementation may require substantial changes to the AMD backend.