vllm - 💡(How to fix) Fix [Bug]: MTP DeepSeek and Eagle Flash Attention Failures in Spec Decode Unit Tests [1 participants]

  Failure 1: test_eagle_correctness_light[FLASH_ATTN-deepseek_eagle]
                             
  Model: eagle618/deepseek-v3-random with FLASH_ATTN backend     
                                 
  Root cause: Flash Attention 4 (FA4) on SM100/SM110 (Blackwell) does not support head_dim=192 with equal Q and V head dimensions. The validation at vllm/vllm_flash_attn/cute/interface.py:114 enforces:
  - Standard range: head_dim between 8-128, divisible by 8       
  - Special DeepSeek shape: (192, 128) only (asymmetric Q/V)
                                                                 
  The eagle618/deepseek-v3-random model has (head_dim, head_dim_v) = (192, 192) which doesn't match either pattern. The actual DeepSeek-V3 production model uses (192, 128) and would work fine — this is specific to the random test model's
  symmetric head dims.                                                                                                                                                                                                                              
                                                                 
  Error:                                                                                                                                                                                                                                            
  AssertionError: (head_dim, head_dim_v)=(192, 192) is not supported on SM100/SM110.                                                                                                                                                                
  head_dim and head_dim_v must be between 8 and 128 and divisible by 8, or (192, 128) for DeepSeek.                                                                                                                                                 
                                                                                                                                                                                                                                                    
  Call chain: _run_eagle_correctness → LLM() → EngineCore init → CUDA graph capture → flash_attn_varlen_func → _validate_head_dims                                                                                                                  
                                                                                                                                                                                                                                                    
  ---                                                                                                                                                                                                                                               
  Failure 2: test_mtp_correctness[deepseek]                                                                                                                                                                                                         
                                                                                                                                                                                                                                                    
  Model: ZixiQi/DeepSeek-V3-4layers-MTP-FP8 with auto attention backend                                                                                                                                                                             
                                                                                                                                                                                                                                                    
  Root cause: FlashInfer's TRTLLM fused MoE kernel on SM100 has a strict inequality check: top_k < (topk_group * num_experts / n_group). The DeepSeek-V3 model's routing config hits this boundary exactly with 4 == 4, causing the check to fail.  
  The upstream code in flashinfer/csrc/trtllm_fused_moe_kernel_launcher.cu:928 uses < instead of <=.
                                                                                                                                                                                                                                                    
  Error:                                                                                                                                                                                                                                            
  RuntimeError: Check failed: args->top_k < (args->topk_group * args->num_experts / args->n_group) (4 vs. 4) :                                                                                                                                      
  top_k must be less than total number of experts in selected groups                                                                                                                                                                                
                                                                                                                                                                                                                                                    
  Call chain: test_mtp_correctness → LLM() → EngineCore init → moe_forward_shared → TRTLLM fused MoE kernel → strict top_k check fails                                                                                                              
                                                                                   
  This is an upstream FlashInfer bug (off-by-one: should be <= not <). Workaround would be to force moe_backend="triton" for this model on SM100+ or skip the test on Blackwell.