pytorch - 💡(How to fix) Fix Extend nvMatmulHeuristics config key tuple for stages/split_k [1 participants]

Fix Plan

The fix involves extending the kernel config key tuple in torch/_inductor/template_heuristics/nv_universal_gemm.py to include stages and split_k parameters.

Steps

• Update the kernel_config_key function to include stages and split_k:

def kernel_config_key(operator, operand_shapes, operand_dtypes, 
                      operator_type, tile_n, tile_m, tile_k, 
                      stages, split_k):
    # ...
    key = (operator_type, tile_n, tile_m, stages, split_k)
    # ...
    return key

• Ensure that stages and split_k are properly handled in the nvMatmulHeuristics and cutlass_api to avoid conflicts with tile_k.

Verification

• Test the updated kernel_config_key function with different stages and split_k values to ensure it generates the correct key tuples.
• Verify that the updated key tuples are correctly used in the nv_universal_gemm.py file.

Extra Tips

• Be cautious when updating the kernel_config_key function to avoid introducing regressions.
• Consider adding tests to cover different scenarios and ensure the fix works as expected.