pytorch - 💡(How to fix) Fix NVGEMM: drop hand-picked supplement configs once nvMatmulHeuristics covers them [1 comments, 1 participants]

TL;DR

Remove the nvgemm_supplement_configs config flag and the associated hand-picked list of tile/cluster configurations once nvMatmulHeuristics' search space is extended to include these configurations.

Guidance

• Verify that nvMatmulHeuristics' search space includes the configurations in the hand-picked list at template_heuristics/nv_universal_gemm.py:179-211.
• Remove the if config.nvgemm_supplement_configs: block once the verification is successful.
• Update the config.nvgemm_supplement_configs flag to be removed or disabled.
• Test the performance with the updated nvMatmulHeuristics to ensure it matches or exceeds the previous performance.

Notes

This fix assumes that the upstream update to nvMatmulHeuristics will include the necessary configurations to replace the hand-picked list.

Recommendation

Apply workaround: Remove the nvgemm_supplement_configs config flag and the associated hand-picked list of tile/cluster configurations once nvMatmulHeuristics' search space is extended, as this will simplify the code and reduce maintenance overhead.