Fix Plan

To address the issue, we will refactor the test cases to use the instantiate_device_type_tests mechanism. Here are the steps:

• Identify the template classes that need to be refactored, starting with the lowest difficulty ones:
  • EfficientConvBNEvalTemplate
  • BinaryFoldingTemplate
  • ProvenanceTracingKernelContextTemplate
• Modify the template classes to accept a device parameter in their test methods:

class EfficientConvBNEvalTemplate:
    def test_custom_op_add(self, device) -> None:
        # test implementation
        pass

• Replace the copy_tests calls with instantiate_device_type_tests:

instantiate_device_type_tests(EfficientConvBNEvalTemplate, globals())

• Repeat the process for the remaining template classes.

Example Code

Here's an example of how the EfficientConvBNEvalTemplate class could be refactored:

# Before
class EfficientConvBNEvalTemplate:
    def test_custom_op_add(self) -> None:
        # test implementation
        pass

copy_tests(
    EfficientConvBNEvalTemplate,
    EfficientConvBNEvalTestABICompatibleCpu,
    "cpu",
    CPU_TEST_FAILURES,
)

# After
class EfficientConvBNEvalTemplate:
    def test_custom_op_add(self, device) -> None:
        # test implementation
        pass

instantiate_device_type_tests(EfficientConvBNEvalTemplate, globals())

Verification

To verify that the fix worked, run the test cases and ensure that they pass without errors. Additionally, check that the instantiate_device_type_tests mechanism is correctly instantiating the test cases for each device type.

Extra Tips

• Make sure to update the test cases to handle any device-specific logic or errors.
• Consider adding additional logging or debugging statements to help diagnose any issues that may arise during testing.