Issue Station
Sign In
Tools hereToolbox

pytorch - ✅(Solved) Fix Multi-level Skip Mechanism for backends(operator level based on @ops) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#177256•Fetched 2026-04-08 00:42:37
View on GitHub
Comments
0
Participants
1
Timeline
13
Reactions
0
Author
fffrogfffrog
Participants
fffrogfffrog
Timeline (top)
cross-referenced ×4labeled ×3project_v2_item_status_changed ×2added_to_project_v2 ×1

Fix Action

Fixed

PR fix notes

PR #176264: [Test] Add device type test support for op decorators and skips in DeviceTypeTestBase

Description (problem / solution / changelog)

Summary

This pull request enhances the device type test infrastructure to allow device-specific decorators and skips for operator tests, and demonstrates this with a new test for the Open Registration (PrivateUse1) extension. The main improvements are the introduction of op_decorators and op_skips hooks, and a new test file that exercises these features for custom backend scenarios.

Device-specific test customization:

  • Added op_decorators and op_skips class attributes to DeviceTypeTestBase, enabling device-specific decorators (like expectedFailure) and skips to be applied to operator tests.
  • Implemented update_op_list class method to apply these decorators and skips to the relevant operator tests, handling device type mapping (including custom backends like PrivateUse1).
  • Integrated update_op_list into the test parametrization process to ensure operator test configurations are updated before test instantiation.

Testing and demonstration:

  • Added a new test file test/cpp_extensions/open_registration_extension/torch_openreg/tests/test_testing.py that demonstrates the use of op_decorators and op_skips for custom backend operator tests, including examples with dummy ops and expected failure/skipped tests.
(pytorch-dev-cuda) ➜  pytorch-cuda git:(testing/op-filter) ✗ python test/cpp_extensions/open_registration_extension/torch_openreg/tests/test_testing.py --verbose
test_normal_openreg (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_normal_openreg) ... ok
test_op_dummy_op1_openreg_bfloat16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_bfloat16) ... ok
test_op_dummy_op1_openreg_bool (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_bool) ... ok
test_op_dummy_op1_openreg_complex128 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_complex128) ... ok
test_op_dummy_op1_openreg_complex64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_complex64) ... ok
test_op_dummy_op1_openreg_float16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_float16) ... ok
test_op_dummy_op1_openreg_float32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_float32) ... ok
test_op_dummy_op1_openreg_float64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_float64) ... ok
test_op_dummy_op1_openreg_int16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_int16) ... ok
test_op_dummy_op1_openreg_int32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_int32) ... ok
test_op_dummy_op1_openreg_int64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_int64) ... ok
test_op_dummy_op1_openreg_int8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_int8) ... ok
test_op_dummy_op1_openreg_uint8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op1_openreg_uint8) ... ok
test_op_dummy_op2_openreg_bfloat16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_bfloat16) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_bool (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_bool) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_complex128 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_complex128) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_complex64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_complex64) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_float16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_float16) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_float32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_float32) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_float64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_float64) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_int16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_int16) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_int32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_int32) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_int64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_int64) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_int8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_int8) ... skipped 'skip dummy_op2'
test_op_dummy_op2_openreg_uint8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op2_openreg_uint8) ... skipped 'skip dummy_op2'
test_op_dummy_op3_openreg_bfloat16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_bfloat16) ... expected failure
test_op_dummy_op3_openreg_bool (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_bool) ... expected failure
test_op_dummy_op3_openreg_complex128 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_complex128) ... expected failure
test_op_dummy_op3_openreg_complex64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_complex64) ... expected failure
test_op_dummy_op3_openreg_float16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_float16) ... expected failure
test_op_dummy_op3_openreg_float32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_float32) ... expected failure
test_op_dummy_op3_openreg_float64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_float64) ... expected failure
test_op_dummy_op3_openreg_int16 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_int16) ... expected failure
test_op_dummy_op3_openreg_int32 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_int32) ... expected failure
test_op_dummy_op3_openreg_int64 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_int64) ... expected failure
test_op_dummy_op3_openreg_int8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_int8) ... expected failure
test_op_dummy_op3_openreg_uint8 (__main__.TestDeviceTypeOpenRegPRIVATEUSE1.test_op_dummy_op3_openreg_uint8) ... expected failure

----------------------------------------------------------------------
Ran 37 tests in 0.281s

OK (skipped=12, expected failures=12)

Changed files

  • test/cpp_extensions/open_registration_extension/torch_openreg/tests/test_testing.py (added, +50/-0)
  • torch/testing/_internal/common_device_type.py (modified, +32/-0)

Code Example

class DeviceTypeTestBase(TestCase):
    op_decorators = None  # dict[op_full_name, list[DecorateInfo]]
    op_skips = None

    @classmethod
    def update_op_list(cls, ops_parametrizer): # new
        if cls.op_decorators is not None:
            for op in ops_parametrizer.op_list:
                if op.full_name in cls.op_decorators:
                    op.decorators = (*op.decorators, *cls.op_decorators[op.full_name])
        # the same logic for op_skips

class ops(_TestParametrizer):
    def _parametrize_test(self, test, generic_cls, device_cls):
        device_cls.update_op_list(self)  # New
        for op in self.op_list:
            ...
RAW_BUFFERClick to expand / collapse

PyTorch uses op_db and instantiate_device_type_tests to dynamically instantiate operator test cases. This greatly simplifies the development and maintenance of operator tests.

However, since new backends may differ in progress of operator support and numerical precision and so on, so we propose adding a configuration-based mechanism to skip operator tests.

class DeviceTypeTestBase(TestCase):
    op_decorators = None  # dict[op_full_name, list[DecorateInfo]]
    op_skips = None

    @classmethod
    def update_op_list(cls, ops_parametrizer): # new
        if cls.op_decorators is not None:
            for op in ops_parametrizer.op_list:
                if op.full_name in cls.op_decorators:
                    op.decorators = (*op.decorators, *cls.op_decorators[op.full_name])
        # the same logic for op_skips

class ops(_TestParametrizer):
    def _parametrize_test(self, test, generic_cls, device_cls):
        device_cls.update_op_list(self)  # New
        for op in self.op_list:
            ...

cc @mruberry

extent analysis

Fix Plan

To add a configuration-based mechanism to skip operator tests, we will introduce a new attribute op_skips in the DeviceTypeTestBase class and modify the update_op_list method to handle skipped operators.

Code Changes

class DeviceTypeTestBase(TestCase):
    op_decorators = None  # dict[op_full_name, list[DecorateInfo]]
    op_skips = None  # dict[op_full_name, bool]

    @classmethod
    def update_op_list(cls, ops_parametrizer):
        if cls.op_decorators is not None:
            for op in ops_parametrizer.op_list:
                if op.full_name in cls.op_decorators:
                    op.decorators = (*op.decorators, *cls.op_decorators[op.full_name])
        if cls.op_skips is not None:
            for op in ops_parametrizer.op_list:
                if op.full_name in cls.op_skips and cls.op_skips[op.full_name]:
                    op.skip = True  # or raise SkipTest("Operator skipped")

class ops(_TestParametrizer):
    def _parametrize_test(self, test, generic_cls, device_cls):
        device_cls.update_op_list(self)
        for op in self.op_list:
            if not hasattr(op, 'skip') or not op.skip:
                # proceed with the test
                ...

Verification

To verify the fix, create a test case with op_skips attribute set to skip a specific operator and check that the test is indeed skipped.

Extra Tips

  • Make sure to update the documentation to reflect the new op_skips attribute and its usage.
  • Consider adding a logging mechanism to track skipped tests for better debugging and monitoring.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#cache error#pipeline error#runtime error#dependency conflict#environment setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING