pytorch - 💡(How to fix) Fix [Testing] Expand Testing for Non-Contiguous Tensor Ops Across All Platforms [1 pull requests]

🚀 The feature, motivation and pitch

Currently the test_ops.py::test_noncontiguous_samples is used to verify correctness of tensors ops on non-contiguous tensors across all platforms. The test operates by generating a contiguous and a non-contiguous tensor with the same shape and values, and comparing the results of the tensor ops on both tensors.

The noncontiguous tensor generation however (common_utils.py::noncontiguous_like) is a relatively simple case of uniformly scaling all strides by 2 and filling the gaps with zero or nans. This leaves out several more complicated cases of non-contiguity that often shows up on real world workloads.

We need to expand the test to cover more complicated cases as well as edge cases to ensure correctness across all platforms.

Alternatives

Expand non-contiguous tensor generation utility function with the following cases that cover a broader range of real world applications:

• Uniform Scale: existing - uniformly scale strides by 2x and fill the gaps with zero/nan
• Channels_last: apply non-contiguity for 4D and 5D tensors through torch.channels_last or torch.channels_last_3d
• Transpose: transpose last two dimensions, clone and then transpose back to preserve shape to create non-contiguity
• Slice: Embed into a larger tensor with gaps, and slice to get a strided non-contiguous view
• Channels_last + Sliced: combine the slicing and channels_last methods
• Permute: permute dimensions with random values, clone and permute back with inverse dimensions to preserve shape and create non-contiguity

Parametrize the test_noncontiguous_samples to run across all the expanded test cases above

Additional context

Motivated by Issue #181946 that modifies handling of non-contiguous tensors for a number of MPS tensor ops, and may introduce bugs that will not be caught with the limited existing non-contiguous testing.

This change is likely to uncover existing bugs and introduce failing tests across all platforms. Contributors familiar with non-MPS platforms should weight in before these expansions become part of official CIs.