pytorch - 💡(How to fix) Fix DISABLED test_comprehensive_linalg_multi_dot_cuda_float16 (__main__.TestInductorOpInfoCUDA) [2 comments, 1 participants]

Traceback (most recent call last): File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1165, in test_wrapper return test(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1499, in only_fn return fn(self, *args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2504, in wrapper fn(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1245, in dep_fn return fn(slf, *args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1245, in dep_fn return fn(slf, *args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1766, in wrapper fn(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1666, in wrapper fn(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/mock.py", line 1379, in patched return func(*newargs, **newkeywargs) File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor_opinfo.py", line 1154, in inner raise e File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor_opinfo.py", line 1146, in inner fn(self, device, dtype, op) File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor_opinfo.py", line 1420, in test_comprehensive raise e File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor_opinfo.py", line 1393, in test_comprehensive self.check_model_gpu( File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor.py", line 748, in check_model_gpu check_model( File "/var/lib/jenkins/workspace/test/inductor/test_torchinductor.py", line 703, in check_model self.assertEqual( File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/test_case.py", line 113, in assertEqual return super().assertEqual(x, y, *args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 4365, in assertEqual raise error_metas.pop()[0].to_error( # type: ignore[index] AssertionError: The values for attribute 'stride()' do not match: (8, 1) != (2, 1).

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 59, in testPartExecutor yield File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 591, in run self._callTestMethod(testMethod) File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 549, in _callTestMethod method() File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 3370, in wrapper method(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 3370, in wrapper method(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 430, in instantiated_test result = test(self, **param_kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1766, in wrapper fn(*args, **kwargs) File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1177, in test_wrapper raise e_tracked from e Exception: The values for attribute 'stride()' do not match: (8, 1) != (2, 1).

Caused by sample input at index 6: SampleInput(input=TensorList[Tensor[size=(2, 4), device="cuda:0", dtype=torch.float16], Tensor[size=(4, 3), device="cuda:0", dtype=torch.float16], Tensor[size=(3, 5), device="cuda:0", dtype=torch.float16], Tensor[size=(5, 3), device="cuda:0", dtype=torch.float16], Tensor[size=(3, 2), device="cuda:0", dtype=torch.float16]], args=(), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir: PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 python test/inductor/test_torchinductor_opinfo.py TestInductorOpInfoCUDA.test_comprehensive_linalg_multi_dot_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

Fix Name: Force contiguous layout for the tensors passed to torch.linalg.multi_dot (float16 CUDA) in the Inductor path.

Why this works

The failure is a stride mismatch:

expected stride (8, 1)   ← result from the reference implementation
actual   stride (2, 1)   ← result from the Inductor‑compiled graph

When the input list contains tensors with non‑default strides (e.g., a view created by a transpose), the Inductor lowering currently keeps the original stride, while the reference implementation calls .contiguous() internally. Making the inputs contiguous before the multi_dot call forces both paths to use the same memory layout, eliminating the stride discrepancy.

Step‑by‑step fix

1. Add a small wrapper in the test (or in the library) that makes every operand contiguous before calling multi_dot.
   This is the quickest way to stop the flaky test without touching the core compiler.

   import torch
   
   def safe_multi_dot(tensors):
       # Ensure each operand is contiguous on CUDA and in float16
       contig = [t.contiguous() for t in tensors]
       return torch.linalg.multi_dot(contig)
   
   # In the test replace the original call:
   # result = torch.linalg.multi_dot(tensor_list)
   # with
   result = safe_multi_dot(tensor_list)

2. If you prefer a permanent library‑level fix, patch the Inductor lowering for aten.linalg_multi_dot (found in torch/_inductor/ir/ops.py or similar). Insert a contiguous call on each operand before the kernel is generated:

   # Inside the lowering for multi_dot (pseudo‑code)
   def lower_multi_dot(self, tensors):
       # tensors is a list of IR values
       contig_tensors = [self.graph.call_method("contiguous", [t]) for t in tensors]
       return self.graph.call_method("linalg_multi_dot", contig_tensors)

   Only add the contiguous call for the CUDA‑float16 variant to avoid unnecessary copies on other back‑ends.

3. Run the test locally to confirm the fix.

   PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=6 \
   python test/inductor/test_torchinductor_opinfo.py \
       TestInductorOpInfoCUDA.test_comprehensive_linalg_multi_dot_cuda_float16

   The test should now pass consistently.

Verification checklist

• The specific flaky test (test_comprehensive_linalg_multi_dot_cuda_float16) runs without a stride‑mismatch assertion.
• No new failures appear in the surrounding TestInductorOpInfoCUDA suite.
• Performance impact is negligible (contiguous copy of a few small tensors is cheap).

Extra tips / caveats

• Adding .contiguous() incurs a copy only when the tensor is not already contiguous; the overhead is minimal for the tiny matrices used in the OpInfo tests.
• If you later see similar stride‑related flakes for other ops, consider adding a generic “make inputs contiguous for CUDA‑float16 kernels” guard in the Inductor lowering pipeline.
• Keep the wrapper (safe_multi_dot) confined to the test suite until the core compiler fix lands, to avoid changing public API semantics.