pytorch - 💡(How to fix) Fix DISABLED test_lennard_jones_batched_jac_jac_jacfwd_cpu (__main__.TestExamplesCorrectnessCPU) [1 comments, 1 participants]

Traceback (most recent call last): File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_dynamo/compiled_autograd.py", line 357, in begin_capture inputs[idx] = self.wrap_fake(x, self.source("inputs", idx)) ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_dynamo/compiled_autograd.py", line 303, in wrap_fake return self.fake_tensor_mode.from_tensor(x, source=source) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_subclasses/fake_tensor.py", line 3187, in from_tensor return self.fake_tensor_converter.from_real_tensor( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ self, ^^^^^ ...<4 lines>... trace=trace, ^^^^^^^^^^^^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_subclasses/fake_tensor.py", line 448, in from_real_tensor out = self.meta_converter( t, ...<4 lines>... trace=trace, ) File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_subclasses/meta_utils.py", line 2235, in call r = self.meta_tensor( t_desc, ...<3 lines>... symbolic_context, ) File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_subclasses/meta_utils.py", line 1897, in meta_tensor r = view_from_base(base, t) File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_subclasses/meta_utils.py", line 1342, in view_from_base base.as_strided(sizes, strides, storage_offset) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: tls_on_entry.has_value() INTERNAL ASSERT FAILED at "/var/lib/jenkins/workspace/aten/src/ATen/core/PythonFallbackKernel.cpp":53, please report a bug to PyTorch.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4530, in test_lennard_jones_batched_jac r = torch.linspace(0.5, 2 * sigma, steps=100, requires_grad=True, device=device) File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4539, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4530 nn.Linear(1, 16), File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4541, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4539 nn.Linear(16, 16), File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4543, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4541 nn.Linear(16, 16), File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4545, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4543 nn.Linear(16, 16), File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4547, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4545 nn.Linear(16, 1), File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 4590, in torch_dynamo_resume_in_test_lennard_jones_batched_jac_at_4547 signs = torch.randint(0, 2, (n_samples,)) * 2 - 1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~ File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/autograd/init.py", line 530, in grad result = _engine_run_backward( outputs, ...<5 lines>... accumulate_grad=False, ) File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/autograd/graph.py", line 882, in _engine_run_backward return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ t_outputs, *args, **kwargs ^^^^^^^^^^^^^^^^^^^^^^^^^^ ) # Calls into the C++ engine to run the backward pass ^ File "/opt/conda/envs/py_3.14/lib/python3.14t/site-packages/torch/_dynamo/compiled_autograd.py", line 359, in begin_capture raise NotImplementedError( f"Found tensor of type {type(x)}, which is not supported by FakeTensorMode. {TURN_OFF_MSG}" ) from e NotImplementedError: Found tensor of type <class 'torch.Tensor'>, which is not supported by FakeTensorMode. You can turn off compiled autograd by either:

1. Moving the unsupported autograd call outside of the torch.compile'd region.
2. Wrapping the unsupported autograd call in the torch._dynamo.compiled_autograd._disable() context manager.
3. Setting torch._dynamo.config.compiled_autograd=False for the torch.compile call containing the unsupported autograd call.
4. Setting torch._dynamo.config.compiled_autograd=False at the start of the program.

To execute this test, run the following from the base repo dir: PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestExamplesCorrectnessCPU.test_lennard_jones_batched_jac_jac_jacfwd_cpu

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TL;DR

The test test_lennard_jones_batched_jac_jac_jacfwd_cpu is failing due to an issue with compiled autograd in PyTorch, and a potential fix is to disable compiled autograd for the specific test or region of code causing the issue.

Guidance

• The error message suggests that the issue is related to compiled autograd not supporting a certain type of tensor, and provides several potential workarounds, including moving the unsupported autograd call outside of the torch.compile'd region, wrapping the call in a context manager, or setting torch._dynamo.config.compiled_autograd=False.
• To debug the issue, follow the provided instructions to find relevant log snippets in the workflow logs, and study the logs to understand the specific cause of the failure.
• Consider setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 to suppress the error message and make it easier to debug.
• The test can be executed manually using the provided command PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestExamplesCorrectnessCPU.test_lennard_jones_batched_jac_jac_jacfwd_cpu to reproduce the issue.

Example

# Example of how to disable compiled autograd for a specific region of code
with torch._dynamo.compiled_autograd._disable():
    # Code that uses unsupported autograd call
    pass

Notes

• The issue is specific to the test_lennard_jones_batched_jac_jac_jacfwd_cpu test, and the fix may need to be applied only to this test or region of code.
• The provided workarounds may have performance implications, and should be carefully evaluated before applying them to production code.

Recommendation

Apply workaround: Disable compiled autograd for the specific test or region of code causing the issue, using one of the methods suggested in the error message, such as wrapping the call in a context manager or setting torch._dynamo.config.compiled_autograd=False. This will allow the test to run without the compiled autograd issue, but may have performance implications.