pytorch - 💡(How to fix) Fix DISABLED test_vjp_doesnt_support_saved_tensor_hooks_cpu (__main__.TestComposabilityCPU) [1 comments, 1 participants]

Traceback (most recent call last): File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_utils.py", line 3448, in wrapper method(*args, **kwargs) ~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_device_type.py", line 459, in instantiated_test raise rte File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_device_type.py", line 439, in instantiated_test result = test(self, **param_kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 2480, in call result = self._torchdynamo_orig_backend( frame, cache_entry, self.hooks, frame_state, skip=1 ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 2198, in call result = self._inner_convert( frame, cache_entry, hooks, frame_state, skip=skip + 1 ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 758, in call result = _compile( frame.f_code, ...<16 lines>... convert_frame_box=self._box, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1983, in _compile guarded_code, tracer_output = compile_inner(code, one_graph, hooks) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_utils_internal.py", line 96, in wrapper_function return function(*args, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1593, in compile_inner result = _compile_inner(code, one_graph, hooks) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1652, in _compile_inner dynamo_output = compile_frame( code, ...<11 lines>... package=package, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1500, in compile_frame bytecode, tracer_output = transform_code_object(code, transform) ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/bytecode_transformation.py", line 1626, in transform_code_object tracer_output = transformations(instructions, code_options) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1472, in transform tracer_output = trace_frame( code, ...<14 lines>... package=package, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 365, in _fn return fn(*args, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 933, in trace_frame run_tracer() ~~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 914, in run_tracer tracer.run() ~~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1839, in run while self.step(): ~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1506, in step self.dispatch_table[inst.opcode](self, inst) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1043, in wrapper return inner_fn(self, inst) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 4431, in CALL_KW self._call(inst, call_kw=True) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 4207, in _call self.call_function(fn, args, kwargs) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1407, in call_function self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/lazy.py", line 306, in realize_and_forward return getattr(self.realize(), name)(args, **kwargs) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/torch.py", line 2724, in call_function tensor_variable = wrap_fx_proxy( tx=tx, ...<4 lines>... ), ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3115, in wrap_fx_proxy return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3190, in wrap_fx_proxy_cls out: VTTypeAlias = _wrap_fx_proxy( ~~~~~~~~~~~~~~^ target_cls, tx, proxy, example_value, subclass_type, **options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3314, in _wrap_fx_proxy example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3751, in get_fake_value return _get_fake_value_impl(node, tx, allow_non_graph_fake) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3942, in _get_fake_value_impl _wrap_graph_break_with_torch_runtime_err( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ lambda: unimplemented( ^^^^^^^^^^^^^^^^^^^^^^ ...<5 lines>... ) ^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3740, in _wrap_graph_break_with_torch_runtime_err raise exc.with_traceback(e.traceback) from None File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3737, in _wrap_graph_break_with_torch_runtime_err gb_fn() ~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3943, in <lambda> lambda: unimplemented( ~~~~~~~~~~~~~^ gb_type="RuntimeError when making fake tensor call", ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<3 lines>... from_exc=cause, ^^^^^^^^^^^^^^^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/exc.py", line 653, in unimplemented raise Unsupported( msg, gb_type, skip_frame, real_stack=past_real_stack ) from from_exc torch._dynamo.exc.TorchRuntimeError: RuntimeError when making fake tensor call Explanation: Dynamo failed to run FX node with fake tensors: call_function <built-in method randn of type object at 0x7fbb06575c00>((3,), **{'device': 'cpu'}): got RuntimeError('tls_on_entry.has_value() INTERNAL ASSERT FAILED at "/var/lib/jenkins/workspace/aten/src/ATen/core/PythonFallbackKernel.cpp":53, please report a bug to PyTorch. ') Hint: Your code may result in an error when running in eager. Please double check that your code doesn't contain a similar error when actually running eager/uncompiled. You can do this by removing the torch.compile call, or by using torch.compiler.set_stance("force_eager").

Developer debug context:

For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb4315.html

from user code: File "/var/lib/jenkins/workspace/test/functorch/test_eager_transforms.py", line 3835, in test_vjp_doesnt_support_saved_tensor_hooks x = torch.randn(3, device=device)

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"

To execute this test, run the following from the base repo dir: PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestComposabilityCPU.test_vjp_doesnt_support_saved_tensor_hooks_cpu

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TL;DR

The test test_vjp_doesnt_support_saved_tensor_hooks_cpu is failing due to a TorchRuntimeError caused by Dynamo's failure to run FX node with fake tensors, suggesting an issue with the torch.randn call.

Guidance

• The error message indicates a problem with the torch.randn function call, specifically when trying to create a tensor with fake values.
• To debug this issue, try removing the torch.compile call or setting torch.compiler.set_stance("force_eager") to run the code in eager mode and see if the error persists.
• Set TORCHDYNAMO_VERBOSE=1 to get the internal stack trace and TORCH_LOGS="+dynamo" for more developer context.
• Run the test with PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestComposabilityCPU.test_vjp_doesnt_support_saved_tensor_hooks_cpu to reproduce the error.

Example

No specific code example is provided, but the issue seems to be related to the line x = torch.randn(3, device=device) in the test_eager_transforms.py file.

Notes

The error message suggests that the issue might be related to the torch.randn function call, but the root cause is not immediately clear. More debugging and investigation are needed to determine the exact cause of the problem.

Recommendation

Apply workaround: Try setting torch.compiler.set_stance("force_eager") or removing the torch.compile call to run the code in eager mode and see if the error persists. This will help determine if the issue is specific to the compiled mode or if it's a more general problem.