pytorch - 💡(How to fix) Fix DISABLED test_basic_sum_cpu (__main__.TestHigherOrderOperatorInteractionCPU) [1 comments, 1 participants]

Traceback (most recent call last): File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_utils.py", line 3514, in wrapper method(*args, **kwargs) ~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_device_type.py", line 459, in instantiated_test raise rte File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/testing/_internal/common_device_type.py", line 439, in instantiated_test result = test(self, **param_kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 2480, in call result = self._torchdynamo_orig_backend( frame, cache_entry, self.hooks, frame_state, skip=1 ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 2198, in call result = self._inner_convert( frame, cache_entry, hooks, frame_state, skip=skip + 1 ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 758, in call result = _compile( frame.f_code, ...<16 lines>... convert_frame_box=self._box, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1983, in _compile guarded_code, tracer_output = compile_inner(code, one_graph, hooks) ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_utils_internal.py", line 96, in wrapper_function return function(*args, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1593, in compile_inner result = _compile_inner(code, one_graph, hooks) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1652, in _compile_inner dynamo_output = compile_frame( code, ...<11 lines>... package=package, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1500, in compile_frame bytecode, tracer_output = transform_code_object(code, transform) ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/bytecode_transformation.py", line 1626, in transform_code_object tracer_output = transformations(instructions, code_options) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 1472, in transform tracer_output = trace_frame( code, ...<14 lines>... package=package, ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 365, in _fn return fn(*args, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 933, in trace_frame run_tracer() ~~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/convert_frame.py", line 914, in run_tracer tracer.run() ~~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1839, in run while self.step(): ~~~~~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1506, in step self.dispatch_table[inst.opcode](self, inst) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1043, in wrapper return inner_fn(self, inst) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 4431, in CALL_KW self._call(inst, call_kw=True) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 4207, in _call self.call_function(fn, args, kwargs) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/symbolic_convert.py", line 1407, in call_function self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/lazy.py", line 306, in realize_and_forward return getattr(self.realize(), name)(args, **kwargs) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/torch.py", line 2724, in call_function tensor_variable = wrap_fx_proxy( tx=tx, ...<4 lines>... ), ) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3117, in wrap_fx_proxy return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3192, in wrap_fx_proxy_cls out: VTTypeAlias = _wrap_fx_proxy( ~~~~~~~~~~~~~~^ target_cls, tx, proxy, example_value, subclass_type, **options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/variables/builder.py", line 3316, in _wrap_fx_proxy example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3757, in get_fake_value return _get_fake_value_impl(node, tx, allow_non_graph_fake) File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3948, in _get_fake_value_impl _wrap_graph_break_with_torch_runtime_err( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ lambda: unimplemented( ^^^^^^^^^^^^^^^^^^^^^^ ...<5 lines>... ) ^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3746, in _wrap_graph_break_with_torch_runtime_err raise exc.with_traceback(e.traceback) from None File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3743, in _wrap_graph_break_with_torch_runtime_err gb_fn() ~~~~~^^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/utils.py", line 3949, in <lambda> lambda: unimplemented( ~~~~~~~~~~~~~^ gb_type="RuntimeError when making fake tensor call", ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<3 lines>... from_exc=cause, ^^^^^^^^^^^^^^^ ) ^ File "/opt/conda/envs/py_3.14/lib/python3.14/site-packages/torch/_dynamo/exc.py", line 653, in unimplemented raise Unsupported( msg, gb_type, skip_frame, real_stack=past_real_stack ) from from_exc torch._dynamo.exc.TorchRuntimeError: RuntimeError when making fake tensor call Explanation: Dynamo failed to run FX node with fake tensors: call_function <built-in method randn of type object at 0x7f3c2fd75c40>((2, 3, 4), **{'device': 'cpu'}): got RuntimeError('tls_on_entry.has_value() INTERNAL ASSERT FAILED at "/__w/pytorch/pytorch/aten/src/ATen/core/PythonFallbackKernel.cpp":53, please report a bug to PyTorch. ') Hint: Your code may result in an error when running in eager. Please double check that your code doesn't contain a similar error when actually running eager/uncompiled. You can do this by removing the torch.compile call, or by using torch.compiler.set_stance("force_eager").

Developer debug context:

For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb4315.html

from user code: File "/__w/pytorch/pytorch/test/functorch/test_eager_transforms.py", line 5283, in test_basic_sum x = torch.randn(2, 3, 4, device=device)

Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace (please do this especially if you're reporting a bug to PyTorch). For even more developer context, set TORCH_LOGS="+dynamo"

To execute this test, run the following from the base repo dir: PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestHigherOrderOperatorInteractionCPU.test_basic_sum_cpu

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TL;DR

The test test_basic_sum_cpu is failing due to a TorchRuntimeError caused by Dynamo's failure to run FX node with fake tensors, and the issue can be investigated by checking the logs and debugging the test.

Guidance

• To debug the issue, follow the provided debugging instructions to find relevant log snippets and study the logs.
• Remove the torch.compile call or use torch.compiler.set_stance("force_eager") to check if the code contains a similar error when running in eager mode.
• Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace and TORCH_LOGS="+dynamo" for more developer context.
• Run the test using PYTORCH_TEST_WITH_DYNAMO=1 python test/functorch/test_eager_transforms.py TestHigherOrderOperatorInteractionCPU.test_basic_sum_cpu to reproduce the issue.

Example

No code snippet is provided as the issue is related to a specific test failure and requires debugging and investigation.

Notes

The issue is specific to the test_basic_sum_cpu test and may be related to the use of torch.randn with fake tensors. The provided debugging instructions and logs should help identify the root cause of the issue.

Recommendation

Apply workaround: Remove the torch.compile call or use torch.compiler.set_stance("force_eager") to check if the code contains a similar error when running in eager mode, and investigate the logs to identify the root cause of the issue.