pytorch - 💡(How to fix) Fix [compile] Handle shallow_copy_data_ device mutation in ProxyTensor instead of save/restore workaround

Official PRs (…)
ON THIS PAGE

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

During make_fx tracing and FakeTensorProp, in-graph shallow_copy_data_ (used for cross-device tensor.data = ...) mutates the fake_device attribute on input FakeTensors. This requires a save/restore workaround in graph_capture.py and compile_fx.py to prevent the backend compiler from seeing incorrect input devices.

Root Cause

During make_fx tracing and FakeTensorProp, in-graph shallow_copy_data_ (used for cross-device tensor.data = ...) mutates the fake_device attribute on input FakeTensors. This requires a save/restore workaround in graph_capture.py and compile_fx.py to prevent the backend compiler from seeing incorrect input devices.

Fix Action

Fix / Workaround

During make_fx tracing and FakeTensorProp, in-graph shallow_copy_data_ (used for cross-device tensor.data = ...) mutates the fake_device attribute on input FakeTensors. This requires a save/restore workaround in graph_capture.py and compile_fx.py to prevent the backend compiler from seeing incorrect input devices.

Current workaround

RAW_BUFFERClick to expand / collapse

Summary

During make_fx tracing and FakeTensorProp, in-graph shallow_copy_data_ (used for cross-device tensor.data = ...) mutates the fake_device attribute on input FakeTensors. This requires a save/restore workaround in graph_capture.py and compile_fx.py to prevent the backend compiler from seeing incorrect input devices.

Current workaround

We save fake_device on all FakeTensor args before make_fx / FakeTensorProp, then restore them after. See:

  • torch/_functorch/_aot_autograd/graph_capture.py (make_fx tracing)
  • torch/_inductor/compile_fx.py (FakeTensorProp)

Desired behavior

Handle shallow_copy_data_ correctly in ProxyTensor, following how t_() works:

  • ProxyTensor snapshots placeholder meta["val"] via set_meta -> extract_val -> snapshot_fake -> fast_detach, creating an independent FakeTensor copy
  • The live FakeTensor gets mutated by the op, but the placeholder snapshot is unaffected
  • track_tensor_tree updates the proxy slot to point at the mutation node

The snapshot mechanism should already protect placeholder metadata (sizes, strides, storage_offset) since fast_detach creates a new TensorImpl via aten.detach. However, fake_device is a Python attribute on the FakeTensor wrapper, not on the TensorImpl. The fast_detach path does capture fake_device at snapshot time (FakeTensor(fake_mode, out, x.device)), so placeholder snapshots should be independent.

The remaining issue is that the arg FakeTensors themselves are mutated and reused after make_fx returns. This may require either:

  1. Confirming the snapshot mechanism is sufficient and removing the placeholder restoration (keeping only arg restoration)
  2. Making shallow_copy_data_'s FakeTensor kernel not mutate in-place during tracing

Dynamo graph inputs may need a separate fix since they don't go through ProxyTensor's snapshot path.

cc @ezyang @EikanWang @jgong5 @wenzhe-nrv @chauhang @penguinwu @eellison @aorenste @voznesenskym @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @jiayisunx @kadeng @amjames @jataylo @azahed98 @bdhirsh @bobrenjc93 @zou3519

Related PR: https://github.com/pytorch/pytorch/pull/185980

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING