pytorch - 💡(How to fix) Fix `torch.compile` silently produces wrong gradient for `atan2(x, y).amin(dim)` on float32

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

torch.atan2(x, y).amin(dim=-1).sum().backward() produces a wrong x.grad under torch.compile (Inductor backend). The gradient at one or more argmin positions is dropped (becomes 0.0 or -0.0) instead of receiving the correct upstream value.

Forward output matches exactly between eager and compiled. Only the backward is wrong, and only in Inductor — aot_eager is correct.

Distinct from existing scatter-family stride-0 grad bugs (#179561, #180164, #182012, #183986) — those involve scatter ops feeding into reductions. This bug involves a plain elementwise binary op (atan2) feeding into an amin/amax reduction, no scatter required.

Root Cause

Root cause hypothesis

Code Example

import torch

torch.manual_seed(0)
x_init = torch.randn(2, 3)
y = torch.randn(2, 3)

def fn(x, y):
    return torch.atan2(x, y).amin(dim=-1)

x_e = x_init.detach().clone().requires_grad_(True)
fn(x_e, y).sum().backward()

x_c = x_init.detach().clone().requires_grad_(True)
torch.compile(fn)(x_c, y).sum().backward()

print("eager grad   :", x_e.grad.flatten().tolist())
print("compiled grad:", x_c.grad.flatten().tolist())
print("max diff:", (x_e.grad - x_c.grad).abs().max().item())

---

eager grad   : [0.0, 0.0, -0.13662527501583099, -0.0, -0.38940736651420593, 0.0]
compiled grad: [0.0, 0.0, -0.0,                 -0.0, -0.38940736651420593, 0.0]
max diff: 0.13662527501583099
                ^^^^^^^^^^^^^^^^^
                argmin position of row 0 — eager has -0.1366, compiled has -0.0

---

aot_eager   : x.grad diff = 0.000000e+00   OK
inductor    : x.grad diff = 1.366253e-01   BUG

---

$ python -c "import torch; print(torch.__version__)"
2.13.0.dev20260608+cpu
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

Summary

torch.atan2(x, y).amin(dim=-1).sum().backward() produces a wrong x.grad under torch.compile (Inductor backend). The gradient at one or more argmin positions is dropped (becomes 0.0 or -0.0) instead of receiving the correct upstream value.

Forward output matches exactly between eager and compiled. Only the backward is wrong, and only in Inductor — aot_eager is correct.

Distinct from existing scatter-family stride-0 grad bugs (#179561, #180164, #182012, #183986) — those involve scatter ops feeding into reductions. This bug involves a plain elementwise binary op (atan2) feeding into an amin/amax reduction, no scatter required.

Minimal reproducer

import torch

torch.manual_seed(0)
x_init = torch.randn(2, 3)
y = torch.randn(2, 3)

def fn(x, y):
    return torch.atan2(x, y).amin(dim=-1)

x_e = x_init.detach().clone().requires_grad_(True)
fn(x_e, y).sum().backward()

x_c = x_init.detach().clone().requires_grad_(True)
torch.compile(fn)(x_c, y).sum().backward()

print("eager grad   :", x_e.grad.flatten().tolist())
print("compiled grad:", x_c.grad.flatten().tolist())
print("max diff:", (x_e.grad - x_c.grad).abs().max().item())

Output on 2.13.0.dev20260608+cpu:

eager grad   : [0.0, 0.0, -0.13662527501583099, -0.0, -0.38940736651420593, 0.0]
compiled grad: [0.0, 0.0, -0.0,                 -0.0, -0.38940736651420593, 0.0]
max diff: 0.13662527501583099
                ^^^^^^^^^^^^^^^^^
                argmin position of row 0 — eager has -0.1366, compiled has -0.0

Forward output is identical between eager and compiled (no fwd diff).

Backend isolation

aot_eager   : x.grad diff = 0.000000e+00   OK
inductor    : x.grad diff = 1.366253e-01   BUG

Bug is exclusive to Inductor.

Robustness (random sweep)

ConditionResult
100 random (seed, shape) combinations on float3231 / 100 reproduce the divergence
float32BUG (consistent 30%+ hit rate)
float64OK (diff=0.0)
float16 / bfloat16NaN in compiled gradient (likely a related lowering bug)
dim=0 reductionOK
dim=-1 reduction (= dim=1 for 2D)BUG
1D tensorsOK

What does and doesn't trigger the bug

Forward chainBackward Bug?
atan2(x, y).amin(dim=-1)BUG
atan2(x, y).amax(dim=-1)BUG
atan2(x, y).abs().amin(dim=-1)BUG
atan2(x, y).square().amin(dim=-1)BUG
atan2(x, y).min(dim=-1)[0]OK (tuple-returning min!)
(x - y).abs().amin(dim=-1)OK
(x + y).amin(dim=-1)OK
atan2(x, y).relu().amin(dim=-1)OK
atan2(x, y).sigmoid().amin(dim=-1)OK
atan2(x, y).sum()OK (any non-amin/amax reduction)

Two distinguishing details:

  1. .min(dim=-1)[0] is OK, .amin(dim=-1) is BUG. Same semantics, different lowering path in Inductor. The amin/amax lowering must be missing something the min/max lowering has.
  2. atan2 is required. Replacing it with add, sub, mul, or div makes the bug disappear. Other binary primitives also need their derivative to flow through the reduction — atan2's derivative ∂atan2(a,b)/∂a = b/(a²+b²) is the specific shape Inductor is mishandling.

Root cause hypothesis

Looking at the output: at the row-0 argmin position, Inductor produces -0.0 (signed zero) instead of the correct finite value -0.1366. This is the signature of a fused argmin + scatter codegen path that writes the chain-rule contribution to the wrong location, or writes zero where it should write the derivative.

The fact that .min(dim)[0] works but .amin(dim) doesn't strongly suggests this is a regression in Inductor's amin/amax backward decomposition — likely a specialized lowering of amin that doesn't correctly recompose with non-trivial elementwise predecessors like atan2.

Versions

Versions

$ python -c "import torch; print(torch.__version__)"
2.13.0.dev20260608+cpu

Also reproduces on 2.11.0+cpu.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING