Issue Station
Sign In
Tools hereToolbox

pytorch - ✅(Solved) Fix [inductor] torch.compile's RNG state is inconsistent with eager on non-contiguous tensor when size >= 16 [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#177652•Fetched 2026-04-08 00:52:45
View on GitHub
Comments
2
Participants
2
Timeline
17
Reactions
0
Author
maybeLeemaybeLee
Participants
cleonard530cleonard530maybeLeemaybeLee
Timeline (top)
labeled ×4mentioned ×3subscribed ×3commented ×2

Error Message

Error logs

Fix Action

Fixed

PR fix notes

PR #177994: Fix randn_like inconsistency between eager and compile with fallback_random = True

Description (problem / solution / changelog)

Fixes #177652

When fallback_random = True, torch.randn_like was producing different results in eager vs compiled when the manual_seed was the same, the tensors where non-contiguous, and the total size >= 16.

e.g.:

x = torch.zeros((6, 3)).t()

torch.manual_seed(0)
out_eager = torch.randn_like(x)

cfunc = torch.compile(torch.randn_like, backend='inductor')
torch.manual_seed(0)
out_compile = cfunc(x)

produced different results for out_eager and out_compile.

Cause The reason for this is because the normal_ generator for eager mode has a special fast path when the size of the tensor is >= 16 and the tensor is contiguous. In eager mode torch.randn_like(x) did not use this fast path since x was not contiguous, but the decomposition for inductor used torch.randn to generate a contiguous vector then transformed it after it was generated, so it did use the fast path.

Solution When fallback_random=True, remove randn_like, rand_like, and randint_like overloads from the copied decomposition table (avoid mutating the global decompositions dict). rand_like and randint_like were also removed to be consistent and because they used the same pattern as randn_like in the decomposition.py file.

Register ATen fallbacks in lowering.py for those op packets so graphs keep aten.*_like and execute the same kernels as eager.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo @mlazos

Changed files

  • test/inductor/test_torchinductor.py (modified, +15/-0)
  • torch/_inductor/decomposition.py (modified, +8/-1)
  • torch/_inductor/lowering.py (modified, +3/-0)

Code Example

import torch
import itertools
torch._inductor.config.fallback_random = True

def fn(x):
    torch.manual_seed(0)
    return torch.randn_like(x)

# shape[0] * shape[1] >= 16 and shape[0], shape[1] != 1
for shape in itertools.product(range(1, 17), range(1, 17)):
    shape = shape
    x1 = torch.zeros(shape).permute(1, 0)
    x2 = torch.zeros(shape).permute(1, 0)
    
    cfunc = torch.compile(fn, backend='inductor')
    
    torch.manual_seed(0)
    out1 = fn(x1)
    # print(f'{out1=}')
    
    torch.manual_seed(0)
    out2 = fn(x1)
    # print(f'{out2=}')
    
    torch.manual_seed(0)
    out3 = cfunc(x2)
    # print(f'{out3=}')
    
    torch.manual_seed(0)
    out4 = cfunc(x2)
    # print(f'{out4=}')
    
    try:
        torch.testing.assert_close(out1, out3, equal_nan=True)
    except AssertionError as e:
        print(f'Shape {shape} will cause inconsistency.')

---

Shape (2, 8) will cause inconsistency.
Shape (2, 9) will cause inconsistency.
Shape (2, 10) will cause inconsistency.
Shape (2, 11) will cause inconsistency.
Shape (2, 12) will cause inconsistency.
Shape (2, 13) will cause inconsistency.
Shape (2, 14) will cause inconsistency.
Shape (2, 15) will cause inconsistency.
Shape (2, 16) will cause inconsistency.
Shape (3, 6) will cause inconsistency.
Shape (3, 7) will cause inconsistency.
Shape (3, 8) will cause inconsistency.
Shape (3, 9) will cause inconsistency.
...
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

When compiling a function with torch.randn_like, the compiled function's return result is inconsistent with eager even when the random seed is fixed and set torch._inductor.config.fallback_random = True. This issue only happens when the input of the function is a non-contiguous tensor (created via permute(1,0)), which size larger than 16 and shape[0], shape[1]!=1.

Here is the code to reproduce:

import torch
import itertools
torch._inductor.config.fallback_random = True

def fn(x):
    torch.manual_seed(0)
    return torch.randn_like(x)

# shape[0] * shape[1] >= 16 and shape[0], shape[1] != 1
for shape in itertools.product(range(1, 17), range(1, 17)):
    shape = shape
    x1 = torch.zeros(shape).permute(1, 0)
    x2 = torch.zeros(shape).permute(1, 0)
    
    cfunc = torch.compile(fn, backend='inductor')
    
    torch.manual_seed(0)
    out1 = fn(x1)
    # print(f'{out1=}')
    
    torch.manual_seed(0)
    out2 = fn(x1)
    # print(f'{out2=}')
    
    torch.manual_seed(0)
    out3 = cfunc(x2)
    # print(f'{out3=}')
    
    torch.manual_seed(0)
    out4 = cfunc(x2)
    # print(f'{out4=}')
    
    try:
        torch.testing.assert_close(out1, out3, equal_nan=True)
    except AssertionError as e:
        print(f'Shape {shape} will cause inconsistency.')

And the output shows that only specific shape can trigger this inconsistency:

Shape (2, 8) will cause inconsistency.
Shape (2, 9) will cause inconsistency.
Shape (2, 10) will cause inconsistency.
Shape (2, 11) will cause inconsistency.
Shape (2, 12) will cause inconsistency.
Shape (2, 13) will cause inconsistency.
Shape (2, 14) will cause inconsistency.
Shape (2, 15) will cause inconsistency.
Shape (2, 16) will cause inconsistency.
Shape (3, 6) will cause inconsistency.
Shape (3, 7) will cause inconsistency.
Shape (3, 8) will cause inconsistency.
Shape (3, 9) will cause inconsistency.
...

Note that this issue does not happen on backend eager and aot_eager. This issue also does not happen when running on cuda (RTX 6000).

Error logs

No response

Versions

Versions of relevant libraries: [pip3] numpy==2.4.2 [pip3] torch==2.12.0.dev20260316+cpu [pip3] torchvision==0.26.0.dev20260223+cpu [pip3] triton==3.6.0 [conda] numpy 2.4.2 pypi_0 pypi [conda] torch 2.12.0.dev20260316+cpu pypi_0 pypi [conda] torchvision 0.26.0.dev20260223+cpu pypi_0 pypi [conda] triton 3.6.0 pypi_0 pypi

cc @chauhang @penguinwu

extent analysis

Fix Plan

To fix the inconsistency issue with torch.randn_like when compiling a function with the inductor backend, we need to ensure that the input tensor is contiguous. We can achieve this by adding a contiguous() call before passing the tensor to the compiled function.

Here are the steps:

  • Modify the input tensor to be contiguous before passing it to the compiled function.
  • Update the fn function to handle non-contiguous tensors.

Code Changes

import torch
import itertools
torch._inductor.config.fallback_random = True

def fn(x):
    torch.manual_seed(0)
    return torch.randn_like(x)

# shape[0] * shape[1] >= 16 and shape[0], shape[1] != 1
for shape in itertools.product(range(1, 17), range(1, 17)):
    shape = shape
    x1 = torch.zeros(shape).permute(1, 0)
    x2 = torch.zeros(shape).permute(1, 0)
    
    cfunc = torch.compile(fn, backend='inductor')
    
    torch.manual_seed(0)
    out1 = fn(x1)
    
    torch.manual_seed(0)
    out2 = fn(x1)
    
    torch.manual_seed(0)
    out3 = cfunc(x2.contiguous())  # Add contiguous() call here
    torch.manual_seed(0)
    out4 = cfunc(x2.contiguous())  # Add contiguous() call here
    
    try:
        torch.testing.assert_close(out1, out3, equal_nan=True)
    except AssertionError as e:
        print(f'Shape {shape} will cause inconsistency.')

Verification

To verify that the fix worked, run the modified code and check that the assert_close call no longer raises an AssertionError for the previously problematic shapes.

Extra Tips

  • When working with non-contiguous tensors, it's essential to ensure that they are properly handled to avoid inconsistencies.
  • The contiguous() method can be used to create a contiguous copy of a tensor, which can help resolve issues like this one.
  • Always test your code thoroughly to catch any potential issues before deploying it to production.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#dependency conflict#environment setup#docker error#permission error#memory optimization

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING