pytorch - ✅(Solved) Fix `fresh_cache()` ignores user setting of `TORCHINDUCTOR_CACHE_DIR` [2 pull requests, 1 comments, 2 participants]

pytorch2026-03-31 07:45:07

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

pytorch/pytorch#178858•Fetched 2026-04-08 01:57:34

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Participants

Assignees

Timeline (top)

mentioned ×36subscribed ×36cross-referenced ×4labeled ×4

Root Cause

ignores the TORCHINDUCTOR_CACHE_DIR=my_folder set by the user and always make the cache into /tmp/tmp_xxx. So we witnessed a recompilation between tests, because they can't find the cache despite that the key are the same:

Fix Action

Fixed

Fixed by PR: [fix] Fix that fresh_cache() ignores TORCHINDUCTOR_CACHE_DIR global setting (https://github.com/pytorch/pytorch/pull/178860)
Fixed by PR: [inductor] Respect TORCHINDUCTOR_CACHE_DIR in fresh_cache (https://github.com/pytorch/pytorch/pull/178905)

PR fix notes

PR #178860: [fix] Fix that fresh_cache() ignores TORCHINDUCTOR_CACHE_DIR global setting

Repository: pytorch/pytorch
Author: Stonepia
State: open | merged: False
Link: https://github.com/pytorch/pytorch/pull/178860

Description (problem / solution / changelog)

Fixes #178858

The fresh_cache() call ignores the TORCHINDUCTOR_CACHE_DIR, so during the tests in helion, we witnessed that even if the TORCHINDUCTOR_CACHE_DIR is set to custom folder, we still find that the /tmp/tmp_xxx is created, so we got cache miss like the following:

test 1:
/tmp/tmpu055r7oy/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...
test 2: (suppose to hit cache, but the cache missed, because tmp folder not the same)
/tmp/tmpoygmne2a/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo

Changed files

test/inductor/test_codecache.py (modified, +21/-0)
torch/_inductor/utils.py (modified, +3/-1)

PR #178905: [inductor] Respect TORCHINDUCTOR_CACHE_DIR in fresh_cache

Repository: pytorch/pytorch
Author: keon
State: open | merged: False
Link: https://github.com/pytorch/pytorch/pull/178905

Description (problem / solution / changelog)

Summary

Fixes fresh_cache() so it reuses an explicitly configured TORCHINDUCTOR_CACHE_DIR when dir is not passed, instead of always replacing it with a new temp directory.

This keeps user-selected cache roots stable while preserving the existing fresh-temp behavior for callers that do not pin a cache directory.

Fixes #178858

Test Plan

Temporary standalone reproducer for #178858 failed before the patch and passed after it
Added regression coverage in test/inductor/test_caching.py
python -m py_compile torch/_inductor/utils.py test/inductor/test_caching.py

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo

Changed files

test/inductor/test_caching.py (modified, +34/-0)
torch/_inductor/utils.py (modified, +16/-5)

Code Example

test1: Compile kernel path:
/tmp/tmpu055r7oy/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...
test2: Compile kernel path:
/tmp/tmpoygmne2a/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...

---

import os
import tempfile

from torch._inductor.utils import fresh_cache

# User sets a custom cache directory
custom_dir = tempfile.mkdtemp(prefix="my_custom_cache_")
os.environ["TORCHINDUCTOR_CACHE_DIR"] = custom_dir

print(f"User-set TORCHINDUCTOR_CACHE_DIR: {custom_dir}")

with fresh_cache(delete=False):
    actual = os.environ["TORCHINDUCTOR_CACHE_DIR"]
    print(f"Inside fresh_cache:              {actual}")
    if actual == custom_dir:
        print("PASS: fresh_cache respected the user-set directory.")
    else:
        print("FAIL: fresh_cache ignored the user-set directory!")
        print(f"  Expected: {custom_dir}")
        print(f"  Got:      {actual}")

os.rmdir(custom_dir)

---

User-set TORCHINDUCTOR_CACHE_DIR: /tmp/my_custom_cache_90ofnto5
Inside fresh_cache:              /tmp/tmp627aqti_
FAIL: fresh_cache ignored the user-set directory!
  Expected: /tmp/my_custom_cache_90ofnto5
  Got:      /tmp/tmp627aqti_

RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

The fresh_cache() call at

https://github.com/pytorch/pytorch/blob/3848ae68ce2a6c5e45541bf77346bc01aabd7286/torch/_inductor/utils.py#L1343-L1349

test1: Compile kernel path:
/tmp/tmpu055r7oy/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...
test2: Compile kernel path:
/tmp/tmpoygmne2a/triton/4SQ645TW7ISAHZEXTEURYAESZMDMDC4BKNH3CAV7DKEGHU6GXLPQ  key: 4SQ645TW...

reproducer


import os
import tempfile

from torch._inductor.utils import fresh_cache

# User sets a custom cache directory
custom_dir = tempfile.mkdtemp(prefix="my_custom_cache_")
os.environ["TORCHINDUCTOR_CACHE_DIR"] = custom_dir

print(f"User-set TORCHINDUCTOR_CACHE_DIR: {custom_dir}")

with fresh_cache(delete=False):
    actual = os.environ["TORCHINDUCTOR_CACHE_DIR"]
    print(f"Inside fresh_cache:              {actual}")
    if actual == custom_dir:
        print("PASS: fresh_cache respected the user-set directory.")
    else:
        print("FAIL: fresh_cache ignored the user-set directory!")
        print(f"  Expected: {custom_dir}")
        print(f"  Got:      {actual}")

os.rmdir(custom_dir)

User-set TORCHINDUCTOR_CACHE_DIR: /tmp/my_custom_cache_90ofnto5
Inside fresh_cache:              /tmp/tmp627aqti_
FAIL: fresh_cache ignored the user-set directory!
  Expected: /tmp/my_custom_cache_90ofnto5
  Got:      /tmp/tmp627aqti_

Versions

Torch: Version: 2.12.0.dev20260317+xpu

cc @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo

extent analysis

TL;DR

The fresh_cache function ignores the user-set TORCHINDUCTOR_CACHE_DIR environment variable, causing recompilation between tests due to cache directory mismatch.

Guidance

The issue is likely caused by the fresh_cache function not respecting the TORCHINDUCTOR_CACHE_DIR environment variable set by the user.
To verify, run the provided reproducer code and check if the fresh_cache function uses the expected cache directory.
A potential workaround is to modify the fresh_cache function to respect the TORCHINDUCTOR_CACHE_DIR environment variable.
The user can also try setting the cache directory manually before calling fresh_cache to ensure it uses the correct directory.

Example

import os
from torch._inductor.utils import fresh_cache

# Set the cache directory manually
cache_dir = os.environ.get("TORCHINDUCTOR_CACHE_DIR", "/tmp/default_cache")
os.environ["TORCHINDUCTOR_CACHE_DIR"] = cache_dir

with fresh_cache(delete=False):
    # Check if the cache directory is correct
    actual_dir = os.environ["TORCHINDUCTOR_CACHE_DIR"]
    print(f"Using cache directory: {actual_dir}")

Notes

The provided reproducer code and example are specific to the PyTorch library and may not be applicable to other libraries or frameworks.

Recommendation

Apply workaround: Modify the fresh_cache function or set the cache directory manually to ensure it respects the TORCHINDUCTOR_CACHE_DIR environment variable. This is because the current implementation of fresh_cache ignores the user-set cache directory, causing recompilation issues.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#configuration error #environment variable #network issue #logging issue #authentication issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

pytorch - ✅(Solved) Fix `fresh_cache()` ignores user setting of `TORCHINDUCTOR_CACHE_DIR` [2 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #178860: [fix] Fix that fresh_cache() ignores TORCHINDUCTOR_CACHE_DIR global setting

Description (problem / solution / changelog)

Changed files

PR #178905: [inductor] Respect TORCHINDUCTOR_CACHE_DIR in fresh_cache

Description (problem / solution / changelog)

Summary

Test Plan

Changed files

Code Example

🐛 Describe the bug

reproducer

Versions

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

TRENDING

pytorch - ✅(Solved) Fix `fresh_cache()` ignores user setting of `TORCHINDUCTOR_CACHE_DIR` [2 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #178860: [fix] Fix that fresh_cache() ignores TORCHINDUCTOR_CACHE_DIR global setting

Description (problem / solution / changelog)

Changed files

PR #178905: [inductor] Respect TORCHINDUCTOR_CACHE_DIR in fresh_cache

Description (problem / solution / changelog)

Summary

Test Plan

Changed files

Code Example

🐛 Describe the bug

reproducer

Versions

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING