pytorch - 💡(How to fix) Fix `set_multithreading_enabled` still spawns autograd threads

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

import os, torch

def autograd_threads(): """Return list of native thread names containing 'autograd' (Linux).""" names = [] for tid in os.listdir("/proc/self/task"): try: name = open(f"/proc/self/task/{tid}/comm").read().strip() if "autograd" in name: names.append(name) except OSError: pass return names

torch.autograd.grad_mode.set_multithreading_enabled(False) print(f"autograd threads before backward: {autograd_threads()}") x = torch.ones(100, requires_grad=True) x.mean().backward() threads = autograd_threads() print(f"autograd threads after backward: {threads}")

Code Example

import os, torch

def autograd_threads():
    """Return list of native thread names containing 'autograd' (Linux)."""
    names = []
    for tid in os.listdir("/proc/self/task"):
        try:
            name = open(f"/proc/self/task/{tid}/comm").read().strip()
            if "autograd" in name:
                names.append(name)
        except OSError:
            pass
    return names

torch.autograd.grad_mode.set_multithreading_enabled(False)
print(f"autograd threads before backward: {autograd_threads()}")
x = torch.ones(100, requires_grad=True)
x.mean().backward()
threads = autograd_threads()
print(f"autograd threads after backward:  {threads}")

---

autograd threads before backward: []
autograd threads after backward:  ['pt_autograd_0', 'pt_autograd_1', 'pt_autograd_2', 'pt_autograd_3', 'pt_autograd_4', 'pt_autograd_5', 'pt_autograd_6', 'pt_autograd_7']
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

The documentation for set_multithreading_enabled says:

Ordinarily, when accelerator devices are in use, the backward pass runs on device-specific worker threads. The engine creates these threads based on the number of available devices and reuses them across iterations. When mode=False, the backward pass runs on the calling thread instead

However, it seems to still spawn threads

import os, torch

def autograd_threads():
    """Return list of native thread names containing 'autograd' (Linux)."""
    names = []
    for tid in os.listdir("/proc/self/task"):
        try:
            name = open(f"/proc/self/task/{tid}/comm").read().strip()
            if "autograd" in name:
                names.append(name)
        except OSError:
            pass
    return names

torch.autograd.grad_mode.set_multithreading_enabled(False)
print(f"autograd threads before backward: {autograd_threads()}")
x = torch.ones(100, requires_grad=True)
x.mean().backward()
threads = autograd_threads()
print(f"autograd threads after backward:  {threads}")
autograd threads before backward: []
autograd threads after backward:  ['pt_autograd_0', 'pt_autograd_1', 'pt_autograd_2', 'pt_autograd_3', 'pt_autograd_4', 'pt_autograd_5', 'pt_autograd_6', 'pt_autograd_7']

I feel like these threads should not be spawned at all in https://github.com/pytorch/pytorch/blob/becedcb06e19598cfd1fd82f887dea7044c753ed/torch/csrc/autograd/engine.cpp#L1422

Although then they would need to be lazy initialized later on if the context manager was switched back to mode=True later on.

Is this unexpected or is this a limitation of set_multithreading_enabled and the documentation should mention it?

Thank you

Versions

2.11.0

cc @ezyang @albanD @gqchen @nikitaved @soulitzer @Varal7 @bobrenjc93

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING