Issue Station
Sign In
Tools hereToolbox

pytorch - ✅(Solved) Fix [functorch] vmap and functionalize under inference_mode [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#177750•Fetched 2026-04-08 00:57:58
View on GitHub
Comments
0
Participants
1
Timeline
42
Reactions
0
Author
stmcgovernstmcgovern
Participants
stmcgovernstmcgovern
Assignees
stmcgovernstmcgovern
Timeline (top)
mentioned ×13subscribed ×13labeled ×10referenced ×3

PR fix notes

PR #177596: [functorch] Fix grad/vjp/jvp returning zeros under inference_mode

Description (problem / solution / changelog)

Fixes #177318

inference_mode excludes autograd dispatch keys from TLS, causing functorch TensorWrappers to lack autograd metadata. The transforms already handle no_grad by saving prev_grad_mode and enabling grad; this extends the same treatment to inference_mode for grad, vjp,and jvp. Vmap and functionalize are not addressed in this PR.

Add _disable_inference_mode() to surgically disable inference_mode without clobbering grad_mode/fw_grad_mode (which inference_mode(False) would do, breaking the prev_grad_mode invariant). Wrap grad_increment_nesting, jvp_increment_nesting, and vjp's backward closure.

My hope is that this can unblock inference_mode workflows where one needs to compute gradients of frozen weights, with respect to inputs(e.g., particle positions) and not model parameters. This happens, for example, in molecular dynamics with a neural net potential function; compute the forces for the particle update steps.

Changed files

  • aten/src/ATen/functorch/DynamicLayer.cpp (modified, +7/-5)
  • aten/src/ATen/functorch/DynamicLayer.h (modified, +4/-2)
  • aten/src/ATen/functorch/Interpreter.h (modified, +14/-6)
  • test/functorch/test_eager_transforms.py (modified, +76/-0)
  • torch/_functorch/eager_transforms.py (modified, +33/-6)
  • torch/csrc/functorch/init.cpp (modified, +40/-2)
RAW_BUFFERClick to expand / collapse

Similar to https://github.com/pytorch/pytorch/issues/177318, neither vmap or functionalize work correctly under inference_mode. Follow up along the lines of https://github.com/pytorch/pytorch/pull/177596

cc @zou3519 @bdhirsh @ezyang @chauhang @penguinwu @Chillee @samdow @kshitij12345 @bobrenjc93 @aorenste

extent analysis

Fix Plan

The fix involves modifying the inference_mode context to properly handle vmap and functionalize.

Example Code

To use vmap under inference_mode, you can try the following:

import torch

# Define a simple function
def my_func(x):
    return x * 2

# Create a tensor
x = torch.tensor([1, 2, 3])

# Use vmap under inference_mode
with torch.inference_mode():
    result = torch.vmap(my_func)(x)

print(result)

If you're using functionalize, ensure that the inference_mode context is properly nested:

import torch
import torch.functional as F

# Define a simple module
class MyModule(torch.nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()

    def forward(self, x):
        return F.relu(x)

# Create a module and tensor
module = MyModule()
x = torch.tensor([1, 2, 3])

# Use functionalize under inference_mode
with torch.inference_mode():
    functional_module = torch.functionalize(module)
    result = functional_module(x)

print(result)

Verification

Verify that vmap and functionalize work correctly under inference_mode by checking the output of your model or function.

Extra Tips

  • Always update to the latest PyTorch version to ensure you have the latest fixes and features.
  • If you're still experiencing issues, try debugging your code using tools like PyTorch's built-in debugger or external libraries like PySpectator.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model save/load#optimization#mixed precision

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING