pytorch - 💡(How to fix) Fix [TMA] Failure when running slice+view operations with TMA enabled on H100 [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

Traceback (most recent call last): File "/opt/pytorch/pytorch/tma_test.py", line 19, in <module> torch.compile(f)(x); torch.cuda.synchronize() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pytorch/pytorch/torch/cuda/init.py", line 1215, in synchronize return torch._C._cuda_synchronize() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.AcceleratorError: CUDA error: misaligned address Search for `cudaErrorMisalignedAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1

Fix Action

Fixed

Code Example

import torch, torch._inductor.config as cfg
cfg.triton.use_tensor_descriptor = True
cfg.assume_aligned_inputs = True

offset = 1

def f(x):
    return x[2:].view(dtype=torch.float32) + 1

x = torch.randn((128 + offset) * 2, dtype=torch.bfloat16, device="cuda")
torch.compile(f)(x); torch.cuda.synchronize()

---

Traceback (most recent call last):
  File "/opt/pytorch/pytorch/tma_test.py", line 19, in <module>
    torch.compile(f)(x); torch.cuda.synchronize()
                         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pytorch/pytorch/torch/cuda/__init__.py", line 1215, in synchronize
    return torch._C._cuda_synchronize()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: misaligned address
Search for `cudaErrorMisalignedAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

Repro

import torch, torch._inductor.config as cfg
cfg.triton.use_tensor_descriptor = True
cfg.assume_aligned_inputs = True

offset = 1

def f(x):
    return x[2:].view(dtype=torch.float32) + 1

x = torch.randn((128 + offset) * 2, dtype=torch.bfloat16, device="cuda")
torch.compile(f)(x); torch.cuda.synchronize()

fails when enabling TMA on H100

Error logs

Traceback (most recent call last):
  File "/opt/pytorch/pytorch/tma_test.py", line 19, in <module>
    torch.compile(f)(x); torch.cuda.synchronize()
                         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pytorch/pytorch/torch/cuda/__init__.py", line 1215, in synchronize
    return torch._C._cuda_synchronize()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: misaligned address
Search for `cudaErrorMisalignedAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1

Versions

main 090b49d99bb9c26144cedea3b9c566fbe6ede275 H100

cc @ptrblck @msaroufim @eqy @jerryzh168 @tinglvv @nWEIdia @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING