Issue Station
Sign In
Tools hereToolbox

pytorch - 💡(How to fix) Fix [ROCm] MIOpen Gemm solvers return workspace_size=0 on gfx1150, silently excluded from algorithm selection [2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#178934•Fetched 2026-04-08 01:57:00
View on GitHub
Comments
2
Participants
2
Timeline
41
Reactions
0
Author
Peterc3-devPeterc3-dev
Participants
jeffdailyjeffdailyPeterc3-devPeterc3-dev
Timeline (top)
subscribed ×16mentioned ×15labeled ×7commented ×2

MIOpen Gemm convolution solvers return workspace_size=0 on gfx1150 (Radeon 890M, RDNA 3.5) when called through PyTorch's ATen convolution path, causing solvers to be skipped silently.

Root Cause

PyTorch's ATen layer uses the legacy MIOpen Find1 API path (miopenConvolutionForwardGetWorkSpaceSize / miopenFindConvolutionForwardAlgorithm). This API queries workspace requirements, but for Gemm-based solvers on new GPU targets like gfx1150, the workspace query returns 0 bytes.

When ATen sees workspace_size=0, it allocates no workspace buffer. The Gemm solver then fails internally (needs workspace for im2col/GEMM scratch) and gets excluded from the algorithm list. The result is that only ConvDirectNaiveConvFwd remains available — which works but is extremely slow.

Fix Action

Workaround

Setting MIOPEN_DEBUG_CONV_GEMM=0 disables the broken Gemm path, allowing other solvers (legacy CK, Winograd where applicable) to be selected.

RAW_BUFFERClick to expand / collapse

Description

MIOpen Gemm convolution solvers return workspace_size=0 on gfx1150 (Radeon 890M, RDNA 3.5) when called through PyTorch's ATen convolution path, causing solvers to be skipped silently.

Root Cause

PyTorch's ATen layer uses the legacy MIOpen Find1 API path (miopenConvolutionForwardGetWorkSpaceSize / miopenFindConvolutionForwardAlgorithm). This API queries workspace requirements, but for Gemm-based solvers on new GPU targets like gfx1150, the workspace query returns 0 bytes.

When ATen sees workspace_size=0, it allocates no workspace buffer. The Gemm solver then fails internally (needs workspace for im2col/GEMM scratch) and gets excluded from the algorithm list. The result is that only ConvDirectNaiveConvFwd remains available — which works but is extremely slow.

Expected Behavior

Gemm solvers should either:

  1. Report their actual workspace requirements through the Find1 API, or
  2. PyTorch should migrate to the Find2 / Immediate mode API which handles workspace allocation differently

Environment

  • Hardware: AMD Ryzen AI 9 HX 370, Radeon 890M (gfx1150)
  • OS: CachyOS, kernel 6.19.5
  • ROCm: 7.2.0
  • PyTorch: 2.12.0a0 (built from source for gfx1150)
  • MIOpen: Built from amd-develop

Workaround

Setting MIOPEN_DEBUG_CONV_GEMM=0 disables the broken Gemm path, allowing other solvers (legacy CK, Winograd where applicable) to be selected.

Related

  • ROCm/MIOpen#3860 (convolution fails on gfx1150)
  • ROCm/rocm-libraries#6045 (full three-bug analysis)
  • ROCm/MIOpen#3953 (legacy CK whitelist PR)
  • ROCm/composable_kernel#3724 (CK WMMA enablement gap)

cc @jerryzh168 @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @jataylo @hongxiayang @naromero77amd @pragupta @jerrymannil @xinyazhang

extent analysis

TL;DR

Setting MIOPEN_DEBUG_CONV_GEMM=0 can serve as a temporary workaround to disable the broken Gemm path and allow other solvers to be selected.

Guidance

  • The issue arises from PyTorch's ATen layer using the legacy MIOpen Find1 API, which returns incorrect workspace requirements for Gemm-based solvers on new GPU targets like gfx1150.
  • To verify the issue, check if the Gemm solvers are being skipped and if only ConvDirectNaiveConvFwd remains available, resulting in slow performance.
  • As a mitigation step, consider migrating PyTorch to use the Find2 / Immediate mode API, which handles workspace allocation differently and may resolve the issue.
  • Alternatively, disabling the Gemm path using the MIOPEN_DEBUG_CONV_GEMM=0 environment variable can allow other solvers to be selected, providing a temporary workaround.

Example

No code snippet is provided as the issue does not require a code-level fix, but rather a configuration or API change.

Notes

The provided workaround may not be a permanent solution and may have performance implications. It is recommended to track the related issues (ROCm/MIOpen#3860, ROCm/rocm-libraries#6045, ROCm/MIOpen#3953, ROCm/composable_kernel#3724) for updates on a more permanent fix.

Recommendation

Apply the workaround by setting MIOPEN_DEBUG_CONV_GEMM=0 to disable the broken Gemm path and allow other solvers to be selected, as this provides a temporary solution to the issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#output truncation#response parsing#generation error#database connection

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING