Issue Station
Sign In
Tools hereToolbox

pytorch - 💡(How to fix) Fix image diffusion model output changes for the worse after 530ff92655 [7 comments, 4 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#181737•Fetched 2026-04-29 06:11:08
View on GitHub
Comments
7
Participants
4
Timeline
51
Reactions
0
Author
zou3519zou3519
Participants
eqyeqygithub-actions[bot]github-actions[bot]ngimelngimelzou3519zou3519
Timeline (top)
mentioned ×19subscribed ×19commented ×7labeled ×4
RAW_BUFFERClick to expand / collapse

I'm not sure if this is acceptable numerical drift or not. I've been bisecting a change where an image diffusion workflow in SGLang produces the following images from PyTorch 2.10 -> PyTorch 2.11. This is one of the issues blocking SGLang from upgrading to PyTorch 2.11.

I bisected this change down to 530ff92655. Claude tells me that this PR makes some more cases go through cublaslt, which we do know we have some problems with recently (@eqy) Any thoughts here? @ngimel @drisspg @nikitaved

GOOD: (approx PyTorch 2.10)

<img width="519" height="523" alt="Image" src="https://github.com/user-attachments/assets/7ec779fb-0350-463b-96e0-6b62dc213c0d" />

BAD: (approx PyTorch 2.11)

<img width="526" height="520" alt="Image" src="https://github.com/user-attachments/assets/a7681e7b-3895-46ad-94cd-ce650b60eda1" />

cc @ptrblck @msaroufim @jerryzh168 @tinglvv @nWEIdia @csarofeen

extent analysis

TL;DR

Investigate the changes introduced in PyTorch 2.11, specifically the commit 530ff92655, to understand the numerical drift causing different image outputs in the image diffusion workflow.

Guidance

  • Review the PR 530ff92655 to understand how it affects the cublaslt cases and potential issues with numerical stability.
  • Compare the image outputs from PyTorch 2.10 and 2.11 to identify any patterns or differences that could indicate the cause of the numerical drift.
  • Consider testing the image diffusion workflow with different inputs or parameters to see if the issue is reproducible and consistent.
  • Investigate potential workarounds or configuration changes that could mitigate the numerical drift, such as adjusting precision or rounding settings.

Notes

The issue seems to be related to changes in PyTorch 2.11, but more information is needed to determine the root cause and develop a comprehensive fix.

Recommendation

Apply workaround: Investigate and potentially revert or modify the changes introduced in PR 530ff92655 to mitigate the numerical drift, as upgrading to PyTorch 2.11 is currently blocked by this issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#memory leak#API versioning#request timeout

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

pytorch - 💡(How to fix) Fix image diffusion model output changes for the worse after 530ff92655 [7 comments, 4 participants]