Issue Station
Sign In
Tools hereToolbox

pytorch - ✅(Solved) Fix `int32 + bfloat16` type promotion silent num err [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Root Cause

The promotion table in c10/core/ScalarType.cpp maps integer + bf16 → bf16, extrapolated from NumPy's "float absorbs integer" rule. But NumPy has no bfloat16. The rule is safe for fp16/fp32/fp64 but not for bf16's 8-bit mantissa.

Triton handles this differently — int32 + bf16 → f32 — treating bf16 as a special case (triton/language/semantic.py:103).

Fix Action

Fixed

PR fix notes

PR #179847: Promote integer + bfloat16 to float32 in type promotion table

Description (problem / solution / changelog)

bfloat16 has only 8 mantissa bits, so it cannot exactly represent integers above 256. The previous rule (integer + bf16 → bf16) silently rounded values, e.g. int32(257) + bf16(0) = bf16(256). This was extrapolated from NumPy's "float absorbs integer" convention, but NumPy has no bfloat16 type.

Change integer + bf16 (and bool + bf16) to promote to float32, matching Triton's promotion behavior. bf16 + bf16 is unchanged.

Fixes https://github.com/pytorch/pytorch/issues/179845

Changed files

  • c10/core/ScalarType.cpp (modified, +7/-7)

Code Example

import torch

x = torch.tensor(257, dtype=torch.int32)
y = torch.tensor(0.0, dtype=torch.bfloat16)
z = torch.tensor(0.0, dtype=torch.float16)
print(x + y)  # tensor(256., dtype=torch.bfloat16) — 257 silently became 256
print(x + z)  # 257
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

int32 + bfloat16 type promotion silently loses precision

Bug

int32 + bfloat16 promotes to bfloat16, which silently rounds integers above 256:

import torch

x = torch.tensor(257, dtype=torch.int32)
y = torch.tensor(0.0, dtype=torch.bfloat16)
z = torch.tensor(0.0, dtype=torch.float16)
print(x + y)  # tensor(256., dtype=torch.bfloat16) — 257 silently became 256
print(x + z)  # 257

float16 does not have this problem because its 11-bit mantissa covers the int32 range much better than bfloat16's 8-bit mantissa.

Root cause

The promotion table in c10/core/ScalarType.cpp maps integer + bf16 → bf16, extrapolated from NumPy's "float absorbs integer" rule. But NumPy has no bfloat16. The rule is safe for fp16/fp32/fp64 but not for bf16's 8-bit mantissa.

Triton handles this differently — int32 + bf16 → f32 — treating bf16 as a special case (triton/language/semantic.py:103).

Suggested fix

Change integer + bf16 to promote to float32 in the _promoteTypesLookup table, matching Triton's behavior.

Versions

PyTorch 2.12.0a0+gitd733e3b, also reproducible on CPU.

Ref issue

https://github.com/triton-lang/triton/issues/9963

Versions

current main

cc @nairbv @mruberry

extent analysis

TL;DR

Change the type promotion rule for int32 + bfloat16 to promote to float32 instead of bfloat16 to prevent silent precision loss.

Guidance

  • Review the _promoteTypesLookup table in c10/core/ScalarType.cpp to understand the current type promotion rules.
  • Update the promotion rule for integer + bf16 to float32 to match Triton's behavior and prevent precision loss.
  • Verify the fix by testing the example code provided in the issue with the updated promotion rule.
  • Consider reviewing other type promotion rules to ensure consistency and accuracy.

Example

# After applying the fix, the example code should produce the correct result
x = torch.tensor(257, dtype=torch.int32)
y = torch.tensor(0.0, dtype=torch.bfloat16)
print(x + y)  # tensor(257., dtype=torch.float32)

Notes

The fix assumes that updating the _promoteTypesLookup table is sufficient to change the type promotion behavior. Additional changes may be required to ensure consistency across the codebase.

Recommendation

Apply the workaround by updating the type promotion rule to promote int32 + bfloat16 to float32, as this change is consistent with Triton's behavior and prevents silent precision loss.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#installation#tensor shape#autograd error#model save/load

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING