Issue Station
Sign In
Tools hereToolbox

pytorch - 💡(How to fix) Fix [Inductor] Remove fp8 special handling in `triton_utils.py` because Triton supports PyTorch fp8 dtypes [4 comments, 4 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
pytorch/pytorch#178938•Fetched 2026-04-08 01:56:56
View on GitHub
Comments
4
Participants
4
Timeline
180
Reactions
0
Author
oonyshchoonyshch
Participants
Divya1SDivya1Sjanseljanselogulcanaydoganogulcanaydoganoonyshchoonyshch
Timeline (top)
mentioned ×75subscribed ×75referenced ×12labeled ×9

torch/_inductor/codegen/triton_utils.py contains a TODO comment added over two years ago that no longer applies:

def signature_of(arg: KernelArgType, *, size_dtype: str | None) -> str:
    if isinstance(arg, TensorArg):
        # TODO: Remove fp8 special handling when Triton supports PyTorch fp8 dtypes.
        # Related PR: https://github.com/triton-lang/triton/pull/2279/
        if arg.dtype == torch.float8_e4m3fn:
            typ = "*fp8e4nv"
        elif arg.dtype == torch.float8_e5m2:
            typ = "*fp8e5"
        elif arg.dtype == torch.float8_e4m3fnuz:
            typ = "*fp8e4b8"
        elif arg.dtype == torch.float8_e5m2fnuz:
            typ = "*fp8e5b16"
        else:
            ...

The referenced PR (triton-lang/triton#2279) "Add PyTorch fp8 dtypes to Triton" was merged. Triton now natively maps PyTorch fp8 dtype names to its own internal type names, so the manual mapping in signature_of is redundant.

Root Cause

torch/_inductor/codegen/triton_utils.py contains a TODO comment added over two years ago that no longer applies:

def signature_of(arg: KernelArgType, *, size_dtype: str | None) -> str:
    if isinstance(arg, TensorArg):
        # TODO: Remove fp8 special handling when Triton supports PyTorch fp8 dtypes.
        # Related PR: https://github.com/triton-lang/triton/pull/2279/
        if arg.dtype == torch.float8_e4m3fn:
            typ = "*fp8e4nv"
        elif arg.dtype == torch.float8_e5m2:
            typ = "*fp8e5"
        elif arg.dtype == torch.float8_e4m3fnuz:
            typ = "*fp8e4b8"
        elif arg.dtype == torch.float8_e5m2fnuz:
            typ = "*fp8e5b16"
        else:
            ...

The referenced PR (triton-lang/triton#2279) "Add PyTorch fp8 dtypes to Triton" was merged. Triton now natively maps PyTorch fp8 dtype names to its own internal type names, so the manual mapping in signature_of is redundant.

Code Example

def signature_of(arg: KernelArgType, *, size_dtype: str | None) -> str:
    if isinstance(arg, TensorArg):
        # TODO: Remove fp8 special handling when Triton supports PyTorch fp8 dtypes.
        # Related PR: https://github.com/triton-lang/triton/pull/2279/
        if arg.dtype == torch.float8_e4m3fn:
            typ = "*fp8e4nv"
        elif arg.dtype == torch.float8_e5m2:
            typ = "*fp8e5"
        elif arg.dtype == torch.float8_e4m3fnuz:
            typ = "*fp8e4b8"
        elif arg.dtype == torch.float8_e5m2fnuz:
            typ = "*fp8e5b16"
        else:
            ...
RAW_BUFFERClick to expand / collapse

Summary

torch/_inductor/codegen/triton_utils.py contains a TODO comment added over two years ago that no longer applies:

def signature_of(arg: KernelArgType, *, size_dtype: str | None) -> str:
    if isinstance(arg, TensorArg):
        # TODO: Remove fp8 special handling when Triton supports PyTorch fp8 dtypes.
        # Related PR: https://github.com/triton-lang/triton/pull/2279/
        if arg.dtype == torch.float8_e4m3fn:
            typ = "*fp8e4nv"
        elif arg.dtype == torch.float8_e5m2:
            typ = "*fp8e5"
        elif arg.dtype == torch.float8_e4m3fnuz:
            typ = "*fp8e4b8"
        elif arg.dtype == torch.float8_e5m2fnuz:
            typ = "*fp8e5b16"
        else:
            ...

The referenced PR (triton-lang/triton#2279) "Add PyTorch fp8 dtypes to Triton" was merged. Triton now natively maps PyTorch fp8 dtype names to its own internal type names, so the manual mapping in signature_of is redundant.

What should happen

The four hand-rolled if/elif branches should be removed and the fp8 dtypes should fall through to the existing _type_of(arg.dtype) path, just like every other dtype — provided _type_of already handles them correctly (which it does via the same mapping Triton itself now exposes).

If _type_of needs a small update to handle these four variants, that is the right place for the fix rather than signature_of.

File

torch/_inductor/codegen/triton_utils.py, lines 37–46

References

Good first issue?

Cleanup that is safe to pick up as a good-first-issue, assuming the removal is paired with a quick test verifying fp8 Inductor codegen still produces correct Triton kernel signatures.

Therefore, I will reference it in new PR adressing the issue.

cc @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo

extent analysis

TL;DR

Remove the redundant if/elif branches for fp8 dtypes in signature_of function and rely on the existing _type_of path.

Guidance

  • Verify that _type_of correctly handles fp8 dtypes by checking its implementation and documentation.
  • Remove the four if/elif branches for fp8 dtypes in signature_of and test the function with fp8 inputs to ensure correct output.
  • Update the test suite to include cases for fp8 Inductor codegen to verify that Triton kernel signatures are still produced correctly.
  • Review the changes in the context of the larger codebase to ensure no unintended side effects.

Example

def signature_of(arg: KernelArgType, *, size_dtype: str | None) -> str:
    if isinstance(arg, TensorArg):
        # Remove the redundant if/elif branches and rely on _type_of
        typ = _type_of(arg.dtype)
        # ... rest of the function remains the same

Notes

The removal of the redundant branches assumes that _type_of correctly handles fp8 dtypes. If _type_of needs updates, they should be made in its implementation rather than in signature_of.

Recommendation

Apply workaround: Remove the redundant if/elif branches and rely on the existing _type_of path, as the referenced PR has already added support for PyTorch fp8 dtypes to Triton.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#indexing error#inference speed#output truncation#response parsing#generation error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

pytorch - 💡(How to fix) Fix [Inductor] Remove fp8 special handling in `triton_utils.py` because Triton supports PyTorch fp8 dtypes [4 comments, 4 participants]