pytorch - 💡(How to fix) Fix Request to support fp8 x4 packed dtypes in PT [1 comments, 2 participants]

TL;DR

Adding support for float8_e4m3fn_x4 and float8_e5m2_x4 dtypes to PT Core may require modifying the existing kernel handling to accommodate the packed FP8 elements.

Guidance

• Investigate the current kernel implementation to understand how it handles packing and unpacking of FP8 elements.
• Determine the requirements for adding the new dtypes to PT Core, including any necessary changes to the kernel or other components.
• Consider the impact of adding these dtypes on existing functionality and performance.
• Evaluate the need for additional testing or validation to ensure correct handling of the new dtypes.

Notes

The proposal to add new dtypes to PT Core may involve significant changes to the underlying implementation, and careful consideration of the potential impact is necessary.

Recommendation

Apply workaround: Modify the existing kernel handling to accommodate the packed FP8 elements, as adding new dtypes to PT Core may require significant changes and testing.