pytorch - 💡(How to fix) Fix The smallest values of E4M3 for two-level quantization is incorrect [5 comments, 2 participants]

🐛 Describe the bug

In ao/torchao/prototype/mx_formats/nvfp4_tensor.py, the smallest E4M3 for two-level quantization is given by

E4M3_EPS = torch.finfo(torch.float8_e4m3fn).tiny

which is the smallest positive normal number of E4M3, i.e., $$2^{1-7}=0.015625,$$ However, the smallest positive number of E4M3 is $$2^{-9}=0.001953125$$ which is a subnormal. Note that a sufficient small scale factor with type of E4M3 can be a subnormal number .

Versions

PyTorch version: 2.8.0+cpu Is debug build: False CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 企业版 (10.0.26100 64 位) GCC version: Could not collect Clang version: Could not collect CMake version: Could not collect Libc version: N/A

Python version: 3.12.1 (tags/v3.12.1:2305ca5, Dec 7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] (64-bit runtime) Python platform: Windows-11-10.0.26100-SP0 Is CUDA available: False CUDA runtime version: No CUDA CUDA_MODULE_LOADING set to: N/A GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA Is XPU available: False HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Caching allocator config: N/A

CPU: Name: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz Manufacturer: GenuineIntel Family: 198 Architecture: 9 ProcessorType: 3 DeviceID: CPU0 CurrentClockSpeed: 2803 MaxClockSpeed: 2803 L2CacheSize: 5120 L2CacheSpeed: None Revision: None

Versions of relevant libraries: [pip3] numpy==2.3.2 [pip3] torch==2.8.0 [pip3] torchaudio==2.8.0 [pip3] torchvision==0.23.0 [conda] Could not collect

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel