pytorch - 💡(How to fix) Fix CPU wheels from download.pytorch.org/whl/cpu missing torch/csrc/ and c10/ headers needed by torch.compile (inductor) [5 comments, 2 participants]

Description

The CPU-only PyTorch wheels distributed via https://download.pytorch.org/whl/cpu ship a stripped-down torch/include/ directory that only contains ATen/ headers (57 files). The torch/csrc/ and c10/ header directories are missing entirely.

This breaks torch.compile (inductor backend) at runtime because the C++ code generation step needs torch/csrc/inductor/cpp_prefix.h and its transitive includes from c10/.

The regular PyPI wheels (CUDA) include the full torch/include/ tree (~9,300+ files) and work fine with torch.compile.

Steps to Reproduce

pip install torch --index-url https://download.pytorch.org/whl/cpu
python -c "
import torch, os
inc = os.path.join(os.path.dirname(torch.__file__), 'include')
cpp = os.path.join(inc, 'torch', 'csrc', 'inductor', 'cpp_prefix.h')
print('torch version:', torch.__version__)
print('include/ exists:', os.path.isdir(inc))
print('include/ contents:', os.listdir(inc))
print('cpp_prefix.h exists:', os.path.exists(cpp))
"

Output:

torch version: 2.11.0+cpu
include/ exists: True
include/ contents: ['ATen']
cpp_prefix.h exists: False

Expected Behavior

The CPU wheel should include the same torch/include/ headers as the CUDA wheel, or at minimum the headers required by torch.compile (inductor):

• torch/csrc/inductor/cpp_prefix.h
• torch/csrc/inductor/aoti_torch/
• c10/util/BFloat16.h, c10/util/Half.h, etc.
• c10/macros/ and c10/core/

Impact

Any project using torch.compile with a CPU wheel hits this at runtime:

fatal error: torch/csrc/inductor/cpp_prefix.h: No such file or directory

This affects vLLM CPU inference (which uses torch.compile by default since v0.17.0), and likely any other project relying on inductor's C++ backend on CPU-only installations.

Current Workaround

We download the matching CUDA wheel from PyPI and extract only the include/ directory into the CPU installation. This adds ~10 seconds to Docker builds and feels like something that should be handled at the wheel packaging level.

Environment

• PyTorch: 2.11.0+cpu (also confirmed on 2.10.0+cpu)
• Platform: linux x86_64 and aarch64
• Install: pip install torch --index-url https://download.pytorch.org/whl/cpu
• CUDA wheels from pypi.org/project/torch/ include the full headers and work correctly

cc @seemethere @malfet @atalman @tinglvv @nWEIdia @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @jerryzh168 @aditew01 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo