pytorch - 💡(How to fix) Fix [CD] Drop CPython 3.13t from binary build matrix (manylinux upstream removed it 2026-05-07)

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Upstream pypa/manylinux dropped CPython 3.13t (free-threaded) from the manylinux images on 2026-05-07. PyTorch's binary-build matrix still includes 3.13t, and .ci/docker/manywheel/Dockerfile_2_28 references /opt/python/cp313-cp313t/bin/python, so the next fresh rebuild of that Docker image fails. Cached image layers on main mask the issue today; any PR that triggers a rebuild (e.g. by changing anything under .ci/docker/) hits the wall.

Root Cause

Upstream pypa/manylinux dropped CPython 3.13t (free-threaded) from the manylinux images on 2026-05-07. PyTorch's binary-build matrix still includes 3.13t, and .ci/docker/manywheel/Dockerfile_2_28 references /opt/python/cp313-cp313t/bin/python, so the next fresh rebuild of that Docker image fails. Cached image layers on main mask the issue today; any PR that triggers a rebuild (e.g. by changing anything under .ci/docker/) hits the wall.

Fix Action

Fix / Workaround

Alternative (workaround, not recommended)

Code Example

$ docker run --rm quay.io/pypa/manylinux_2_28_x86_64:latest ls /opt/python/
cp310-cp310  cp311-cp311  cp312-cp312  cp313-cp313  cp314-cp314
cp314-cp314t  cp315-cp315  cp315-cp315t  cp39-cp39  pp311-pypy311_pp73

---

RUN for cpython_version in "cp312-cp312" "cp313-cp313" "cp313-cp313t"; do \
    /opt/python/${cpython_version}/bin/python -m pip install setuptools wheel; \
    done;

---

/bin/sh: /opt/python/cp313-cp313t/bin/python: No such file or directory
exit code: 127

---

FULL_PYTHON_VERSIONS = ["3.10", "3.11", "3.12", "3.13", "3.13t", "3.14", "3.14t"]
RAW_BUFFERClick to expand / collapse

Summary

Upstream pypa/manylinux dropped CPython 3.13t (free-threaded) from the manylinux images on 2026-05-07. PyTorch's binary-build matrix still includes 3.13t, and .ci/docker/manywheel/Dockerfile_2_28 references /opt/python/cp313-cp313t/bin/python, so the next fresh rebuild of that Docker image fails. Cached image layers on main mask the issue today; any PR that triggers a rebuild (e.g. by changing anything under .ci/docker/) hits the wall.

Reproducer

$ docker run --rm quay.io/pypa/manylinux_2_28_x86_64:latest ls /opt/python/
cp310-cp310  cp311-cp311  cp312-cp312  cp313-cp313  cp314-cp314
cp314-cp314t  cp315-cp315  cp315-cp315t  cp39-cp39  pp311-pypy311_pp73

No cp313-cp313t. Hence the loop in .ci/docker/manywheel/Dockerfile_2_28:115:

RUN for cpython_version in "cp312-cp312" "cp313-cp313" "cp313-cp313t"; do \
    /opt/python/${cpython_version}/bin/python -m pip install setuptools wheel; \
    done;

fails on the third iteration with:

/bin/sh: /opt/python/cp313-cp313t/bin/python: No such file or directory
exit code: 127

This breaks every manylinux2_28-builder:* job (cpu, cuda12.6, cuda13.0, cuda13.2, rocm7.1, rocm7.2, xpu) and cascades into every wheel-py3_*-{cpu,cuda,xpu}-build that depends on those builder images.

Observed first on #180237 (which forces a rebuild via an unrelated .ci/docker/requirements-ci.txt change). Failing run: https://github.com/pytorch/pytorch/actions/runs/25554099840.

Upstream context

Quoting #1882: "CPython 3.13t (free-threading/no-gil) was experimental and has become less relevant now that CPython 3.14t is available and no longer experimental. Free-threading fixes are not always backported in CPython 3.13 given its experimental status."

So this isn't an upstream regression to revert — it's a planned EOL announced ~5 months in advance. The free-threaded variants in the new image are cp314-cp314t and cp315-cp315t.

What needs to change in PyTorch

  1. .github/scripts/generate_binary_build_matrix.py:309 — drop "3.13t" from FULL_PYTHON_VERSIONS:

    FULL_PYTHON_VERSIONS = ["3.10", "3.11", "3.12", "3.13", "3.13t", "3.14", "3.14t"]
  2. .ci/docker/manywheel/Dockerfile_2_28:115 — drop "cp313-cp313t" from the loop.

  3. Any downstream config that special-cases 3.13t (Windows installer URLs, macOS _MACOS_SETUP_PYTHON_VERSION, release scripts, etc.) — needs an audit.

Alternative (workaround, not recommended)

Pin the manylinux base image in the manywheel Dockerfiles to a pre-drop tag (quay.io/pypa/manylinux_2_28_x86_64:2026.05.02-2). This preserves 3.13t builds for now but defers the upstream-EOL alignment and accumulates technical debt against future image updates.

Impact

  • 3.13 free-threaded was an experimental ABI; users targeting free-threaded should migrate to 3.14t (which is non-experimental).
  • PyTorch 2.x already shipped 3.13t wheels for 2.6+; dropping for the next release follows upstream convention but is a release-notes-worthy change.

cc @seemethere @malfet @atalman @tinglvv @nWEIdia @pytorch/pytorch-dev-infra

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

pytorch - 💡(How to fix) Fix [CD] Drop CPython 3.13t from binary build matrix (manylinux upstream removed it 2026-05-07)