Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Bug]: dependency `nixl<0.10.0` installs `nixl-cu12==0.10.1` [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#36676•Fetched 2026-04-08 00:35:26
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
1
Author
cjackalcjackal
Participants
cjackalcjackal
Timeline (top)
cross-referenced ×1labeled ×1mentioned ×1subscribed ×1

Fix Action

Fixed

PR fix notes

PR #36735: [Docker] Resolve NIXL runtime wheel for kv connectors

Description (problem / solution / changelog)

Purpose

Fixes #36676 by resolving requirements/kv_connectors.txt to the CUDA-runtime-specific NIXL wheel during Docker builds instead of relying on the nixl meta package.

This keeps the existing <0.10.0 compatibility bound while preventing the pre-0.10.1 meta package from pulling a newer nixl-cu12 or nixl-cu13 wheel through its transitive >= dependency.

The PR also updates the NIXL user docs to recommend installing nixl-cu12 or nixl-cu13 directly on NVIDIA platforms.

Test Plan

  • python3 -m py_compile tools/resolve_kv_connector_requirements.py
  • pytest tests/tools/test_resolve_kv_connector_requirements.py
  • python3 tools/resolve_kv_connector_requirements.py --input requirements/kv_connectors.txt --cuda-major 12 --output /tmp/kv_connectors_cuda12.txt
  • python3 tools/resolve_kv_connector_requirements.py --input requirements/kv_connectors.txt --cuda-major 13 --output /tmp/kv_connectors_cuda13.txt

Test Result

  • pytest tests/tools/test_resolve_kv_connector_requirements.py: passed (4 tests)
  • CUDA 12 resolution output rewrites nixl to nixl-cu12 while preserving the version bound and inline comment
  • CUDA 13 resolution output rewrites nixl to nixl-cu13 while preserving the version bound and inline comment

Changed files

  • docker/Dockerfile (modified, +7/-5)
  • docs/contributing/dockerfile/dockerfile.md (modified, +5/-0)
  • docs/features/nixl_connector_usage.md (modified, +11/-1)
  • docs/serving/expert_parallel_deployment.md (modified, +1/-1)
  • tests/tools/test_resolve_kv_connector_requirements.py (added, +43/-0)
  • tools/resolve_kv_connector_requirements.py (added, +102/-0)

Code Example

# Inside vllm/vllm-openai:v0.17.0 container

$ cat /usr/local/lib/python3.12/dist-packages/nixl-0.9.0-dist-info/METADATA | grep Requires-Dist
# Requires-Dist: nixl-cu12>=0.9.0
# Requires-Dist: nixl-cu12>=0.9.0; extra == "cu12"
# Requires-Dist: nixl-cu13>=0.9.0; extra == "cu13"

$ uv pip show nixl-cu12
# Using Python 3.12.13 environment at: /usr
# Name: nixl-cu12
# Version: 0.10.1
# Location: /usr/local/lib/python3.12/dist-packages
# Requires: numpy, torch
# Required-by: nixl
RAW_BUFFERClick to expand / collapse

Your current environment

<details> <summary>The output of <code>python collect_env.py</code></summary>

This issue is about vllm packaging, independent of my worker spec.

</details>

🐛 Describe the bug

vllm uses nixl as the primary kv connector for PDD, and as such vllm/vllm-openai docker image includes nixl as kv connector dependencies. Since #35495 we pinned the nixl version as nixl >= 0.7.1, < 0.10.0, but this does not work as intended:

nixl python package is just a meta package which directs to nixl-cu12 or nixl-cu13 depending on the desired CUDA runtime, and before nixl==0.10.1, it directs to nixl-cu1{2,3}>=@VERSION@, not nixl-cu1{2,3}==@VERSION@. One can check the package dependency like the following:

# Inside vllm/vllm-openai:v0.17.0 container

$ cat /usr/local/lib/python3.12/dist-packages/nixl-0.9.0-dist-info/METADATA | grep Requires-Dist
# Requires-Dist: nixl-cu12>=0.9.0
# Requires-Dist: nixl-cu12>=0.9.0; extra == "cu12"
# Requires-Dist: nixl-cu13>=0.9.0; extra == "cu13"

$ uv pip show nixl-cu12
# Using Python 3.12.13 environment at: /usr
# Name: nixl-cu12
# Version: 0.10.1
# Location: /usr/local/lib/python3.12/dist-packages
# Requires: numpy, torch
# Required-by: nixl

I haven't yet faced a bug from nixl-cu12==0.10.1, but it is worth fixing in CI build pipeline I think. The pain point for the fix is that we now need to manually edit CUDA runtime version to the kv_connectors.txt, some sort of sed is needed.

cc. @NickLucche

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To fix the issue, we need to update the kv_connectors.txt file to specify the correct version of nixl-cu12 or nixl-cu13 depending on the desired CUDA runtime. We can use sed to manually edit the CUDA runtime version.

  • Update the kv_connectors.txt file to include the correct version of nixl-cu12 or nixl-cu13. For example:
sed -i 's/nixl >= 0.7.1, < 0.10.0/nixl-cu12==0.9.0/' kv_connectors.txt

Alternatively, you can use a more robust approach by using a Python script to update the kv_connectors.txt file:

import re

with open('kv_connectors.txt', 'r') as f:
    lines = f.readlines()

with open('kv_connectors.txt', 'w') as f:
    for line in lines:
        if 'nixl' in line:
            line = re.sub(r'nixl >= 0.7.1, < 0.10.0', 'nixl-cu12==0.9.0', line)
        f.write(line)
  • Update the CI build pipeline to use the updated kv_connectors.txt file.

Verification

To verify that the fix worked, you can check the package dependency using the following command:

cat /usr/local/lib/python3.12/dist-packages/nixl-0.9.0-dist-info/METADATA | grep Requires-Dist

This should show the updated version of nixl-cu12 or nixl-cu13.

Extra Tips

  • Make sure to test the updated kv_connectors.txt file in a development environment before deploying it to production.
  • Consider using a more robust package management system, such as pip-compile, to manage dependencies and avoid manual editing of the kv_connectors.txt file.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#container setup#orchestration issue#cache issue#memory leak#API versioning

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - ✅(Solved) Fix [Bug]: dependency `nixl<0.10.0` installs `nixl-cu12==0.10.1` [1 pull requests, 1 participants]