Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Feature]: upstream nightly rocm vllm [3 pull requests, 3 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#36703•Fetched 2026-04-08 00:35:19
View on GitHub
Comments
3
Participants
3
Timeline
20
Reactions
0
Author
functionstackxfunctionstackx
Participants
functionstackxfunctionstackxgithub-actions[bot]github-actions[bot]hongxiayanghongxiayang
Timeline (top)
mentioned ×5subscribed ×5commented ×3labeled ×2

Fix Action

Fixed

PR fix notes

PR #37283: [Releases] [ROCm] Enable Nightly Docker Image and Wheel Releases for ROCm

Description (problem / solution / changelog)

Purpose

Address https://github.com/vllm-project/vllm/issues/36703

This PR enables Nightly Docker Image and Wheel Releases for ROCm

This PR can still stand on its own as it will build the base image and populate the cache if the base docker image is not found in the cache.

User Experience Details

Docker

Following the CUDA Release pipeline, we are only keeping the docker image of latest 14 commits.

The docker image on dockerhub will be released with the following tag pattern:

  • Base Docker Image (for traceability):
    • vllm/vllm-openai-rocm:base-nightly
    • vllm/vllm-openai-rocm:base-nightly-<commit>
  • vLLM OpenAI Docker Image:
    • vllm/vllm-openai-rocm:nightly
    • vllm/vllm-openai-rocm:nightly-<commit>

Example log:

[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:1227c9527d573e09-rocm-base vllm/vllm-openai-rocm:base-nightly
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:1227c9527d573e09-rocm-base vllm/vllm-openai-rocm:base-nightly-57e207873b521dcbaba50b37153b0dd0b5883636
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:57e207873b521dcbaba50b37153b0dd0b5883636-rocm vllm/vllm-openai-rocm:nightly
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:57e207873b521dcbaba50b37153b0dd0b5883636-rocm vllm/vllm-openai-rocm:nightly-57e207873b521dcbaba50b37153b0dd0b5883636

Wheel

The ROCm wheels are located at the path:

  • https://wheels.vllm.ai/rocm/nightly/<variant> now https://wheels.vllm.ai/rocm/nightly/rocm700
  • https://wheels.vllm.ai/rocm/36c72b2191380fa3809928f7b29880a499177457/rocm700/

Test plan

Trigger the pipeline

Test results

Mock results https://buildkite.com/vllm/release-pipeline-shadow/builds/3264/steps/canvas

Enhancement (Future plan)

Enhancement: ci-infra PR https://github.com/vllm-project/ci-infra/pull/297 to ensure the Dockerfile.rocm_base cache is always pre-populated.

<details> <summary> Archive </summary>

Test Plan

Step 1: Trigger Ci-Infra PR https://github.com/vllm-project/ci-infra/pull/297 is triggered to populate the cache

Step 2: Trigger this PR

  • Ensure that this Pipeline reuses the cached docker image and wheels from Step 1.

Test Result

Step 1: Trigger Ci-Infra PR https://github.com/vllm-project/ci-infra/pull/297 is triggered to populate the cache

https://buildkite.com/vllm/amd-ci/builds/6593/steps/canvas?sid=019cfb35-7dd4-4c22-8543-1b403a11356e&tab=output

The generated based docker image and wheels are

[2026-03-17T09:56:01Z] ROCm Base Image Build/Reuse
[2026-03-17T09:56:01Z]   Cache Key: b58dc988fa0856d2-9d3bce57
[2026-03-17T09:56:01Z]   ECR Cache Tag: public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base
[2026-03-17T09:56:01Z]   ECR Commit Tag: public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base

[2026-03-17T09:56:04Z] Tagged public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base as public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base in ECR (no pull required)
[2026-03-17T09:56:04Z] Base image ready: public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base

Step 2: Trigger this PR

The results are in https://buildkite.com/vllm/release-pipeline-shadow/builds/3262/steps/canvas

Building docker vLLM image, the cached base docker image is downloaded and reused https://buildkite.com/vllm/release-pipeline-shadow/builds/3262/steps/canvas?sid=019d0e39-a357-44d8-a6f8-43ff0e15b1ac&tab=output

[2026-03-21T05:59:30Z] Pulling base Docker image from ECR: public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base
[2026-03-21T05:59:30Z] b58dc988fa0856d2-9d3bce57-rocm-base: Pulling from q9t5s3a7/vllm-release-repo

Implementation Details

Challenges:

The ROCm releases depends on custom dependencies that are specified in Dockerfile.rocm_base. The build time of this docker file takes more than 3 hrs even after sccache enabled. The majority of the build time comes from torch and amd-aiter (if we choose to enable prebuilt again).

Current implementation reuses the caching logic from https://github.com/vllm-project/vllm/pull/32264 to cache the base docker image and dependency wheels (created from Dockerfile.rocm_base).

Ci-infra PR https://github.com/vllm-project/ci-infra/pull/297 to ensure the Dockerfile.rocm_base cache is always pre-populated.

</details>
<details> <summary> Essential Elements of an Effective PR Description Checklist </summary>
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.
</details>

Changed files

  • .buildkite/release-pipeline.yaml (modified, +158/-219)
  • .buildkite/scripts/annotate-release.sh (modified, +4/-2)
  • .buildkite/scripts/annotate-rocm-release.sh (modified, +6/-5)
  • .buildkite/scripts/cache-rocm-base-wheels.sh (modified, +7/-16)
  • .buildkite/scripts/cleanup-nightly-builds.sh (modified, +10/-7)
  • .buildkite/scripts/push-nightly-builds-rocm.sh (added, +62/-0)
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

currently only cuda has nightly pip install vllm wheels . if rocm wants to be first class, it should have it too

<img width="578" height="530" alt="Image" src="https://github.com/user-attachments/assets/90c92903-fa27-431c-87a1-63962444c388" />

Alternatives

build from source

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To add nightly pip install for ROCm, follow these steps:

  • Create a new GitHub Actions workflow to build and deploy ROCm wheels
  • Update the setup.py file to include ROCm as a supported platform
  • Configure the workflow to upload the wheels to PyPI

Example Code

# setup.py
import setuptools

setuptools.setup(
    name='vllm',
    version='1.0',
    platforms=['cuda', 'rocm'],  # Add 'rocm' as a supported platform
    # ...
)
# .github/workflows/build_and_deploy_rocm.yml
name: Build and Deploy ROCm Wheels

on:
  push:
    branches:
      - main

jobs:
  build_and_deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Setup ROCm
        uses: rocmetrics/rocm-actions@v1
      - name: Build wheel
        run: |
          python setup.py bdist_wheel
      - name: Upload to PyPI
        uses: pypa/gh-action-pypi-deploy@v1
        with:
          username: ${{ secrets.PYPI_USERNAME }}
          password: ${{ secrets.PYPI_PASSWORD }}

Verification

To verify that the fix worked, check that the ROCm wheels are available on PyPI and can be installed using pip:

pip install --pre vllm

If the installation is successful, the fix has worked.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#serialization error#model compatibility#GPU setup#container setup#orchestration issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - ✅(Solved) Fix [Feature]: upstream nightly rocm vllm [3 pull requests, 3 comments, 3 participants]