Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Feature]: vllm.ai docs should should instructions for rocm nightly docker & rocm nightly wheel [3 pull requests, 7 comments, 5 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38304•Fetched 2026-04-08 01:36:40
View on GitHub
Comments
7
Participants
5
Timeline
34
Reactions
0
Author
functionstackxfunctionstackx
Participants
esmeetuesmeetufunctionstackxfunctionstackxgithub-actions[bot]github-actions[bot]hongxiayanghongxiayangtjtanaatjtanaa
Assignees
hongxiayanghongxiayang
Timeline (top)
mentioned ×10subscribed ×10commented ×7labeled ×2

PR fix notes

PR #37283: [Releases] [ROCm] Enable Nightly Docker Image and Wheel Releases for ROCm

Description (problem / solution / changelog)

Purpose

Address https://github.com/vllm-project/vllm/issues/36703

This PR enables Nightly Docker Image and Wheel Releases for ROCm

This PR can still stand on its own as it will build the base image and populate the cache if the base docker image is not found in the cache.

User Experience Details

Docker

Following the CUDA Release pipeline, we are only keeping the docker image of latest 14 commits.

The docker image on dockerhub will be released with the following tag pattern:

  • Base Docker Image (for traceability):
    • vllm/vllm-openai-rocm:base-nightly
    • vllm/vllm-openai-rocm:base-nightly-<commit>
  • vLLM OpenAI Docker Image:
    • vllm/vllm-openai-rocm:nightly
    • vllm/vllm-openai-rocm:nightly-<commit>

Example log:

[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:1227c9527d573e09-rocm-base vllm/vllm-openai-rocm:base-nightly
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:1227c9527d573e09-rocm-base vllm/vllm-openai-rocm:base-nightly-57e207873b521dcbaba50b37153b0dd0b5883636
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:57e207873b521dcbaba50b37153b0dd0b5883636-rocm vllm/vllm-openai-rocm:nightly
[2026-03-23T17:42:32Z] + docker tag public.ecr.aws/q9t5s3a7/vllm-release-repo:57e207873b521dcbaba50b37153b0dd0b5883636-rocm vllm/vllm-openai-rocm:nightly-57e207873b521dcbaba50b37153b0dd0b5883636

Wheel

The ROCm wheels are located at the path:

  • https://wheels.vllm.ai/rocm/nightly/<variant> now https://wheels.vllm.ai/rocm/nightly/rocm700
  • https://wheels.vllm.ai/rocm/36c72b2191380fa3809928f7b29880a499177457/rocm700/

Test plan

Trigger the pipeline

Test results

Mock results https://buildkite.com/vllm/release-pipeline-shadow/builds/3264/steps/canvas

Enhancement (Future plan)

Enhancement: ci-infra PR https://github.com/vllm-project/ci-infra/pull/297 to ensure the Dockerfile.rocm_base cache is always pre-populated.

<details> <summary> Archive </summary>

Test Plan

Step 1: Trigger Ci-Infra PR https://github.com/vllm-project/ci-infra/pull/297 is triggered to populate the cache

Step 2: Trigger this PR

  • Ensure that this Pipeline reuses the cached docker image and wheels from Step 1.

Test Result

Step 1: Trigger Ci-Infra PR https://github.com/vllm-project/ci-infra/pull/297 is triggered to populate the cache

https://buildkite.com/vllm/amd-ci/builds/6593/steps/canvas?sid=019cfb35-7dd4-4c22-8543-1b403a11356e&tab=output

The generated based docker image and wheels are

[2026-03-17T09:56:01Z] ROCm Base Image Build/Reuse
[2026-03-17T09:56:01Z]   Cache Key: b58dc988fa0856d2-9d3bce57
[2026-03-17T09:56:01Z]   ECR Cache Tag: public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base
[2026-03-17T09:56:01Z]   ECR Commit Tag: public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base

[2026-03-17T09:56:04Z] Tagged public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base as public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base in ECR (no pull required)
[2026-03-17T09:56:04Z] Base image ready: public.ecr.aws/q9t5s3a7/vllm-release-repo:0e4701ff0f6802aef64d449f7e8ab3c6599ca6e3-b58dc988fa0856d2-9d3bce57-rocm-base

Step 2: Trigger this PR

The results are in https://buildkite.com/vllm/release-pipeline-shadow/builds/3262/steps/canvas

Building docker vLLM image, the cached base docker image is downloaded and reused https://buildkite.com/vllm/release-pipeline-shadow/builds/3262/steps/canvas?sid=019d0e39-a357-44d8-a6f8-43ff0e15b1ac&tab=output

[2026-03-21T05:59:30Z] Pulling base Docker image from ECR: public.ecr.aws/q9t5s3a7/vllm-release-repo:b58dc988fa0856d2-9d3bce57-rocm-base
[2026-03-21T05:59:30Z] b58dc988fa0856d2-9d3bce57-rocm-base: Pulling from q9t5s3a7/vllm-release-repo

Implementation Details

Challenges:

The ROCm releases depends on custom dependencies that are specified in Dockerfile.rocm_base. The build time of this docker file takes more than 3 hrs even after sccache enabled. The majority of the build time comes from torch and amd-aiter (if we choose to enable prebuilt again).

Current implementation reuses the caching logic from https://github.com/vllm-project/vllm/pull/32264 to cache the base docker image and dependency wheels (created from Dockerfile.rocm_base).

Ci-infra PR https://github.com/vllm-project/ci-infra/pull/297 to ensure the Dockerfile.rocm_base cache is always pre-populated.

</details>
<details> <summary> Essential Elements of an Effective PR Description Checklist </summary>
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.
</details>

Changed files

  • .buildkite/release-pipeline.yaml (modified, +158/-219)
  • .buildkite/scripts/annotate-release.sh (modified, +4/-2)
  • .buildkite/scripts/annotate-rocm-release.sh (modified, +6/-5)
  • .buildkite/scripts/cache-rocm-base-wheels.sh (modified, +7/-16)
  • .buildkite/scripts/cleanup-nightly-builds.sh (modified, +10/-7)
  • .buildkite/scripts/push-nightly-builds-rocm.sh (added, +62/-0)
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

hi @powderluv @chunfangamd @andyluo7 @hongxiayang

https://github.com/vllm-project/vllm/pull/37283 added rocm nightly docker & rocm nightly wheel but unfortunately vllm.ai docs dont show it. can u look into fixing this small issue?

ROCm vllm.ai vs CUDA vllm.ai for nightly

<img width="616" height="316" alt="Image" src="https://github.com/user-attachments/assets/cf412b4c-265a-45b4-aa6e-c2d301308659" /> <img width="583" height="563" alt="Image" src="https://github.com/user-attachments/assets/4522852f-177b-4e62-9a78-975065d51d38" />

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To fix the issue of the vllm.ai docs not showing the ROCm nightly docker and wheel, we need to update the documentation to include the new ROCm nightly builds.

Steps to Fix

  • Update the README.md file to include information about the ROCm nightly docker and wheel.
  • Add a new section to the documentation to describe how to use the ROCm nightly builds.
  • Update the CI/CD pipeline to automatically generate documentation for the ROCm nightly builds.

Example Code

### ROCm Nightly Builds
To use the ROCm nightly builds, follow these steps:

1. Pull the latest ROCm nightly docker image: `docker pull vllm/rocm-nightly`
2. Install the ROCm nightly wheel: `pip install vllm-rocm-nightly`
# .github/workflows/docs.yml
name: Docs

on:
  push:
    branches:
      - main

jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Generate docs
        run: |
          # Generate documentation for CUDA builds
          # ...
          # Generate documentation for ROCm nightly builds
          echo "### ROCm Nightly Builds" >> README.md
          echo "To use the ROCm nightly builds, follow these steps:" >> README.md
          echo "1. Pull the latest ROCm nightly docker image: `docker pull vllm/rocm-nightly`" >> README.md
          echo "2. Install the ROCm nightly wheel: `pip install vllm-rocm-nightly`" >> README.md
      - name: Deploy docs
        uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./docs

Verification

To verify that the fix worked, check the vllm.ai documentation to ensure that the ROCm nightly builds are listed and the instructions for using them are correct.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#index setup#retrieval issue#search optimization#API routing#API middleware

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING