Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [CI Failure]: mi325_1: Language Models Test (Extended Generation) [2 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#41584•Fetched 2026-05-04 04:58:36
View on GitHub
Comments
2
Participants
1
Timeline
10
Reactions
0
Author
AndreasKaratzasAndreasKaratzas
Participants
AndreasKaratzasAndreasKaratzas
Timeline (top)
added_to_project_v2 ×2commented ×2renamed ×2labeled ×1

Root Cause

  • Flaky test
  • Can reproduce locally
  • Caused by external libraries (e.g. bug in transformers)

Code Example

FAILED models/language/generation/test_common.py::test_models[True-False-5-32-naver-hyperclovax/HyperCLOVAX-SEED-Think-14B]
FAILED models/language/generation/test_common.py::test_models[False-False-5-32-naver-hyperclovax/HyperCLOVAX-SEED-Think-14B]
RAW_BUFFERClick to expand / collapse

Name of failing test

(command rocm-smi || true) && export VLLM_TEST_GROUP_NAME=mi325_1-language-models-test-extended-generation && export VLLM_ALLOW_DEPRECATED_BEAM_SEARCH=1 && cd /vllm-workspace/tests && uv pip install --system --no-build-isolation 'git+https://github.com/AndreasKaratzas/[email protected]' && uv pip install --system --no-build-isolation 'git+https://github.com/Dao-AILab/[email protected]' && pytest -v -s models/language/generation -m '(not core_model) and (not hybrid_model)'

Basic information

  • Flaky test
  • Can reproduce locally
  • Caused by external libraries (e.g. bug in transformers)

🧪 Describe the failing test

FAILED models/language/generation/test_common.py::test_models[True-False-5-32-naver-hyperclovax/HyperCLOVAX-SEED-Think-14B]
FAILED models/language/generation/test_common.py::test_models[False-False-5-32-naver-hyperclovax/HyperCLOVAX-SEED-Think-14B]

📝 History of failing test

  • Current streak start: 2026-04-16
  • First failure in 60d window: 2026-04-09
  • Last successful nightly: 2026-04-15
  • Break frequency (60d, pass↔fail flips): 3
  • Latest nightly date: 2026-05-03
  • Latest build(s): amd-ci #8177
  • Latest hardware status: mi325_1=fail

extent analysis

TL;DR

The failing test can be mitigated by investigating the test_common.py file and the HyperCLOVAX-SEED-Think-14B model configuration.

Guidance

  • Review the test_common.py file to identify potential issues with the test_models function, specifically the test cases that are failing.
  • Investigate the HyperCLOVAX-SEED-Think-14B model configuration to see if there are any changes or updates that could be causing the test failures.
  • Check the build logs for the latest nightly build (amd-ci #8177) to see if there are any errors or warnings related to the failing test.
  • Consider running the test in isolation to see if it's a problem with the test itself or if it's related to the overall test suite.

Notes

The issue seems to be related to a specific test case and model configuration, so the solution may involve updating the test or the model configuration.

Recommendation

Apply workaround: Investigate and update the test_common.py file and the HyperCLOVAX-SEED-Think-14B model configuration to resolve the test failures.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#memory leak#API versioning#request timeout

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - 💡(How to fix) Fix [CI Failure]: mi325_1: Language Models Test (Extended Generation) [2 comments, 1 participants]