Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [RFC]: Reduce CI Cost by Pruning Low-Value Unit Tests [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#39829•Fetched 2026-04-16 06:36:20
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
2
Author
yewentao256yewentao256
Participants
yewentao256yewentao256
Assignees
yewentao256yewentao256
Timeline (top)
assigned ×1cross-referenced ×1labeled ×1subscribed ×1

Fix Action

Fixed

PR fix notes

PR #39832: [Tests] Remove deprecated and compatibility-only coverage

Description (problem / solution / changelog)

Purpose

Part of https://github.com/vllm-project/vllm/issues/39829

eg. drop pooling tests that only asserted deprecated warning paths

Changed files

  • tests/entrypoints/pooling/classify/test_offline.py (modified, +1/-14)
  • tests/entrypoints/pooling/embed/test_offline.py (modified, +0/-11)
  • tests/entrypoints/pooling/token_classify/test_offline.py (modified, +7/-13)
  • tests/entrypoints/pooling/token_embed/test_offline.py (modified, +7/-13)
  • tests/v1/kv_connector/unit/test_backwards_compatibility.py (removed, +0/-275)
RAW_BUFFERClick to expand / collapse

Motivation.

Our CI unit test suite continues to grow, and the cost of a full CI run grows with it. While comprehensive coverage is important, not all unit tests provide enough signal to justify their runtime and maintenance cost.

The goal of this RFC is not to weaken test quality, but to improve the signal-to-cost ratio of our CI unit tests while preserving confidence in important behaviors and regressions.

Proposed Change.

From my understanding, we can choose to update:

  1. tests cover outdated functionality
  2. tests rely on overly exhaustive parameter combinations

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

Review and refine the existing unit test suite to improve the signal-to-cost ratio by updating outdated tests and optimizing parameter combinations.

Guidance

  • Identify and update tests that cover outdated functionality to ensure they remain relevant and effective.
  • Analyze tests with overly exhaustive parameter combinations and consider optimizing them to reduce runtime and maintenance costs.
  • Evaluate the trade-off between test coverage and runtime cost to determine the optimal balance for the project.
  • Consider categorizing tests based on their signal-to-cost ratio to prioritize and optimize the most critical ones.

Notes

The proposed change aims to improve the efficiency of the unit test suite without compromising test quality, but the optimal approach may vary depending on the specific project requirements and constraints.

Recommendation

Apply workaround: Refine the existing test suite by updating outdated tests and optimizing parameter combinations to improve the signal-to-cost ratio.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#tokenizer error#prompt formatting#chain error#conversation history#tool integration

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - ✅(Solved) Fix [RFC]: Reduce CI Cost by Pruning Low-Value Unit Tests [1 pull requests, 1 participants]