Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Bug]: Possible warm start compile time issue for Deepseek V3.2 and Kimi K2.5 [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38051•Fetched 2026-04-08 01:26:55
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
zou3519zou3519
Participants
zou3519zou3519
Timeline (top)
labeled ×2added_to_project_v2 ×1cross-referenced ×1project_v2_item_status_changed ×1

Fix Action

Fixed

PR fix notes

PR #38046: [compile] Add some more startup tests for top models

Description (problem / solution / changelog)

This PR adds some more counter-based torch.compile startup time tests for top models. The goal is for us to continuously refresh these tests with the top models. I chose an older version of GLM because vLLM's HF range does not fully support the newer GLM models.

The way the vLLM-torch.compile integration works right now is that we will do graph splitting to create subgraphs and then only produce compiled artifacts for the unique subgraphs. For these models, we check that the number of unique subgraphs is much less than the total number of layers, otherwise something has gone wrong.

These tests make smaller versions of the models with dummy weights to keep the compile times low while still being able to test the unique subgraph behavior.

There are two followups after this PR:

  • early-exit the inductor compilation, will make these tests run 2x faster
  • figure out what is going on with deepseek v3.2 and kimi k2.5 warm start time.

Changed files

  • tests/compile/test_startup.py (modified, +161/-1)
RAW_BUFFERClick to expand / collapse

Your current environment

main

🐛 Describe the bug

see https://github.com/vllm-project/vllm/pull/38046 I'm not sure if the issue is that the test is wrong or if it actually manifests in real life. Need to go benchmarking later

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

The fix involves updating the test to accurately reflect real-life scenarios and benchmarking to verify the issue.

Example code snippet to add benchmarking:

import timeit

def benchmark_test():
    # Test code here
    start_time = timeit.default_timer()
    # Code to be benchmarked
    end_time = timeit.default_timer()
    print(f"Execution time: {end_time - start_time} seconds")

benchmark_test()

Verification

Run the updated test and benchmarking to verify that the issue is resolved.

Extra Tips

  • Use a profiling tool to identify performance bottlenecks in the code.
  • Consider adding more test cases to cover different scenarios.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#API middleware#SSR setup#ISR setup#authentication setup#request error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING