Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [Bug]: gemma4-31B-AWQ Running: 0 reqs, Waiting: 4 reqs [10 comments, 4 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#39037•Fetched 2026-04-08 02:52:54
View on GitHub
Comments
10
Participants
4
Timeline
18
Reactions
1
Author
maxin9966maxin9966
Participants
benjamin-mariebenjamin-marieCyrus-HeiCyrus-Heimaxin9966maxin9966zuukyzuuky
Timeline (top)
commented ×10mentioned ×3subscribed ×3closed ×1

Error Message

(vllm) ma@gpu4090d:~$ python collect_env.py Traceback (most recent call last): File "/home/ma/collect_env.py", line 20, in <module> from vllm.envs import environment_variables File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/init.py", line 14, in <module> import vllm.env_override # noqa: F401 ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/env_override.py", line 87, in <module> import torch File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/init.py", line 431, in <module> from torch._C import * # noqa: F403 ^^^^^^^^^^^^^^^^^^^^^^ ImportError: /home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLogSize_12_9, version libnvJitLink.so.12

Code Example

(vllm) ma@gpu4090d:~$ python collect_env.py
Traceback (most recent call last):
  File "/home/ma/collect_env.py", line 20, in <module>
    from vllm.envs import environment_variables
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/__init__.py", line 14, in <module>
    import vllm.env_override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/env_override.py", line 87, in <module>
    import torch
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/__init__.py", line 431, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: /home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLogSize_12_9, version libnvJitLink.so.12
RAW_BUFFERClick to expand / collapse

Your current environment

4090 vllm 0.19.0 transformer 5.5.0 Driver Version: 570.211.01 CUDA Version: 12.8

<details> <summary>The output of <code>python collect_env.py</code></summary> 
(vllm) ma@gpu4090d:~$ python collect_env.py
Traceback (most recent call last):
  File "/home/ma/collect_env.py", line 20, in <module>
    from vllm.envs import environment_variables
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/__init__.py", line 14, in <module>
    import vllm.env_override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/vllm/env_override.py", line 87, in <module>
    import torch
  File "/home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/__init__.py", line 431, in <module>
    from torch._C import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^
ImportError: /home/ma/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkGetErrorLogSize_12_9, version libnvJitLink.so.12
</details>

🐛 Describe the bug

Apr 05 18:07:38 gpu4090d bash[59857]: (APIServer pid=59857) INFO 04-05 18:07:38 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 4 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 30.3%

模型： cyankiwi/gemma-4-31B-it-AWQ-4bit

Running: 0 reqs, Waiting: 4 reqs，vLLM日志不再定时打印(APIServer pid=xxxxx) INFO，期间GPU零负载，过很长很长时间之后能恢复。

<img width="2490" height="792" alt="Image" src="https://github.com/user-attachments/assets/bde7c842-52f6-44cd-9669-81a3561923ce" />

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The most likely fix is to ensure compatibility between the CUDA version and the driver version, potentially by upgrading or downgrading one of them to match the requirements of the torch library.

Guidance

  • Verify the compatibility of the CUDA version (12.8) with the Driver Version (570.211.01) to ensure they are compatible with the torch library.
  • Check the torch library documentation for specific version requirements and compatibility with CUDA and driver versions.
  • Consider upgrading or downgrading the CUDA version or driver version to match the requirements of the torch library.
  • Investigate the error message undefined symbol: __nvJitLinkGetErrorLogSize_12_9 to determine if it is related to a specific version mismatch.

Notes

The issue seems to be related to a version mismatch between the CUDA version, driver version, and the torch library. However, without more information about the specific requirements of the torch library and the vllm environment, it is difficult to provide a more specific solution.

Recommendation

Apply workaround: The best course of action would be to try to match the CUDA version with the driver version to ensure compatibility with the torch library, as this seems to be the most likely cause of the issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#cache issue#memory leak#API versioning#request timeout

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - 💡(How to fix) Fix [Bug]: gemma4-31B-AWQ Running: 0 reqs, Waiting: 4 reqs [10 comments, 4 participants]