vllm - 💡(How to fix) Fix [Bug]: vLLM 0.15.0 startup on H200 failed at deep_gemm [2 comments, 3 participants]

vllm2026-03-10 22:41:34

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#36718•Fetched 2026-04-08 00:35:16

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Participants

Timeline (top)

commented ×2labeled ×1

Error Message

Hi Team,

We’re using v0.15.0, and it failed to run on a H200 instance because of the DeepGEMM assertion error: RuntimeError: Assertion error (csrc/apis/../jit_kernels/impls/../../jit/kernel_runtime.hpp:45): exit_code == 0

DeepGEMM's JIT kernel is crashing when vLLM tries to run FP8 GEMM operations (fp8_gemm_nt) during startup profiling.

Can you help take a look ?

Root Cause

Code Example

Hi Team,

We’re using v0.15.0, and it failed to run on a H200 instance because of the DeepGEMM assertion error: RuntimeError: Assertion error (csrc/apis/../jit_kernels/impls/../../jit/kernel_runtime.hpp:45): exit_code == 0

DeepGEMM's JIT kernel is crashing when vLLM tries to run FP8 GEMM operations (fp8_gemm_nt) during startup profiling.

Can you help take a look ?

---

0.15.1 
4:57:02 PM [algo-1-1772841836] [vllm.server] Traceback (most recent call last):
4:57:02 PM [algo-1-1772841836] [vllm.server] RuntimeError: Worker failed with error 'Assertion error (csrc/apis/../jit_kernels/impls/../../jit/kernel_runtime.hpp:45): exit_code == 0', please check the stack trace above for the root cause
4:57:03 PM [algo-1-1772841836] [vllm.server] Traceback (most recent call last):
4:57:03 PM [algo-1-1772841836] [vllm.server] RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] ✗ Configuration failed: VLLM: DP=1, TP=8, KV=0.7, eager
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] ✗ All vLLM configurations failed
4:57:10 PM [algo-1-1772841836] /opt/wrapper/libfarm/lib/python3.11/site-packages/watchtower/__init__.py:464: WatchtowerWarning: Received message after logging system shutdown warnings.warn("Received message after logging system shutdown", WatchtowerWarning)
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] Stopping server server (PID: 68574)

RAW_BUFFERClick to expand / collapse

Your current environment

<details> <summary>vLLM 0.15.0 and 0.15.1 both failed on H200 instance with DeepGEMM assertion error</summary>

Hi Team,

We’re using v0.15.0, and it failed to run on a H200 instance because of the DeepGEMM assertion error: RuntimeError: Assertion error (csrc/apis/../jit_kernels/impls/../../jit/kernel_runtime.hpp:45): exit_code == 0

DeepGEMM's JIT kernel is crashing when vLLM tries to run FP8 GEMM operations (fp8_gemm_nt) during startup profiling.

Can you help take a look ?

</details>

🐛 Describe the bug


0.15.1 
4:57:02 PM [algo-1-1772841836] [vllm.server] Traceback (most recent call last):
4:57:02 PM [algo-1-1772841836] [vllm.server] RuntimeError: Worker failed with error 'Assertion error (csrc/apis/../jit_kernels/impls/../../jit/kernel_runtime.hpp:45): exit_code == 0', please check the stack trace above for the root cause
4:57:03 PM [algo-1-1772841836] [vllm.server] Traceback (most recent call last):
4:57:03 PM [algo-1-1772841836] [vllm.server] RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] ✗ Configuration failed: VLLM: DP=1, TP=8, KV=0.7, eager
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] ✗ All vLLM configurations failed
4:57:10 PM [algo-1-1772841836] /opt/wrapper/libfarm/lib/python3.11/site-packages/watchtower/__init__.py:464: WatchtowerWarning: Received message after logging system shutdown warnings.warn("Received message after logging system shutdown", WatchtowerWarning)
4:57:10 PM [algo-1-1772841836] [VLLM_STATUS] Stopping server server (PID: 68574)

Before submitting a new issue...

Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To resolve the DeepGEMM assertion error, we need to update the JIT kernel configuration to handle FP8 GEMM operations correctly. Here are the steps:

Update the vllm configuration to disable FP8 GEMM operations:

import vllm

# Disable FP8 GEMM operations
vllm.config.fp8_gemm_enabled = False

Alternatively, you can also update the DeepGEMM kernel to handle FP8 GEMM operations:

// Update the DeepGEMM kernel to handle FP8 GEMM operations
void fp8_gemm_nt(...) {
    // Add a check for FP8 GEMM operations
    if (fp8_gemm_enabled) {
        // Handle FP8 GEMM operations correctly
    } else {
        // Fall back to default GEMM operations
    }
}

If you are using a vllm version older than 0.15.1, you may need to upgrade to the latest version to get the fix.

Verification

To verify that the fix worked, you can run the vllm server with the updated configuration and check for any errors:

vllm-server --config updated_config.json

Check the server logs for any errors related to DeepGEMM or FP8 GEMM operations.

Extra Tips

Make sure to update the vllm configuration and kernel code carefully to avoid introducing any new errors.
If you are using a custom vllm build, you may need to rebuild the kernel with the updated code.
You can also try disabling FP8 GEMM operations temporarily to see if it resolves the issue.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #request error #file not found #serialization error #model compatibility

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - 💡(How to fix) Fix [Bug]: vLLM 0.15.0 startup on H200 failed at deep_gemm [2 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

TRENDING

vllm - 💡(How to fix) Fix [Bug]: vLLM 0.15.0 startup on H200 failed at deep_gemm [2 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING