Your current environment

Key info: vLLM 0.17.1, PyTorch 2.10.0+cu129, GPU: NVIDIA
  GB10, OS: Ubuntu 22.04 aarch64.

🐛 Describe the bug

(if it helps) Environment where we first spotted the bug

• vLLM version: v0.17.0rc1.dev164+gfff3711a2
• GPU: NVIDIA B200 (DGX B200)
• Model: Qwen/Qwen3.5-397B-A17B
• TP: 8
• Docker: yes (container wsd-qwen3.5-397b-a17b-vllm)
• OS: Linux

When serving Qwen3.5-397B-A17B with the recommended MTP speculative decoding config from vLLM recipes (num_speculative_tokens=1), the GDN attention backend crashes with a fatal AssertionError during concurrent request serving. The engine dies and all in-flight requests fail with EngineDeadError. I could reproduce this on my DGX spark and smaller models from the Qwen3.5 family. (see gist)

Root cause

The assertion at vllm/v1/attention/backends/gdn_attn.py:310 fails:

assert not (num_decodes > 0 and num_spec_decodes > 0), (
    f"num_decodes: {num_decodes}, num_spec_decodes: {num_spec_decodes}"
)

The GDN attention backend does not support mixed batches containing both regular decode tokens and speculative decode tokens. However, the V1 scheduler produces exactly this kind of heterogeneous batch under certain conditions.

How to trigger

The crash occurs when the scheduler creates a batch where at least one request is in regular decode mode while other requests are in speculative decode mode. In my case, this happens when a request approaches max_seq_len (262144) one request had num_computed_tokens=262142 and was scheduled for 1 regular decode token, while 4 other requests were scheduled with scheduled_spec_decode_tokens: [-1].

From the scheduler output dump:

num_scheduled_tokens={
    chatcmpl-930d910693e2b9b3: 1,    # <-- regular decode (1 token)
    chatcmpl-988529689500aeab: 2,     # <-- spec decode (1 + 1 MTP)
    chatcmpl-a4414e85c2ae3487: 2,     # <-- spec decode
    chatcmpl-b88f8e337b3083b1: 2,     # <-- spec decode
    chatcmpl-97d27dc64ac3ad97: 2      # <-- spec decode
}
scheduled_spec_decode_tokens={
    chatcmpl-97d27dc64ac3ad97: [-1],
    chatcmpl-a4414e85c2ae3487: [-1],
    chatcmpl-988529689500aeab: [-1],
    chatcmpl-b88f8e337b3083b1: [-1]
}
# Note: chatcmpl-930d910693e2b9b3 is NOT in scheduled_spec_decode_tokens

Request chatcmpl-930d910693e2b9b3 has num_computed_tokens=262142, only 2 tokens away from max_seq_len=262144. With MTP num_spec_tokens=1, a speculative step would need to allocate 2 tokens (1 regular + 1 speculative), landing at exactly 262144. The verification of the speculative token could require going to 262145, which exceeds the limit. So the scheduler correctly disables spec decode for this individual request because there isn't enough headroom for producing these tokens.

TLDR : The scheduler doesn't account for the GDN attention backend's constraint that all requests in a batch must use the same decode mode. It produces a valid per-request schedule but an invalid batch-level state.

Expected behavior

The scheduler should not produce batches that violate the attention backend's invariants. Either:

1. The scheduler should ensure all requests in a batch use the same decode mode when the GDN backend is active (split into separate batches or disable spec decode for all requests in the batch when any request can't use it), or
2. The GDN attention backend should handle mixed decode/spec_decode batches gracefully.

How to reproduce

Run the two files from the following gist <script src="https://gist.github.com/lulmer/8e8cb558c430bbe70f749aeea784d02f.js"></script>

run ./trigger_bug_init.sh then ./trigger_bug.sh

Suggested unit test approach

A faster way to reproduce would be a unit test that directly constructs a SchedulerOutput with mixed decode/spec_decode requests and passes it to the GDN attention metadata builder, or a test that sets max_seq_len to a very small value (e.g., 128) so the boundary is reached quickly (see gist).

Relevant config from the engine dump

model='Qwen/Qwen3.5-397B-A17B'
speculative_config=SpeculativeConfig(method='mtp', model='...', num_spec_tokens=1)
dtype=torch.bfloat16
max_seq_len=262144
tensor_parallel_size=8
enable_chunked_prefill=True
enable_prefix_caching=False

Full stack trace

(Worker_TP4 pid=826) AssertionError: num_decodes: 1, num_spec_decodes: 4

(Worker_TP4 pid=826) File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3671, in execute_model
(Worker_TP4 pid=826)     self._build_attention_metadata(
(Worker_TP4 pid=826)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 2056, in _build_attention_metadata
(Worker_TP4 pid=826)     _build_attn_group_metadata(kv_cache_gid, attn_gid, cm)
(Worker_TP4 pid=826)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 2007, in _build_attn_group_metadata
(Worker_TP4 pid=826)     attn_metadata_i = builder.build(
(Worker_TP4 pid=826)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/gdn_attn.py", line 310, in build
(Worker_TP4 pid=826)     assert not (num_decodes > 0 and num_spec_decodes > 0), (

All 8 TP workers hit the same assertion simultaneously. The EngineCore then dies with EngineDeadError, killing all in-flight requests.

Current Workaround

Disable MTP speculative decoding entirely:

vllm serve Qwen/Qwen3.5-397B-A17B \
    --speculative-config '{"method": "mtp", "num_speculative_tokens": 0}' \
    ...

Related issues

• #24660 : different GDN + MTP bug (cudagraph padding IndexError), but same general area

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.