Summary

MTP speculative decoding × TurboQuant KV cache × CUDA graph capture produces degenerate output on Qwen3-Next hybrid models. The four upstream PRs that fixed the ngram path (#40738 + #36138 + #40783 + #39055, plus the prompt_lookup_min=8 config trick from #40875) do not close this for MTP — the MTP forward path goes through a different proposer (EagleProposer configured with method="mtp") that the ngram-scoped fixes don't cover.

Filed at @Sandermage's explicit suggestion: "we did not test MTP at all in the v7.13 cycle... your data shows that assumption is wrong."

Reproducer

Image: vllm/vllm-openai@sha256:9bba4628a3b943e0dd33caefb94b811569ba1e97bdf23bee19a265c31b947ccb (v0.19.2rc1.dev21+g893611813). Hardware: 1× RTX 3090 (Ampere SM 8.6). Model: Lorbus/Qwen3.6-27B-int4-AutoRound. Genesis v7.13 backports applied via Sandermage/genesis-vllm-patches@852b73f — log confirms Genesis Results: 25 applied, 16 skipped, 0 failed including P60 (#40738 Phase 1) + P60b (#40738 Phase 2 Triton kernel offset).

vllm serve Lorbus/Qwen3.6-27B-int4-AutoRound \
  --quantization auto_round --dtype float16 \
  --tensor-parallel-size 1 --max-model-len 125000 \
  --gpu-memory-utilization 0.92 \
  --kv-cache-dtype turboquant_3bit_nc \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
  # cudagraph + torch.compile both ON (default — no overrides)

Test request:

import json, urllib.request
body = {
  "model": "qwen3.6-27b-autoround",
  "messages": [{"role": "user", "content": "What is the weather in Paris in Celsius? Use the tool."}],
  "tools": [{"type": "function", "function": {
    "name": "get_weather",
    "parameters": {"type": "object", "properties": {
      "city": {"type": "string"}, "unit": {"type": "string"}}, "required": ["city","unit"]}}}],
  "tool_choice": "auto", "max_tokens": 256, "chat_template_kwargs": {"enable_thinking": False},
}
r = urllib.request.urlopen(urllib.request.Request(
  "http://localhost:8020/v1/chat/completions",
  data=json.dumps(body).encode(), headers={"content-type":"application/json"}))
print(json.loads(r.read())["choices"][0]["message"])

Expected: tool_calls=[{name=get_weather, args={"city":"Paris","unit":"Celsius"}}]

Actual:

tool_calls=[]
content="<tool_call>\n<function=...><parameter=...>...</parameter>...</parameter>...
        (cascading inline-text emission, never populates tool_calls[])"

Same shape on the long-context needle test — model retrieves first token of the secret then output ends:

expected: 'silver platypus 22'
got:      'silver '

Why this is separate from the v7.13 ngram fix

• The four backported PRs scope to the ngram path's GDN state recovery (gdn_attn.py + gdn_linear_attn.py). Genesis P60/P60b explicitly target the ngram-affected paths.
• MTP runs through vllm/v1/spec_decode/eagle.py with method="mtp" and Qwen3_5MTP model class — distinct machinery from ngram's path.
• Empirically: same config, same Genesis loaded, same image. Switching method from mtp to ngram + prompt_lookup_min=8 flips outcome from broken to clean. So the four backports are working; they just don't cover this code path.

Confirmed in #40831 probe ladder Test A vs Test C on the same rig.

Working configurations that avoid this bug

1. --compilation-config '{"cudagraph_mode":"NONE"}' — keeps torch.compile inductor on, disables CUDA graphs. Restores correctness at ~33 TPS (vs ~85 broken).
2. Switch to ngram + prompt_lookup_min=8 per #40875.
3. Drop TurboQuant entirely (use fp8_e5m2 KV) — caps context at ~32-40K on a 24 GB card but unaffected.

Suggested investigation entry-point

Whether Qwen3_5MTP.forward correctly reads SSM/conv state from block[num_accepted-1] after spec-decode acceptance — same shape of bug #40738 fixed for ngram, possibly fixable with the same spec_decode_src_indices pattern adapted to the MTP path.

References

• #40831 — broader bug class; closed for ngram path; this issue is the scoped MTP follow-up
• #40875 — ngram prompt_lookup_min=8 config-only fix
• #40738 — @tdoublep's GDN+ngram state recovery (scope-limited to ngram)
• Genesis v7.13 tree — backport tree applied for this test

cc @Sandermage @tdoublep — per the handoff in #40831.