Your current environment

• vLLM version: 0.18.1rc1.dev43+gdebd6e768 (also reproducible on latest nightly 0.18.1rc1.dev236)
• GPU: NVIDIA GH200 480GB
• Model: Qwen3.5-9b (Qwen3.5 architecture, model_type: qwen3_5_text, hybrid GDN + full attention)

Model description

Qwen3.5-based model with hybrid architecture: 24 GatedDeltaNet (linear attention) layers + 8 full attention layers. FP8 quantized via compressed-tensors. Config includes layer_types: [linear_attention, linear_attention, linear_attention, full_attention, ...] repeating pattern.

How to reproduce the error

Without ngram (correct output):

# Start vLLM with no speculative decoding
vllm serve model-fp8 \
  --trust-remote-code \
  --enforce-eager \
  --enable-chunked-prefill \
  --max-model-len 131072 \
  --max-num-batched-tokens 131072 \
  --additional-config '{"gdn_prefill_backend": "triton"}'

# Test
curl -s http://localhost:8080/v1/completions -H "Content-Type: application/json" -d '{
  "model": "model-fp8",
  "prompt": "<code>\nclass Calculator:\n    def add(self, a, b):\n        return a + b\n</code>\n<update>\nAdd subtract and multiply methods\n</update>",
  "max_tokens": 300, "temperature": 0
}'

Output (correct):

class Calculator:
    def add(self, a, b):
        return a + b

    def subtract(self, a, b):
        return a - b

    def multiply(self, a, b):
        return a * b

With ngram (corrupted output):

vllm serve model-fp8 \
  --trust-remote-code \
  --enforce-eager \
  --enable-chunked-prefill \
  --max-model-len 131072 \
  --max-num-batched-tokens 131072 \
  --additional-config '{"gdn_prefill_backend": "triton"}' \
  --speculative-config '{"method": "ngram", "num_speculative_tokens": 64, "prompt_lookup_max": 10, "prompt_lookup_min": 2}'

Same prompt produces degenerate output with repeated fragments:

class Calculator:
    def add(self, a, b):
        return a + b

    def add(self, a, b):
        return a + b

    add(self, a, b):
        return a + b

, a, b):
        return a + b
...

Relevant logs or traceback

No crash or error — the model runs but produces corrupted output. The corruption pattern shows repeated/truncated fragments that degrade progressively, suggesting SSM state corruption rather than a sampling issue.

Root cause analysis

We traced this to how mamba_utils.postprocess_mamba handles token rejection during ngram speculative decoding on hybrid GDN models.

The problem (vllm/v1/worker/mamba_utils.py, lines ~242-243):

When ngram proposes N speculative tokens and some are rejected:

1. The forward pass runs on all N proposed tokens, advancing GDN SSM state by N steps
2. The rejection sampler correctly identifies which tokens were accepted (e.g., 2 of 4)
3. postprocess_mamba computes num_tokens_running_state and new_num_computed_tokens to determine which SSM state block to copy:

num_tokens_running_state = (
    num_computed_tokens + num_scheduled_tokens - num_draft_tokens
)
new_num_computed_tokens = num_tokens_running_state + num_accepted_tokens - 1

1. But the SSM state was already evolved for all N tokens during the forward pass. There is no mechanism to rollback/revert the GDN state to the position after only the accepted tokens. The state copy uses accept_token_bias to select which intermediate state to preserve, but this relies on the block-aligned state checkpoints being correct — which they aren't, because the SSM kernel (fused_recurrent) wrote states for all N tokens into contiguous block slots.

Why MTP doesn't have this issue: MTP draft tokens are generated after the base model step completes, so the SSM state evolution and draft proposal are decoupled. With ngram, drafts are pre-computed from prompt history but the forward pass still speculatively evolves SSM state for all proposed tokens.

Key files involved:

• vllm/v1/worker/mamba_utils.py — postprocess_mamba() state copy logic
• vllm/v1/attention/backends/gdn_attn.py — GDN metadata passes num_accepted_tokens but no state rollback
• vllm/v1/worker/gpu_model_runner.py — _update_states_after_model_execute()
• vllm/model_executor/layers/mamba/abstract.py — num_speculative_blocks allocation

Expected behavior

Ngram speculative decoding should produce identical output to non-speculative decoding (with temperature=0, output should be deterministic and match).

Before submitting a new issue...

• I have searched existing issues and confirmed this is not a duplicate
• I have verified the issue persists on the latest nightly build
• I have included reproduction steps and root cause analysis