Summary

Full CUDA graph metadata can include stale CPU request-state values for padded rows. For Mamba backends this can make padding rows look like prefills, so a decode-only CUDA graph replay can be classified as mixed decode/prefill metadata.

The relevant path is:

• GPUModelRunner._build_attention_metadata slices num_computed_tokens_cpu_tensor[:num_reqs_padded] and num_prompt_tokens_cpu_tensor[:num_reqs_padded].
• Other padded metadata is explicitly neutralized, e.g. padded block table entries are filled with NULL_BLOCK_ID and padded sequence lengths are zeroed.
• The request-state tensors are not neutralized beyond num_reqs.
• is_prefilling = num_computed_tokens_cpu < num_prompt_tokens_cpu can therefore be true for inactive padded rows if those rows contain stale values from prior requests.
• Mamba calls split_decodes_and_prefills(..., treat_short_extends_as_decodes=False), which ORs is_prefilling into the split. A zero-token padded row can become a fake prefill.

Minimal Repro

This repro is synthetic and does not require a model or dataset. It demonstrates the metadata classification issue directly.

import torch

from vllm.v1.attention.backend import CommonAttentionMetadata
from vllm.v1.attention.backends.utils import split_decodes_and_prefills

num_reqs = 2
num_reqs_padded = 4

# Two real decode rows, then two padded rows with zero query length.
query_start_loc = torch.tensor([0, 1, 2, 2, 2], dtype=torch.int32)
seq_lens = torch.tensor([8, 9, 0, 0], dtype=torch.int32)

# Real rows are decode-only, but padded rows contain stale state from older requests.
num_computed_tokens = torch.tensor([7, 8, 1, 2], dtype=torch.int32)
num_prompt_tokens = torch.tensor([4, 4, 3, 4], dtype=torch.int32)

is_prefilling = num_computed_tokens[:num_reqs_padded] < num_prompt_tokens[:num_reqs_padded]
metadata = CommonAttentionMetadata(
    query_start_loc=query_start_loc,
    query_start_loc_cpu=query_start_loc,
    seq_lens=seq_lens,
    num_reqs=num_reqs_padded,
    num_actual_tokens=2,
    max_query_len=1,
    max_seq_len=9,
    block_table_tensor=torch.zeros((num_reqs_padded, 1), dtype=torch.int32),
    slot_mapping=torch.arange(2, dtype=torch.int64),
    is_prefilling=is_prefilling,
)

print(split_decodes_and_prefills(metadata, treat_short_extends_as_decodes=False))

Current output:

(2, 2, 2, 0)

Expected output for a padded decode-only full CUDA graph batch:

(4, 0, 2, 0)

If the padded CPU request-state rows are cleared before computing is_prefilling, the split returns the expected decode-only result:

fixed_num_computed = num_computed_tokens[:num_reqs_padded].clone()
fixed_num_prompt = num_prompt_tokens[:num_reqs_padded].clone()
fixed_num_computed[num_reqs:num_reqs_padded] = 0
fixed_num_prompt[num_reqs:num_reqs_padded] = 0
fixed_metadata = metadata.replace(
    is_prefilling=fixed_num_computed < fixed_num_prompt,
)
print(split_decodes_and_prefills(fixed_metadata, treat_short_extends_as_decodes=False))
# (4, 0, 2, 0)

Why this matters

This only affects padded CUDA graph metadata rows. It is most likely to surface in workloads that combine:

• Mamba or hybrid Mamba models
• full CUDA graph replay with padded request rows
• prefix caching / Mamba align mode or any path sensitive to Mamba decode-vs-prefill metadata
• long decode batches where request rows churn and stale inactive-row state is more likely to matter

Possible Fix

When building common attention metadata with num_reqs_padded > num_reqs, clone and zero num_computed_tokens_cpu and num_prompt_tokens_cpu for padded rows before computing is_prefilling.