vllm - 💡(How to fix) Fix [Bug]: Structured outputs + pipeline parallelism on the V1 model runner: xgrammar FSM desync ("Failed to advance FSM") [1 pull requests]

Your current environment

env is not needed

🐛 Describe the bug

Used model : qwen 3.5 27b

PP (pipeline_parallel_size > 1) and structured outputs (xgrammar), requests intermittently fail with a 500.

backend_xgrammar.py: Failed to advance FSM for request <id> for tokens 4754
scheduler.py: Unexpected: grammar rejected tokens [4754]. Terminating request.

The failure is most reliable right after server start; once the engine is warm it becomes intermittent.

This only happens when the engine uses the V1 model runner. Models that run on the V2 model runner are not affected.

Environment

• vLLM 0.22.1rc1.dev266+g469f3dcf1; code paths below are unchanged on current main (c66b19800).
• Model: cyankiwi/Qwen3.5-27B-AWQ-4bit (hybrid / Mamba-style, Qwen3.5).
• 3x A100-40GB.
• Flags:

  --structured-outputs '{"backend":"xgrammar","disable_any_whitespace":true}'
  --reasoning-parser qwen3 --attention-backend FLASHINFER

-Async scheduling enabled (default, and forced-on by PP).

The model is hybrid with mamba_cache_mode="align" (set automatically when prefix caching is on), which is listed as unsupported by the V2 model runner (config/vllm.py: _get_v2_model_runner_unsupported_features), so the engine falls back to the V1 model runner. That fallback is what exposes the bug.

how to reproduce the bug

Send multiple concurrent chat-completion requests with json schema, right after the server becomes ready A single request is enough on a cold engine.

100% failure on the first burst, then intermittent.

• PP=3, V1 runner, structured: fails (FSM reject, 500). Reliable on first burst.
• TP=2, V1 runner, structured: no failures (105 requests).
• TP=3: cannot start for this model (16 attention heads not divisible by 3), which is why PP is used to span 3 GPUs in the first place.
• Plain (non-structured) generation under PP=3: no failures.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.