PR #40063: fix: allow max_num_batched_tokens > max_model_len with chunked prefill

• Repository: vllm-project/vllm
• Author: ianliuy
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/40063

Description (problem / solution / changelog)

What's broken?

When chunked prefill is enabled, the server may fail to start or emit misleading warnings if max_num_batched_tokens exceeds max_model_len. Users who explicitly set a large max_num_batched_tokens (which is valid for chunked prefill since tokens are processed in chunks across multiple requests) encounter unnecessary validation friction.

Who is affected?

Users who enable chunked prefill and either:

• Explicitly set max_num_batched_tokens > max_num_seqs * max_model_len
• Rely on default max_num_batched_tokens with small max_num_seqs or max_model_len values

This does not affect users with chunked prefill disabled — all existing validation for that case is preserved.

Why does it happen?

Two validation/clamping paths did not account for chunked prefill:

1. SchedulerConfig.verify_max_model_len: The warning for max_num_batched_tokens > max_num_seqs * max_model_len fired unconditionally, even though with chunked prefill this is a valid configuration (tokens come from partial prefills across multiple requests).

2. EngineArgs._set_default_max_num_seqs_and_batched_tokens_args: The default max_num_batched_tokens was clamped to min(max_num_seqs * max_model_len, default) even when chunked prefill was enabled. This unnecessarily restricted the default batch size.

How did we fix it?

1. vllm/config/scheduler.py: Added and not self.enable_chunked_prefill guard to the warning at line 281, consistent with the existing guard on the error check at line 261.

2. vllm/engine/arg_utils.py: Moved the default max_num_batched_tokens clamping (min(max_num_seqs * max_model_len, ...)) inside the existing if not self.enable_chunked_prefill block, so it only applies when chunked prefill is disabled.

Both changes are minimal and surgical — no unrelated code is modified.

How do we know it works?

• All existing validation for non-chunked-prefill cases is preserved (the guards only skip checks when enable_chunked_prefill=True)
• ruff check and ruff format --check pass on both changed files
• The change is consistent with the existing pattern at line 261-264 of scheduler.py which already has the same and not self.enable_chunked_prefill guard

Fixes #39976

Changed files

• vllm/config/scheduler.py (modified, +4/-1)
• vllm/engine/arg_utils.py (modified, +9/-7)