PR #40299: Register parsed config classes before tokenizer init

• Repository: vllm-project/vllm
• Author: Bortlesboat
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/40299

Description (problem / solution / changelog)

Summary

Register the already-parsed HF config class with AutoConfig before cached_tokenizer_from_config() reads the tokenizer config again.

This fixes #40249, where worker-side tokenizer init could still fail on qwen3_5_moe even though vLLM had already parsed the model config successfully.

Why this is not a duplicate

Checked before posting:

• gh issue view 40249 --repo vllm-project/vllm --comments
• gh pr list --repo vllm-project/vllm --state open --search "40249 in:body"
• gh pr list --repo vllm-project/vllm --state open --search "qwen3_5_moe tokenizer trust_remote_code worker init"

Those checks did not find an open PR for this lane. This is also different from #39554: that change keeps HFConfigParser.parse() on the right config class, while this patch fixes the later tokenizer init path inside workers.

Tests

• uvx ruff check vllm/transformers_utils/config.py vllm/tokenizers/registry.py tests/tokenizers_/test_registry.py -> passed
• uv run --isolated python -m py_compile vllm/transformers_utils/config.py vllm/tokenizers/registry.py tests/tokenizers_/test_registry.py -> passed
• uv run --isolated --with pytest python - with direct calls to test_customized_tokenizer() and test_cached_tokenizer_from_config_registers_local_config(...) -> PASS

AI assistance

I used AI assistance for issue triage and an initial patch/test draft, then reviewed the final diff and verified it locally.

Changed files

• tests/tokenizers_/test_registry.py (modified, +64/-0)
• vllm/tokenizers/registry.py (modified, +3/-1)
• vllm/transformers_utils/config.py (modified, +17/-2)