vllm - ✅(Solved) Fix [Bug]: `resolve_chat_template_kwargs` silently drops chat template kwargs when `chat_template` is passed in as chat template name instead of Jinja [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#36907Fetched 2026-04-08 00:43:41
View on GitHub
Comments
0
Participants
1
Timeline
8
Reactions
0
Author
Participants
Timeline (top)
cross-referenced ×2referenced ×2closed ×1labeled ×1

Root Cause

The issue spans two functions in vllm/renderers/hf.py:

1. resolve_chat_template (L110) — Priority 1 returns the chat_template string as-is without checking whether it's a Jinja template or a template name:

def resolve_chat_template(tokenizer, chat_template, tools, *, model_config):
    # 1st priority: The given chat template
    if chat_template is not None:
        return chat_template  # returns "tool_use" verbatim, not the Jinja content
    ...

HuggingFace tokenizers support named templates via tokenizer.chat_template being a dict (e.g. {"default": "<jinja...>", "tool_use": "<jinja...>"}). The tokenizer.get_chat_template(name) method resolves a name to the actual Jinja string. But resolve_chat_template short-circuits at Priority 1 and never calls get_chat_template when chat_template is not None.

2. resolve_chat_template_kwargs (L433) — receives the unresolved name string (e.g. "tool_use") and passes it to _resolve_chat_template_kwargs, which tries to parse it as Jinja:

def _resolve_chat_template_kwargs(chat_template: str) -> Set[str]:
    env = jinja2.sandbox.ImmutableSandboxedEnvironment(...)
    parsed_content = env.parse(chat_template)  # parses "tool_use" as literal text
    template_vars = jinja2.meta.find_undeclared_variables(parsed_content)
    return template_vars  # returns empty set — no Jinja variables in plain text

Since "tool_use" is valid (trivial) Jinja that contains no undeclared variables, template_vars is set(). This means kwargs that the actual template expects (like tools, documents, etc.) are not detected, and resolve_chat_template_kwargs silently drops them from the accepted kwargs.

The only kwargs that survive are those in fn_kw (explicit params of tokenizer.apply_chat_template) and hf_base_params. Template-specific variables that are only discoverable by parsing the actual Jinja content are lost.

Fix Action

Fixed

PR fix notes

PR #36937: fix: resolve chat template names before kwargs detection

Description (problem / solution / changelog)

Summary

Fixes #36907

When a user passes a template name (e.g. "tool_use") instead of actual Jinja content via the chat_template parameter, resolve_chat_template returned it verbatim. The downstream resolve_chat_template_kwargs then parsed "tool_use" as trivial Jinja, found no template variables, and silently dropped kwargs like tools and documents that the actual template expects.

Root Cause

resolve_chat_template (Priority 1) returned chat_template as-is without checking whether it's a template name or actual Jinja content. HF tokenizers support named templates via tokenizer.chat_template being a dict (e.g. {"default": "<jinja>", "tool_use": "<jinja>"}), but the name was never resolved.

Fix

In resolve_chat_template Priority 1, use tokenizer.get_chat_template(chat_template, tools=tools) to resolve template names to Jinja content. This method:

  • Resolves names from the tokenizer's template dict to actual Jinja
  • Returns literal Jinja strings as-is
  • Falls back gracefully on error

Testing

Added two unit tests:

  • test_resolve_chat_template_resolves_name — verifies template names are resolved via get_chat_template
  • test_resolve_chat_template_kwargs_with_template_name — verifies kwargs like tools and documents are not dropped when using resolved Jinja content

Changed files

  • tests/renderers/test_hf.py (modified, +56/-0)
  • vllm/renderers/hf.py (modified, +3/-1)

Code Example

def resolve_chat_template(tokenizer, chat_template, tools, *, model_config):
    # 1st priority: The given chat template
    if chat_template is not None:
        return chat_template  # returns "tool_use" verbatim, not the Jinja content
    ...

---

def _resolve_chat_template_kwargs(chat_template: str) -> Set[str]:
    env = jinja2.sandbox.ImmutableSandboxedEnvironment(...)
    parsed_content = env.parse(chat_template)  # parses "tool_use" as literal text
    template_vars = jinja2.meta.find_undeclared_variables(parsed_content)
    return template_vars  # returns empty set — no Jinja variables in plain text
RAW_BUFFERClick to expand / collapse

🐛 Describe the bug

When a user passes a chat template name (e.g. "tool_use", "default") instead of an actual Jinja template string via the chat_template parameter, the resolve_chat_templateresolve_chat_template_kwargs pipeline silently drops valid template kwargs like tools, documents, etc. This happens because the template name string is never resolved to the actual Jinja content before being parsed for variable detection.

Root Cause

The issue spans two functions in vllm/renderers/hf.py:

1. resolve_chat_template (L110) — Priority 1 returns the chat_template string as-is without checking whether it's a Jinja template or a template name:

def resolve_chat_template(tokenizer, chat_template, tools, *, model_config):
    # 1st priority: The given chat template
    if chat_template is not None:
        return chat_template  # returns "tool_use" verbatim, not the Jinja content
    ...

HuggingFace tokenizers support named templates via tokenizer.chat_template being a dict (e.g. {"default": "<jinja...>", "tool_use": "<jinja...>"}). The tokenizer.get_chat_template(name) method resolves a name to the actual Jinja string. But resolve_chat_template short-circuits at Priority 1 and never calls get_chat_template when chat_template is not None.

2. resolve_chat_template_kwargs (L433) — receives the unresolved name string (e.g. "tool_use") and passes it to _resolve_chat_template_kwargs, which tries to parse it as Jinja:

def _resolve_chat_template_kwargs(chat_template: str) -> Set[str]:
    env = jinja2.sandbox.ImmutableSandboxedEnvironment(...)
    parsed_content = env.parse(chat_template)  # parses "tool_use" as literal text
    template_vars = jinja2.meta.find_undeclared_variables(parsed_content)
    return template_vars  # returns empty set — no Jinja variables in plain text

Since "tool_use" is valid (trivial) Jinja that contains no undeclared variables, template_vars is set(). This means kwargs that the actual template expects (like tools, documents, etc.) are not detected, and resolve_chat_template_kwargs silently drops them from the accepted kwargs.

The only kwargs that survive are those in fn_kw (explicit params of tokenizer.apply_chat_template) and hf_base_params. Template-specific variables that are only discoverable by parsing the actual Jinja content are lost.

Expected behavior

When chat_template is a template name (not a Jinja string), resolve_chat_template_kwargs should still be able to correctly detect which kwargs the template expects.

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To fix this issue, we need to modify the resolve_chat_template function to resolve the template name to the actual Jinja content before returning it. We can use the tokenizer.get_chat_template(name) method to achieve this.

Here are the steps:

  • Check if chat_template is a string and not a Jinja template
  • If it's a string, use tokenizer.get_chat_template(name) to resolve it to the actual Jinja content
  • If chat_template is not a string or tokenizer.get_chat_template(name) returns None, return the original chat_template

Example code:

def resolve_chat_template(tokenizer, chat_template, tools, *, model_config):
    # 1st priority: The given chat template
    if chat_template is not None:
        if isinstance(chat_template, str):
            # Try to resolve the template name to the actual Jinja content
            resolved_template = tokenizer.get_chat_template(chat_template)
            if resolved_template is not None:
                return resolved_template
        return chat_template  # returns the actual Jinja content or the original chat_template
    ...

Verification

To verify that the fix worked, you can test the resolve_chat_template function with a template name and check if it returns the actual Jinja content. You can also test the resolve_chat_template_kwargs function to ensure that it correctly detects the expected kwargs.

Example test code:

tokenizer = ...  # initialize the tokenizer
chat_template_name = "tool_use"
resolved_template = resolve_chat_template(tokenizer, chat_template_name, ...)
print(resolved_template)  # should print the actual Jinja content

expected_kwargs = resolve_chat_template_kwargs(resolved_template)
print(expected_kwargs)  # should print the expected kwargs

Extra Tips

  • Make sure to handle the case where tokenizer.get_chat_template(name) returns None to avoid errors.
  • Consider adding a check to ensure that the resolved template is a valid Jinja template to prevent errors downstream.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

When chat_template is a template name (not a Jinja string), resolve_chat_template_kwargs should still be able to correctly detect which kwargs the template expects.

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING