Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [Feature Request] Support chat_template in tokenizer_config.json for DeepSeekV32 [5 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#37839•Fetched 2026-04-08 01:17:46
View on GitHub
Comments
5
Participants
2
Timeline
5
Reactions
0
Author
xlshaoscuxlshaoscu
Participants
chaunceyjiangchaunceyjiangxlshaoscuxlshaoscu
Timeline (top)
commented ×5

Currently, the DeepSeekV32 tokenizer's chat template is hardcoded in the apply_chat_template method of the DeepseekV32Tokenizer class. It would be beneficial to allow users to customize the chat template through the tokenizer_config.json file, similar to how other tokenizers work in vLLM.

Root Cause

Currently, the DeepSeekV32 tokenizer's chat template is hardcoded in the apply_chat_template method of the DeepseekV32Tokenizer class. It would be beneficial to allow users to customize the chat template through the tokenizer_config.json file, similar to how other tokenizers work in vLLM.

Code Example

{
  "chat_template": "{% for message in messages %}{% if message.role == 'user' %}{{ message.content }}{% elif message.role == 'assistant' %}{{ message.content }}{% endif %}{% endfor %}"
}
RAW_BUFFERClick to expand / collapse

Feature Request: Support chat_template in tokenizer_config.json for DeepSeekV32

Description

Currently, the DeepSeekV32 tokenizer's chat template is hardcoded in the apply_chat_template method of the DeepseekV32Tokenizer class. It would be beneficial to allow users to customize the chat template through the tokenizer_config.json file, similar to how other tokenizers work in vLLM.

Current Behavior

The chat template logic is hardcoded in vllm/tokenizers/deepseek_v32.py, specifically in the apply_chat_template method. Users cannot easily modify the chat template without modifying the source code.

Expected Behavior

Users should be able to define a custom chat template in the tokenizer_config.json file using the standard chat_template field, and the DeepSeekV32 tokenizer should respect this configuration.

Use Case

  • Users may want to customize the chat template for specific use cases
  • Different DeepSeekV32 models might require slightly different template formats
  • Consistency with other tokenizers that already support chat_template configuration

Proposed Solution

  1. Modify the DeepseekV32Tokenizer class to check for a chat_template field in the tokenizer config
  2. If present, use this template instead of the hardcoded logic
  3. Maintain backward compatibility by falling back to the current hardcoded template if no chat_template is provided

Implementation Notes

  • The current apply_chat_template method handles special cases like tool calls and thinking mode, which should be considered when implementing the chat_template support
  • The chat_template should support the same variables and formatting as other tokenizers in vLLM

Example Tokenizer Config

{
  "chat_template": "{% for message in messages %}{% if message.role == 'user' %}{{ message.content }}{% elif message.role == 'assistant' %}{{ message.content }}{% endif %}{% endfor %}"
}

Additional Context

This feature would make DeepSeekV32 tokenizer more flexible and consistent with other tokenizers in the vLLM ecosystem, allowing users to easily customize the chat template without code changes.

extent analysis

Fix Plan

To support a customizable chat template in the tokenizer_config.json file for the DeepSeekV32 tokenizer, follow these steps:

  • Modify the DeepseekV32Tokenizer class to load the chat_template from the tokenizer_config.json file.
  • Update the apply_chat_template method to use the loaded chat_template if it exists, otherwise fall back to the current hardcoded template.

Example Code

import json

class DeepseekV32Tokenizer:
    def __init__(self, config_file):
        with open(config_file, 'r') as f:
            self.config = json.load(f)
        self.chat_template = self.config.get('chat_template')

    def apply_chat_template(self, messages):
        if self.chat_template:
            # Render the chat template with the given messages
            # This can be done using a templating engine like Jinja2
            from jinja2 import Template
            template = Template(self.chat_template)
            return template.render(messages=messages)
        else:
            # Fall back to the current hardcoded template
            # ... (existing code)

Verification

To verify that the fix worked, create a tokenizer_config.json file with a custom chat_template and test the DeepseekV32Tokenizer with it. The tokenizer should apply the custom template correctly.

Extra Tips

  • Make sure to handle any potential errors when loading the tokenizer_config.json file or rendering the chat template.
  • Consider adding validation for the chat_template field to ensure it is a valid template string.
  • Use a templating engine like Jinja2 to render the chat template, as it provides a secure and flexible way to render templates with variables.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#training loop#device allocation#model download#tokenizer error#prompt formatting

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - 💡(How to fix) Fix [Feature Request] Support chat_template in tokenizer_config.json for DeepSeekV32 [5 comments, 2 participants]