transformers - 💡(How to fix) Fix Gemma4: chat_template missing from tokenizer_config.json, requires manual loading from separate file [2 comments, 2 participants]

ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed!

TL;DR

To fix the issue with Gemma4 models, manually load the chat_template.jinja file and set it on the tokenizer before applying the chat template.

Guidance

• Load the chat_template.jinja file from the Hugging Face hub using hf_hub_download and set it as the chat_template attribute of the tokenizer.
• Verify that tokenizer.chat_template is set correctly before calling apply_chat_template.
• Consider updating the tokenizer_config.json to include the chat_template for future models, if possible.
• Test the workaround with different Gemma4 variants to ensure it works consistently.

Example

import os
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-E2B-it")
chat_template_path = hf_hub_download("google/gemma-4-E2B-it", "chat_template.jinja")
with open(chat_template_path) as f:
    tokenizer.chat_template = f.read()
tokenizer.apply_chat_template([{"role": "user", "content": "Hello"}], tokenize=False, add_generation_prompt=True)

Notes

This workaround assumes that the chat_template.jinja file is available on the Hugging Face hub for the specific model being used. If the file is not available, an alternative solution will be needed.

Recommendation

Apply the workaround by manually loading the chat_template.jinja file and setting it on the tokenizer, as this is the most straightforward solution given the current implementation of the Gemma4 models.