Issue Station
Sign In
Tools hereToolbox

transformers - 💡(How to fix) Fix [Gemma-3] Inconsistent eos_token_id configuration: tokenizer has single value (1) but model.config has list [1, 106] [1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#44871•Fetched 2026-04-08 01:03:20
View on GitHub
Comments
1
Participants
2
Timeline
3
Reactions
0
Author
IvanFan-VanIvanFan-Van
Participants
ashwin3005ashwin3005IvanFan-VanIvanFan-Van
Timeline (top)
closed ×1commented ×1labeled ×1

Code Example

from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL = "google/gemma-3-1b-it"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL)
print(tokenizer.eos_token_id) # 1
print(model.config.eos_token_id) # [1, 106]

---

for _ in range(max_new_tokens):
    ...
    if new_token_id.item() == tokenizer.eos_token_id:
        break  # should break the loop
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 5.3.0
  • Platform: Windows-11-10.0.26100-SP0
  • Python version: 3.12.11
  • Huggingface_hub version: 1.7.1
  • Safetensors version: 0.7.0
  • Accelerate version: not installed
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.7.1+cu118 (CUDA)
  • Using distributed or parallel set-up in script?: <fill in>
  • Using GPU in script?: <fill in>
  • GPU type: NVIDIA GeForce RTX 4060 Laptop GPU

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL = "google/gemma-3-1b-it"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL)
print(tokenizer.eos_token_id) # 1
print(model.config.eos_token_id) # [1, 106]

Expected behavior

Expected Behavior

Tokenizer and Model Config should be consistent on eos_token_id so that the following generation loop can work correctly:

for _ in range(max_new_tokens):
    ...
    if new_token_id.item() == tokenizer.eos_token_id:
        break  # should break the loop

extent analysis

Fix Plan

To fix the inconsistency between the tokenizer's eos_token_id and the model's eos_token_id, we need to ensure they are aligned.

  • Update the model's eos_token_id to match the tokenizer's eos_token_id:

model.config.eos_token_id = tokenizer.eos_token_id

*   Alternatively, update the tokenizer's `eos_token_id` to match one of the model's `eos_token_id` values:
    ```python
tokenizer.eos_token_id = model.config.eos_token_id[0]  # or model.config.eos_token_id[1]

Verification

After applying the fix, verify that the eos_token_id values are consistent:

print(tokenizer.eos_token_id)  # Should match model.config.eos_token_id
print(model.config.eos_token_id)  # Should match tokenizer.eos_token_id

Extra Tips

When working with pre-trained models and tokenizers, it's essential to ensure their configurations are consistent to avoid issues like this. Always verify the consistency of key configuration values like eos_token_id before using them in your application.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#agent setup#task chaining#parallel task#integration issue#index setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - 💡(How to fix) Fix [Gemma-3] Inconsistent eos_token_id configuration: tokenizer has single value (1) but model.config has list [1, 106] [1 comments, 2 participants]