Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix resize_token_embeddings does not effect to output_embeddings [1 pull requests, 1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#45292•Fetched 2026-04-08 03:00:46
View on GitHub
Comments
1
Participants
1
Timeline
6
Reactions
0
Author
KoichiYasuokaKoichiYasuoka
Participants
KoichiYasuokaKoichiYasuoka
Timeline (top)
mentioned ×2subscribed ×2commented ×1labeled ×1

PR fix notes

PR #45311: resize_token_embeddings does not effect to output_embeddings

Description (problem / solution / changelog)

What does this PR do?

Fixes #45292 (seems to come from #41580)

Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by code agents. We are currently bottlenecked by our ability to review and respond to them. As a result, we ask that new users do not submit pure code agent PRs at this time. You may use code agents in drafting or to help you diagnose issues. We'd also ask autonomous "OpenClaw"-like agents not to open any PRs or issues for the moment.

PRs that appear to be fully agent-written will probably be closed without review, and we may block users who do this repeatedly or maliciously.

This is a rapidly-evolving situation that's causing significant shockwaves in the open-source community. As a result, this policy is likely to be updated regularly in the near future. For more information, please read CONTRIBUTING.md.

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

  • text models: @ArthurZucker @Cyrilvallez

Changed files

  • src/transformers/modeling_utils.py (modified, +2/-0)

Code Example

from transformers import AutoModelForMaskedLM
mdl = AutoModelForMaskedLM.from_pretrained("google-bert/bert-base-uncased")
e = mdl.get_input_embeddings()
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 5.5.0
  • Platform: Linux-6.6.113+-x86_64-with-glibc2.35
  • Python version: 3.12.13
  • Huggingface_hub version: 1.8.0
  • Safetensors version: 0.7.0
  • Accelerate version: 1.13.0
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.10.0+cpu (NA)
  • Using distributed or parallel set-up in script?: <fill in>

(Google Colaboratory)

Who can help?

@ArthurZucker @Cyrilvallez

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Quick reproduce:

from transformers import AutoModelForMaskedLM
mdl = AutoModelForMaskedLM.from_pretrained("google-bert/bert-base-uncased")
e = mdl.get_input_embeddings()
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features

Expected behavior

Both e.num_embeddings and g.out_features should be increased to 30523.

extent analysis

TL;DR

The issue can be resolved by ensuring that the output embeddings are resized accordingly after modifying the input embeddings.

Guidance

  • The provided reproduction code snippet resizes the input embeddings but does not update the output embeddings, which could be the cause of the discrepancy.
  • After resizing the input embeddings using mdl.resize_token_embeddings(e.num_embeddings + 1), also update the output embeddings using a similar method if available.
  • Verify that both input and output embeddings have the same number of embeddings after resizing by checking e.num_embeddings and g.out_features.
  • If a method to resize output embeddings is not directly available, consider checking the documentation for the specific model or library version being used.

Example

from transformers import AutoModelForMaskedLM
mdl = AutoModelForMaskedLM.from_pretrained("google-bert/bert-base-uncased")
e = mdl.get_input_embeddings()
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features
e = mdl.resize_token_embeddings(e.num_embeddings + 1)
# Assuming a method to resize output embeddings exists
# g = mdl.resize_output_embeddings(g.out_features + 1)  # This line is hypothetical
g = mdl.get_output_embeddings()
print(e.num_embeddings, g.out_features)
assert e.num_embeddings == g.out_features

Notes

The exact method to resize output embeddings is not specified in the provided issue and may vary depending on the library version or model architecture. It's essential to consult the documentation for the specific library and model being used.

Recommendation

Apply workaround: The workaround involves ensuring that both input and output embeddings are resized and updated correctly. This might require additional steps or methods not directly mentioned in the issue, necessitating further research into the library's documentation.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Both e.num_embeddings and g.out_features should be increased to 30523.

#device allocation#model download#tokenizer error#prompt formatting#chain error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix resize_token_embeddings does not effect to output_embeddings [1 pull requests, 1 comments, 1 participants]