Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix Bug: TypeError when loading model with `init_empty_weights` in transformers >= 5.0.0rc0 due to unexpected `_is_hf_initialized` argument [1 pull requests, 5 comments, 6 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#44291•Fetched 2026-04-08 00:29:20
View on GitHub
Comments
5
Participants
6
Timeline
21
Reactions
0
Author
chenyushuochenyushuo
Participants
chenyushuochenyushuocynricfucynricfuCyrilvallezCyrilvallezeval-deveval-devRocketknight1Rocketknight1SunMarcSunMarc
Timeline (top)
subscribed ×7mentioned ×6commented ×5closed ×1

Error Message

Traceback (most recent call last): File "/mnt/data/chenyushuo.cys/Trinity/tmpfiles/test20260225.py", line 11, in <module> model = transformers.AutoModelForCausalLM.from_pretrained(model_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained return model_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4072, in from_pretrained loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4191, in _load_pretrained_model loading_info, disk_offload_index = convert_and_load_state_dict_in_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model set_param_for_module( File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 922, in set_param_for_module setattr(module_obj, param_name, param_value) File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1989, in setattr self.register_parameter(name, value) File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/accelerate/big_modeling.py", line 135, in register_empty_parameter module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Parameter.new() got an unexpected keyword argument '_is_hf_initialized'

Root Cause

model_path = 'Qwen/Qwen3-0.6B' with init_empty_weights(): model = transformers.AutoModelForCausalLM.from_pretrained(model_path)

2. I have also tried loading other models, but it always encounter the following error:

Traceback (most recent call last): File "/mnt/data/chenyushuo.cys/Trinity/tmpfiles/test20260225.py", line 11, in <module> model = transformers.AutoModelForCausalLM.from_pretrained(model_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained return model_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4072, in from_pretrained loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4191, in _load_pretrained_model loading_info, disk_offload_index = convert_and_load_state_dict_in_model( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model set_param_for_module( File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 922, in set_param_for_module setattr(module_obj, param_name, param_value) File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1989, in setattr self.register_parameter(name, value) File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/accelerate/big_modeling.py", line 135, in register_empty_parameter module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: Parameter.new() got an unexpected keyword argument '_is_hf_initialized'

3. **Investigation Findings:**
The error occurs because `transformers.core_model_loading.set_param_for_module` attaches a custom `_is_hf_initialized` attribute to the parameter object. When `accelerate.big_modeling.register_empty_parameter` later tries to reconstruct this parameter, it indiscriminately passes all existing attributes (including `_is_hf_initialized`) to `torch.nn.Parameter.__new__()`. As `torch.nn.Parameter` does not recognize this keyword argument, the process fails with a `TypeError`.

Fix Action

Fixed

PR fix notes

PR #3943: fix: pop unexpected argument before initializing empty parameter

Description (problem / solution / changelog)

What does this PR do?

<!-- Congratulations! You've made it this far! You're not quite done yet though. Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution. Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change. Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost. --> <!-- Remove if not applicable -->

A simple fix for this issue from the transformers repo.

This fix simply pops the _is_hf_initialized argument before initializing the empty parameter.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @ If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**. - Big modeling: @SunMarc - Fully-Sharded Data Parallism: @SunMarc - DeepSpeed: @SunMarc - Command Line Interface: @SunMarc - Documentation: @SunMarc - Core parts of the library: @BenjaminBossan @SunMarc - Maintained examples: @SunMarc -->

Changed files

  • src/accelerate/big_modeling.py (modified, +4/-0)

Code Example

import transformers
from accelerate import init_empty_weights

model_path = 'Qwen/Qwen3-0.6B'
with init_empty_weights():
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)

---

Traceback (most recent call last):
  File "/mnt/data/chenyushuo.cys/Trinity/tmpfiles/test20260225.py", line 11, in <module>
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4072, in from_pretrained
    loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4191, in _load_pretrained_model
    loading_info, disk_offload_index = convert_and_load_state_dict_in_model(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model
    set_param_for_module(
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 922, in set_param_for_module
    setattr(module_obj, param_name, param_value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1989, in __setattr__
    self.register_parameter(name, value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/accelerate/big_modeling.py", line 135, in register_empty_parameter
    module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Parameter.__new__() got an unexpected keyword argument '_is_hf_initialized'
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 5.2.0
  • Platform: Linux-5.10.134-013.5.kangaroo.al8.x86_64-x86_64-with-glibc2.39
  • Python version: 3.12.12
  • Huggingface_hub version: 1.4.1
  • Safetensors version: 0.7.0
  • Accelerate version: 1.12.0
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.9.1+cu128 (CUDA)
  • Using distributed or parallel set-up in script?: No
  • Using GPU in script?: No
  • GPU type: NVIDIA H20

Who can help?

@Cyrilvallez

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

  1. Run the following scripts:
import transformers
from accelerate import init_empty_weights

model_path = 'Qwen/Qwen3-0.6B'
with init_empty_weights():
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)
  1. I have also tried loading other models, but it always encounter the following error:
Traceback (most recent call last):
  File "/mnt/data/chenyushuo.cys/Trinity/tmpfiles/test20260225.py", line 11, in <module>
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4072, in from_pretrained
    loading_info, disk_offload_index = cls._load_pretrained_model(model, state_dict, checkpoint_files, load_config)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4191, in _load_pretrained_model
    loading_info, disk_offload_index = convert_and_load_state_dict_in_model(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 1225, in convert_and_load_state_dict_in_model
    set_param_for_module(
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/transformers/core_model_loading.py", line 922, in set_param_for_module
    setattr(module_obj, param_name, param_value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1989, in __setattr__
    self.register_parameter(name, value)
  File "/mnt/data/chenyushuo.cys/Trinity/.venv/lib/python3.12/site-packages/accelerate/big_modeling.py", line 135, in register_empty_parameter
    module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Parameter.__new__() got an unexpected keyword argument '_is_hf_initialized'
  1. Investigation Findings: The error occurs because transformers.core_model_loading.set_param_for_module attaches a custom _is_hf_initialized attribute to the parameter object. When accelerate.big_modeling.register_empty_parameter later tries to reconstruct this parameter, it indiscriminately passes all existing attributes (including _is_hf_initialized) to torch.nn.Parameter.__new__(). As torch.nn.Parameter does not recognize this keyword argument, the process fails with a TypeError.

Expected behavior

I also run the above code on transformers<=4.57.6, and the model loaded normally.

extent analysis

Fix Plan

Update transformers and accelerate to compatible versions

The issue arises from a compatibility problem between transformers 5.2.0 and accelerate 1.12.0. We need to update transformers to a version that is compatible with accelerate.

Solution Steps

  1. Update transformers to version 4.28.0 (compatible with accelerate 1.12.0)
pip install transformers==4.28.0
  1. Update accelerate to version 1.11.0 (compatible with transformers 4.28.0)
pip install accelerate==1.11.0
  1. Verify the fix by running the original code with the updated packages.

Example Code

import transformers
from accelerate import init_empty_weights

model_path = 'Qwen/Qwen3-0.6B'
with init_empty_weights():
    model = transformers.AutoModelForCausalLM.from_pretrained(model_path)

This should load the model without any errors.

Verification

To verify that the fix worked, you can run the original code and check if the model is loaded successfully. You can also try loading other models to ensure that the issue is resolved.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

I also run the above code on transformers<=4.57.6, and the model loaded normally.

#api#ssr#installation#tensor shape#autograd error#API middleware#SSR setup#ISR setup#authentication setup#request error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING