Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix Recent transformers versions break models using `remote_code` [1 pull requests, 4 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#45020•Fetched 2026-04-08 01:36:03
View on GitHub
Comments
4
Participants
2
Timeline
11
Reactions
0
Author
fxmarty-amdfxmarty-amd
Participants
ArthurZuckerArthurZuckerfxmarty-amdfxmarty-amd
Timeline (top)
commented ×4labeled ×2mentioned ×2subscribed ×2

Error Message

Traceback (most recent call last): File "/felmarty/scripts/test_kimi_k25.py", line 10, in <module> model = AutoModelForCausalLM.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/auto_factory.py", line 356, in from_pretrained model_class = get_class_from_dynamic_module( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 583, in get_class_from_dynamic_module return get_class_in_module(class_name, final_module, force_reload=force_download) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 309, in get_class_in_module module_spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 999, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 67, in <module> from .modeling_deepseek import DeepseekV3ForCausalLM File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_deepseek.py", line 47, in <module> from transformers.utils.import_utils import is_torch_fx_available ImportError: cannot import name 'is_torch_fx_available' from 'transformers.utils.import_utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py). Did you mean: 'is_torch_available'?

PR fix notes

PR #3562: upgrade transformers to 5.4.0

Description (problem / solution / changelog)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

Summary by CodeRabbit

  • Chores
    • Updated core library dependency to a newer version for improved stability and performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Changed files

  • requirements.txt (modified, +1/-1)
  • tests/e2e/multigpu/patched/test_sp.py (modified, +3/-0)
  • tests/prompt_strategies/test_dpo_chat_templates.py (modified, +2/-0)

Code Example

- `transformers` version: 5.3.0
- Platform: Linux-5.15.0-70-generic-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.8.0
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+git8907517 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type:

---

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "moonshotai/Kimi-K2.5"

print(f"Loading tokenizer from {model_id}...")
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

print(f"Loading model from {model_id}...")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

---

Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/auto_factory.py", line 356, in from_pretrained
    model_class = get_class_from_dynamic_module(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 583, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 309, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 67, in <module>
    from .modeling_deepseek import DeepseekV3ForCausalLM
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_deepseek.py", line 47, in <module>
    from transformers.utils.import_utils import is_torch_fx_available
ImportError: cannot import name 'is_torch_fx_available' from 'transformers.utils.import_utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py). Did you mean: 'is_torch_available'?

---

A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- tool_declaration_ts.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Loading model from moonshotai/Kimi-K2.5...
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- configuration_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- modeling_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
You are attempting to use Flash Attention 2 without specifying a torch dtype. This might lead to unexpected behaviour
Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/models/auto/auto_factory.py", line 597, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/modeling_utils.py", line 277, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/modeling_utils.py", line 4971, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 836, in __init__
    self.vision_tower = MoonViT3dPretrainedModel(vt_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 657, in __init__
    self.encoder = MoonViT3dEncoder(hidden_dim=config.hidden_size,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 575, in __init__
    use_deterministic_attn=self.use_deterministic_attn)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1964, in __getattr__
    raise AttributeError(
AttributeError: 'MoonViT3dEncoder' object has no attribute 'use_deterministic_attn'

---

Loading tokenizer from moonshotai/Kimi-K2.5...
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- tool_declaration_ts.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 7, in <module>
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/models/auto/tokenization_auto.py", line 1107, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/dynamic_module_utils.py", line 581, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/dynamic_module_utils.py", line 276, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi-K2.5/54383e83fa343a1331754112fb9e3410c55efa2f/tokenization_kimi.py", line 15, in <module>
    from .tool_declaration_ts import encode_tools_to_typescript_style
ModuleNotFoundError: No module named 'transformers_modules.moonshotai.Kimi-K2'
RAW_BUFFERClick to expand / collapse

System Info

- `transformers` version: 5.3.0
- Platform: Linux-5.15.0-70-generic-x86_64-with-glibc2.35
- Python version: 3.12.12
- Huggingface_hub version: 1.8.0
- Safetensors version: 0.7.0
- Accelerate version: 1.13.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1+git8907517 (CUDA)
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type:

Who can help?

@ArthurZucker @Cyrilvallez moonshotai folks

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Hi, https://huggingface.co/moonshotai/Kimi-K2.5 seems to be not loadable with Transformers.

Using transformers==5.3:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "moonshotai/Kimi-K2.5"

print(f"Loading tokenizer from {model_id}...")
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

print(f"Loading model from {model_id}...")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

results in

Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/auto_factory.py", line 356, in from_pretrained
    model_class = get_class_from_dynamic_module(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 583, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 309, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 67, in <module>
    from .modeling_deepseek import DeepseekV3ForCausalLM
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_deepseek.py", line 47, in <module>
    from transformers.utils.import_utils import is_torch_fx_available
ImportError: cannot import name 'is_torch_fx_available' from 'transformers.utils.import_utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py). Did you mean: 'is_torch_available'?

Using v4.57:

A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- tool_declaration_ts.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Loading model from moonshotai/Kimi-K2.5...
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- configuration_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- modeling_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
You are attempting to use Flash Attention 2 without specifying a torch dtype. This might lead to unexpected behaviour
Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 10, in <module>
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/models/auto/auto_factory.py", line 597, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/modeling_utils.py", line 277, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/modeling_utils.py", line 4971, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 836, in __init__
    self.vision_tower = MoonViT3dPretrainedModel(vt_config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 657, in __init__
    self.encoder = MoonViT3dEncoder(hidden_dim=config.hidden_size,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi_hyphen_K2_dot_5/54383e83fa343a1331754112fb9e3410c55efa2f/modeling_kimi_k25.py", line 575, in __init__
    use_deterministic_attn=self.use_deterministic_attn)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1964, in __getattr__
    raise AttributeError(
AttributeError: 'MoonViT3dEncoder' object has no attribute 'use_deterministic_attn'

Using v4.56:

Loading tokenizer from moonshotai/Kimi-K2.5...
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- tool_declaration_ts.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
Traceback (most recent call last):
  File "/felmarty/scripts/test_kimi_k25.py", line 7, in <module>
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/models/auto/tokenization_auto.py", line 1107, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/dynamic_module_utils.py", line 581, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/felmarty/repos/transformers/src/transformers/dynamic_module_utils.py", line 276, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/root/.cache/huggingface/modules/transformers_modules/moonshotai/Kimi-K2.5/54383e83fa343a1331754112fb9e3410c55efa2f/tokenization_kimi.py", line 15, in <module>
    from .tool_declaration_ts import encode_tools_to_typescript_style
ModuleNotFoundError: No module named 'transformers_modules.moonshotai.Kimi-K2'

Are there plans to support Kimi models natively in Transformers?

New changes in Transformers releases are likely breaking its support with trust_remote_code=True.

Related: https://github.com/huggingface/transformers/issues/43726

Thank you!

Expected behavior

No error

extent analysis

Fix Plan

The issue arises from compatibility problems between the transformers library and the Kimi-K2.5 model. To fix this, we'll try the following steps:

  • Downgrade the transformers library: The error occurs with transformers version 5.3.0. Let's try downgrading to an earlier version that might be compatible with the Kimi-K2.5 model.
  • Pin the model revision: To avoid downloading new versions of the model files, we can pin the revision of the Kimi-K2.5 model.

Here's an example code snippet that demonstrates these steps:

# Downgrade the transformers library to version 4.56
!pip install transformers==4.56

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Pin the model revision
model_id = "moonshotai/Kimi-K2.5"
revision = "main"  # Replace with the desired revision

print(f"Loading tokenizer from {model_id}...")
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True, revision=revision)

print(f"Loading model from {model_id}...")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    revision=revision,
    trust_remote_code=True,
)
model.eval()

Verification

To verify that the fix worked, run the code snippet above and check if the model loads successfully without any errors.

Extra Tips

  • Make sure to check the compatibility of the transformers library with the Kimi-K2.5 model before upgrading or downgrading the library.
  • Pinning the model revision can help avoid unexpected changes to the model files.
  • If you're still encountering issues, try checking the Hugging Face Transformers GitHub page for any open issues or discussions related to the Kimi-K2.5 model.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

No error

#cache error#pipeline error#runtime error#dependency conflict#environment setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix Recent transformers versions break models using `remote_code` [1 pull requests, 4 comments, 2 participants]