Issue Station
Sign In
Tools hereToolbox

transformers - ✅(Solved) Fix Many models started showing UNEXPECTED Key with position id [2 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#44493•Fetched 2026-04-08 00:28:03
View on GitHub
Comments
2
Participants
3
Timeline
8
Reactions
1
Author
weathonweathon
Participants
KartikPawadeKartikPawadeRocketknight1Rocketknight1weathonweathon
Timeline (top)
commented ×2subscribed ×2closed ×1cross-referenced ×1

Fix Action

Fix / Workaround

Many models, including google/owlvit-base-patch32 and a few LLMs started showing unexpecting keys. For example, I got this

Prepare processor and model

model_id = "google/owlv2-large-patch14-ensemble"

model_id = "google/owlvit-base-patch32" device = "cuda" if torch.cuda.is_available() else "cpu" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)

OwlViTForObjectDetection LOAD REPORT from: google/owlvit-base-patch32 Key | Status | | --------------------------------------------+------------+--+- owlvit.text_model.embeddings.position_ids | UNEXPECTED | | owlvit.vision_model.embeddings.position_ids | UNEXPECTED | |

Prepare processor and model

model_id = "google/owlv2-large-patch14-ensemble"

model_id = "google/owlvit-base-patch32" device = "cuda" if torch.cuda.is_available() else "cpu" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)

PR fix notes

PR #44508: Fix unexpected position_ids keys when loading OwlViT models

Description (problem / solution / changelog)

What does this PR do?

Older OwlViT checkpoints stored position_ids as buffers in the text and vision embedding modules. These tensors are simple integer ranges (0 → max sequence length) and are now recomputed dynamically during initialization.

As a result, when loading models such as google/owlvit-base-patch32, the loader reports the following unexpected keys:

  • owlvit.text_model.embeddings.position_ids
  • owlvit.vision_model.embeddings.position_ids

These buffers are not required for correct model behavior and can be safely ignored.

This PR suppresses the warnings by adding the corresponding patterns to _keys_to_ignore_on_load_unexpected in OwlViTPreTrainedModel.

This change does not affect model performance or weights and only prevents unnecessary loading warnings when using older checkpoints.

Fixes #44493

Before submitting

  • This PR fixes a bug (unexpected keys warning when loading OwlViT checkpoints).
  • Did you read the contributor guideline? Yes.
  • Was this discussed/approved via a Github issue or the forum? Yes, discussed in #44493.
  • Did you make sure to update the documentation with your changes? Not required.
  • Did you write any new necessary tests? Not required (warning suppression only).

Who can review?

@Rocketknight1 @yonigozlan

Changed files

  • src/transformers/models/owlv2/modeling_owlv2.py (modified, +4/-0)
  • src/transformers/models/owlvit/modeling_owlvit.py (modified, +4/-0)

PR #45385: Ignore CLIP position_ids in unexpected key loading report

Description (problem / solution / changelog)

What does this PR do?

Loading openai/clip-vit-base-patch32 currently reports the following keys as unexpected:

  • text_model.embeddings.position_ids
  • vision_model.embeddings.position_ids

In the current CLIP implementation, these buffers are registered with persistent=False, so they are not part of the current serialization contract and are not expected to be restored from checkpoints.

Older CLIP checkpoints may still contain these keys, but they are harmless historical buffers. This PR adds them to _keys_to_ignore_on_load_unexpected in CLIPPreTrainedModel so they no longer appear as misleading unexpected keys during loading.

This follows the same approach as #44508 for OwlViT.

Related to #44493.

I reproduced the issue locally, verified that the two keys disappear from unexpected_keys after the change, and added a targeted regression test.

Code Agent Policy

  • I confirm that this is not a pure code agent PR.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Test plan

  • Reproduced the issue locally with:
    • CLIPModel.from_pretrained("openai/clip-vit-base-patch32", output_loading_info=True)
  • Added a slow test covering the loading report
  • Ran:
    • RUN_SLOW=1 python -m pytest tests/models/clip/test_modeling_clip.py -k position_ids -sv -rs

Who can review?

@CyrilVallez

Changed files

  • src/transformers/models/clip/modeling_clip.py (modified, +4/-0)
  • tests/models/clip/test_modeling_clip.py (modified, +11/-0)

Code Example

import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image


# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)

---

OwlViTForObjectDetection LOAD REPORT from: google/owlvit-base-patch32
Key                                         | Status     |  | 
--------------------------------------------+------------+--+-
owlvit.text_model.embeddings.position_ids   | UNEXPECTED |  | 
owlvit.vision_model.embeddings.position_ids | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.

---

import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image


# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 5.0.0
  • Platform: Linux-6.6.113+-x86_64-with-glibc2.35
  • Python version: 3.12.12
  • Huggingface_hub version: 1.5.0
  • Safetensors version: 0.7.0
  • Accelerate version: 1.12.0
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.10.0+cu128 (CUDA)
  • Using distributed or parallel set-up in script?: <fill in>
  • Using GPU in script?: <fill in>
  • GPU type: NVIDIA A100-SXM4-40GB

Who can help?

Many models, including google/owlvit-base-patch32 and a few LLMs started showing unexpecting keys. For example, I got this

import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image


# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)
OwlViTForObjectDetection LOAD REPORT from: google/owlvit-base-patch32
Key                                         | Status     |  | 
--------------------------------------------+------------+--+-
owlvit.text_model.embeddings.position_ids   | UNEXPECTED |  | 
owlvit.vision_model.embeddings.position_ids | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Loading some models that could trigger this, example code as below

import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image


# Prepare processor and model
# model_id = "google/owlv2-large-patch14-ensemble"
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)

Expected behavior

It should load correctly with no unexpected keys as they are the same unmodified model

extent analysis

Fix Plan

Update Transformers Version

The issue is likely due to a bug in the transformers library version 5.0.0. We can try updating to the latest version.

Steps:

  1. Install the latest version of transformers using pip:
pip install --upgrade transformers
  1. Update the transformers version in your code to the latest version:
import transformers
print(transformers.__version__)

This should print the latest version number.

Verify Model Loading

After updating the transformers version, try loading the model again:

import torch
from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from transformers.image_utils import load_image

# Prepare processor and model
model_id = "google/owlvit-base-patch32"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)

If the model loads correctly without any unexpected keys, the issue should be resolved.

Verify Expected Behavior

To verify that the fix worked, try loading the model in different scenarios and check if the unexpected keys are still present. If not, the fix is successful.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

It should load correctly with no unexpected keys as they are the same unmodified model

#api#ssr#installation#tensor shape#autograd error#tokenizer error#prompt formatting#chain error#conversation history#tool integration

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix Many models started showing UNEXPECTED Key with position id [2 pull requests, 2 comments, 3 participants]