transformers - 💡(How to fix) Fix Add automatic dtype alignment and validation for model inputs and pipelines [2 comments, 2 participants]

Feature request

Currently, users frequently encounter runtime errors caused by dtype mismatches (e.g., float32 vs float16) when working with Transformers models and pipelines—especially in mixed precision or when integrating with libraries like diffusers and accelerate.

These errors are often:

non-obvious difficult to debug caused by implicit assumptions about dtype handling

Motivation

❗ Problem

Example failure pattern:

model = AutoModel.from_pretrained(..., torch_dtype=torch.float16) inputs = tokenizer(..., return_tensors="pt") # defaults to float32

outputs = model(**inputs) # RuntimeError: expected Half but got Float

This leads to errors such as:

expected scalar type Half but found Float silent failures or crashes in pipelines

Your contribution

I’d be happy to work on this feature and submit a PR if it aligns with the maintainers’ direction. Please let me know if there are any preferred approaches or existing plans.