transformers - 💡(How to fix) Fix Transformers v5.9.0 breaks support for pytorch with USE_DISTRIBUTED=0

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

uv run --with torch --with transformers python -c " import torch from transformers import AutoModelForImageToImage try: # Using a tiny swin2SR model for fast verification model = AutoModelForImageToImage.from_pretrained('caidas/swin2SR-lightweight-x2-64') print('\nSuccess! AutoModelForImageToImage initialized perfectly.') print(f'Model class: {model.class.name}') except Exception as e: print(f'Test failed: {e}') "

Code Example

uv run --with torch --with transformers python -c "
import torch
from transformers import AutoModelForImageToImage
try:
    # Using a tiny swin2SR model for fast verification
    model = AutoModelForImageToImage.from_pretrained('caidas/swin2SR-lightweight-x2-64')
    print('\nSuccess! AutoModelForImageToImage initialized perfectly.')
    print(f'Model class: {model.__class__.__name__}')
except Exception as e:
    print(f'Test failed: {e}')
"

---

Traceback (most recent call last):
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2249, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2483, in _get_module
    raise e
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2481, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/python/cpython-3.14.4-macos-x86_64-none/lib/python3.14/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1406, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 759, in exec_module
  File "<frozen importlib._bootstrap>", line 491, in _call_with_frames_removed
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/utils.py", line 73, in <module>
    from .continuous_batching import ContinuousMixin
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/__init__.py", line 14, in <module>
    from .cache import PagedAttentionCache
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/cache.py", line 24, in <module>
    from .distributed import DistributedHelper
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/distributed.py", line 19, in <module>
    from torch.distributed.tensor.device_mesh import DeviceMesh
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/__init__.py", line 4, in <module>
    import torch.distributed.tensor._ops  # force import all built-in dtensor ops
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_ops/__init__.py", line 2, in <module>
    from ._conv_ops import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_ops/_conv_ops.py", line 7, in <module>
    from torch.distributed.tensor._dtensor_spec import DTensorSpec, TensorMeta
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_dtensor_spec.py", line 10, in <module>
    from torch.distributed.tensor.placement_types import (
    ...<7 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/placement_types.py", line 11, in <module>
    import torch.distributed._functional_collectives as funcol
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/_functional_collectives.py", line 10, in <module>
    import torch.distributed.distributed_c10d as c10d
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/distributed_c10d.py", line 25, in <module>
    from torch._C._distributed_c10d import (
    ...<22 lines>...
    )
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2249, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2483, in _get_module
    raise e
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2481, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/python/cpython-3.14.4-macos-x86_64-none/lib/python3.14/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1406, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 759, in exec_module
  File "<frozen importlib._bootstrap>", line 491, in _call_with_frames_removed
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/models/auto/modeling_auto.py", line 21, in <module>
    from .auto_factory import (
    ...<4 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/models/auto/auto_factory.py", line 43, in <module>
    from ...generation import GenerationMixin
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2337, in __getattr__
    raise ModuleNotFoundError(
        f"Could not import module '{name}'. Are this object's requirements defined correctly?"
    ) from e
ModuleNotFoundError: Could not import module 'GenerationMixin'. Are this object's requirements defined correctly?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/msdarwish/.local/bin/docling", line 4, in <module>
    from docling.cli.main import app
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/cli/main.py", line 109, in <module>
    from docling.document_converter import (
    ...<10 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/document_converter.py", line 68, in <module>
    from docling.pipeline.asr_pipeline import AsrPipeline
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/pipeline/asr_pipeline.py", line 33, in <module>
    from docling.pipeline.base_pipeline import BasePipeline
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/pipeline/base_pipeline.py", line 36, in <module>
    from docling.models.stages.chart_extraction.granite_vision import (
    ...<2 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/models/stages/chart_extraction/granite_vision.py", line 27, in <module>
    from transformers import AutoModelForImageTextToText, AutoProcessor
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2337, in __getattr__
    raise ModuleNotFoundError(
        f"Could not import module '{name}'. Are this object's requirements defined correctly?"
    ) from e
ModuleNotFoundError: Could not import module 'AutoModelForImageTextToText'. Are this object's requirements defined correctly?

---

uv run --with torch --with transformers==5.8.1 ...
RAW_BUFFERClick to expand / collapse

System Info

Version: 5.9.0 Platform: Intel-based MacOS 26.5 Python Version: 3.14 Pytorch: 2.12.0 (self-built for Intel MacOS)

Who can help?

@remi-or

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Run the following command on macOS version of pytorch (especially USE_DISTRIBUTED=0 during compilation):

uv run --with torch --with transformers python -c "
import torch
from transformers import AutoModelForImageToImage
try:
    # Using a tiny swin2SR model for fast verification
    model = AutoModelForImageToImage.from_pretrained('caidas/swin2SR-lightweight-x2-64')
    print('\nSuccess! AutoModelForImageToImage initialized perfectly.')
    print(f'Model class: {model.__class__.__name__}')
except Exception as e:
    print(f'Test failed: {e}')
"

The execution fails with the following error:

Traceback (most recent call last):
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2249, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2483, in _get_module
    raise e
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2481, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/python/cpython-3.14.4-macos-x86_64-none/lib/python3.14/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1406, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 759, in exec_module
  File "<frozen importlib._bootstrap>", line 491, in _call_with_frames_removed
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/utils.py", line 73, in <module>
    from .continuous_batching import ContinuousMixin
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/__init__.py", line 14, in <module>
    from .cache import PagedAttentionCache
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/cache.py", line 24, in <module>
    from .distributed import DistributedHelper
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/generation/continuous_batching/distributed.py", line 19, in <module>
    from torch.distributed.tensor.device_mesh import DeviceMesh
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/__init__.py", line 4, in <module>
    import torch.distributed.tensor._ops  # force import all built-in dtensor ops
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_ops/__init__.py", line 2, in <module>
    from ._conv_ops import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_ops/_conv_ops.py", line 7, in <module>
    from torch.distributed.tensor._dtensor_spec import DTensorSpec, TensorMeta
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/_dtensor_spec.py", line 10, in <module>
    from torch.distributed.tensor.placement_types import (
    ...<7 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/tensor/placement_types.py", line 11, in <module>
    import torch.distributed._functional_collectives as funcol
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/_functional_collectives.py", line 10, in <module>
    import torch.distributed.distributed_c10d as c10d
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/torch/distributed/distributed_c10d.py", line 25, in <module>
    from torch._C._distributed_c10d import (
    ...<22 lines>...
    )
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2249, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2483, in _get_module
    raise e
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2481, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/msdarwish/.local/share/uv/python/cpython-3.14.4-macos-x86_64-none/lib/python3.14/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1406, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1371, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1342, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 938, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 759, in exec_module
  File "<frozen importlib._bootstrap>", line 491, in _call_with_frames_removed
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/models/auto/modeling_auto.py", line 21, in <module>
    from .auto_factory import (
    ...<4 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/models/auto/auto_factory.py", line 43, in <module>
    from ...generation import GenerationMixin
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2337, in __getattr__
    raise ModuleNotFoundError(
        f"Could not import module '{name}'. Are this object's requirements defined correctly?"
    ) from e
ModuleNotFoundError: Could not import module 'GenerationMixin'. Are this object's requirements defined correctly?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/msdarwish/.local/bin/docling", line 4, in <module>
    from docling.cli.main import app
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/cli/main.py", line 109, in <module>
    from docling.document_converter import (
    ...<10 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/document_converter.py", line 68, in <module>
    from docling.pipeline.asr_pipeline import AsrPipeline
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/pipeline/asr_pipeline.py", line 33, in <module>
    from docling.pipeline.base_pipeline import BasePipeline
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/pipeline/base_pipeline.py", line 36, in <module>
    from docling.models.stages.chart_extraction.granite_vision import (
    ...<2 lines>...
    )
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/docling/models/stages/chart_extraction/granite_vision.py", line 27, in <module>
    from transformers import AutoModelForImageTextToText, AutoProcessor
  File "/Users/msdarwish/.local/share/uv/tools/docling-slim/lib/python3.14/site-packages/transformers/utils/import_utils.py", line 2337, in __getattr__
    raise ModuleNotFoundError(
        f"Could not import module '{name}'. Are this object's requirements defined correctly?"
    ) from e
ModuleNotFoundError: Could not import module 'AutoModelForImageTextToText'. Are this object's requirements defined correctly?

The command runs successfully when using:

uv run --with torch --with transformers==5.8.1 ...

It seems to be coming from the newly introduced transformers/generation/continuous_batching/distributed.py.

Expected behavior

The command runs successfully

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

The command runs successfully

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING