vllm - ✅(Solved) Fix [Bug]: `ImportError: cannot import name 'GenerationConfig' from 'transformers'`, possible race condition on import [1 pull requests, 2 comments, 2 participants]

vllm2026-04-21 07:26:59

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#40442•Fetched 2026-04-22 07:45:37

View on GitHub

Comments

Participants

Timeline

Reactions

Author

dbari

Participants

DarkLight1337

dbari

Timeline (top)

commented ×2cross-referenced ×2closed ×1labeled ×1

Error Message

$ docker run --rm -it vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d Unable to find image 'vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d' locally nightly-b47840019e61a3983c8144066a99c843d177947d: Pulling from vllm/vllm-openai Digest: sha256:befc9151fea3ffc042abb0949bff1a1c9a2f1295c6eaef58bae63301f0c6981b Status: Downloaded newer image for vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d Traceback (most recent call last): File "/usr/local/bin/vllm", line 10, in <module> sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 87, in main import vllm.entrypoints.cli.benchmark.main File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/benchmark/main.py", line 10, in <module> from vllm.entrypoints.utils import VLLM_SUBCMD_PARSER_EPILOG File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/utils.py", line 19, in <module> from vllm.engine.arg_utils import EngineArgs File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 35, in <module> from vllm.config import ( File "/usr/local/lib/python3.12/dist-packages/vllm/config/init.py", line 20, in <module> from vllm.config.model import ( File "/usr/local/lib/python3.12/dist-packages/vllm/config/model.py", line 30, in <module> from vllm.transformers_utils.config import ( File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/config.py", line 18, in <module> from transformers import GenerationConfig, PretrainedConfig ImportError: cannot import name 'GenerationConfig' from 'transformers' (/usr/local/lib/python3.12/dist-packages/transformers/init.py)

Fix Action

Fixed

Fixed by PR: Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm" (https://github.com/vllm-project/vllm/pull/40438)

PR fix notes

PR #40438: Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm"

Repository: vllm-project/vllm
Author: noooop
State: closed | merged: True
Link: https://github.com/vllm-project/vllm/pull/40438

Description (problem / solution / changelog)

Reverts vllm-project/vllm#40331

FIX https://github.com/vllm-project/vllm/issues/40442

I think the following code is causing the recent main failures.

Failures can be reproduced locally.

pytest -s -vvv tests/entrypoints/pooling/basic/test_truncation.py

After removing this code, the local test passed.

    with contextlib.suppress(Exception):
        import transformers  # noqa: F401

3 passed

Changed files

vllm/entrypoints/cli/main.py (modified, +1/-71)
vllm/entrypoints/openai/api_server.py (modified, +2/-142)
vllm/envs.py (modified, +3/-7)

Code Example

$ docker run --rm -it vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d
Unable to find image 'vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d' locally
nightly-b47840019e61a3983c8144066a99c843d177947d: Pulling from vllm/vllm-openai
Digest: sha256:befc9151fea3ffc042abb0949bff1a1c9a2f1295c6eaef58bae63301f0c6981b
Status: Downloaded newer image for vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d
Traceback (most recent call last):
  File "/usr/local/bin/vllm", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 87, in main
    import vllm.entrypoints.cli.benchmark.main
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/benchmark/main.py", line 10, in <module>
    from vllm.entrypoints.utils import VLLM_SUBCMD_PARSER_EPILOG
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/utils.py", line 19, in <module>
    from vllm.engine.arg_utils import EngineArgs
  File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 35, in <module>
    from vllm.config import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/__init__.py", line 20, in <module>
    from vllm.config.model import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/model.py", line 30, in <module>
    from vllm.transformers_utils.config import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/config.py", line 18, in <module>
    from transformers import GenerationConfig, PretrainedConfig
ImportError: cannot import name 'GenerationConfig' from 'transformers' (/usr/local/lib/python3.12/dist-packages/transformers/__init__.py)

RAW_BUFFERClick to expand / collapse

Your current environment

Using the vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d image.

🐛 Describe the bug

The current nightly fails with the following error:

$ docker run --rm -it vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d
Unable to find image 'vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d' locally
nightly-b47840019e61a3983c8144066a99c843d177947d: Pulling from vllm/vllm-openai
Digest: sha256:befc9151fea3ffc042abb0949bff1a1c9a2f1295c6eaef58bae63301f0c6981b
Status: Downloaded newer image for vllm/vllm-openai:nightly-b47840019e61a3983c8144066a99c843d177947d
Traceback (most recent call last):
  File "/usr/local/bin/vllm", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 87, in main
    import vllm.entrypoints.cli.benchmark.main
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/benchmark/main.py", line 10, in <module>
    from vllm.entrypoints.utils import VLLM_SUBCMD_PARSER_EPILOG
  File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/utils.py", line 19, in <module>
    from vllm.engine.arg_utils import EngineArgs
  File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 35, in <module>
    from vllm.config import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/__init__.py", line 20, in <module>
    from vllm.config.model import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/config/model.py", line 30, in <module>
    from vllm.transformers_utils.config import (
  File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/config.py", line 18, in <module>
    from transformers import GenerationConfig, PretrainedConfig
ImportError: cannot import name 'GenerationConfig' from 'transformers' (/usr/local/lib/python3.12/dist-packages/transformers/__init__.py)

The cause seems to be the background thread loading transformers here and the main thread using an incomplete module here. Adding a time.sleep(0.05) before the second import prevents the error, pointing to a possible race condition.

This does not happen with the previous vllm/vllm-openai:nightly-8936118134d0547fa1cc78adab2d03edd6d3dc48

Before submitting a new issue...

Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The most likely fix is to address the race condition causing the ImportError by ensuring that the transformers module is fully loaded before attempting to import GenerationConfig.

Guidance

Verify that the issue is indeed a race condition by consistently reproducing the error and checking if adding a small delay before the import resolves the issue.
Review the code changes between the working nightly-8936118134d0547fa1cc78adab2d03edd6d3dc48 and the current nightly-b47840019e61a3983c8144066a99c843d177947d images to identify any changes that might have introduced the race condition.
Consider using a synchronization mechanism, such as a lock, to ensure that the transformers module is fully loaded before attempting to import GenerationConfig.
If possible, test the code with a newer version of the transformers library to see if the issue has been resolved.

Example

No code example is provided as the issue seems to be related to the timing of imports and not a specific code snippet.

Notes

The issue might be specific to the combination of the vllm library and the transformers library versions used in the nightly-b47840019e61a3983c8144066a99c843d177947d image. Further investigation is needed to determine the root cause and a reliable fix.

Recommendation

Apply a workaround, such as adding a small delay before the import, until a more permanent fix can be implemented. This is because the issue seems to be a race condition, and adding a delay can help ensure that the transformers module is fully loaded before attempting to import GenerationConfig.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#prompt template #agent execution #callback error #memory management #API rate limit

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [Bug]: `ImportError: cannot import name 'GenerationConfig' from 'transformers'`, possible race condition on import [1 pull requests, 2 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #40438: Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm"

Description (problem / solution / changelog)

Changed files

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [Bug]: `ImportError: cannot import name 'GenerationConfig' from 'transformers'`, possible race condition on import [1 pull requests, 2 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #40438: Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm"

Description (problem / solution / changelog)

Changed files

Code Example

Your current environment

🐛 Describe the bug

Before submitting a new issue...

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING