Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Transformers v5] IsaacForConditionalGeneration [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38389•Fetched 2026-04-08 01:41:35
View on GitHub
Comments
2
Participants
2
Timeline
7
Reactions
0
Author
hmellorhmellor
Participants
amitmodiamitmodihmellorhmellor
Timeline (top)
commented ×2labeled ×2connected ×1cross-referenced ×1

Error Message

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[IsaacForConditionalGeneration] ... AttributeError: 'str' object has no attribute 'args'

PR fix notes

PR #38461: Fixed issues

Description (problem / solution / changelog)

Purpose

Closes #38389

Fixes two test failures for IsaacForConditionalGeneration that are sub-issues of #38379.

Fix 1 — AttributeError: 'str' object has no attribute '__args__'
(tests/models/test_initialization.py::test_can_initialize_large_subset[IsaacForCo nditionalGeneration])

from __future__ import annotations in vllm/transformers_utils/configs/isaac.py defers all annotations to strings at runtime, breaking any introspection of __args__ on int | None / str | None type annotations. Removing it restores
runtime type objects — valid without the future import on Python 3.10+, which vLLM already requires.

Fix 2 — ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (tests/models/multimodal/generation/test_common.py Isaac HF-runner tests)

The Isaac HF reference model (invoked via patch_hf_runner) imports SlidingWindowCache from transformers.cache_utils, which was removed in transformers v5. A guard is added to detect availability at import time and skip the Isaac HF-runner tests when the symbol is absent. The vLLM inference path for Isaac is unaffected.

This PR does not duplicate any existing open PR (checked via gh pr list search for isaac and #38379). AI assistance (Claude) was used; all changed lines reviewed by the human submitter.

Test Plan

# Install vLLM and transformers from source (v5)
git clone https://github.com/huggingface/transformers.git
cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .                                        
uv pip install -e ../transformers
                                                                                  
# Fix 1 — initialization test                                                   
pytest tests/models/test_initialization.py::test_can_initialize_large_subset[Isaac
ForConditionalGeneration] -v                                                      
 
# Fix 2 — multimodal generation tests (expect SKIP on transformers v5)            
pytest tests/models/multimodal/generation/test_common.py -k "isaac" -v          
                                                                                  
Test Result
                                                                                  
Test: test_can_initialize_large_subset[IsaacForConditionalGeneration]           
Before: AttributeError: 'str' object has no attribute '__args__'
After: PASSED            
────────────────────────────────────────
Test: test_single_image_models[isaac-*]                                           
Before: ImportError: cannot import name 'SlidingWindowCache'                      
After: SKIPPED (transformers v5) / PASSED (older transformers)                    
────────────────────────────────────────                                          
Test: test_multi_image_models[isaac-*]                                            
Before: ImportError: cannot import name 'SlidingWindowCache'
After: SKIPPED (transformers v5) / PASSED (older transformers)

## Changed files

- `tests/models/multimodal/generation/test_common.py` (modified, +15/-0)
- `vllm/transformers_utils/configs/isaac.py` (modified, +0/-1)

Code Example

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[IsaacForConditionalGeneration]
...
AttributeError: 'str' object has no attribute '__args__'

---

[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case35] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case36] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case37] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case35] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case36] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case37] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)

---

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers
RAW_BUFFERClick to expand / collapse

This is a sub-issue forming part of the work in https://github.com/vllm-project/vllm/issues/38379, please read the description of this issue before beginning to work on this one.

Which test is failing?

This one has lots of issues beyond this test, it should be upstreamed so that the code is correct and stays maintained

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[IsaacForConditionalGeneration]
...
AttributeError: 'str' object has no attribute '__args__'

Related tests:

[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case35] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case36] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_single_image_models[isaac-test_case37] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case35] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case36] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)
[2026-03-27T01:46:05Z] FAILED models/multimodal/generation/test_common.py::test_multi_image_models[isaac-test_case37] - ImportError: cannot import name 'SlidingWindowCache' from 'transformers.cache_utils' (/usr/local/lib/python3.12/dist-packages/transformers/cache_utils.py)

How to configure my environment?

It's very important that you install both vLLM and Transformers from source so that your test results reflect the current state of both libraries.

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

extent analysis

Fix Plan

The fix involves updating the test_initialization.py file to handle the AttributeError caused by the 'str' object has no attribute '__args__' issue.

  • Update the test_can_initialize_large_subset function in test_initialization.py to check the type of the object before trying to access its __args__ attribute.
  • Ensure that the SlidingWindowCache is properly imported from transformers.cache_utils in the related tests.

Code Changes

# In test_initialization.py
def test_can_initialize_large_subset(self):
    # ...
    if isinstance(obj, str):
        # Handle the case where obj is a string
        pass
    else:
        # Access the __args__ attribute
        obj.__args__
    # ...

# In test_common.py
from transformers.cache_utils import SlidingWindowCache

Verification

Run the tests again using pytest tests/models/test_initialization.py to verify that the AttributeError is resolved. Also, run the related tests in test_common.py to ensure that the SlidingWindowCache is properly imported.

Extra Tips

  • Make sure to install both vLLM and Transformers from source as described in the issue body to reflect the current state of both libraries.
  • If the issue persists, try updating the transformers library to the latest version.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#memory management#API rate limit#retriever error#indexing error#inference speed

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING