Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Transformers v5] Tarsier2ForConditionalGeneration [2 pull requests, 3 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38736•Fetched 2026-04-08 02:23:03
View on GitHub
Comments
3
Participants
3
Timeline
15
Reactions
0
Author
hmellorhmellor
Participants
ahmedabbas104ahmedabbas104h1yuolh1yuolhmellorhmellor
Timeline (top)
referenced ×4commented ×3cross-referenced ×2labeled ×2

Error Message

[2026-04-01T10:33:31Z] Value error, The text_config extracted from the model config does not have num_attention_heads attribute. This indicates a mismatch between the model config and vLLM's expectations. Please ensure that the model config is compatible with vLLM. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]

Fix Action

Fix / Workaround

Tarsier 2 has a malformed config.json in its checkpoint and vLLM already has some workarounds to account for this (the Tarsier2Config that gets registered in vLLM). This appears to not be working correctly with the latest Transformers.

PR fix notes

PR #38821: [Transformers v5] Fix Tarsier2Config text_config nesting (#38736)

Description (problem / solution / changelog)

Summary

Tarsier2's config.json reports model_type: "llava", which causes AutoConfig.from_pretrained to instantiate LlavaConfig instead of Qwen2VLConfig. LlavaConfig.get_text_config() returns a Qwen2VLConfig sub-object whose num_attention_heads is delegated to the inner Qwen2VLTextConfig — so hasattr(text_cfg, "num_attention_heads") is False, triggering the ValueError in get_hf_text_config.

Root cause: AutoConfig.from_pretrained reads model_type from config.json and ignores the registered Tarsier2Config. Registering a custom config class is not sufficient when the file reports a different model_type.

Fix (two changes):

  1. HFConfigParser.parse (config.py): When model_type in _CONFIG_REGISTRY, call config_class.from_pretrained directly instead of routing through AutoConfig.from_pretrained. This ensures the registered class is always used, regardless of what model_type the config.json reports. This also benefits any future model with a similar model_type mismatch.

  2. Tarsier2Config (tarsier2.py): Simplified to just model_type = "tarsier2". The __post_init__ override was dead code — the text_config dict in config.json is flat (not doubly nested), so the unwrap condition never fired. Qwen2VLConfig.__post_init__ correctly converts the flat dict into Qwen2VLTextConfig with all fields intact (num_attention_heads=28, hidden_size=3584, etc.).

Test Plan

  • Loaded omni-research/Tarsier2-Recap-7b with the patched config class via config_class.from_pretrained; confirmed:
    • type(config).__name__ == "Tarsier2Config"
    • type(config.get_text_config()).__name__ == "Qwen2VLTextConfig"
    • config.get_text_config().num_attention_heads == 28

Checklist

  • Format: ruff check / ruff format pass
  • No new dependencies introduced

AI assistance was used in developing this fix.

Fixes #38736

Changed files

  • vllm/transformers_utils/config.py (modified, +31/-22)
  • vllm/transformers_utils/configs/tarsier2.py (modified, +12/-12)

PR #39239: WIP: Tarsier2ForConditionalGeneration (begin work)

Description (problem / solution / changelog)

Work-in-progress branch for issue #38736. I will push fixes to this branch.

Changed files

Code Example

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[Tarsier2ForConditionalGeneration]
...
[2026-04-01T10:33:31Z]   Value error, The text_config extracted from the model config does not have `num_attention_heads` attribute. This indicates a mismatch between the model config and vLLM's expectations. Please ensure that the model config is compatible with vLLM. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]

---

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers
RAW_BUFFERClick to expand / collapse

This is a sub-issue forming part of the work in https://github.com/vllm-project/vllm/issues/38379, please read the description of this issue before beginning to work on this one.

Which test is failing?

Tarsier 2 has a malformed config.json in its checkpoint and vLLM already has some workarounds to account for this (the Tarsier2Config that gets registered in vLLM). This appears to not be working correctly with the latest Transformers.

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[Tarsier2ForConditionalGeneration]
...
[2026-04-01T10:33:31Z]   Value error, The text_config extracted from the model config does not have `num_attention_heads` attribute. This indicates a mismatch between the model config and vLLM's expectations. Please ensure that the model config is compatible with vLLM. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]

How to configure my environment?

It's very important that you install both vLLM and Transformers from source so that your test results reflect the current state of both libraries.

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

extent analysis

TL;DR

The issue can be resolved by ensuring compatibility between the model config and vLLM's expectations, potentially by updating the Tarsier2Config workaround to match the latest Transformers.

Guidance

  • Verify that the Tarsier2Config is correctly registered in vLLM and compatible with the latest Transformers.
  • Check the config.json in the Tarsier 2 checkpoint for any missing or malformed attributes, such as num_attention_heads.
  • Ensure that the environment is configured correctly by installing both vLLM and Transformers from source using the provided commands.
  • Review the test case test_can_initialize_large_subset in tests/models/test_initialization.py to understand the specific requirements for model initialization.

Example

No code snippet is provided as the issue does not contain sufficient information to create a specific example.

Notes

The issue may be related to a mismatch between the model config and vLLM's expectations, and resolving this compatibility issue may require updates to the Tarsier2Config workaround.

Recommendation

Apply workaround: Update the Tarsier2Config to ensure compatibility with the latest Transformers, as this is likely to resolve the issue with the malformed config.json in the Tarsier 2 checkpoint.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#tensor shape#autograd error#model save/load#optimization#mixed precision

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - ✅(Solved) Fix [Transformers v5] Tarsier2ForConditionalGeneration [2 pull requests, 3 comments, 3 participants]