Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Transformers v5] SarvamMLAForCausalLM [1 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#38734•Fetched 2026-04-08 02:23:06
View on GitHub
Comments
2
Participants
3
Timeline
13
Reactions
0
Author
hmellorhmellor
Participants
ahmedabbas104ahmedabbas104hmellorhmellorVikrantpalleVikrantpalle
Assignees
VikrantpalleVikrantpalle
Timeline (top)
cross-referenced ×4referenced ×3commented ×2labeled ×2

Error Message

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[SarvamMLAForCausalLM] ... TypeError: RotaryEmbeddingConfigMixin.validate_rope() got an unexpected keyword argument 'ignore_keys'

Fix Action

Fixed

PR fix notes

PR #38767: [Transformers v5] Add SarvamMLAConfig to fix SarvamMLAForCausalLM (#38734)

Description (problem / solution / changelog)

What's the problem?

Fixes #38734

SarvamMLAForCausalLM (sarvamai/sarvam-105b) fails to load under transformers v5 because the remote configuration_sarvam_mla.py calls validate_rope(ignore_keys=...). Upstream PR huggingface/transformers#41250 (transformers v5) removed the ignore_keys parameter, causing:

TypeError: RotaryEmbeddingConfigMixin.validate_rope() got an unexpected
keyword argument 'ignore_keys'

Fix

Add SarvamMLAConfig to vLLM's local config registry. Once registered, HFConfigParser.parse() matches model_type="sarvam_mla" and loads this local class instead of the remote code — bypassing trust_remote_code and the broken API call entirely.

SarvamMLAConfig exposes all fields accessed by vllm/model_executor/models/sarvam.py and handles the rope_scaling → rope_parameters conversion (including the "type" → "rope_type" key rename required by transformers v5). The single intermediate_size field from the model's config.json is exposed as both intermediate_size and moe_intermediate_size to satisfy direct accesses in the model code.

Why this is not a duplicate

There are no open PRs addressing this issue. A user asked to be assigned in the issue comments but has not submitted a PR.

Tests

This change is pure configuration — no runtime logic is added or removed. The fix can be verified by loading sarvamai/sarvam-105b with trust_remote_code=True under transformers v5; before this PR the TypeError is raised during get_config(); after it the config loads cleanly.

CI note: The model is gated and cannot be tested in CI without credentials. The pattern follows the established approach used by NemotronConfig, IsaacConfig, and every other entry in _CONFIG_REGISTRY.

AI assistance was used to research and author this fix.

Changed files

  • vllm/transformers_utils/config.py (modified, +1/-0)
  • vllm/transformers_utils/configs/__init__.py (modified, +2/-0)
  • vllm/transformers_utils/configs/sarvam.py (added, +114/-0)

Code Example

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[SarvamMLAForCausalLM]
...
TypeError: RotaryEmbeddingConfigMixin.validate_rope() got an unexpected keyword argument 'ignore_keys'

---

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers
RAW_BUFFERClick to expand / collapse

This is a sub-issue forming part of the work in https://github.com/vllm-project/vllm/issues/38379, please read the description of this issue before beginning to work on this one.

Which test is failing?

As of https://github.com/huggingface/transformers/pull/41250 the ignore_keys argument to validate_rope was removed in favour of ClassVars (ignore_keys_at_rope_validation) attached to the config classes themselves.

$ pytest tests/models/test_initialization.py::test_can_initialize_large_subset[SarvamMLAForCausalLM]
...
TypeError: RotaryEmbeddingConfigMixin.validate_rope() got an unexpected keyword argument 'ignore_keys'

How to configure my environment?

It's very important that you install both vLLM and Transformers from source so that your test results reflect the current state of both libraries.

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

extent analysis

TL;DR

Update the test_can_initialize_large_subset test in tests/models/test_initialization.py to remove the ignore_keys argument when calling validate_rope due to its removal in favor of ClassVars.

Guidance

  • Review the changes made in https://github.com/huggingface/transformers/pull/41250 to understand the removal of the ignore_keys argument and the introduction of ignore_keys_at_rope_validation ClassVars.
  • Update the test to use the new ignore_keys_at_rope_validation attribute on the config class instead of passing ignore_keys as an argument.
  • Ensure that the test environment is set up correctly by installing both vLLM and Transformers from source using the provided commands.
  • Verify that the test passes after making the necessary changes.

Example

# Before
config.validate_rope(ignore_keys=["some_key"])

# After
config.ignore_keys_at_rope_validation = ["some_key"]
config.validate_rope()

Notes

The solution assumes that the ignore_keys_at_rope_validation attribute is correctly defined on the config class and that the validate_rope method is updated to use this attribute.

Recommendation

Apply workaround: Update the test to use the new ignore_keys_at_rope_validation attribute to ensure compatibility with the latest changes in the Transformers library.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#model save/load#optimization#mixed precision#training loop#device allocation

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING