vllm - ✅(Solved) Fix [Transformers v5] MiniCPMV cannot apply processor [1 pull requests, 8 comments, 4 participants]

vllm2026-03-27 18:40:51

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#38385•Fetched 2026-04-08 01:41:41

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Participants

Assignees

Timeline (top)

subscribed ×15mentioned ×14commented ×8labeled ×2

Error Message

$ pytest tests/lora/test_minicpmv_tp.py::test_minicpmv_lora ... ValueError: Failed to apply MiniCPMVProcessor on data={'text': '(<image>./</image>)', 'images': [<PIL.Image.Image image mode=RGB size=448x4032 at 0x7F7695579D60>]} with kwargs={'truncation': False, 'return_tensors': 'pt'}

PR fix notes

PR #38437: Draft: Vendor MiniCPMV and MiniCPMO processors to unblock Transformers v5 upgrade

Repository: vllm-project/vllm
Author: guanwei-wu
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/38437

Description (problem / solution / changelog)

Purpose

Fixes #38385 and MiniCPMO skip in #30566 This PR vendors and adapts the MiniCPMVProcessor and MiniCPMOProcessor into vLLM to unblock the Transformers v5 upgrade.

Test Plan

I performed a local functional test using the specific test case for MiniCPMV LoRA to verify that the AttributeError is resolved. Also, I removes the skip markers in minicpmv25 and minicpmo_26.

pytest tests/lora/test_minicpmv_tp.py::test_minicpmv_lora
pytest tests/models/multimodal/generation/test_common.py -k minicpmv_25
pytest tests/models/multimodal/generation/test_common.py -k minicpmo_26

Test Result

Before

Failed with AttributeError: 'LlamaTokenizerFast' object has no attribute 'im_start_id' during processor initialization.
ValueError: Failed to apply MiniCPMVProcessor on data={'text': '(<image>./</image>)', ...

After

The AttributeError and ValueError are resolved.
Encountered torch.OutOfMemoryError: CUDA out of memory. due to local resource limitations.

<details> <summary> Essential Elements of an Effective PR Description Checklist </summary>

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

</details>

Changed files

tests/models/multimodal/generation/test_common.py (modified, +2/-2)
vllm/model_executor/models/minicpmo.py (modified, +34/-0)
vllm/model_executor/models/minicpmv.py (modified, +9/-3)
vllm/transformers_utils/processors/__init__.py (modified, +4/-0)
vllm/transformers_utils/processors/minicpmo.py (added, +597/-0)
vllm/transformers_utils/processors/minicpmv.py (added, +354/-0)

Code Example

$ pytest tests/lora/test_minicpmv_tp.py::test_minicpmv_lora
...
ValueError: Failed to apply MiniCPMVProcessor on data={'text': '(<image>./</image>)', 'images': [<PIL.Image.Image image mode=RGB size=448x4032 at 0x7F7695579D60>]} with kwargs={'truncation': False, 'return_tensors': 'pt'}

---

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

RAW_BUFFERClick to expand / collapse

This is a sub-issue forming part of the work in https://github.com/vllm-project/vllm/issues/38379, please read the description of this issue before beginning to work on this one.

Which test is failing?

If the model is relevant enough for us to keep supporting, this should be upstreamed to Transformers.

$ pytest tests/lora/test_minicpmv_tp.py::test_minicpmv_lora
...
ValueError: Failed to apply MiniCPMVProcessor on data={'text': '(<image>./</image>)', 'images': [<PIL.Image.Image image mode=RGB size=448x4032 at 0x7F7695579D60>]} with kwargs={'truncation': False, 'return_tensors': 'pt'}

How to configure my environment?

It's very important that you install both vLLM and Transformers from source so that your test results reflect the current state of both libraries.

# Or your fork
git clone https://github.com/huggingface/transformers.git
git clone https://github.com/vllm-project/vllm.git

cd vllm
VLLM_USE_PRECOMPILED=1 uv pip install -e .
uv pip install -e ../transformers

extent analysis

Fix Plan

The fix involves updating the MiniCPMVProcessor to handle the given data format.

Update the MiniCPMVProcessor to check for the presence of images in the input data.
Modify the MiniCPMVProcessor to handle the image data correctly.

Example code:

from transformers import AutoProcessor

class MiniCPMVProcessor(AutoProcessor):
    def __call__(self, data, **kwargs):
        if 'images' in data:
            # Handle image data
            images = data['images']
            # Preprocess images
            preprocessed_images = self.preprocess_images(images)
            # Update data with preprocessed images
            data['images'] = preprocessed_images
        # Apply default processing
        return super().__call__(data, **kwargs)

    def preprocess_images(self, images):
        # Implement image preprocessing logic here
        # For example:
        return [self.image_preprocessor(image) for image in images]

Verification

To verify the fix, run the test again:

pytest tests/lora/test_minicpmv_tp.py::test_minicpmv_lora

If the test passes, the fix is successful.

Extra Tips

Ensure that the image_preprocessor is correctly implemented to handle the image data.
Test the MiniCPMVProcessor with different input data formats to ensure it handles all cases correctly.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#indexing error #inference speed #output truncation #response parsing #generation error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [Transformers v5] MiniCPMV cannot apply processor [1 pull requests, 8 comments, 4 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

PR fix notes

PR #38437: Draft: Vendor MiniCPMV and MiniCPMO processors to unblock Transformers v5 upgrade

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Which test is failing?

How to configure my environment?

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [Transformers v5] MiniCPMV cannot apply processor [1 pull requests, 8 comments, 4 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

PR fix notes

PR #38437: Draft: Vendor MiniCPMV and MiniCPMO processors to unblock Transformers v5 upgrade

Description (problem / solution / changelog)

Purpose

Test Plan

Test Result

Changed files

Code Example

Which test is failing?

How to configure my environment?

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING