PR #37247: [Model] Implement LoRA support for Qwen3ASRForConditionalGeneration

• Repository: vllm-project/vllm
• Author: petern48
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/37247

Description (problem / solution / changelog)

Purpose

This PR adds LoRA support for Qwen3ASRForConditionalGeneration model.

For this to work for the audio tower, I had to make a few additional changes:

• Implement get_num_mm_encoder_tokens()
• Replaced some nn.Linears with ReplicatedLinear along the audio tower path.
• Qwen3ASR seems to be our first model with a tower, but no connector. In gpu_model_runner.py, I found that the hasattr(self.model, "get_num_mm_connector_tokens") was improperly evaluating to True due to inheritance, despite the model not implementing get_num_mm_connector_tokens(). This was leading us to incorrectly go down that path and encounter an error. I've modified the condition to check if connector actually exists in the mapping.

Fixes #37223

Test Plan

I tested on the following public adaptor available on HuggingFace: ha0yuan/Qwen3-ASR-LoRa-ChineseAviation-Tiny.

vllm serve Qwen/Qwen3-ASR-1.7B \
  --enable-lora \
  --enable-tower-connector-lora \
  --lora-modules aviation=ha0yuan/Qwen3-ASR-LoRa-ChineseAviation-Tiny \
  --port 8000

I also double-checked that the adapters are properly shown when querying the /v1/models endpoint:

curl localhost:8000/v1/models | jq .

{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen3-ASR-1.7B",
      "object": "model",
      ...
      "root": "Qwen/Qwen3-ASR-1.7B",
      ...
    },
    {
      "id": "medpl",
       ...
      "root": "AleksanderObuchowski/Qwen3-ASR-1.7B-med-pl-lora-decoder-only",
      ...
    }
}

Then I used a Python script to load in a .wav file and query the /v1/chat/completions endpoint. Specifically, I used this audio file as input.

Test Result

Before this PR, the server would error with ValueError: Qwen3ASRForConditionalGeneration does not support LoRA yet.

After this change, the server starts up properly, and I successfully queried the /v1/chat/completions endpoint.

Querying both the raw model and the adaptor, I verified that the output differs when the LoRa adaptor is enabled. The outputs are below. (Notice, the raw model transcribes to numbers (e.g 9 and 10), while the adaptor transcribes the numbers to words ("nine" and "ten).

== no LoRA ==
language English<asr_text>November the 10th, Wednesday, 9 p.m. I'm standing in a dark alley. After waiting several hours, the time has come. A woman with long dark hair approaches. I have to act, and fast, before she realizes what has happened. I must find out.

== with LoRA ==
language English<asr_text>November the tenth, Wednesday, nine p.m. I'm standing in a dark alley. After waiting several hours, the time has come. A woman with long dark hair approaches. I have to act, and fast, before she realizes what has happened. I must find out.

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Changed files

• docs/models/supported_models.md (modified, +1/-1)
• vllm/model_executor/models/qwen3_asr.py (modified, +26/-0)
• vllm/model_executor/models/qwen3_omni_moe_thinker.py (modified, +22/-3)
• vllm/v1/worker/gpu_model_runner.py (modified, +14/-1)