vllm - 💡(How to fix) Fix [Bug]: AttributeError in mla_attention.py L2094 (_compute_prefill_context) on long prefill with AWQ model — regression after PR #34695

vLLM version: 0.21.1rc1.dev110+g129019f33.d20260519
Model: cyankiwi/GLM-4.7-Flash-AWQ-4bit (AWQ 4-bit, compressed-tensors)
Quantization: compressed-tensors
GPU: 2x NVIDIA DGX Spark (GB10)
Deployment: dual-node, tensor-parallel-size=2, no-Ray (multiproc)
OS: Ubuntu 24.04 ARM64
Python: 3.12

---

vLLM version: 0.21.1rc1.dev110+g129019f33.d20260519
Model: cyankiwi/GLM-4.7-Flash-AWQ-4bit (AWQ 4-bit, compressed-tensors)
Quantization: compressed-tensors
GPU: 2x NVIDIA DGX Spark (GB10)
Deployment: dual-node, tensor-parallel-size=2, no-Ray (multiproc)
OS: Ubuntu 24.04 ARM64
Python: 3.12

---

AttributeError: 'ColumnParallelLinear' object has no attribute 'weight'

---

# For quantized layers (AWQ/GPTQ) that lack a .weight attribute,
# use params_dtype which is the expected input dtype.
_kv_b_proj_w_dtype = (
    self.kv_b_proj.weight.dtype
    if hasattr(self.kv_b_proj, "weight")
    else self.kv_b_proj.params_dtype
)
# For NVFP4, weights are packed uint8 — keep input in model dtype
# since the NVFP4 linear layer quantizes internally.
if (
    use_fp8_prefill or _kv_b_proj_w_dtype != current_platform.fp8_dtype()
) and _kv_b_proj_w_dtype != torch.uint8:
    kv_c_normed = kv_c_normed.to(self.kv_b_proj.weight.dtype)   # ← line 2094, BUG

---

(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962] Traceback (most recent call last):
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/v1/executor/multiproc_executor.py", line 957, in worker_busy_loop
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/v1/worker/gpu_worker.py", line 843, in execute_model
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/v1/worker/gpu_model_runner.py", line 4203, in execute_model
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/v1/worker/gpu_model_runner.py", line 3680, in _model_forward
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/model_executor/models/glm4_moe_lite.py", line 606, in forward
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/model_executor/layers/attention/mla_attention.py", line 1062, in unified_mla_attention_with_output
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]     layer.forward_impl(
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/model_executor/layers/attention/mla_attention.py", line 691, in forward_impl
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]     self.impl.forward_mha(...)
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/model_executor/layers/attention/mla_attention.py", line 2300, in forward_mha
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]     context_output, context_lse = self._compute_prefill_context(...)
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]   File ".../vllm/model_executor/layers/attention/mla_attention.py", line 2094, in _compute_prefill_context
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962]     kv_c_normed = kv_c_normed.to(self.kv_b_proj.weight.dtype)
(Worker_TP0 pid=142) ERROR 05-20 23:24:00 [multiproc_executor.py:962] AttributeError: 'ColumnParallelLinear' object has no attribute 'weight'

---

vllm serve cyankiwi/GLM-4.7-Flash-AWQ-4bit \
     --trust-remote-code \
     --tool-call-parser glm47 --reasoning-parser glm45 \
     --enable-auto-tool-choice \
     --served-model-name glm-4.7-flash \
     --max-model-len 202752 \
     --tensor-parallel-size 2 \
     --host 0.0.0.0 --port 8000 \
     --gpu-memory-utilization 0.7

---

filler = "The quick brown fox jumps over the lazy dog. " * 2700
   payload = {
       "model": "glm-4.7-flash",
       "messages": [{"role": "user", "content": filler + "\n\nSummarize."}],
       "max_tokens": 50,
   }

---

- kv_c_normed = kv_c_normed.to(self.kv_b_proj.weight.dtype)
+ kv_c_normed = kv_c_normed.to(_kv_b_proj_w_dtype)