PR #37622: [Bugfix] Fix Step3 pipeline parallel KeyError for residual tensor

• Repository: vllm-project/vllm
• Author: JMonde
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/37622

Description (problem / solution / changelog)

Summary

This PR fixes a KeyError: 'residual' that occurs when running Step3 models (e.g., Qwen3.5) with pipeline parallel enabled.

Root Cause

The Step3TextModel.make_empty_intermediate_tensors_factory() was only creating tensors for ["hidden_states"], but the forward() method expects both hidden_states and residual in intermediate_tensors for non-first pipeline ranks.

When running with pipeline parallel:

• First rank: sets residual = None
• Non-first ranks: tries to access intermediate_tensors["residual"] → KeyError

Fix

Added "residual" to the list of keys in make_empty_intermediate_tensors_factory():

# Before
self.make_empty_intermediate_tensors = make_empty_intermediate_tensors_factory(
    ["hidden_states"], config.hidden_size
)

# After  
self.make_empty_intermediate_tensors = make_empty_intermediate_tensors_factory(
    ["hidden_states", "residual"], config.hidden_size
)

Related Issues

• Fixes #37543

Note on Issue #37350

During investigation, I also checked issue #37350 (placeholder_block_size is not defined). This issue has already been fixed in the current codebase - the variable is properly defined at line 597-612 in vllm/v1/worker/gpu_model_runner.py.

Testing

This fix follows the same pattern used in other models like step1.py which already includes both keys:

# vllm/model_executor/models/step1.py (line 312-315)
self.make_empty_intermediate_tensors = make_empty_intermediate_tensors_factory(
    ["hidden_states", "residual"],
    config.hidden_size,
)

Models with similar pipeline parallel patterns that correctly handle residuals:

• step1.py: ✅ Already has both keys
• step3p5.py: ✅ Correctly only uses hidden_states (doesn't use residual in forward)
• step3_text.py: 🔧 Fixed in this PR

Changed files

• vllm/model_executor/models/step3_text.py (modified, +1/-1)