vllm - 💡(How to fix) Fix [Tracking Issue]: NIXL P/D Disaggregation for Hybrid Models [1 participants]

TL;DR

Complete the implementation of the "Unify Transfer Topology" refactor to potentially resolve inconsistencies and improve model compatibility.

Guidance

• Review the "Unify Transfer Topology" refactor task to ensure it addresses the differences between Nemotron/Mamba2 and other models like Gemma4 and Qwen3.5.
• Verify that the "Heterogeneous TP" feature is correctly implemented for Nemotron/Mamba2 and consider its applicability to other models.
• Check the status of the "Keep model-specific logics clean" task to prevent logic duplication and ensure a unified approach across models.
• Investigate how the "prefix caching" and "spec decode" features for Nemotron/Mamba2 could be generalized for other models.

Notes

The provided information lacks specific technical details about the implementation, so the guidance is focused on high-level suggestions for refactoring and feature implementation.

Recommendation

Apply workaround: Focus on completing the "Unify Transfer Topology" refactor to create a more consistent and adaptable codebase for various models.