Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [Feature]: Support NixlConnector with Pipeline Parallelism for disaggregated serving [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#40674•Fetched 2026-04-24 05:52:14
View on GitHub
Comments
0
Participants
1
Timeline
1
Reactions
0
Author
qichingqiching
Participants
qichingqiching
Timeline (top)
labeled ×1
  • Model: DeepSeek-R1 (FP4) on Blackwell (GB300), 128K/8K, disaggregated serving.
  • Goal: shrink the prefill pool via PP on prefill workers while keeping decode on TP/DP+EP, matching the deployment pattern already validated on SGLang.
  • Current vLLM status: aggregated-mode PP works; disaggregated-mode PP with NixlConnector does not support.

Root Cause

today this path is blocked in vLLM because NixlConnector does not support pipeline-parallel-size > 1. Any disaggregated deployment that wants PP on the prefill workers cannot use vLLM's KV-transfer layer. This forces us (and other users building long-context / disagg stacks) to either:

Fix Action

Fix / Workaround

  • drop PP on prefill entire (losing the prefill-pool compression benefit), or
  • fork or patch NixlConnector internally, which is not sustainable.
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

On the prefill side of high-throughput long-context disaggregated serving, Pipeline Parallelism (PP) is an important scaling axis: it compresses the prefill pool (fewer prefill GPUs feeding the same number of decode GPUs) and, when combined with chunked-pipeline techniques, helps hide the long-context attention bubble.

today this path is blocked in vLLM because NixlConnector does not support pipeline-parallel-size > 1. Any disaggregated deployment that wants PP on the prefill workers cannot use vLLM's KV-transfer layer. This forces us (and other users building long-context / disagg stacks) to either:

  • drop PP on prefill entire (losing the prefill-pool compression benefit), or
  • fork or patch NixlConnector internally, which is not sustainable.

Reference comparison: SGLang already supports PP + NIXL in disaggregated mode and reports sizable long-context prefill gains from PP when combined with their chunked-PP path (SGLang blog, 2026-01-15). vLLM currently has no equivalent path.

Alternatives

Proposed feature

Add first-class support for NixlConnector when pipeline-parallel-size > 1, on both the prefill and decode side of disaggregated serving:

  • KV cache handoff correctly addresses the PP stage that owns each layer's KV
  • Engine bring-up with NixlConnector + PP does not deadlock on handshake.
  • Works in combination with TP and DP/EP on both ends

context

  • Model: DeepSeek-R1 (FP4) on Blackwell (GB300), 128K/8K, disaggregated serving.
  • Goal: shrink the prefill pool via PP on prefill workers while keeping decode on TP/DP+EP, matching the deployment pattern already validated on SGLang.
  • Current vLLM status: aggregated-mode PP works; disaggregated-mode PP with NixlConnector does not support.

Related work

A separate (also important) gap is that vLLM PP today lacks the long-context-optimized pipeline path, Chunked Pipeline Parallelism (CPP) + Async P2P + Dynamic Chunking, that SGLang uses to make PP actually win on long-context prefill. (https://www.lmsys.org/blog/2026-01-15-chunked-pipeline/)

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

Add support for pipeline-parallel-size > 1 in NixlConnector to enable Pipeline Parallelism (PP) on prefill workers in disaggregated serving.

Guidance

  • Modify NixlConnector to handle KV cache handoff correctly for each PP stage.
  • Ensure engine bring-up with NixlConnector and PP does not deadlock on handshake.
  • Test the updated NixlConnector with pipeline-parallel-size > 1 in combination with TP and DP/EP on both ends.

Example

No code snippet is provided as the issue lacks specific implementation details.

Notes

The proposed solution requires modifications to the NixlConnector component, which may involve significant development and testing efforts.

Recommendation

Apply a workaround by forking or patching NixlConnector internally to add support for pipeline-parallel-size > 1, as a temporary solution until the official fix is available. This is because dropping PP on prefill workers or waiting for an official fix may not be feasible for users who need to deploy disaggregated serving with PP.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#indexing error#inference speed#output truncation#response parsing#generation error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING