Issue Station
Sign In
Tools hereToolbox

vllm - 💡(How to fix) Fix [RFC] [Feature]: PD Disaggregation with MooncakeConnector Roadmap [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#36948•Fetched 2026-04-08 00:43:26
View on GitHub
Comments
0
Participants
1
Timeline
9
Reactions
9
Author
dtccccdtcccc
Participants
dtccccdtcccc
Timeline (top)
subscribed ×6mentioned ×2labeled ×1

Error Message

  • Other improvements and fixes(better error handling, physical block != config block, different hidden size per layer, HND-NHD transfer, test cases, kv events & stats, etc...)
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

Following the successful initial integration of Mooncake Transfer Engine into vLLM (#24718 #31034), this RFC proposes a feature roadmap to unlock more advanced capabilities, enhance performance, and improve flexibility. The foundational work allows for basic KV cache offloading, but to fully leverage the potential of Mooncake and support more complex, large-scale inference scenarios, we need to implement more sophisticated parallelism strategies and optimizations.

Core Features:

  • Mooncake store integration
  • Heterogeneous TP support (#36869)
  • PP support
  • Layer-wise Transfer
  • Cross-Layer KV Support
  • PCP/DCP Support
  • hybrid attention
  • Other improvements and fixes(better error handling, physical block != config block, different hidden size per layer, HND-NHD transfer, test cases, kv events & stats, etc...)

Related issue #33702 cc @NickLucche @LCAIZJ

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To address the proposed feature roadmap for Mooncake Transfer Engine integration, we will focus on implementing the core features.

Step-by-Step Solution

  1. Mooncake Store Integration: Implement a Mooncake store interface to handle KV cache offloading.
  2. Heterogeneous TP Support: Develop a heterogeneous TP strategy to support various parallelism scenarios.
  3. PP Support: Integrate PP support for enhanced performance.
  4. Layer-wise Transfer: Implement layer-wise transfer for efficient data transfer.
  5. Cross-Layer KV Support: Develop cross-layer KV support for improved flexibility.

Example Code Snippet (Python)

# Mooncake Store Interface
class MooncakeStore:
    def __init__(self, config):
        self.config = config

    def offload_kv_cache(self, data):
        # Implement KV cache offloading logic
        pass

# Heterogeneous TP Strategy
class HeterogeneousTP:
    def __init__(self, config):
        self.config = config

    def execute(self, data):
        # Implement heterogeneous TP logic
        pass

# PP Support
class PPSupport:
    def __init__(self, config):
        self.config = config

    def execute(self, data):
        # Implement PP support logic
        pass

Verification

To verify the implementation, test each core feature individually and ensure they function as expected.

Extra Tips

  • Ensure thorough testing and validation of each feature before merging into the main branch.
  • Consider implementing automated tests for regression testing.
  • Review and refine the implementation based on code review feedback.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#optimization#ISR setup#authentication setup#request error#file not found

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING