Issue Station
Sign In
Tools hereToolbox

vllm - ✅(Solved) Fix [Feature]: Add kv_transfer_params to Responses API for PD disaggregation [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#37422•Fetched 2026-04-08 00:57:34
View on GitHub
Comments
1
Participants
2
Timeline
13
Reactions
0
Author
bongwoobakbongwoobak
Participants
bongwoobakbongwoobakgithub-actions[bot]github-actions[bot]
Timeline (top)
mentioned ×3subscribed ×3project_v2_item_status_changed ×2added_to_project_v2 ×1

Fix Action

Fixed

PR fix notes

PR #37424: [Responses API] Add kv_transfer_params for PD disaggregation

Description (problem / solution / changelog)

Purpose

Add kv_transfer_params to the Responses API (/v1/responses) for PD disaggregation support. The Chat Completions API already supports this, but the Responses API does not, preventing PD disaggregation through it.

Follows the same pattern as ChatCompletionRequest/ChatCompletionResponse:

  • ResponsesRequest: add kv_transfer_params field, inject into SamplingParams.extra_args
  • ResponsesResponse: add kv_transfer_params field, populate from engine output
  • All 4 context types (SimpleContext, ParsableContext, HarmonyContext, StreamingHarmonyContext): track kv_transfer_params from RequestOutput

Closes #37422

Test Plan

Tested E2E PD disaggregation on AMD MI250 with NixlConnector:

  • SimpleContext (Llama-3.1-8B-Instruct): non-streaming + streaming
  • ParsableContext (DeepSeek-R1-Distill-Llama-8B)
  • HarmonyContext (gpt-oss-20b-bf16): non-streaming + streaming
  • E2E Prefill→Decode NIXL KV transfer confirmed (cached_tokens=109/110)

Changed files

  • vllm/entrypoints/openai/responses/context.py (modified, +12/-1)
  • vllm/entrypoints/openai/responses/protocol.py (modified, +16/-1)
  • vllm/entrypoints/openai/responses/serving.py (modified, +1/-0)
RAW_BUFFERClick to expand / collapse

🚀 The feature, motivation and pitch

The Chat Completions API (/v1/chat/completions) supports kv_transfer_params on both request and response, enabling PD disaggregation. However, the Responses API (/v1/responses) does not have this field, so PD disaggregation cannot be used through it.

Since the Responses API is becoming more widely adopted, it would be valuable to add kv_transfer_params support for feature parity with Chat Completions.

The change is straightforward: add the field to ResponsesRequest/ResponsesResponse, inject it into SamplingParams.extra_args in to_sampling_params(), and track it through all context types (SimpleContext, ParsableContext, HarmonyContext, StreamingHarmonyContext). This follows the exact same pattern already used in ChatCompletionRequest/ChatCompletionResponse.

I have an implementation ready and have tested E2E PD disaggregation (NixlConnector, AMD MI250) across all context types — will submit a PR shortly.

Alternatives

Users must use the Chat Completions API instead of the Responses API when PD disaggregation is needed.

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

Fix Plan

To add kv_transfer_params support to the Responses API, follow these steps:

  • Add kv_transfer_params field to ResponsesRequest and ResponsesResponse data models.
  • Update to_sampling_params() method to inject kv_transfer_params into SamplingParams.extra_args.
  • Track kv_transfer_params through all context types: SimpleContext, ParsableContext, HarmonyContext, StreamingHarmonyContext.

Example code snippet:

class ResponsesRequest:
    # ... existing fields ...
    kv_transfer_params: Optional[Dict[str, str]] = None

class ResponsesResponse:
    # ... existing fields ...
    kv_transfer_params: Optional[Dict[str, str]] = None

def to_sampling_params(self) -> SamplingParams:
    # ... existing code ...
    extra_args = {
        # ... existing args ...
        'kv_transfer_params': self.kv_transfer_params
    }
    return SamplingParams(extra_args=extra_args)

Verification

Verify the fix by testing PD disaggregation using the Responses API with kv_transfer_params across all context types.

Extra Tips

  • Ensure to update documentation and tests to reflect the added kv_transfer_params support.
  • Review existing code for any potential conflicts or inconsistencies with the new field.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#request error#file not found#serialization error#model compatibility

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING