[Roadmap] 2026 Q2 vLLM × RL Roadmap

This tracks the Q2 2026 vLLM-side work needed to make RL workloads (training & rollout) first-class. Each item links its own RFC / issue / PR — please discuss there, and use this thread for cross-cutting prioritization.

Training-Inference Consistency

• Support R3 routing replay: #39701

• Fix logprobs / logits surface consistency: #37737

Runtime State Switching

• Standardize weight sync lifecycle: #31848

• Make pause / resume coordinator-safe: #32103

• NCCL context offload / resume

Rollout Performance / Efficiency

• Improve KV cache / prefix reuse: #40244

• Stabilize P/D rollout throughput: verl-project/verl#6243

• Add phase-aware performance modes to dynamically switch between throughput-optimized and latency-optimized configs

• More mature FP8 W8A8 KV-cache rollout support

• RDMA-based cross-cluster transport for vLLM intermediate results — vLLM internally produces large per-request artifacts (expert routing indices are one example, but it generalizes — any large per-step or per-layer signal a downstream RL system might need). We need a generic plugin mechanism to export these artifacts. Doing it over RDMA in a distributed fashion (peer-to-peer between vLLM workers and downstream consumers) instead of going through a host-side aggregation buffer scales much better when multiple nodes pull at once

Framework & Workload Enablement

• Stabilize RL framework serving contract: verl-project/verl#5737

• Support multimodal RL: verl-project/verl#5916

• Support teacher / OPD server pluggability: verl-project/verl#5897

Misc

• Add rollout liveness and debug signals: #38147

• Publish more up-to-date vLLM Docker for RL Framework

Sync / Discussion

• Slack channel: #sig-reinforcement-learning on the vLLM Slack workspace

• Weekly call: every Friday 06:30 Beijing / 15:30 Pacific (Thursday) PDT / 18:30 Eastern (Thursday) EDT — https://meet.google.com/hpi-znch-gcx?hs=224

• New contributors / observers welcome — leave a comment on this thread or ping in Slack to be added to the agenda