PR #40096: [Frontend][Core] Add sparse NCCL weight transfer support for in-place updates

• Repository: vllm-project/vllm
• Author: bedeks
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/40096

Description (problem / solution / changelog)

Purpose

Implements an MVP sparse NCCL weight transfer path for online RL weight sync. Instead of resending full dense tensors, the trainer can send (indices, values) patches that are applied in-place to existing runtime GPU parameters.

This addresses #39451.

Current scope:

• NCCL backend only
• sparse updates use kernel-format/runtime parameter names
• not composable with packed=True
• not composable with is_checkpoint_format=True
• restricted to TP=1, PP=1

Why this is not duplicating an existing PR:

• I checked issue #39451 and open PRs for the same area; no open PR was already implementing this sparse NCCL MVP.

Test Plan

.venv/bin/python -m pytest tests/distributed/test_weight_transfer.py -v -k 'valid_sparse_update_info or sparse_update_requires_nnz_list or sparse_update_rejects_checkpoint_format or sparse_update_rejects_packed or sparse_update_rejects_non_int32_indices or dense_update_rejects_sparse_metadata or nccl_receive_sparse_weights_without_init_raises or nccl_sparse_weight_transfer_between_processes or sparse_update_kind_rejected'

.venv/bin/python -m pytest tests/entrypoints/weight_transfer/test_weight_transfer_llm.py -v -k 'test_update_weights_passes_sparse_metadata'

.venv/bin/python -m pytest tests/v1/worker/test_gpu_model_runner.py -v -k 'apply_sparse_weight_patches'

.venv/bin/python -m pytest tests/v1/worker/test_gpu_worker_weight_transfer.py -v -k 'sparse_dispatches or sparse_rejects_tp_or_pp'

Test Result

• tests/distributed/test_weight_transfer.py

  • 9 passed, 27 deselected in 13.59s
  • includes test_nccl_sparse_weight_transfer_between_processes on a 2-GPU pod

• tests/entrypoints/weight_transfer/test_weight_transfer_llm.py

  • 1 passed, 5 deselected in 28.24s

• tests/v1/worker/test_gpu_model_runner.py

  • 3 passed, 25 deselected in 2.40s

• tests/v1/worker/test_gpu_worker_weight_transfer.py

  • 2 passed in 2.17s

Additional Validation

Outside this PR branch, I also ran temporary repro/debug harnesses on a 2-GPU pod to validate dense-vs-sparse equivalence for the same deterministic patch on Qwen/Qwen3-1.7B.

Observed results:

• trainer patch digests matched
• full server-side parameter digest maps matched
• controlled max_tokens=1, greedy outputs matched between dense and sparse updates

Performance validation:

• for a patch affecting ~0.3% of model elements on Qwen/Qwen3-1.7B, the sparse payload was ~30.97 MB versus ~3.44 GB for the dense full-model resend path
• in one pod validation run, trainer-side send time decreased from ~175 ms for dense resend to ~4 ms for sparse patch transfer

This validation is supplementary and is not part of the submitted branch.

AI assistance was used to help develop and validate this change.

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Changed files

• tests/distributed/test_weight_transfer.py (modified, +247/-0)
• tests/entrypoints/weight_transfer/test_weight_transfer_llm.py (modified, +67/-0)
• tests/v1/worker/test_gpu_model_runner.py (modified, +69/-0)
• tests/v1/worker/test_gpu_worker_weight_transfer.py (added, +99/-0)
• vllm/distributed/weight_transfer/base.py (modified, +16/-1)
• vllm/distributed/weight_transfer/ipc_engine.py (modified, +4/-0)
• vllm/distributed/weight_transfer/nccl_engine.py (modified, +95/-0)
• vllm/v1/worker/gpu_model_runner.py (modified, +39/-0)
• vllm/v1/worker/gpu_worker.py (modified, +21/-2)