pytorch - 💡(How to fix) Fix [vllm] [2.12 regression] Sequence Parallel test_tp_sp_generation: RayChannelTimeoutError on tp=2 setups (Llama) [2 comments, 1 participants]

Summary

Under torch 2.12.0 + triton 3.7.0, vLLM's test_tp_sp_generation fails on three parametrized configurations because the Ray-backed engine times out waiting for an inter-process object:

ray.exceptions.RayChannelTimeoutError: System error: Timed out waiting for object available to read.

Three parametrizations fail (tp_size=2, ray distributed backend):

test_tp_sp_generation[False-True-hmellor/tiny-random-LlamaForCausalLM-parallel_setup1-ray-auto-test_options1]
test_tp_sp_generation[False-True-hmellor/tiny-random-LlamaForCausalLM-parallel_setup3-ray-auto-test_options3]
test_tp_sp_generation[False-True-RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-parallel_setup17-ray-auto-test_options17]

The pytest-side error is the wrapper:

AssertionError: function test_tp_sp_generation failed when called with args () and kwargs {'model_id': 'hmellor/tiny-random-LlamaForCausalLM', 'parallel_setup': ParallelSetup(tp_size=2, pp_size=1, ...), 'distributed_backend': 'ray', ...}

Passes on torch 2.11. Blocking the torch 2.12 upgrade for vLLM (vllm-project/vllm#40077).

Environment

• torch: 2.12.0+cu130 (test channel)
• triton: 3.7.0
• CUDA: 13.0
• Python: 3.12.13
• GPU: 2× NVIDIA H100 (test name says 2xH100)
• Distributed backend: ray
• Models: hmellor/tiny-random-LlamaForCausalLM, RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8

Traceback (abridged)

File "vllm/v1/engine/core.py", line 1129, in run_engine_core
  engine_core.run_busy_loop()
...
File "python/ray/_raylet.pyx", line 3194, in ray._raylet.CoreWorker.get_objects
File "python/ray/includes/common.pxi", line 106, in ray._raylet.check_status
ray.exceptions.RayChannelTimeoutError: System error: Timed out waiting for object available to read.
ObjectID: 003d5d11e415fe881ac38fca360ef9053beab8b0010000000ae1f505

Reproducibility

• torch-2.12 branch: failed https://buildkite.com/vllm/ci/builds/63095#019dcf15-7f5c-4f36-a0fb-53e4cf05b5e7
• main (torch 2.11) — passes on every recent build:
  • 2026-04-25 daily: https://buildkite.com/vllm/ci/builds/62981 (job was interrupted by signal — not the same RayChannelTimeoutError signature)
  • 2026-04-26 nightly: https://buildkite.com/vllm/ci/builds/62990
  • 2026-04-26 daily: https://buildkite.com/vllm/ci/builds/63026
  • 2026-04-27 nightly: https://buildkite.com/vllm/ci/builds/63061

Diagnosis request

RayChannelTimeoutError on tp=2 ray-backed engine startup suggests something in the engine's worker-to-worker communication path is now significantly slower or hanging on torch 2.12 — possibly compile/AOT-cache time spent on each worker before they synchronize. Could a maintainer (or the Ray + torch.compile interaction owner) look at whether torch 2.12 introduces additional per-worker compilation latency that pushes Ray's default channel timeout over the limit?

Links

• vLLM PR: vllm-project/vllm#40077
• Umbrella: pytorch/pytorch#180899

cc @chauhang @penguinwu @oulgen @jamesjwu @aorenste @anijain2305 @laithsakka @masnesral @coconutruben @aditvenk