PR #41423: [Bugfix] Fix spawn_new_process_for_each_test silently swallowing test failures

• Repository: vllm-project/vllm
• Author: dzhengAP
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/41423

Description (problem / solution / changelog)

Problem

spawn_new_process_for_each_test was broken — it always passed regardless of what the test function did, causing silent test coverage failures.

The previous implementation ran python -m <module_name> in the child process, which re-executed the module's __main__ block instead of actually calling the test function. check_returncode() always saw exit 0, so any exception raised inside the test was silently swallowed.

Repro from the issue:

@create_new_process_for_each_test("spawn")
def test_failing():
    raise ValueError  # always passed before this fix

Fix

Serialize the test function and its arguments with cloudpickle and pass them to a minimal child script via stdin. The child writes its full traceback to a temp file on failure; the parent reads it and raises RuntimeError with the traceback included, making CI failures actionable.

This matches the robustness of fork_new_process_for_each_test which already handled exception propagation correctly.

Verification

Tested locally with the exact repro from #41415:

Testing failure propagation...
FIXED: failure correctly propagated
Test subprocess 'test_failing' failed (exit code 1):
Traceback (most recent call last):
  File "<string>", line 4, in <module>
    f(*args, **kwargs)
  File "/tmp/test_fix3.py", line 45, in test_failing
    raise ValueError('should propagate')
ValueError: should propagate

Testing success case...
FIXED: success case passes cleanly

• Failing tests now correctly raise RuntimeError with the full traceback from the child process
• Passing tests continue to work normally

Fixes #41415

Changed files

• tests/test_spawn_decorator.py (added, +38/-0)
• tests/utils.py (modified, +49/-35)

PR #41425: [Bugfix] Fix spawn decorator never executing the test function

• Repository: vllm-project/vllm
• Author: sriharshamudumba
• State: open | merged: False
• Link: https://github.com/vllm-project/vllm/pull/41425

Description (problem / solution / changelog)

Purpose

Fix spawn_new_process_for_each_test always passing tests regardless of whether the test function raised an exception. Fixes #41415, spotted in #36823.

Two bugs in the decorator:

1. Subprocess was launched with python -m <module> which only imports the module and exits. stdin was never read so f was never called.
2. args and kwargs were missing from the pickled payload, so parametrized tests would not receive their arguments.

Test Plan

bash python -c " from tests.utils import create_new_process_for_each_test

@create_new_process_for_each_test('spawn') def test_failing(): raise ValueError('intentional failure')

try: test_failing() print('BROKEN: passed when it should have failed') except RuntimeError: print('FIXED: correctly raised RuntimeError')

Test Result

Testing failure propagation... FIXED: failure correctly propagated

Testing success case... FIXED: success case passes cleanly

Testing args and kwargs... FIXED: args and kwargs forwarded correctly

Changed files

• tests/utils.py (modified, +22/-27)