vllm - ✅(Solved) Fix [ROCm][CI]: `create_new_process_for_each_test("spawn")` may silently skip tests without `main` entrypoint [1 pull requests, 1 comments, 2 participants]

AndreasKaratzas · 2026-03-25T10:31:26Z

[vllm] create new process for each test "spawn" runs tests by invoking python -m ` in a subprocess. If the test file has no `if __name__ == "__main__":` block, the subprocess exits successfully without running any tests, and the decorator interprets this as a pass. So, tests silently appear to pass when they are not actually executed. This affects ROCm since the default method is `"spawn"` when no argument is provided - Related issue: #34323 cc @DarkLight1337 # PR #38335: [ROCm][CI] Fix spawn test decorator silently skipping tests without __main__ - Repository: vllm-project/vllm - Author: westers - State: open | merged: False - Link: https://github.com/vllm-project/vllm/pull/38335 ## Description (problem / solution / changelog) ## Summary - `create_new_process_for_each_test("spawn")` ran tests via `python -m `, which requires test files to have `if __name__ == "__main__":` blocks - Without that block, the subprocess exits 0 without running the test, and the decorator treats it as a pass — tests silently never execute - This is especially impactful on ROCm since `spawn` is the default method (`use_spawn = current_platform.is_rocm()`) ## Fix Replace `python -m ` with `python -c` using an inline script that reads the cloudpickle-serialized test function from stdin and calls it directly. The function was already being serialized and piped — this just ensures it's actually invoked regardless of the file's `__main__` block. Fixes #38097 ## Test plan - [ ] Run existing spawn-based tests on ROCm CI — should still pass - [ ] Verify that a test file without `__main__` now properly executes (instead of silently passing) - [ ] Run on CUDA CI to ensure no regression (CUDA uses fork, not spawn) 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Changed files - `tests/utils.py` (modified, +11/-4) ## Fix / Workaround 1. Fix the spawn implementation and instead of running `python -m `, use a wrapper entrypoint that deserializes and calls the test function directly (similar to how `fork` works). This would eliminate the `__main__` requirement entirely. 2. Add `__main__` blocks to all 36 affected test files, which is quick workaround but doesn't prevent future regressions. ### Summary `create_new_process_for_each_test("spawn")` runs tests by invoking `python -m ` in a subprocess. If the test file has no `if __name__ == "__main__":` block, the subprocess exits successfully without running any tests, and the decorator interprets this as a pass. So, tests silently appear to pass when they are not actually executed. This affects ROCm since the default method is `"spawn"` when no argument is provided - Related issue: #34323 cc @DarkLight1337 ### Root cause In [`tests/utils.py:1405`](https://github.com/vllm-project/vllm/blob/main/tests/utils.py#L1405), the spawn path runs: ```python cmd = [sys.executable, "-m", f"{module_name}"] returned = subprocess.run(cmd, input=input_bytes, capture_output=True, env=env) returned.check_returncode() # exit code 0 (means "passed") ``` When a test file has no `__main__` block, `python -m tests.some.test_file` imports the module, does nothing, and exits with code 0. The decorator sees a successful return code and considers the test passed. The `"fork"` path does not have this problem because it calls the test function directly in the forked process. ### Possible fixes 1. Fix the spawn implementation and instead of running `python -m `, use a wrapper entrypoint that deserializes and calls the test function directly (similar to how `fork` works). This would eliminate the `__main__` requirement entirely. 2. Add `__main__` blocks to all 36 affected test files, which is quick workaround but doesn't prevent future regressions.

vllm2026-03-25 10:31:26

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

vllm-project/vllm#38097•Fetched 2026-04-08 01:26:41

View on GitHub

Comments

Participants

Timeline

Reactions

Author

AndreasKaratzas

Participants

AndreasKaratzas

github-actions[bot]

Timeline (top)

mentioned ×4subscribed ×4added_to_project_v2 ×2labeled ×2

create_new_process_for_each_test("spawn") runs tests by invoking python -m <module_name> in a subprocess. If the test file has no if __name__ == "__main__": block, the subprocess exits successfully without running any tests, and the decorator interprets this as a pass. So, tests silently appear to pass when they are not actually executed.

This affects ROCm since the default method is "spawn" when no argument is provided

Related issue: #34323

cc @DarkLight1337

Root Cause

In tests/utils.py:1405, the spawn path runs:

cmd = [sys.executable, "-m", f"{module_name}"]
returned = subprocess.run(cmd, input=input_bytes, capture_output=True, env=env)
returned.check_returncode()  # exit code 0 (means "passed")

When a test file has no __main__ block, python -m tests.some.test_file imports the module, does nothing, and exits with code 0. The decorator sees a successful return code and considers the test passed.

The "fork" path does not have this problem because it calls the test function directly in the forked process.

Fix Action

Fix / Workaround

Fix the spawn implementation and instead of running python -m <module>, use a wrapper entrypoint that deserializes and calls the test function directly (similar to how fork works). This would eliminate the __main__ requirement entirely.
Add __main__ blocks to all 36 affected test files, which is quick workaround but doesn't prevent future regressions.

PR fix notes

PR #38335: [ROCm][CI] Fix spawn test decorator silently skipping tests without main

Repository: vllm-project/vllm
Author: westers
State: open | merged: False
Link: https://github.com/vllm-project/vllm/pull/38335

Description (problem / solution / changelog)

Summary

create_new_process_for_each_test("spawn") ran tests via python -m <module>, which requires test files to have if __name__ == "__main__": blocks
Without that block, the subprocess exits 0 without running the test, and the decorator treats it as a pass — tests silently never execute
This is especially impactful on ROCm since spawn is the default method (use_spawn = current_platform.is_rocm())

Fix

Replace python -m <module> with python -c using an inline script that reads the cloudpickle-serialized test function from stdin and calls it directly. The function was already being serialized and piped — this just ensures it's actually invoked regardless of the file's __main__ block.

Fixes #38097

Test plan

Run existing spawn-based tests on ROCm CI — should still pass
Verify that a test file without __main__ now properly executes (instead of silently passing)
Run on CUDA CI to ensure no regression (CUDA uses fork, not spawn)

🤖 Generated with Claude Code

Changed files

tests/utils.py (modified, +11/-4)

Code Example

cmd = [sys.executable, "-m", f"{module_name}"]
returned = subprocess.run(cmd, input=input_bytes, capture_output=True, env=env)
returned.check_returncode()  # exit code 0 (means "passed")

RAW_BUFFERClick to expand / collapse

Summary

This affects ROCm since the default method is "spawn" when no argument is provided

Related issue: #34323

cc @DarkLight1337

Root cause

In tests/utils.py:1405, the spawn path runs:

cmd = [sys.executable, "-m", f"{module_name}"]
returned = subprocess.run(cmd, input=input_bytes, capture_output=True, env=env)
returned.check_returncode()  # exit code 0 (means "passed")

The "fork" path does not have this problem because it calls the test function directly in the forked process.

Possible fixes

Fix the spawn implementation and instead of running python -m <module>, use a wrapper entrypoint that deserializes and calls the test function directly (similar to how fork works). This would eliminate the __main__ requirement entirely.
Add __main__ blocks to all 36 affected test files, which is quick workaround but doesn't prevent future regressions.

extent analysis

Fix Plan

To address the issue, we will implement a wrapper entrypoint to deserialize and call the test function directly in the spawn implementation. This approach eliminates the need for an __main__ block in test files.

Step-by-Step Solution:

Create a wrapper module: Create a new Python module, e.g., test_wrapper.py, with a function that deserializes and calls the test function.
Modify the spawn implementation: Update the spawn path in tests/utils.py to use the test_wrapper module instead of running python -m <module_name> directly.
Update the test execution: Modify the subprocess.run call to execute the test_wrapper module with the necessary arguments.

Example Code:

# test_wrapper.py
import importlib
import sys

def run_test(module_name, test_name):
    module = importlib.import_module(module_name)
    test_func = getattr(module, test_name)
    test_func()

# tests/utils.py
# ...

def create_new_process_for_each_test(method):
    # ...
    if method == "spawn":
        # Create a wrapper command
        cmd = [sys.executable, "-m", "test_wrapper", module_name, test_name]
        returned = subprocess.run(cmd, input=input_bytes, capture_output=True, env=env)
        returned.check_returncode()
    # ...

Verification

To verify the fix, run the tests with the spawn method and ensure that tests without an __main__ block are executed correctly and reported as failed if they contain assertions or other test code.

Extra Tips

Consider adding a check to ensure that all test files have a corresponding test function in the test_wrapper module to prevent future regressions.
Review the test_wrapper module to ensure it handles edge cases, such as test functions with arguments or complex setup/teardown logic.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#prompt issue #agent setup #task chaining #parallel task #integration issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

vllm - ✅(Solved) Fix [ROCm][CI]: `create_new_process_for_each_test("spawn")` may silently skip tests without `main` entrypoint [1 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fix / Workaround

PR fix notes

PR #38335: [ROCm][CI] Fix spawn test decorator silently skipping tests without main

Description (problem / solution / changelog)

Summary

Fix

Test plan

Changed files

Code Example

Summary

Root cause

Possible fixes

extent analysis

Fix Plan

Step-by-Step Solution:

Example Code:

Verification

Extra Tips

Still need to ship something?

TRENDING

vllm - ✅(Solved) Fix [ROCm][CI]: `create_new_process_for_each_test("spawn")` may silently skip tests without `__main__` entrypoint [1 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fix / Workaround

PR fix notes

PR #38335: [ROCm][CI] Fix spawn test decorator silently skipping tests without main

Description (problem / solution / changelog)

Summary

Fix

Test plan

Changed files

Code Example

Summary

Root cause

Possible fixes

extent analysis

Fix Plan

Step-by-Step Solution:

Example Code:

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING

vllm - ✅(Solved) Fix [ROCm][CI]: `create_new_process_for_each_test("spawn")` may silently skip tests without `main` entrypoint [1 pull requests, 1 comments, 2 participants]