vllm - 💡(How to fix) Fix [Bug]: # vLLM v0.19.0 - WorkerProc initialization failed with Tensor Parallelism [2 comments, 2 participants]

(EngineCore pid=335) ERROR 04-11 22:28:52 [core.py:1108] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause. (EngineCore pid=335) Traceback (most recent call last): File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in self._target(*self._args, **self._kwargs) ... (APIServer pid=1) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

TL;DR

The issue can be potentially resolved by adjusting the configuration to use Pipeline Parallelism (PP) instead of Tensor Parallelism (TP) or investigating the compatibility of the model with TP.

Guidance

• Verify that the model is compatible with Tensor Parallelism (TP) by checking the documentation or testing with a smaller model.
• Try reducing the tensor-parallel-size to a smaller value (e.g., 2) to see if the issue persists.
• Use Pipeline Parallelism (PP) as a workaround, as mentioned in the issue, by setting --pipeline-parallel-size 3 and adjusting the GPU allocation accordingly.
• Check the GPU memory utilization and adjust the --gpu-memory-utilization parameter to ensure it's not causing issues with the model loading.

Example

No code snippet is provided as the issue seems to be related to configuration and model compatibility rather than code.

Notes

The issue might be specific to the Qwen3-Coder-Next-REAM-AWQ-4bit model or the vLLM version 0.19.0. Further investigation into the model's compatibility with Tensor Parallelism or adjusting the configuration might be necessary.

Recommendation

Apply the workaround by using Pipeline Parallelism (PP) instead of Tensor Parallelism (TP), as it has been reported to work for the user. This allows the model to load successfully, albeit with a different parallelism strategy.