vllm - 💡(How to fix) Fix [Bug]: vLLM fails to start with LMCache + Qwen3-Coder-Next-FP8 (nightly image) [1 participants]

(Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] Traceback (most recent call last): (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 944, in worker_busy_loop (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] output = func(*args, **kwargs) (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] return func(*args, **kwargs) (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 381, in determine_available_memory (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] cudagraph_memory_estimate = self.model_runner.profile_cudagraph_memory() (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] return func(*args, **kwargs) (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5873, in profile_cudagraph_memory (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] self._init_minimal_kv_cache_for_profiling() (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5813, in _init_minimal_kv_cache_for_profiling (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] kv_cache_groups = get_kv_cache_groups(self.vllm_config, kv_cache_spec) (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1236, in get_kv_cache_groups (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] unify_hybrid_kv_cache_specs(kv_cache_spec) (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/core/kv_cache_utils.py", line 1216, in unify_hybrid_kv_cache_specs (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] raise ValueError( (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] ValueError: Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type. (Worker_TP0 pid=1143) ERROR 04-01 07:17:10 [multiproc_executor.py:949] (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] EngineCore failed to start. (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] Traceback (most recent call last): (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 1082, in run_engine_core (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs) (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] return func(*args, **kwargs) (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 848, in init (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] super().init( (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 124, in init (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] kv_cache_config = self._initialize_kv_caches(vllm_config) (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] return func(*args, **kwargs) (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/engine/core.py", line 247, in _initialize_kv_caches (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] available_gpu_memory = self.model_executor.determine_available_memory() (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/abstract.py", line 136, in determine_available_memory (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] return self.collective_rpc("determine_available_memory") (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 397, in collective_rpc (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] return aggregate(get_response()) (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] ^^^^^^^^^^^^^^ (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 380, in get_response (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] raise RuntimeError( (EngineCore pid=603) ERROR 04-01 07:17:10 [core.py:1108] RuntimeError: Worker failed with error 'Hybrid KV cache manager is disabled but failed to convert the KV cache specs to one unified type.', please check the stack trace above for the root cause (Worker_TP0 pid=1143) WARNING 04-01 07:17:10 [multiproc_executor.py:871] WorkerProc was terminated (Worker_TP1 pid=1144) WARNING 04-01 07:17:10 [multiproc_executor.py:871] WorkerProc was terminated (EngineCore pid=603) ERROR 04-01 07:17:13 [multiproc_executor.py:273] Worker proc VllmWorker-0 died unexpectedly, shutting down executor.

TL;DR

The error message indicates a failure to convert KV cache specs to a unified type, suggesting an issue with the KV cache configuration, which can be resolved by adjusting the kv_transfer_config argument.

Guidance

• Review the kv_transfer_config argument to ensure it is correctly formatted and compatible with the LMCacheConnectorV1 connector.
• Verify that the kv_role is set to a valid value, such as "kv_both", and that the kv_connector is correctly specified as "LMCacheConnectorV1".
• Consider checking the documentation for the LMCacheConnectorV1 connector to ensure that the configuration is correct and up-to-date.
• If using a hybrid KV cache manager, ensure that it is properly enabled and configured.

Example

No code example is provided as the issue seems to be related to configuration rather than code.

Notes

The fact that GLM-4.7-flash with lmcache/vllm-openai works successfully suggests that the issue may be specific to the current configuration or environment.

Recommendation

Apply a workaround by adjusting the kv_transfer_config argument to ensure compatibility with the LMCacheConnectorV1 connector. This may involve updating the configuration to match the requirements of the connector.