vllm - 💡(How to fix) Fix [Bug]: Mimo v2.5 model loading fails from s3/remote locations [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#41172Fetched 2026-04-29 06:11:56
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Participants
Timeline (top)
labeled ×1renamed ×1

Error Message

(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start. (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last): (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] worker = WorkerProc(*args, **kwargs) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in init (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.worker.load_model() (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.model_runner.load_model(load_dummy_weights=load_dummy_weights) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.model = model_loader.load_model( (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] model = initialize_model( (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] model = model_class(vllm_config=vllm_config, prefix=prefix) (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in init (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.audio_encoder = MimoAudioEncoder( (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in init (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.audio_tokenizer = self._load_audio_tokenizer( (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] raise FileNotFoundError( (Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start. (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last): (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] worker = WorkerProc(*args, **kwargs) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in init (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.worker.load_model() (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.model_runner.load_model(load_dummy_weights=load_dummy_weights) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.model = model_loader.load_model( (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] model = initialize_model( (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] return func(*args, **kwargs) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] model = model_class(vllm_config=vllm_config, prefix=prefix) (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in init (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.audio_encoder = MimoAudioEncoder( (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in init (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] self.audio_tokenizer = self._load_audio_tokenizer( (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] raise FileNotFoundError( (Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin)

Root Cause

When trying to load mimo v2.5 from s3 location, we get a failure because of the /audioTokenizer folder not being downloaded to local disk/streamed. This is the same if we use runai_streamer or not.

Code Example

vllm serve s3://path-to-mimo/mimo-v2.5 \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --generation-config vllm

---

(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     worker = WorkerProc(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.worker.load_model()
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model = model_loader.load_model(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = initialize_model(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_encoder = MimoAudioEncoder(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                          ^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_tokenizer = self._load_audio_tokenizer(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     raise FileNotFoundError(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     worker = WorkerProc(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.worker.load_model()
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model = model_loader.load_model(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = initialize_model(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_encoder = MimoAudioEncoder(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                          ^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_tokenizer = self._load_audio_tokenizer(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     raise FileNotFoundError(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin)
RAW_BUFFERClick to expand / collapse

Your current environment

using the mimov2.5 docker container: vllm/vllm-openai:mimov25-cu129

🐛 Describe the bug

When trying to load mimo v2.5 from s3 location, we get a failure because of the /audioTokenizer folder not being downloaded to local disk/streamed. This is the same if we use runai_streamer or not.

vllm serve s3://path-to-mimo/mimo-v2.5 \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --generation-config vllm

Fails with:

(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     worker = WorkerProc(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.worker.load_model()
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model = model_loader.load_model(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = initialize_model(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_encoder = MimoAudioEncoder(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                          ^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in __init__
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_tokenizer = self._load_audio_tokenizer(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     raise FileNotFoundError(
(Worker_TP0 pid=1083) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] WorkerProc failed to start.
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] Traceback (most recent call last):
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 837, in worker_main
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     worker = WorkerProc(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 619, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.worker.load_model()
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 4793, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.model = model_loader.load_model(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                  ^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = initialize_model(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     return func(*args, **kwargs)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_v2_omni.py", line 1220, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_encoder = MimoAudioEncoder(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                          ^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1243, in __init__
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     self.audio_tokenizer = self._load_audio_tokenizer(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]   File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/mimo_audio.py", line 1269, in _load_audio_tokenizer
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870]     raise FileNotFoundError(
(Worker_TP1 pid=1084) ERROR 04-28 22:40:19 [multiproc_executor.py:870] FileNotFoundError: No model weights found in /home/ray/.cache/vllm/assets/model_streamer/123ab828/audio_tokenizer (expected model.safetensors or pytorch_model.bin)

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The issue is likely due to the /audio_tokenizer folder not being downloaded to the local disk, causing a FileNotFoundError when trying to load the model weights.

Guidance

  • Verify that the audio_tokenizer folder exists in the S3 location s3://path-to-mimo/mimo-v2.5 and that it contains the expected model weights (model.safetensors or pytorch_model.bin).
  • Check the permissions and accessibility of the S3 location to ensure that the Docker container has the necessary access to download the model weights.
  • Consider adding a flag or option to the vllm serve command to specify the location of the audio_tokenizer folder or to download it manually before running the command.
  • If using runai_streamer, check its configuration to ensure that it is properly set up to stream the audio_tokenizer folder.

Example

No code snippet is provided as the issue seems to be related to the configuration and accessibility of the model weights rather than a code error.

Notes

The error message indicates that the audio_tokenizer folder is not being downloaded to the local disk, which suggests a configuration or accessibility issue rather than a code error. Further investigation is needed to determine the root cause of the issue.

Recommendation

Apply a workaround by manually downloading the audio_tokenizer folder to the local disk and specifying its location in the vllm serve command, or by configuring runai_streamer to properly stream the folder.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

vllm - 💡(How to fix) Fix [Bug]: Mimo v2.5 model loading fails from s3/remote locations [1 participants]