vllm - ✅(Solved) Fix [Bug]: [CPU] AttributeError: '_OpNamespace' '_C' object has no attribute 'gelu_tanh_and_mul' [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
vllm-project/vllm#39334Fetched 2026-04-09 07:51:51
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
Participants
Timeline (top)
cross-referenced ×1labeled ×1

Error Message

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it ... (APIServer pid=674180) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig (APIServer pid=674180) Value error, The checkpoint you are trying to load has model type gemma4 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. (APIServer pid=674180) (APIServer pid=674180) You can update Transformers with the command pip install --upgrade transformers. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command pip install git+https://github.com/huggingface/transformers.git [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs] (APIServer pid=674180) For further information visit https://errors.pydantic.dev/2.12/v/value_error

Root Cause

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it
...
(APIServer pid=674180) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=674180)   Value error, The checkpoint you are trying to load has model type `gemma4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
(APIServer pid=674180)
(APIServer pid=674180) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=674180)     For further information visit https://errors.pydantic.dev/2.12/v/value_error

Fix Action

Fixed

PR fix notes

PR #39342: [CPU] Fix AttributeError when loading GeluAndMul and similar activations

Description (problem / solution / changelog)

When the vLLM CPU C++ extension fails to load (e.g. due to a PyTorch ABI mismatch), torch.ops._C activation ops like gelu_tanh_and_mul are never registered, causing GeluAndMul, NewGELU, FastGELU, and QuickGELU to crash at model init with an AttributeError. The fix removes is_cpu() from the op-access guard in each class since the CPU forward path already falls back to the native PyTorch implementation through forward_cpu --> forward_native, so self.op was dead code on CPU anyway. Added a regression test that verifies the classes initialize and produce correct output on CPU even when the C extension is absent. Fixes #39334

Changed files

  • tests/kernels/core/test_activation.py (modified, +44/-0)
  • vllm/model_executor/layers/activation.py (modified, +4/-20)

Code Example

Collecting environment information...
uv is set
==============================
        System Info
==============================
OS                           : x86_64
GCC version                  : (GCC) 13.3.1 20240611 (Red Hat 13.3.1-2.2.0.1)
Clang version                : 12.0.1 (Red Hat 12.0.1-4.0.1.module+el8.5.0+20428+2b4ecd47)
CMake version                : version 4.3.1
Libc version                 : glibc-2.28

==============================
       PyTorch Info
==============================
PyTorch version              : 2.10.0+cpu
Is debug build               : False
CUDA used to build PyTorch   : None
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.12 (main, Dec 17 2025, 21:10:06) [Clang 21.1.4 ] (64-bit runtime)
Python platform              : Linux-5.15.0-315.193.2.el8uek.x86_64-x86_64-with-glibc2.28

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : False
CUDA runtime version         : No CUDA
CUDA_MODULE_LOADING set to   : N/A
GPU models and configuration : No CUDA
Nvidia driver version        : No CUDA
cuDNN version                : No CUDA
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC 7742 64-Core Processor
Stepping:            0
CPU MHz:             2245.778
BogoMIPS:            4491.55
Virtualization:      AMD-V
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid amd_dcm tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 nt_good clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities

==============================
Versions of relevant libraries
==============================
[pip3] numpy==2.4.4
[pip3] pyzmq==27.1.0
[pip3] torch==2.10.0+cpu
[pip3] torchaudio==2.11.0+cpu
[pip3] torchvision==0.25.0+cpu
[pip3] transformers==5.5.0
[pip3] triton==3.6.0
[conda] _anaconda_depends         2024.10             py312_mkl_0
[conda] blas                      1.0                         mkl
[conda] intel-extension-for-pytorch 2.7.0                    pypi_0    pypi
[conda] mkl                       2023.1.0         h213fc3f_46344
[conda] mkl-service               2.4.0           py312h5eee18b_1
[conda] mkl_fft                   1.3.10          py312h5eee18b_0
[conda] mkl_random                1.2.7           py312h526ad5a_0
[conda] numpy                     1.26.4          py312hc5e2394_0
[conda] numpy-base                1.26.4          py312h0da6c21_0
[conda] numpydoc                  1.7.0           py312h06a4308_0
[conda] nvidia-cublas-cu12        12.6.4.1                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.6.80                  pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.6.77                  pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.6.77                  pypi_0    pypi
[conda] nvidia-cudnn-cu12         9.5.1.17                 pypi_0    pypi
[conda] nvidia-cufft-cu12         11.3.0.4                 pypi_0    pypi
[conda] nvidia-cufile-cu12        1.11.1.6                 pypi_0    pypi
[conda] nvidia-curand-cu12        10.3.7.77                pypi_0    pypi
[conda] nvidia-cusolver-cu12      11.7.1.2                 pypi_0    pypi
[conda] nvidia-cusparse-cu12      12.5.4.2                 pypi_0    pypi
[conda] nvidia-cusparselt-cu12    0.6.3                    pypi_0    pypi
[conda] nvidia-nccl-cu12          2.26.2                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.85                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.6.77                  pypi_0    pypi
[conda] pyzmq                     25.1.2          py312h6a678d5_0
[conda] torch                     2.7.1                    pypi_0    pypi
[conda] torchaudio                2.7.0+cpu                pypi_0    pypi
[conda] torchvision               0.22.0+cpu               pypi_0    pypi
[conda] transformers              4.52.3                   pypi_0    pypi
[conda] triton                    3.3.1                    pypi_0    pypi

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.19.1rc1.dev56+g419e73cdf (git sha: 419e73cdf)
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled
GPU Topology:
  Could not collect

==============================
     Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1

---

uv pip install -r requirements/cpu-build.txt --torch-backend cpu
uv pip install -r requirements/cpu.txt --torch-backend cpu
VLLM_TARGET_DEVICE=cpu uv pip install . --no-build-isolation

---

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it
...
(APIServer pid=674180) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=674180)   Value error, The checkpoint you are trying to load has model type `gemma4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
(APIServer pid=674180)
(APIServer pid=674180) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=674180)     For further information visit https://errors.pydantic.dev/2.12/v/value_error

---

(gemma4) (base) bash-4.4$ uv pip install --upgrade transformers
...
Installed 3 packages in 4.35s
 - huggingface-hub==0.36.2
 + huggingface-hub==1.9.0
 - numpy==2.2.6
 + numpy==2.4.4
 - transformers==4.57.6
 + transformers==5.5.0

---

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it
INFO 04-06 14:37:54 [importing.py:44] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors.
INFO 04-06 14:37:54 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 04-06 14:37:54 [cpu.py:515] Failed to import from vllm._C_AVX2: ImportError('/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/_C_AVX2.abi3.so: undefined symbol: _ZN3c1013MessageLoggerC1EPKcii')
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]        █     █     █▄   ▄█
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.19.1rc1.dev56+g419e73cdf
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]   █▄█▀ █     █     █     █  model   google/gemma-4-E2B-it
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:233] non-default args: {'model_tag': 'google/gemma-4-E2B-it', 'model': 'google/gemma-4-E2B-it'}
(APIServer pid=670199) INFO 04-06 14:37:57 [model.py:554] Resolved architecture: Gemma4ForConditionalGeneration
(APIServer pid=670199) INFO 04-06 14:37:57 [model.py:1684] Using max model len 131072
(APIServer pid=670199) WARNING 04-06 14:37:57 [cpu.py:139] VLLM_CPU_KVCACHE_SPACE not set. Using 62.64 GiB for KV cache.
...
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] WorkerProc failed to start.
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] Traceback (most recent call last):
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 840, in worker_main
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     worker = WorkerProc(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 627, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.worker.load_model()
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/worker/cpu_model_runner.py", line 72, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model = get_model(vllm_config=self.vllm_config)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/__init__.py", line 138, in get_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return loader.load_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = initialize_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/utils.py", line 57, in initialize_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4_mm.py", line 908, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.language_model: Gemma4ForCausalLM = init_vllm_registered_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 379, in init_vllm_registered_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return initialize_model(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/utils.py", line 57, in initialize_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 1392, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model = Gemma4Model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                  ^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/compilation/decorators.py", line 379, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     old_init(self, *args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 924, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.start_layer, self.end_layer, self.layers = make_layers(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                                                     ^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 646, in make_layers
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     + get_offloader().wrap_modules(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/offloader/base.py", line 90, in wrap_modules
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return list(modules_generator)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 647, in <genexpr>
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 926, in <lambda>
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     lambda prefix: Gemma4DecoderLayer(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                    ^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 493, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.mlp = Gemma4MLP(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                ^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 117, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.act_fn = GeluAndMul(approximate="tanh")
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/layers/activation.py", line 277, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.op = torch.ops._C.gelu_tanh_and_mul
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/torch/_ops.py", line 1319, in __getattr__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     raise AttributeError(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] AttributeError: '_OpNamespace' '_C' object has no attribute 'gelu_tanh_and_mul'
RAW_BUFFERClick to expand / collapse

Your current environment

<details> <summary>The output of <code>python collect_env.py</code></summary>
Collecting environment information...
uv is set
==============================
        System Info
==============================
OS                           : x86_64
GCC version                  : (GCC) 13.3.1 20240611 (Red Hat 13.3.1-2.2.0.1)
Clang version                : 12.0.1 (Red Hat 12.0.1-4.0.1.module+el8.5.0+20428+2b4ecd47)
CMake version                : version 4.3.1
Libc version                 : glibc-2.28

==============================
       PyTorch Info
==============================
PyTorch version              : 2.10.0+cpu
Is debug build               : False
CUDA used to build PyTorch   : None
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.12 (main, Dec 17 2025, 21:10:06) [Clang 21.1.4 ] (64-bit runtime)
Python platform              : Linux-5.15.0-315.193.2.el8uek.x86_64-x86_64-with-glibc2.28

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : False
CUDA runtime version         : No CUDA
CUDA_MODULE_LOADING set to   : N/A
GPU models and configuration : No CUDA
Nvidia driver version        : No CUDA
cuDNN version                : No CUDA
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC 7742 64-Core Processor
Stepping:            0
CPU MHz:             2245.778
BogoMIPS:            4491.55
Virtualization:      AMD-V
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid amd_dcm tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 nt_good clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities

==============================
Versions of relevant libraries
==============================
[pip3] numpy==2.4.4
[pip3] pyzmq==27.1.0
[pip3] torch==2.10.0+cpu
[pip3] torchaudio==2.11.0+cpu
[pip3] torchvision==0.25.0+cpu
[pip3] transformers==5.5.0
[pip3] triton==3.6.0
[conda] _anaconda_depends         2024.10             py312_mkl_0
[conda] blas                      1.0                         mkl
[conda] intel-extension-for-pytorch 2.7.0                    pypi_0    pypi
[conda] mkl                       2023.1.0         h213fc3f_46344
[conda] mkl-service               2.4.0           py312h5eee18b_1
[conda] mkl_fft                   1.3.10          py312h5eee18b_0
[conda] mkl_random                1.2.7           py312h526ad5a_0
[conda] numpy                     1.26.4          py312hc5e2394_0
[conda] numpy-base                1.26.4          py312h0da6c21_0
[conda] numpydoc                  1.7.0           py312h06a4308_0
[conda] nvidia-cublas-cu12        12.6.4.1                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.6.80                  pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.6.77                  pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.6.77                  pypi_0    pypi
[conda] nvidia-cudnn-cu12         9.5.1.17                 pypi_0    pypi
[conda] nvidia-cufft-cu12         11.3.0.4                 pypi_0    pypi
[conda] nvidia-cufile-cu12        1.11.1.6                 pypi_0    pypi
[conda] nvidia-curand-cu12        10.3.7.77                pypi_0    pypi
[conda] nvidia-cusolver-cu12      11.7.1.2                 pypi_0    pypi
[conda] nvidia-cusparse-cu12      12.5.4.2                 pypi_0    pypi
[conda] nvidia-cusparselt-cu12    0.6.3                    pypi_0    pypi
[conda] nvidia-nccl-cu12          2.26.2                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.85                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.6.77                  pypi_0    pypi
[conda] pyzmq                     25.1.2          py312h6a678d5_0
[conda] torch                     2.7.1                    pypi_0    pypi
[conda] torchaudio                2.7.0+cpu                pypi_0    pypi
[conda] torchvision               0.22.0+cpu               pypi_0    pypi
[conda] transformers              4.52.3                   pypi_0    pypi
[conda] triton                    3.3.1                    pypi_0    pypi

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.19.1rc1.dev56+g419e73cdf (git sha: 419e73cdf)
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled
GPU Topology:
  Could not collect

==============================
     Environment Variables
==============================
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
</details>

🐛 Describe the bug

1. Build the vllm from the source against the main branch (as of 04/07/2026)

uv pip install -r requirements/cpu-build.txt --torch-backend cpu
uv pip install -r requirements/cpu.txt --torch-backend cpu
VLLM_TARGET_DEVICE=cpu uv pip install . --no-build-isolation

2. Serve Gemma4

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it
...
(APIServer pid=674180) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=674180)   Value error, The checkpoint you are trying to load has model type `gemma4` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
(APIServer pid=674180)
(APIServer pid=674180) You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git` [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]
(APIServer pid=674180)     For further information visit https://errors.pydantic.dev/2.12/v/value_error

3. Upgrade transformers

(gemma4) (base) bash-4.4$ uv pip install --upgrade transformers
...
Installed 3 packages in 4.35s
 - huggingface-hub==0.36.2
 + huggingface-hub==1.9.0
 - numpy==2.2.6
 + numpy==2.4.4
 - transformers==4.57.6
 + transformers==5.5.0

4. Try again

(gemma4) (base) bash-4.4$ vllm serve google/gemma-4-E2B-it
INFO 04-06 14:37:54 [importing.py:44] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors.
INFO 04-06 14:37:54 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 04-06 14:37:54 [cpu.py:515] Failed to import from vllm._C_AVX2: ImportError('/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/_C_AVX2.abi3.so: undefined symbol: _ZN3c1013MessageLoggerC1EPKcii')
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]        █     █     █▄   ▄█
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.19.1rc1.dev56+g419e73cdf
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]   █▄█▀ █     █     █     █  model   google/gemma-4-E2B-it
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:299]
(APIServer pid=670199) INFO 04-06 14:37:56 [utils.py:233] non-default args: {'model_tag': 'google/gemma-4-E2B-it', 'model': 'google/gemma-4-E2B-it'}
(APIServer pid=670199) INFO 04-06 14:37:57 [model.py:554] Resolved architecture: Gemma4ForConditionalGeneration
(APIServer pid=670199) INFO 04-06 14:37:57 [model.py:1684] Using max model len 131072
(APIServer pid=670199) WARNING 04-06 14:37:57 [cpu.py:139] VLLM_CPU_KVCACHE_SPACE not set. Using 62.64 GiB for KV cache.
...
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] WorkerProc failed to start.
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] Traceback (most recent call last):
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 840, in worker_main
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     worker = WorkerProc(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 627, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.worker.load_model()
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 323, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/v1/worker/cpu_model_runner.py", line 72, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model = get_model(vllm_config=self.vllm_config)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/__init__.py", line 138, in get_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return loader.load_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/base_loader.py", line 55, in load_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = initialize_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/utils.py", line 57, in initialize_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4_mm.py", line 908, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.language_model: Gemma4ForCausalLM = init_vllm_registered_model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 379, in init_vllm_registered_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return initialize_model(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/tracing/otel.py", line 178, in sync_wrapper
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return func(*args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/model_loader/utils.py", line 57, in initialize_model
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 1392, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.model = Gemma4Model(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                  ^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/compilation/decorators.py", line 379, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     old_init(self, *args, **kwargs)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 924, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.start_layer, self.end_layer, self.layers = make_layers(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                                                     ^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 646, in make_layers
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     + get_offloader().wrap_modules(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/offloader/base.py", line 90, in wrap_modules
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     return list(modules_generator)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]            ^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 647, in <genexpr>
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     layer_fn(prefix=f"{prefix}.{idx}") for idx in range(start_layer, end_layer)
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 926, in <lambda>
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     lambda prefix: Gemma4DecoderLayer(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                    ^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 493, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.mlp = Gemma4MLP(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                ^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/models/gemma4.py", line 117, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.act_fn = GeluAndMul(approximate="tanh")
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/vllm/model_executor/layers/activation.py", line 277, in __init__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     self.op = torch.ops._C.gelu_tanh_and_mul
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]   File "/scratch/my/uvvenv/gemma4/lib/python3.12/site-packages/torch/_ops.py", line 1319, in __getattr__
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871]     raise AttributeError(
(Worker pid=670627) ERROR 04-06 14:39:30 [multiproc_executor.py:871] AttributeError: '_OpNamespace' '_C' object has no attribute 'gelu_tanh_and_mul'

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

extent analysis

TL;DR

The most likely fix is to ensure that the PyTorch and Transformers versions are compatible with the Gemma4 model.

Guidance

  • Verify that the PyTorch version (2.10.0+cpu) is compatible with the Transformers version (5.5.0) and the Gemma4 model.
  • Check the documentation for the Gemma4 model to ensure that it supports the current PyTorch and Transformers versions.
  • Consider updating PyTorch or Transformers to a version that is known to be compatible with the Gemma4 model.
  • Review the error message to identify the specific attribute ('gelu_tanh_and_mul') that is missing from the '_C' object, which may indicate a version mismatch or an issue with the PyTorch installation.

Example

No specific code example is provided, as the issue appears to be related to version compatibility rather than a code error.

Notes

The error message suggests that there is an issue with the PyTorch installation or version compatibility, rather than a problem with the code itself. It is recommended to review the documentation for the Gemma4 model and the PyTorch and Transformers libraries to ensure that all versions are compatible.

Recommendation

Apply a workaround by checking the version compatibility of PyTorch and Transformers with the Gemma4 model, and update or downgrade as necessary to ensure compatibility.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING