transformers - 💡(How to fix) Fix transformers serve crashes with AttributeError: 'Gemma4Processor' object has no attribute '_tokenizer' [4 comments, 2 participants]

System Info

transformers version: 5.5.3 Python version: 3.12.3 PyTorch version: 2.11.0+cu130 Platform: Linux (Ubuntu 24.04)

Who can help?

@ArthurZucker ? idk I will open an PR once i find time. I "fixed" it locally for now.

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

pip install transformers[serving] transformers serve TrevorJS/gemma-4-31B-it-uncensored Then send any chat completion request: bashcurl http://localhost:8000/v1/chat/completions
-H "Content-Type: application/json"
-d '{"model": "TrevorJS/gemma-4-31B-it-uncensored", "messages": [{"role": "user", "content": "Hello"}]}'

Full Traceback

File ".../transformers/cli/serving/utils.py", line 565, in generate_streaming
    streamer = DirectStreamer(processor._tokenizer, loop, queue, skip_special_tokens=True)
                              ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Gemma4Processor' object has no attribute '_tokenizer'. Did you mean: 'tokenizer'?

The same error also occurs on line 664 for the continuous batching code path: streamer = CBStreamer(self._cb, request_id, processor._tokenizer, loop, text_queue)

Expected behavior

Expected Behavior Chat completions should work without error. DirectStreamer and CBStreamer require the raw Rust tokenizers.Tokenizer object, which is available at processor.tokenizer._tokenizer for Gemma4Processor.

Actual Behavior Every request crashes with AttributeError immediately after the model loads.

Fix Two lines in src/transformers/cli/serving/utils.py need updating: Line 565 (standard streaming path):

# Before
streamer = DirectStreamer(processor._tokenizer, loop, queue, skip_special_tokens=True)

# After
streamer = DirectStreamer(processor.tokenizer._tokenizer, loop, queue, skip_special_tokens=True)

Line 664 (continuous batching path):

# Before
streamer = CBStreamer(self._cb, request_id, processor._tokenizer, loop, text_queue)

# After
streamer = CBStreamer(self._cb, request_id, processor.tokenizer._tokenizer, loop, text_queue)

Alternatively, a more robust fix would be to make both call sites handle both cases: pythonrust_tokenizer = getattr(processor, "_tokenizer", None) or processor.tokenizer._tokenizer This would be defensive against other processor types that may also use the public .tokenizer attribute.

Notes

Affects all Gemma 4 models (google/gemma-4-*) since Gemma4Processor uses the public .tokenizer attribute rather than ._tokenizer Gemma 4 support was added in v5.5.0 (#45192) and transformers serve was not tested against it No existing GitHub issues or public reports found for this specific error as of April 2026 — this appears to be a gap in integration testing between the new serving CLI and Gemma4's processor design