transformers - ✅(Solved) Fix `apply_chat_template(tokenize=True)` crashes on assistant messages with tool calls and no content [1 pull requests, 1 comments, 2 participants]

transformers2026-04-07 13:29:49

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

huggingface/transformers#45290•Fetched 2026-04-08 03:00:48

View on GitHub

Comments

Participants

Timeline

Reactions

Author

qgallouedec

Participants

qgallouedec

zucchini-nlp

Timeline (top)

subscribed ×4mentioned ×3commented ×1cross-referenced ×1

Error Message

KeyError: 'content' File "processing_utils.py", line 1807, in apply_chat_template visuals = [content for content in message["content"] if content["type"] in ["image", "video"]] ~~~~~~~^^^^^^^^^^^

Fix Action

Fixed

Fixed by PR: Narrow prefix-preserving check to the actual requirement (https://github.com/huggingface/trl/pull/5458)

PR fix notes

PR #5458: Narrow prefix-preserving check to the actual requirement

Repository: huggingface/trl
Author: qgallouedec
State: closed | merged: True
Link: https://github.com/huggingface/trl/pull/5458

Description (problem / solution / changelog)

Before #5224, the tool call loop re-tokenized the full conversation at each iteration, requiring prefix-preservation across all role transitions. Now that this is fixed, only _get_tool_suffix_ids relies on prefix-preservation, and only for the narrow [user, assistant] → [user, assistant, tool] case. This PR aligns the check with the actual requirement, which in turn reveals that Qwen3.5 doesn't need patching anymore.

This PR:

is_chat_template_prefix_preserving now tests the exact property that _get_tool_suffix_ids relies on: appending tool messages after [user, assistant] must not change the rendering of earlier messages. (Previously it tested broader user→assistant and assistant→user transitions.)
This revealed that Qwen3.5 was already prefix-preserving for the tool transition, only Qwen3 actually needs the training template patch. Removed the now-dead Qwen3.5 training template code and _patch_qwen3_5_training_template.
Updated tests accordingly (removed Qwen3.5 from TestGetTrainingChatTemplate, added tool role to test templates).

[!NOTE] Medium Risk Changes affect how tool-response suffix tokens are derived and when chat templates are considered safe for tool-calling, which can subtly impact multi-turn/tool formatting across tokenizers. Scope is limited and covered by updated tests, but regressions would show up as incorrect tool token slicing or template patching decisions.

Overview Aligns prefix-preserving validation with the actual tool-calling need. is_chat_template_prefix_preserving now specifically checks that appending a tool message after a [user, assistant(tool_calls)] prefix does not change the rendered prefix, matching what _get_tool_suffix_ids relies on.

Simplifies training-template patching and updates tool suffix extraction. Removes Qwen3.5 training-template patching paths (only Qwen3 is patched now), updates docs/examples accordingly, and adjusts _get_tool_suffix_ids in both AsyncRolloutWorker and GRPOTrainer to use a dummy assistant tool_calls message plus safer EOS trimming when extracting the suffix.

Tests updated for tool-aware templates. Chat-template tests add tool/tool_calls handling in templates and drop Qwen3.5 from TestGetTrainingChatTemplate parametrization.

<sup>Reviewed by Cursor Bugbot for commit 87131e503f85bdc1ea65947f6937da2b5635c6fe. Bugbot is set up for automated code reviews on this repo. Configure here.</sup>

Changed files

tests/test_chat_template_utils.py (modified, +26/-11)
trl/chat_template_utils.py (modified, +33/-52)
trl/experimental/async_grpo/async_rollout_worker.py (modified, +20/-10)
trl/trainer/grpo_trainer.py (modified, +11/-1)

Code Example

from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

messages = [
    [
        {"role": "user", "content": [{"type": "text", "text": "dummy"}]},
        {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "foo", "arguments": {}}}]},
    ]
]

processor.apply_chat_template(messages, tokenize=True)

---

KeyError: 'content'
  File "processing_utils.py", line 1807, in apply_chat_template
    visuals = [content for content in message["content"] if content["type"] in ["image", "video"]]
                                      ~~~~~~~^^^^^^^^^^^

---

for message in conversation:
    content = message.get("content") or []
    visuals = [c for c in content if isinstance(c, dict) and c.get("type") in ["image", "video"]]

RAW_BUFFERClick to expand / collapse

System Info

Transformers 5.5..0

Who can help?

@zucchini-nlp @Rocketknight1

Information

The official example scripts
My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)

Reproduction

ProcessorMixin.apply_chat_template raises KeyError: 'content' when tokenize=True and the conversation contains an assistant message with tool_calls but no content key.

from transformers import AutoProcessor

processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

messages = [
    [
        {"role": "user", "content": [{"type": "text", "text": "dummy"}]},
        {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "foo", "arguments": {}}}]},
    ]
]

processor.apply_chat_template(messages, tokenize=True)

KeyError: 'content'
  File "processing_utils.py", line 1807, in apply_chat_template
    visuals = [content for content in message["content"] if content["type"] in ["image", "video"]]
                                      ~~~~~~~^^^^^^^^^^^

It comes from these lines

https://github.com/huggingface/transformers/blob/52cb0653b48fcb0737a74546911df77034b61732/src/transformers/processing_utils.py#L1801-L1807

where it's assumed that all turns contain a content key. However in the codebase, AFAICT, content is always assumed to be optional for assistant turns with tool calls.

Possible fix

Guard the access with .get("content") or skip messages without content:

for message in conversation:
    content = message.get("content") or []
    visuals = [c for c in content if isinstance(c, dict) and c.get("type") in ["image", "video"]]

Expected behavior

to pass with no content

extent analysis

TL;DR

Modify the apply_chat_template function to handle cases where the 'content' key is missing from assistant messages with 'tool_calls' by using the .get() method to safely access the 'content' key.

Guidance

The error occurs because the apply_chat_template function assumes all messages have a 'content' key, but in the case of assistant messages with 'tool_calls', this key is optional and may be missing.
To fix this, you can modify the apply_chat_template function to use the .get() method to safely access the 'content' key, as shown in the possible fix provided in the issue.
Verify that the fix works by testing the apply_chat_template function with a conversation that contains an assistant message with 'tool_calls' but no 'content' key.
Consider submitting a pull request to the transformers repository to include this fix in the official codebase.

Example

for message in conversation:
    content = message.get("content") or []
    visuals = [c for c in content if isinstance(c, dict) and c.get("type") in ["image", "video"]]

Notes

This fix assumes that the 'content' key is optional for assistant messages with 'tool_calls', as stated in the issue. If this assumption is incorrect, further modifications may be needed.

Recommendation

Apply the workaround by modifying the apply_chat_template function to use the .get() method to safely access the 'content' key, as this will allow the function to handle cases where the 'content' key is missing without raising a KeyError.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

FAQ

Expected behavior

to pass with no content

#model download #tokenizer error #prompt formatting #chain error #conversation history

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

transformers - ✅(Solved) Fix `apply_chat_template(tokenize=True)` crashes on assistant messages with tool calls and no content [1 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #5458: Narrow prefix-preserving check to the actual requirement

Description (problem / solution / changelog)

Changed files

Code Example

System Info

Who can help?

Information

Tasks

Reproduction

Possible fix

Expected behavior

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

FAQ

Expected behavior

Still need to ship something?

TRENDING

transformers - ✅(Solved) Fix `apply_chat_template(tokenize=True)` crashes on assistant messages with tool calls and no content [1 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #5458: Narrow prefix-preserving check to the actual requirement

Description (problem / solution / changelog)

Changed files

Code Example

System Info

Who can help?

Information

Tasks

Reproduction

Possible fix

Expected behavior

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

FAQ

Expected behavior

Still need to ship something?

RELATED_DISCOVERY

TRENDING