transformers - ✅(Solved) Fix [BUG] Different output for google/pegasus-cnn_dailymail between Transformers v4 and v5 [1 pull requests, 3 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
huggingface/transformers#44448Fetched 2026-04-08 00:28:29
View on GitHub
Comments
3
Participants
3
Timeline
16
Reactions
0
Timeline (top)
mentioned ×4subscribed ×4commented ×3cross-referenced ×2

Fix Action

Fixed

PR fix notes

PR #44571: Fix pegasus conversion

Description (problem / solution / changelog)

Fixes #44448

Changed files

  • setup.py (modified, +5/-8)
  • src/transformers/convert_slow_tokenizer.py (modified, +22/-0)
  • src/transformers/dependency_versions_table.py (modified, +2/-2)
  • src/transformers/models/pegasus/tokenization_pegasus.py (modified, +4/-2)

Code Example

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "google/pegasus-cnn_dailymail"
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "google/pegasus-cnn_dailymail",
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)

input_text = """Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems."""
input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)

output = model.generate(**input_ids, cache_implementation="static")
print(tokenizer.decode(output[0], skip_special_tokens=True))

---

Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.

---

And want seek help Jen first gavepist brought well magical am high place go her three Brit Meadows building – event masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows building masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre –

---

Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
RAW_BUFFERClick to expand / collapse

System Info

  • transformers version: 4.57.6 (working) / 5.0.0 (incorrect output)
  • Platform: Linux-6.6.113+-x86_64-with-glibc2.35
  • Python version: 3.12.12
  • Huggingface_hub version: 1.5.0
  • Safetensors version: 0.7.0
  • Accelerate version: 1.12.0
  • Accelerate config: not found
  • DeepSpeed version: not installed
  • PyTorch version (accelerator?): 2.10.0+cpu (NA)
  • Using distributed or parallel set-up in script?: distributed

Who can help?

@ArthurZucker @Cyrilvallez

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

When executing the example from the Pegasus documentation: https://huggingface.co/docs/transformers/main/en/model_doc/pegasus?usage=AutoModel

Transformers v4 produces a reasonable summary, while v5 produces an incoherent / meaningless output.

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "google/pegasus-cnn_dailymail"
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "google/pegasus-cnn_dailymail",
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)

input_text = """Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems."""
input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)

output = model.generate(**input_ids, cache_implementation="static")
print(tokenizer.decode(output[0], skip_special_tokens=True))

output (v4.*)

Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.

output (v5.*)

And want seek help Jen first gavepist brought well magical am high place go her three Brit Meadows building – event masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows building masterre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only nourished recruitmentA am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre – variety source me well only healthy am many high place go her three Brit Meadows buildingpre –

Expected behavior

The model generates a coherent summary of the article. Example output:

Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.

extent analysis

Fix Plan

Downgrade Transformers to v4.57.6

The issue is caused by a regression in Transformers v5.0.0. Downgrading to v4.57.6 should resolve the issue.

Step-by-Step Solution

  1. Uninstall Transformers v5.0.0: Run pip uninstall transformers
  2. Install Transformers v4.57.6: Run pip install transformers==4.57.6
  3. Verify the installation: Run pip show transformers to ensure that the correct version is installed.

Example Code

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "google/pegasus-cnn_dailymail"
)
model = AutoModelForSeq2SeqLM.from_pretrained(
    "google/pegasus-cnn_dailymail",
    dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)

# Generate a summary
input_text = """Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems."""
input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)

output = model.generate(**input_ids, cache_implementation="static")
print(tokenizer.decode(output[0], skip_special_tokens=True))

This code should produce a coherent summary, similar to the output from Transformers v4.57.6.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

The model generates a coherent summary of the article. Example output:

Plants are remarkable organisms that produce their own food using a method called photosynthesis.<n>This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

transformers - ✅(Solved) Fix [BUG] Different output for google/pegasus-cnn_dailymail between Transformers v4 and v5 [1 pull requests, 3 comments, 3 participants]