transformers - 💡(How to fix) Fix [New Model] Add TIGER — Recommender Systems with Generative Retrieval (Google DeepMind, NeurIPS 2023)

Model Description

Paper: Recommender Systems with Generative Retrieval Authors: Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, Maheswaran Sathiamoorthy Affiliations: Google DeepMind, Google, University of Wisconsin-Madison Venue: NeurIPS 2023

Architecture Summary

TIGER (Tokenized Item GEnerative Retrieval) is a fully generative, ID-free approach to sequential recommendation:

1. RQ-VAE Item Tokenizer — maps each item to a short sequence of semantic token IDs derived from content embeddings (no learnable ID lookup tables)
2. Seq2Seq Backbone (T5) — given a user's interaction history as a token ID sequence, autoregressively generates the next item's token ID sequence
3. Constrained Beam Search — decoding is constrained to only valid item token sequences via a prefix trie, guaranteeing only real items are recommended

This eliminates the cold-start and ID-sparsity problems in classical collaborative filtering and ID-based sequential recommendation models.

Why add to Transformers

• NeurIPS 2023 paper from Google DeepMind with 500+ citations
• Currently zero RecSys/LLM intersection models exist in Transformers despite very high community demand from the RecSys research community
• Builds directly on T5 which is already in Transformers — clean integration path
• Would bring the full Transformers API (fine-tuning, pipelines, quantization, PEFT) to the RecSys community automatically

Implementation Plan

• TigerConfig extending T5Config with item vocabulary and RQ-VAE parameters
• RQVAEItemTokenizer for semantic item ID generation
• TigerForSequentialRecommendation (T5 encoder-decoder + constrained decoding)
• Prefix-trie constrained beam search integrated with GenerationMixin
• Conversion script + example notebook on a public RecSys dataset (e.g. Amazon Reviews)

I am actively working on this implementation. Would appreciate maintainer guidance on whether this fits best as a T5 variant, a standalone model, or a Transformers Pipeline — open to whichever approach maintainers prefer.