transformers - 💡(How to fix) Fix Outdated examples about RAG

C:\Users\usuario\AppData\Local\Programs\Python\Python313\python.exe: No module named transformers.__main__; 'transformers' is a package and cannot be directly executed

(.venv) C:\Users\usuario\Documents\aibased>transformers env                                                                       
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\usuario\Documents\aibased\.venv\Scripts\transformers.exe\__main__.py", line 4, in <module>
    from transformers.cli.transformers import main
  File "C:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\cli\transformers.py", line 21, in <module>
    from transformers.cli.serve import Serve
  File "C:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\cli\serve.py", line 27, in <module>
    from .serving.utils import set_torch_seed
  File "C:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\cli\serving\__init__.py", line 16, in <module>
    from .server import build_server
  File "C:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\cli\serving\server.py", line 30, in <module>
    from .chat_completion import ChatCompletionHandler
  File "C:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\cli\serving\chat_completion.py", line 53, in <module>
    class TransformersCompletionCreateParamsStreaming(CompletionCreateParamsStreaming, total=False):
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'CompletionCreateParamsStreaming' is not defined

---

(.venv) C:\Users\usuario\Documents\aibased>C:/Users/usuario/AppData/Local/Programs/Python/Python313/python.exe -m pip list        
Package               Version
--------------------- ------------
accelerate            1.13.0
aiohappyeyeballs      2.6.1
aiohttp               3.13.5
aiosignal             1.4.0
annotated-doc         0.0.4
anyio                 4.13.0
attrs                 26.1.0
bitsandbytes          0.49.2
certifi               2026.4.22
charset-normalizer    3.4.7
click                 8.3.3
colorama              0.4.6
datasets              4.8.5
dill                  0.4.1
dnspython             2.8.0
faiss-cpu             1.13.2
filelock              3.29.0
frozenlist            1.8.0
fsspec                2026.2.0
h11                   0.16.0
hf-xet                1.5.0
httpcore              1.0.9
httpx                 0.28.1
huggingface_hub       1.14.0
idna                  3.14
Jinja2                3.1.6
joblib                1.5.3
markdown-it-py        4.2.0
MarkupSafe            3.0.3
mdurl                 0.1.2
mpmath                1.3.0
multidict             6.7.1
multiprocess          0.70.19
networkx              3.6.1
numpy                 2.4.4
packaging             26.2
pandas                3.0.3
pillow                12.2.0
pip                   25.1.1
propcache             0.5.2
psutil                7.2.2
pyarrow               24.0.0
Pygments              2.20.0
pymongo               4.17.0
python-dateutil       2.9.0.post0
PyYAML                6.0.3
regex                 2026.5.9
requests              2.33.1
rich                  15.0.0
safetensors           0.7.0
scikit-learn          1.8.0
scipy                 1.17.1
sentence-transformers 5.5.0
setuptools            70.2.0
shellingham           1.5.4
six                   1.17.0
sympy                 1.14.0
threadpoolctl         3.6.0
tokenizers            0.22.2
torch                 2.12.0+cu132
torch_scatter         2.1.2
torchvision           0.27.0+cu132
tqdm                  4.67.3
transformers          5.8.0
typer                 0.25.1
typing_extensions     4.15.0
tzdata                2026.2
urllib3               2.7.0
xxhash                3.7.0
yarl                  1.23.0

---

from transformers import RagRetriever, RagSequenceForGeneration, RagTokenizer


tokenizer = RagTokenizer.from_pretrained("facebook/rag-sequence-nq")
retriever = RagRetriever.from_pretrained(
    "facebook/dpr-ctx_encoder-single-nq-base", dataset="wiki_dpr", index_name="compressed"
)

model = RagSequenceForGeneration.from_pretrained(
    "facebook/rag-token-nq",
    retriever=retriever,
    attn_implementation="flash_attention_2",
 device_map="auto")
input_dict = tokenizer.prepare_seq2seq_batch("How many people live in Paris?", return_tensors="pt").to(model.device)
generated = model.generate(input_ids=input_dict["input_ids"])
print(tokenizer.batch_decode(generated, skip_special_tokens=True)[0])

---

(.venv) C:\Users\usuario\Documents\aibased>C:/Users/usuario/AppData/Local/Programs/Python/Python313/python.exe c:/Users/usuario/Documents/aibased/rag/rag1example.py
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] You are using a model of type `dpr` to instantiate a model of type `rag`. This may be expected if you are loading a checkpoint that shares a subset of the architecture (e.g., loading a `sam2_video` checkpoint into `Sam2Model`), but is otherwise not supported and can yield errors. Please verify that the checkpoint is compatible with the model you are instantiating.
Traceback (most recent call last):
  File "c:\Users\usuario\Documents\aibased\rag\rag1example.py", line 5, in <module>
    retriever = RagRetriever.from_pretrained(
        "facebook/dpr-ctx_encoder-single-nq-base", dataset="wiki_dpr", index_name="compressed"
    )
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\transformers\models\rag\retrieval_rag.py", line 446, in from_pretrained
    config = kwargs.pop("config", None) or RagConfig.from_pretrained(retriever_name_or_path, **kwargs)
                                           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\transformers\configuration_utils.py", line 662, in from_pretrained
    return cls.from_dict(config_dict, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\transformers\configuration_utils.py", line 839, in from_dict
    config = cls(**config_dict)
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\huggingface_hub\dataclasses.py", line 275, in init_with_validate
    initial_init(self, *args, **kwargs)  # type: ignore [call-arg]
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\transformers\configuration_utils.py", line 114, in __init__
    self.__post_init__(**additional_kwargs)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\usuario\AppData\Local\Programs\Python\Python313\Lib\site-packages\transformers\models\rag\configuration_rag.py", line 111, in __post_init__
    raise ValueError(
    ...<2 lines>...
    )
ValueError: A configuration of type rag cannot be instantiated because not both `question_encoder` and `generator` sub-configurations are passed, but only {'attention_probs_dropout_prob': 0.1, 'gradient_checkpointing': False, 'hidden_act': 'gelu', 'hidden_dropout_prob': 0.1, 'hidden_size': 768, 'initializer_range': 0.02, 'intermediate_size': 3072, 'layer_norm_eps': 1e-12, 'max_position_embeddings': 512, 'model_type': 'dpr', 'num_attention_heads': 12, 'num_hidden_layers': 12, 'projection_dim': 0, 'type_vocab_size': 2, '_commit_hash': 'bb21a3c2b1656d60c6a8e920283bc40dabddadb8'}

---

# To load your own indexed dataset built with the datasets library. More info on how to build the indexed dataset in examples/rag/use_own_knowledge_dataset.py

---

# To load your own indexed dataset built with the datasets library. More info on how to build the indexed dataset in examples/rag/use_own_knowledge_dataset.py
from transformers import RagRetriever

dataset = (
    ...
)  # dataset must be a datasets.Datasets object with columns "title", "text" and "embeddings", and it must have a supported index (e.g., Faiss or other index types depending on your setup)
retriever = RagRetriever.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base", indexed_dataset=dataset)

---

import torch, json
from transformers import BitsAndBytesConfig, RagRetriever, RagSequenceForGeneration, RagTokenizer
from datasets import load_dataset

fp = open("./dm-dataset-extra-reduced.json", "r")
data = json.load(fp)
fp.close()

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq")

dataset = load_dataset("json", data_files="./datasets/dm-dataset-1.json", split="train")

dataset = dataset.rename_column("instruction", "title")
dataset = dataset.rename_column("output", "text")

tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq")
#retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", indexed_dataset=dataset)

model = RagSequenceForGeneration.from_pretrained(
    "facebook/rag-token-nq",
    #retriever=retriever,
    #quantization_config=bnb,   # quantizes generator weights
    device_map="auto",
)

input_text = "How many people live in Paris?"
input_ids = tokenizer(input_text, return_tensors='pt').input_ids.to(device)

generated = model.generate(input_ids=input_ids)
print(tokenizer.batch_decode(generated, skip_special_tokens=True)[0])

---

(.venv) C:\Users\usuario\Documents\aibased>c:/Users/usuario/Documents/aibased/.venv/Scripts/python.exe c:/Users/usuario/Documents/aibased/rag/ragbug3rd.py
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[transformers] Please make sure the generation config includes `forced_bos_token_id=0`. 
Loading weights: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 711/711 [00:00<00:00, 1289.12it/s]
[transformers] RagSequenceForGeneration LOAD REPORT from: facebook/rag-token-nq
Key                                                                  | Status     |  | 
---------------------------------------------------------------------+------------+--+-
rag.question_encoder.question_encoder.bert_model.pooler.dense.bias   | UNEXPECTED |  | 
rag.question_encoder.question_encoder.bert_model.pooler.dense.weight | UNEXPECTED |  | 

Notes:
- UNEXPECTED:   can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Traceback (most recent call last):
  File "c:\Users\usuario\Documents\aibased\rag\ragbug3rd.py", line 30, in <module>
    generated = model.generate(input_ids=input_ids)
  File "c:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\torch\utils\_contextlib.py", line 124, in decorate_context
    return func(*args, **kwargs)
  File "c:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\models\rag\modeling_rag.py", line 943, in generate
    num_return_sequences if num_return_sequences is not None else self.config.num_return_sequences
                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\usuario\Documents\aibased\.venv\Lib\site-packages\transformers\configuration_utils.py", line 434, in __getattribute__
    return super().__getattribute__(key)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
AttributeError: 'RagConfig' object has no attribute 'num_return_sequences'