transformers - 💡(How to fix) Fix Transformers is trying to call home despite `local_files_only=True` [6 comments, 4 participants]

Traceback (most recent call last): File "/home/user/.local/lib/python3.12/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions yield File "/home/user/.local/lib/python3.12/site-packages/httpcore/backends/sync.py", line 94, in connect_tcp sock = socket.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/socket.py", line 841, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/socket.py", line 978, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/user/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions yield File "/home/user/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 218, in handle_request resp = self._pool.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 253, in handle_request raise exc File "/home/user/.local/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 237, in handle_request response = connection.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 86, in handle_request raise exc File "/home/user/.local/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 63, in handle_request stream = self._connect(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 111, in _connect stream = self._network_backend.connect_tcp(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpcore/backends/sync.py", line 93, in connect_tcp with map_exceptions(exc_map): ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/home/user/.local/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) httpcore.ConnectError: [Errno -3] Temporary failure in name resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/user/projekte/progen/program/programAi/./program.py", line 16, in <module> from program_backend import programProject File "/home/user/projekte/progen/program/programAi/program_backend.py", line 11, in <module> from program_project import programProject File "/home/user/projekte/progen/program/programAi/program_project.py", line 17, in <module> from program_llm import WebLlamaServer_LLM File "/home/user/projekte/progen/program/programAi/program_llm.py", line 25, in <module> from program_context import program_context_generate File "/home/user/projekte/progen/program/programAi/program_context.py", line 11, in <module> import program_memory_embeddings as lme File "/home/user/projekte/progen/program/programAi/program_memory_embeddings.py", line 67, in <module> text_embedding=TextEmbedding() #initialize ^^^^^^^^^^^^^^^ File "/home/user/projekte/progen/program/programAi/program_memory_embeddings.py", line 26, in init self.tokenizer = _load_pretrained_offline_first( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/projekte/progen/program/programAi/program_memory_embeddings.py", line 18, in _load_pretrained_offline_first return loader.from_pretrained(model_name, local_files_only=True, dtype=torch.float16, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 792, in from_pretrained return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1729, in from_pretrained return cls._from_pretrained( ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1918, in _from_pretrained tokenizer = cls(*init_inputs, **init_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/models/qwen2/tokenization_qwen2.py", line 89, in init super().init( File "/home/user/.local/lib/python3.12/site-packages/transformers/tokenization_utils_tokenizers.py", line 477, in init self._tokenizer = self._patch_mistral_regex( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/tokenization_utils_tokenizers.py", line 1297, in _patch_mistral_regex is_local or (not is_local and is_base_mistral(pretrained_model_name_or_path)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/transformers/tokenization_utils_tokenizers.py", line 1287, in is_base_mistral model = model_info(model_id) ^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 88, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/huggingface_hub/hf_api.py", line 3203, in model_info r = get_session().get(path, headers=headers, timeout=timeout, params=params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 1045, in get return self.request( ^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 821, in request return self.send(request, auth=auth, follow_redirects=follow_redirects) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 908, in send response = self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 936, in _send_handling_auth response = self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 973, in _send_handling_redirects response = self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_client.py", line 1009, in _send_single_request response = transport.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 217, in handle_request with map_httpcore_exceptions(): ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/home/user/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno -3] Temporary failure in name resolution

TL;DR

The issue is likely due to a bug in the transformers library where it attempts to connect to the internet even when local_files_only=True, causing a Temporary failure in name resolution error.

Guidance

• The error occurs because the transformers library is trying to fetch model information from the Hugging Face hub even when local_files_only=True.
• To verify this, try setting local_files_only=True and check if the issue persists.
• As a temporary workaround, consider setting revision to a specific version of the model, e.g., TextEmbedding.from_pretrained('model_name', revision='commit_hash', local_files_only=True).
• Check the version of the transformers library and consider upgrading to a newer version if available.

Example

# Try setting revision to a specific version of the model
text_embedding = TextEmbedding.from_pretrained('model_name', revision='commit_hash', local_files_only=True)

Notes

• The issue seems to be a bug in the transformers library, and a fix may be available in a newer version.
• The local_files_only=True parameter is not being respected, causing the library to attempt to connect to the internet.

Recommendation

Apply workaround: Set revision to a specific version of the model to avoid the issue. This is a temporary solution until a fix is available in the transformers library.