Issue Station
Sign In
Tools hereToolbox

llamaIndex - ✅(Solved) Fix llama-index-llms-ipex-llm: Silent fallback to trust_remote_code=True in tokenizer loading [1 pull requests, 4 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
run-llama/llama_index#21464•Fetched 2026-04-25 06:03:20
View on GitHub
Comments
4
Participants
2
Timeline
8
Reactions
0
Author
ryosuke-tokushimaryosuke-tokushima
Participants
dosubot[bot]dosubot[bot]ryosuke-tokushimaryosuke-tokushima
Timeline (top)
commented ×3mentioned ×2subscribed ×2cross-referenced ×1

In llama-index-integrations/llms/llama-index-llms-ipex-llm, the tokenizer loading logic silently falls back to LlamaTokenizer.from_pretrained(..., trust_remote_code=True) whenever AutoTokenizer.from_pretrained() raises any exception. This means trust_remote_code=True is applied without the user's explicit knowledge or consent.

Error Message

try: tokenizer = AutoTokenizer.from_pretrained( tokenizer_name, **tokenizer_kwargs ) except Exception: # ← catches ALL exceptions tokenizer = LlamaTokenizer.from_pretrained( tokenizer_name, trust_remote_code=True # ← hardcoded True )

Root Cause

In llama-index-integrations/llms/llama-index-llms-ipex-llm, the tokenizer loading logic silently falls back to LlamaTokenizer.from_pretrained(..., trust_remote_code=True) whenever AutoTokenizer.from_pretrained() raises any exception. This means trust_remote_code=True is applied without the user's explicit knowledge or consent.

Fix Action

Fixed

PR fix notes

PR #21466: fix(ipex-llm): make trust_remote_code user-controlled, default to False

Description (problem / solution / changelog)

Summary

Fixes #21464

trust_remote_code=True was hardcoded in two places in llama-index-llms-ipex-llm, silently granting arbitrary code execution permission without user consent.

Changes

1. Tokenizer fallback (~line 235)

Before:

except Exception:
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name, trust_remote_code=True  # hardcoded
    )

After:

except Exception:
    logger.warning(
        f"AutoTokenizer failed for '{tokenizer_name}'. "
        "Retrying with LlamaTokenizer. "
        "trust_remote_code is NOT enabled automatically; "
        "pass trust_remote_code=True in tokenizer_kwargs explicitly "
        "if you trust this model source."
    )
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name,
        trust_remote_code=tokenizer_kwargs.get("trust_remote_code", False),
    )

2. Model loading (~line 407)

Before:

load_kwargs = {"use_cache": True, "trust_remote_code": True}  # unconditional

After:

trust_remote_code = model_kwargs.pop("trust_remote_code", False)
if trust_remote_code:
    logger.warning(
        "trust_remote_code=True is enabled. This allows execution of "
        "arbitrary code from the model repository. Only use this if you "
        "trust the model source."
    )
load_kwargs = {"use_cache": True, "trust_remote_code": trust_remote_code}

Behaviour

  • Default is now False (safe) — no breaking change for users who trust the model and already pass trust_remote_code=True explicitly via model_kwargs or tokenizer_kwargs
  • A logger.warning() is emitted whenever trust_remote_code=True is actually applied, consistent with the Gaudi and vLLM integrations

Changed files

  • llama-index-integrations/llms/llama-index-llms-ipex-llm/llama_index/llms/ipex_llm/base.py (modified, +17/-2)

Code Example

try:
    tokenizer = AutoTokenizer.from_pretrained(
        tokenizer_name, **tokenizer_kwargs
    )
except Exception:          # ← catches ALL exceptions
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name, trust_remote_code=True   # ← hardcoded True
    )

---

try:
    tokenizer = AutoTokenizer.from_pretrained(
        tokenizer_name, **tokenizer_kwargs
    )
except Exception:
    logger.warning(
        f"AutoTokenizer failed for '{tokenizer_name}'. "
        "Retrying with LlamaTokenizer. "
        "Note: trust_remote_code is NOT enabled automatically. "
        "Pass trust_remote_code=True explicitly if you trust this model source."
    )
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name,
        trust_remote_code=tokenizer_kwargs.get("trust_remote_code", False),  # honour user intent
    )
RAW_BUFFERClick to expand / collapse

Issue Title

llama-index-llms-ipex-llm: Silent fallback to trust_remote_code=True in tokenizer loading

Summary

In llama-index-integrations/llms/llama-index-llms-ipex-llm, the tokenizer loading logic silently falls back to LlamaTokenizer.from_pretrained(..., trust_remote_code=True) whenever AutoTokenizer.from_pretrained() raises any exception. This means trust_remote_code=True is applied without the user's explicit knowledge or consent.

Affected File

llama-index-integrations/llms/llama-index-llms-ipex-llm/llama_index/llms/ipex_llm/base.py (confirmed on llama-index-core==0.14.21, commit a3aeb31)

Vulnerable Code

try:
    tokenizer = AutoTokenizer.from_pretrained(
        tokenizer_name, **tokenizer_kwargs
    )
except Exception:          # ← catches ALL exceptions
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name, trust_remote_code=True   # ← hardcoded True
    )

Problem

trust_remote_code=True instructs the HuggingFace Hub to download and execute arbitrary Python code bundled with the model repository. This flag should only be set explicitly by the user after reviewing the model's source.

Two specific concerns:

  1. Silent escalation: A user who does not pass trust_remote_code expects the library to behave safely. The fallback silently grants code-execution permission without any log warning or user confirmation.

  2. Exception-triggered escalation: A network hiccup, a misconfigured tokenizer_kwargs, or a model that intentionally raises an error in its AutoTokenizer path can trigger the fallback, causing trust_remote_code=True to be applied to a tokenizer the user never intended to trust.

Impact

If a user loads a model from an untrusted or attacker-controlled HuggingFace Hub repository, arbitrary Python code inside the tokenizer's tokenization_*.py will be executed on the user's machine at model-load time.

Suggested Fix

try:
    tokenizer = AutoTokenizer.from_pretrained(
        tokenizer_name, **tokenizer_kwargs
    )
except Exception:
    logger.warning(
        f"AutoTokenizer failed for '{tokenizer_name}'. "
        "Retrying with LlamaTokenizer. "
        "Note: trust_remote_code is NOT enabled automatically. "
        "Pass trust_remote_code=True explicitly if you trust this model source."
    )
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name,
        trust_remote_code=tokenizer_kwargs.get("trust_remote_code", False),  # honour user intent
    )

Discovery

Found via static analysis of the llama_index codebase.

extent analysis

TL;DR

The most likely fix is to modify the tokenizer loading logic to not silently fall back to trust_remote_code=True and instead honor the user's explicit intent.

Guidance

  • Identify the try-except block in the base.py file and modify it to log a warning when AutoTokenizer.from_pretrained() raises an exception, indicating that LlamaTokenizer will be used instead.
  • Update the LlamaTokenizer.from_pretrained() call to use the trust_remote_code value from tokenizer_kwargs if provided, defaulting to False otherwise.
  • Verify that the modified code behaves as expected by testing it with different scenarios, including network errors and invalid tokenizer_kwargs.
  • Consider adding additional logging or warnings to inform the user when trust_remote_code is not explicitly set.

Example

try:
    tokenizer = AutoTokenizer.from_pretrained(
        tokenizer_name, **tokenizer_kwargs
    )
except Exception:
    logger.warning(
        f"AutoTokenizer failed for '{tokenizer_name}'. "
        "Retrying with LlamaTokenizer. "
        "Note: trust_remote_code is NOT enabled automatically. "
        "Pass trust_remote_code=True explicitly if you trust this model source."
    )
    tokenizer = LlamaTokenizer.from_pretrained(
        tokenizer_name,
        trust_remote_code=tokenizer_kwargs.get("trust_remote_code", False),
    )

Notes

This fix assumes that the tokenizer_kwargs dictionary contains the user's explicit intent regarding trust_remote_code. If this is not the case, additional modifications may be necessary to ensure that the user's intent is properly handled.

Recommendation

Apply the suggested fix to modify the tokenizer loading logic and ensure that trust_remote_code is only enabled when explicitly set by the user. This change will help prevent silent escalation and exception-triggered escalation, improving the security of the llama-index library.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#ssr#cache issue#memory leak#API versioning#request timeout

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

llamaIndex - ✅(Solved) Fix llama-index-llms-ipex-llm: Silent fallback to trust_remote_code=True in tokenizer loading [1 pull requests, 4 comments, 2 participants]