llamaIndex - 💡(How to fix) Fix [Bug]: Silent fallback to OpenAI in Retrievers and Indexes compromises Air-Gapped/Local-First deployments [6 comments, 3 participants]

llamaIndex2026-03-08 01:46:40

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

run-llama/llama_index#20912•Fetched 2026-04-08 00:30:14

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Participants

Timeline (top)

commented ×6labeled ×2mentioned ×2subscribed ×2

This was discovered while building a sovereign, 100% local-first RAG architecture. Because of a missing kwarg, the system attempted to leak locally embedded system-knowledge chunks to OpenAI. Luckily, the environment variables were strictly sanitized, which triggered the 401 Unauthorized exception and exposed the silent fallback behavior.

Error Message

ValueError: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.

Root Cause

Context

Code Example

from llama_index.core.retrievers import QueryFusionRetriever

# Intending to use local environment, but forgot to pass llm=...
hybrid_retriever = QueryFusionRetriever(
    [vector_retriever, bm25_retriever],
    mode="reciprocal_rank"
)

---

ValueError: No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.

---

### Expected Behavior
For enterprise, legal, or privacy-focused implementations, a framework should not default to a commercial cloud API unconditionally.

Ideally, LlamaIndex should:
1. Provide a global setting like `Settings.strict_mode = True` or `Settings.air_gapped = True` that immediately disables all OpenAI commercial defaults and throws a strict `MissingProviderError` if an LLM is not explicitly provided.
2. At the very least, log a `WARNING` when defaulting to OpenAI in deep instantiations: *"Warning: No LLM provided to QueryFusionRetriever. Falling back to default OpenAI models. Your data will be sent to OpenAI servers."*

### Full Traceback
Here is the exact framework crash output when the environment doesn't have an `OPENAI_API_KEY`:


Traceback (most recent call last):
  ...
  File "/app/src/engine_builder.py", line 116, in build_chat_engine
    hybrid_retriever = QueryFusionRetriever(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/retrievers/fusion_retriever.py", line 63, in __init__
    resolve_llm(llm, callback_manager=callback_manager) if llm else Settings.llm
                                                                    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/settings.py", line 36, in llm
    self._llm = resolve_llm("default")
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/llms/utils.py", line 64, in resolve_llm
    raise ValueError(
ValueError: 
******
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys
******


### Context
This was discovered while building a sovereign, 100% local-first RAG architecture. Because of a missing kwarg, the system attempted to leak locally embedded system-knowledge chunks to OpenAI. Luckily, the environment variables were strictly sanitized, which triggered the `401 Unauthorized` exception and exposed the silent fallback behavior.

RAW_BUFFERClick to expand / collapse

Bug Description

When instantiating components like VectorStoreIndex or QueryFusionRetriever without explicitly passing the llm or embed_model kwargs, LlamaIndex silently falls back to OpenAI's models (gpt-3.5-turbo and text-embedding-ada-002).

While this default behavior is convenient for quick prototypes, it creates a critical security/privacy flaw for developers building Local-First, Air-Gapped, or Privacy-Strict architectures (e.g., using local Ollama or vLLM instances). If a developer misses injecting the local LLM into a nested retriever, the framework will silently attempt to send the user's private data/vectors to api.openai.com.

If an old OPENAI_API_KEY happens to exist in the system's environment variables, the data leak occurs completely silently without any warnings.

Version

LlamaIndex Version: (Latest); - Python Version: 3.12; - OS: Linux (Arch)

Steps to Reproduce

Intentionally construct a Local-Only architecture and remove OpenAI keys from the active environment.
Instantiate a QueryFusionRetriever without explicitly passing the llm argument:

from llama_index.core.retrievers import QueryFusionRetriever

# Intending to use local environment, but forgot to pass llm=...
hybrid_retriever = QueryFusionRetriever(
    [vector_retriever, bm25_retriever],
    mode="reciprocal_rank"
)

Observe the crash: The system does not raise an explicit MissingLLMProvider error. Instead, it throws an OpenAI specific error:

ValueError: No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.

Relevant Logs/Tracbacks

### Expected Behavior
For enterprise, legal, or privacy-focused implementations, a framework should not default to a commercial cloud API unconditionally.

Ideally, LlamaIndex should:
1. Provide a global setting like `Settings.strict_mode = True` or `Settings.air_gapped = True` that immediately disables all OpenAI commercial defaults and throws a strict `MissingProviderError` if an LLM is not explicitly provided.
2. At the very least, log a `WARNING` when defaulting to OpenAI in deep instantiations: *"Warning: No LLM provided to QueryFusionRetriever. Falling back to default OpenAI models. Your data will be sent to OpenAI servers."*

### Full Traceback
Here is the exact framework crash output when the environment doesn't have an `OPENAI_API_KEY`:


Traceback (most recent call last):
  ...
  File "/app/src/engine_builder.py", line 116, in build_chat_engine
    hybrid_retriever = QueryFusionRetriever(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/retrievers/fusion_retriever.py", line 63, in __init__
    resolve_llm(llm, callback_manager=callback_manager) if llm else Settings.llm
                                                                    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/settings.py", line 36, in llm
    self._llm = resolve_llm("default")
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/llms/utils.py", line 64, in resolve_llm
    raise ValueError(
ValueError: 
******
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys
******


### Context
This was discovered while building a sovereign, 100% local-first RAG architecture. Because of a missing kwarg, the system attempted to leak locally embedded system-knowledge chunks to OpenAI. Luckily, the environment variables were strictly sanitized, which triggered the `401 Unauthorized` exception and exposed the silent fallback behavior.

extent analysis

Fix Plan

1. Introduce a global setting to enforce strict mode

Add a Settings.strict_mode flag to the Settings class in llama_index/core/settings.py. When strict_mode is enabled, LlamaIndex should raise a MissingProviderError if an LLM is not explicitly provided.

class Settings:
    # ...
    strict_mode = False

    @classmethod
    def set_strict_mode(cls, value: bool):
        cls.strict_mode = value

2. Update `QueryFusionRetriever` to raise an error when no LLM is provided

In llama_index/core/retrievers/fusion_retriever.py, update the __init__ method to raise a MissingProviderError when llm is not provided and Settings.strict_mode is enabled.

class QueryFusionRetriever:
    # ...

    def __init__(self, retrievers, mode, llm=None, **kwargs):
        if Settings.strict_mode and llm is None:
            raise MissingProviderError("No LLM provider provided")
        # ...

3. Log a warning when defaulting to OpenAI models

In llama_index/core/llms/utils.py, update the resolve_llm function to log a warning when defaulting to OpenAI models.

def resolve_llm(llm, callback_manager=None):
    if llm is None:
        logging.warning("No LLM provider provided. Falling back to default OpenAI models.")
        # ...

Verification

Set Settings.strict_mode = True in your application.
Instantiate a QueryFusionRetriever without providing an LLM.
Verify that a MissingProviderError is raised.

Extra Tips

To prevent regressions, ensure that the Settings.strict_mode

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #prompt template #agent execution #callback error #environment variable

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

llamaIndex - 💡(How to fix) Fix [Bug]: Silent fallback to OpenAI in Retrievers and Indexes compromises Air-Gapped/Local-First deployments [6 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Context

Code Example

Bug Description

Version

Steps to Reproduce

Relevant Logs/Tracbacks

extent analysis

Fix Plan

1. Introduce a global setting to enforce strict mode

2. Update `QueryFusionRetriever` to raise an error when no LLM is provided

3. Log a warning when defaulting to OpenAI models

Verification

Extra Tips

Still need to ship something?

TRENDING

llamaIndex - 💡(How to fix) Fix [Bug]: Silent fallback to OpenAI in Retrievers and Indexes compromises Air-Gapped/Local-First deployments [6 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Context

Code Example

Bug Description

Version

Steps to Reproduce

Relevant Logs/Tracbacks

extent analysis

Fix Plan

1. Introduce a global setting to enforce strict mode

2. Update QueryFusionRetriever to raise an error when no LLM is provided

3. Log a warning when defaulting to OpenAI models

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING

2. Update `QueryFusionRetriever` to raise an error when no LLM is provided