Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

LiteLLM currently loads the “local fallback” model cost map from a JSON bundled inside the Python package (via importlib.resources). In packaged/bundled deployments (e.g., PyInstaller, slim Docker images), that bundled JSON may be missing, which can break offline fallback behavior.

Proposal: support user-provided local sources for the model cost map:

• LITELLM_MODEL_COST_MAP_PATH=/path/to/model_cost_map.json (load JSON from file)

Pseudocode

@staticmethod
def load_local_model_cost_map() -> dict:
    path = os.getenv("LITELLM_MODEL_COST_MAP_PATH", "").strip()
    if path:
        with open(path, "r", encoding="utf-8") as f:
            return json.load(f)

    # existing fallback
    return json.loads(
        files("litellm")
        .joinpath("model_prices_and_context_window_backup.json")
        .read_text(encoding="utf-8")
    )

Motivation, pitch

This is mainly for offline/firewalled and packaged environments where:

• remote fetch (GitHub raw) is blocked/unavailable, and
• package data files (backup JSON) may be excluded by packaging/build tooling.

Related but not duplicates:

• #1434 (local-only mode via LITELLM_LOCAL_MODEL_COST_MAP=True) still assumes the bundled backup JSON exists.
• #15274 (custom remote URL env var) is about remote sources, not local file injection.

This change would make LiteLLM more reliable in air-gapped deployments and bundled binaries, while keeping current default behavior unchanged.

What part of LiteLLM is this about?

SDK (litellm Python package)

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response