Issue Station
Sign In
Tools hereToolbox

litellm - 💡(How to fix) Fix [Feature]: Do not store API keys directly in LiteLLM - inject with nono gateway [9 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24578•Fetched 2026-04-08 01:32:36
View on GitHub
Comments
9
Participants
3
Timeline
19
Reactions
3
Author
qdrddrqdrddr
Participants
krrish-berri-2krrish-berri-2lukehindslukehindsqdrddrqdrddr
Timeline (top)
commented ×9subscribed ×4mentioned ×3labeled ×2
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

This proposal applies to both the LiteLLM SDK and Proxy:

  1. Sandboxed execution with placeholder API keys
  • Run LiteLLM inside a sandbox where only placeholder API keys are visible to the process. The real API keys are injected at HTTP request time and are never exposed to LiteLLM itself.
  • Real credentials remain securely stored in OS‑level secret vaults (macOS Keychain, Windows Credential Manager / DPAPI, Linux Secret Service / Keyring) or in widely used secret‑vault tools such as 1Password, Bitwarden, etc.
  1. Restrict outbound network access
  • Prevent LiteLLM from connecting to any internet endpoints other than officially supported LLM provider domains and LiteLLM’s own official endpoints.

Both capabilities are already achievable using the nono sandbox, which supports:

  • Credential injection via a nono gateway
  • Network egress restrictions to block unauthorized endpoints with nono

nono can be used either as a standalone CLI to run LiteLLM in a sandbox or embedded via its Python bindings.

Motivation, pitch

This related issue https://github.com/BerriAI/litellm/issues/24512 demonstrates a real security risk:
Malicious code embedded in LiteLLM could exfiltrate API keys and send them to the attacker.

By sandboxing LiteLLM and providing only placeholder keys (with real keys injected at request time by the gateway), LiteLLM never has access to sensitive credentials, even if compromised. By restricting outbound network access to approved provider endpoints, any attempted exfiltration is blocked by design.

Both measures are immediately implementable using nono and would significantly strengthen LiteLLM’s security posture. Given LiteLLM’s growing adoption in production environments, deploying these protections would provide meaningful, defense‑in‑depth guarantees for all users.

What part of LiteLLM is this about?

SDK (litellm Python package) & Proxy

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

https://www.linkedin.com/in/damien-e/

extent analysis

Fix Plan

To address the security risk, we will implement sandboxed execution with placeholder API keys and restrict outbound network access using the nono sandbox. Here are the steps:

  • Install nono: Install the nono sandbox using pip: pip install nono
  • Configure nono gateway: Set up a nono gateway to inject real API keys at request time
  • Restrict network egress: Use nono to restrict outbound network access to approved provider endpoints
  • Integrate with LiteLLM: Embed nono via its Python bindings in the LiteLLM SDK and Proxy

Example code to integrate nono with LiteLLM:

import nono

# Create a nono gateway
gateway = nono.Gateway()

# Define placeholder API keys
placeholder_keys = {"api_key": "placeholder"}

# Define approved provider endpoints
approved_endpoints = ["https://example.com"]

# Restrict network egress
nono.restrict_egress(approved_endpoints)

# Run LiteLLM in a sandbox with placeholder API keys
def run_litellm():
    # Inject real API keys at request time using the nono gateway
    def inject_keys(request):
        real_keys = gateway.get_real_keys()
        request.api_key = real_keys["api_key"]
        return request

    # Run LiteLLM with placeholder API keys
    litellm = LiteLLM(placeholder_keys)
    litellm.run()

# Run LiteLLM in a sandbox
nono.run_sandboxed(run_litellm)

Verification

To verify that the fix worked, test the LiteLLM SDK and Proxy with the nono sandbox enabled. Ensure that:

  • Placeholder API keys are used by default
  • Real API keys are injected at request time by the nono gateway
  • Outbound network access is restricted to approved provider endpoints

Extra Tips

  • Ensure that the nono gateway is properly configured and secured to store and manage real API keys.
  • Monitor and test the nono sandbox regularly to ensure it is functioning as expected.
  • Consider implementing additional security measures, such as encryption and access controls, to further protect sensitive credentials.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#agent setup#task chaining#parallel task#integration issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING