litellm - 💡(How to fix) Fix Feature Request: EU AI Act compliance checks for LLM gateway traffic [1 comments, 2 participants]

Summary

With the EU AI Act enforcement deadline on August 2, 2026, projects that route LLM traffic (like LiteLLM) will need to demonstrate compliance with Articles 9-15 for high-risk AI systems. LiteLLM is uniquely positioned to help here since it already sits in the request path and handles logging, guardrails, and cost tracking.

What this could look like

LiteLLM already has many of the building blocks: structured logging, retry/fallback logic, and guardrails. A compliance mode could surface how well a deployment covers the key EU AI Act articles:

• Art. 9 (Risk Management): Error handling and fallback patterns across providers
• Art. 10 (Data Governance): PII detection/redaction in prompts before they hit provider APIs
• Art. 11 (Documentation): Auto-generated system cards for multi-provider deployments
• Art. 12 (Record-Keeping): Tamper-evident audit trails for all LLM calls
• Art. 14 (Human Oversight): Budget controls, rate limiting, kill switches
• Art. 15 (Security): Prompt injection scanning, output validation

Context

I ran LiteLLM through AIR Blackbox, an open-source EU AI Act compliance scanner (Apache 2.0). LiteLLM scored 48% on static analysis checks, which is one of the highest scores I've seen across any project. The foundations are already there.

You can run the scan yourselves:

pip install air-blackbox
air-blackbox comply --scan . --no-llm --format table --verbose

Everything runs locally, no data leaves your machine.

Why this matters

LiteLLM is used as an AI gateway by teams that have EU operations. Having compliance-aware features built into the gateway layer means every team using LiteLLM gets compliance tooling for free, which is a strong differentiator against other gateway solutions.

The EU AI Act carries penalties of up to €35M or 7% of global turnover. Teams are starting to evaluate their stack for compliance readiness now.