llamaIndex - 💡(How to fix) Fix feat: Governance instrumentation handler for tool/query security (TealTiger integration)

from llamaindex_tealtiger import TealTigerCallback

# Zero-config: observe tool calls, track cost, detect PII (no blocking)
query_engine = index.as_query_engine(callbacks=[TealTigerCallback()])

# With governance policies
from tealtiger import TealEngine
engine = TealEngine(policies=company_policies, mode="ENFORCE")
query_engine = index.as_query_engine(callbacks=[TealTigerCallback(engine=engine)])


Capabilities:

Pre-tool-call policy evaluation (<5ms, deterministic, no LLM in governance path)
Tool/retriever allowlisting (block unauthorized tools or retrieval sources)
Cost tracking per query, per index, per session with budget enforcement
PII detection in query content and retrieved documents
Structured audit trail with correlation IDs per query lifecycle
Kill switch to halt a specific agent mid-pipeline
References:

OWASP Top 10 for Agentic AI — framework covers 8/10 ASIs
Existing observability handlers: Langfuse, WandB, Argilla — this follows the same callback pattern but for governance rather than observability
TealTiger on PyPI (v1.3.0), npm (v0.1.0)




Reason:


LlamaIndex has excellent observability integrations (Langfuse, WandB, OpenTelemetry) for monitoring what happened, but no built-in mechanism to enforce governance policies _before_ actions execute. Today, if you want to:

- Block a tool call that violates policy
- Enforce a cost budget across a query pipeline
- Detect PII in retrieved documents before returning to users
- Restrict which retrievers/tools an agent can invoke

...you have to write custom logic wrapping every query engine and tool manually. This doesn't scale when you have multiple indices, agents, and pipelines.

The callback/instrumentation system already provides the right interception points — it just lacks a governance handler that can evaluate policies and block actions deterministically.

1. Security for agentic RAG — as LlamaIndex agents gain tool-calling capabilities, governance becomes critical. An agent querying a database tool should have budget limits and allowlisted queries.

2. Compliance — production deployments need audit trails of every tool call and retrieval. A governance handler produces structured evidence records (correlation IDs, decisions, cost) automatically.

3. Zero-config adoption — developers can add `TealTigerCallback()` with no configuration to get immediate visibility into cost, PII exposure, and tool usage patterns. No policy writing required to start.

4. In-process, <5ms — unlike proxy-based solutions, this runs in the same process with no network hop. Deterministic evaluation, works offline.

5. Community pattern — this follows the same integration pattern as Langfuse/WandB callbacks but for a different concern (governance vs. observability). Both are needed in production.

I'm happy to contribute a PR with the callback handler and a docs example once the package is on PyPI. Already integrated with LangChain, Vercel AI SDK, and CrewAI using similar patterns.