llamaIndex - 💡(How to fix) Fix Proposal: Agent Threat Rules detection integration for LlamaIndex

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Fix Action

Fix / Workaround

Where it would fit in LlamaIndex. The cleanest integration point looks like the V2 instrumentation pipeline (llama_index.core.instrumentation, BaseEventHandler). A handler subscribed to AgentRunStepEvent, ToolCallStartEvent, ToolCallEndEvent, and CallStartEvent could match incoming text against the rule set and emit a new ThreatDetectionEvent into the same pipeline, which downstream observability tools like Phoenix, Langfuse, or OpenInference would already capture. The literalai callback at llama-index-integrations/callbacks/llama-index-callbacks-literalai is roughly the shape I am modeling against, including the get_dispatcher and add_event_handler pattern.

Option 1 docs only: a worked example in docs/examples/observability that wires a small inline EventHandler subclass with a handful of regex rules and shows the detection signal flowing through the dispatcher. Useful but not a reusable building block.

RAW_BUFFERClick to expand / collapse

I would like to propose an integration that exposes ATR-style threat detection as an instrumentation EventHandler in LlamaIndex, and I want to check the right shape before opening a PR given that CONTRIBUTING.md says new integration packages with their own pyproject.toml are auto-closed.

What ATR is. The Agent Threat Rules standard at https://github.com/Agent-Threat-Rule/agent-threat-rules is an Apache-2.0 detection catalog covering nine attack categories for LLM agents. It is shipped in production at Cisco AI Defense skill-scanner and Microsoft agent-governance-toolkit PolicyEvaluator. The catalog targets prompt injection, system prompt extraction, indirect injection markers, credential exfiltration, shell command injection, SSRF cloud metadata, and similar attacker patterns specific to tool-using agents.

Where it would fit in LlamaIndex. The cleanest integration point looks like the V2 instrumentation pipeline (llama_index.core.instrumentation, BaseEventHandler). A handler subscribed to AgentRunStepEvent, ToolCallStartEvent, ToolCallEndEvent, and CallStartEvent could match incoming text against the rule set and emit a new ThreatDetectionEvent into the same pipeline, which downstream observability tools like Phoenix, Langfuse, or OpenInference would already capture. The literalai callback at llama-index-integrations/callbacks/llama-index-callbacks-literalai is roughly the shape I am modeling against, including the get_dispatcher and add_event_handler pattern.

Three options I can see, ordered from least to most invasive.

Option 1 docs only: a worked example in docs/examples/observability that wires a small inline EventHandler subclass with a handful of regex rules and shows the detection signal flowing through the dispatcher. Useful but not a reusable building block.

Option 2 small core helper: a thin generic ThreatDetectionEventHandler class in llama-index-core under callbacks or instrumentation that takes a list of rule tuples and a strategy ('block' raises, 'flag' emits an event), with no opinionated rule defaults. Users would supply their own rules or load the ATR catalog with a few lines of code.

Option 3 separately published integration: a PyPI package outside this repo, since CONTRIBUTING.md is clear that new integration packages with new pyproject.toml are auto-closed. I am happy to do this; I would just want to know if a docs example or core helper is welcome alongside it so users discover it.

Question for maintainers: which of these three would you accept a PR for, and does the answer change if no new dependency is added (the implementation needs only stdlib re plus the existing instrumentation API)? Apache-2.0 license either way.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING