langchain - 💡(How to fix) Fix Feature: callback handler for AI agent threat detection (Agent Threat Rules) [1 comments, 2 participants]

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain

Feature Description

LangChain agents using tool calling and chat models are increasingly exposed to prompt injection, role-play jailbreak, MCP tool poisoning, encoding bypass, and instructed SSRF attempts. There is currently no built-in callback handler dedicated to detecting agent-layer threats before or after LLM and tool execution.

Proposing a community-contributable callback handler that screens user input, tool calls, and tool outputs against an open detection rule set. The reference rule set is Agent Threat Rules (ATR), an open Apache-2.0 detection standard with 330 rules covering 9 attack categories, 100 percent NIST AI RMF mapping, 97.1 percent recall on the NVIDIA garak benchmark, and 0 percent false positive on 53,577 real-world MCP skills. ATR is shipped in production at Cisco AI Defense and Microsoft agent-governance-toolkit.

Concretely, a BaseCallbackHandler subclass that runs detection patterns inline (no external network dependencies, no new heavy deps) on on_llm_start, on_tool_start, and on_tool_end. On detection, raise or log based on user configuration. Patterns are drop-in regex strings, decoupled from any specific rule registry, so users can plug in custom rule sets.

Reference implementation: PR #37303 (auto-closed by require_issue_link.yml because no linked feature request issue existed; opening this issue to unblock).

Use Case

A user building a tool-calling agent wants to harden the agent against jailbreak and tool poisoning before LLM evaluation, with a single callback handler line. Today this requires writing a custom CallbackHandler from scratch and finding a rule set; a first-class community handler closes that gap.

Proposed Solution

Add a callback handler in libs/community or as a partner package, following the existing partner package pattern. Hardcode 5 to 10 high-value detection regex patterns inline; expose a hook for user-supplied additional patterns. Include a small README and one cookbook example.

Alternatives Considered

Custom user code (current state, reinvented per project). Heavy external dependency injection (rejected because of supply-chain surface). Network-loaded rules (rejected for cookbook reproducibility).

Additional context

ATR repo: https://github.com/Agent-Threat-Rule/agent-threat-rules License: Apache-2.0 ATR is also a Sigma rule pack candidate (PR open at SigmaHQ/sigma #5994), an in-toto attestation predicate (PR open at in-toto/attestation #552), and a Microsoft Pyrit dataset/scorer candidate (forthcoming).