hermes - 💡(How to fix) Fix Feature request: Pre-response validation hook for capability claims [1 participants]

Problem

Hermes Agent has no pre-response validation mechanism. When an agent makes a capability claim ("can do X", "doesn't support Y", "is implemented"), there is no automated gate that verifies the claim before it is delivered to the user.

Current flow:

Desired flow:

Impact

Without this, every capability claim is a single point of trust. Agents frequently make unverified claims that reach the user before any validation occurs. Corrections are always reactive, never preventive. For power users managing complex multi-platform setups (Telegram, WeChat, Discord, cron jobs, etc.), this creates a pattern of repeated errors that erode trust.

Specific failure patterns observed:

1. Unverified capability claims: Agent states a feature doesn't exist without checking the codebase. Example: claiming WeChat group support "doesn't work" when gateway/platforms/weixin.py contains full group policy logic and the actual limitation is a platform-level iLink Bot account type restriction.

2. Reactive error correction only: When a claim is wrong, the agent learns only after the user flags it. There is no mechanism to self-audit before delivery.

3. No hook for custom validation rules: Users cannot inject domain-specific pre-flight checks (e.g., "before claiming a cron job exists, verify its ID in the scheduler").

Proposed Solutions (in order of implementation complexity)

Option A — Prompt-level pre-response hook (easiest, no code change)

Add a configurable pre-response validation step in the system prompt / AGENTS.md convention. The agent is instructed to run verification checks before delivering any capability claim. This still relies on the agent's self-discipline but is the fastest path.

Option B — Built-in pre-response middleware (recommended)

Add a lightweight hook in run_agent.py (or equivalent) that fires before text responses are returned:

# Conceptual location: run_agent.py, before return in LLM text response path
if text_response:
    for hook in self._pre_response_hooks:
        text_response = hook.run(text_response, context)

Hooks are registered at startup and can implement capability claim validation, tone checks, or any user-defined logic. This is how production agent systems handle reliability guarantees.

Option C — Declarative capability validation schema

A structured schema where skills / platform adapters declare their actual capabilities, and the agent cross-references claims against this schema before outputting them. More complex but most robust.

Environment

• Hermes Agent latest (NousResearch/hermes-agent)
• Python-based, gateway architecture
• Multi-platform (Telegram, WeChat via iLink, Discord, etc.)

References

• Related: capability-claim-gate skill pattern for per-user pre-flight validation
• Similar patterns in other agent frameworks (LangChain callbacks, AutoGPT hooks)