hermes - 💡(How to fix) Fix [Feature]: First-Class Claim Verification & Audit Mechanism for Hermes Agent

Problem or Use Case

Hermes Agent currently lacks a structured, supported mechanism for verifying assistant claims or auditing agent behavior. Today, anyone wanting to add claim verification has to do it out-of-tree — patching or wrapping Hermes in ways that are reversible but ad-hoc, fragile across upgrades, and duplicated across integrators. Concrete gaps: No pre-assistant hook point — there is no documented place to intercept assistant output before it is finalized/emitted to the user. No verifier contract — third-party verifiers have no stable interface to implement against (claim/response context in, verdict + metadata out). No auditor loop — there is no canonical way to re-evaluate session content after the fact against a verifier. No feedback channel — verification results have nowhere defined to land (session record, telemetry, future-turn context). Use case: Operators running Hermes in higher-stakes settings (research workflows, internal tooling, anything user-facing) want to layer claim verification, policy checks, or evaluation harnesses on top of the agent without forking it. Right now, every integrator reinvents the hook points and the audit format.

Note: I am not a coder, i got frustrated with constant hallusinations and got claude opus to do this solution for me. It is working.

Proposed Solution

Introduce optional, first-class primitives in Hermes for verification and audit: Pre-assistant hooks — a documented extension point that runs before the assistant's response is finalized, with stable input/output contracts. Verifier contract — a defined interface (in-tree or pluggable) that consumes claim/response context and returns a verdict plus structured metadata. Auditor loop — an optional process (background or post-hoc) that applies the verifier contract over session content. Feedback channel — a structured surface for verification results to flow into the canonical session record, telemetry, and/or subsequent turns. The exact shape (sync vs. async, per-claim vs. per-response, in-process vs. RPC, sidecar artifact vs. inline session field) is intentionally left open for maintainer discussion — this issue is meant to scope the direction, not prescribe the implementation. Acceptance criteria (draft): Documented pre-assistant hook point with a stable API. Defined verifier contract / interface. Optional auditor loop with a documented invocation model. Feedback channel from verifier → session/state. Example verifier referenced in docs.

Alternatives Considered

Out-of-tree implementation (status quo). Wrap or patch Hermes externally to add hooks and a verifier. Works, but is reversible, fragile across upstream changes, and forces every integrator to reinvent hook points and audit formats. Does not produce a shared ecosystem surface. Fork Hermes. Maintain a downstream fork with verification baked in. High maintenance cost, fragments the ecosystem, and discourages contributions back upstream. External-only audit (post-hoc log scraping). Run verification entirely outside Hermes by parsing session JSONL after the fact. Loses the ability to gate or annotate responses pre-emission, and depends on an undocumented log schema. Tool-layer verification. Implement verification as just another "tool" the agent can call. Possible, but conflates agent-driven tool use with operator-driven policy/audit, and gives no guarantees that verification actually runs. Do nothing. Leave verification entirely to integrators. Viable if maintainers consider this out of scope, but it leaves a recurring papercut for anyone deploying Hermes in settings where claim quality matters.

Feature Type

Performance / reliability

Scope

None

Contribution

• I'd like to implement this myself and submit a PR

Debug Report (optional)