Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix [hindsight plugin] Support update_mode + topic-aware document_id to deduplicate retained memories [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#20115•Fetched 2026-05-06 06:38:40
View on GitHub
Comments
0
Participants
1
Timeline
7
Reactions
0
Author
yunyngyunyng
Participants
yunyngyunyng
Timeline (top)
labeled ×4cross-referenced ×2referenced ×1

Root Cause

Same configuration change (e.g. "nudge_interval changed from 10 to 20") is retained as separate documents across every session that mentions it. Consolidation then produces conflicting facts because the input observations themselves contain stale duplicates. Mitigated by raising retain_every_n_turns from 1 to 3, but the root cause remains.

Fix Action

Fix / Workaround

Workaround Until Fix

PR fix notes

PR #20222: feat(hindsight): probe API for update_mode='append' to dedupe across processes

Description (problem / solution / changelog)

Mirrors the pattern already shipping in hindsight-integrations/openclaw: probe <api_url>/version once per process and gate on Hindsight ≥ 0.5.0. When the API supports update_mode='append', retains use a stable session-scoped document_id (just session_id) so cross-process retains for the same session merge into one document. When unsupported or the probe fails, fall back to the existing per-process unique f"{session_id}-{start_ts}" doc_id and don't pass update_mode — the resume-overwrite fix (#6654) keeps working unchanged on legacy servers.

Closes the dedup half of #20115. The proposed document_id_strategy config knob isn't needed: auto-detection via the same /version probe the OpenClaw plugin already uses gives the same outcome with no extra config burden, and the choice is purely a function of what the server can do.

Plumbing

  • New module-level helpers (_meets_minimum_version, _fetch_hindsight_api_version, _check_api_supports_update_mode_append) cache the result per api_url so every provider in the process pays one /version round-trip total.
  • One-time WARN when the API is older than 0.5.0, nudging the user to upgrade for cross-session dedup.
  • New instance helper _resolve_retain_target(fallback_doc_id) -> (document_id, update_mode) wired into sync_turn and the on_session_switch flush path.
  • For local_embedded the probe URL is taken from the live client (client.url) so we hit the actual daemon port, not the configured default.
  • update_mode is set on the per-item dict; Hindsight.aretain_batch already threads item['update_mode'] through.

Test plan

  • uv run pytest tests/plugins/memory/test_hindsight_provider.py tests/agent/test_memory_session_switch.py — 108/108 passed.
  • New TestUpdateModeAppendCapability (5 cases): legacy fallback, modern stable+append, per-URL cache, one-time WARN, flush-on-switch resolves against the OLD session.

End-to-end against installed ~/.hermes/hermes-agent

unreachable API (legacy fallback)live hindsight-embed 0.5.6
document_idlegacy-session-20260505_144153_663958modern-session
item.update_modeNone (not set)"append"

test_hermes_embedded_smoke.py passes in 90s.

Changed files

  • plugins/memory/hindsight/__init__.py (modified, +145/-6)
  • tests/agent/test_memory_session_switch.py (modified, +8/-0)
  • tests/plugins/memory/test_hindsight_provider.py (modified, +104/-0)
RAW_BUFFERClick to expand / collapse

Background

  • Hindsight 0.5.0+ supports update_mode='append' and update_mode='replace' with same-document-id delete-then-rebuild semantics.
  • The Hermes hindsight plugin currently hardcodes document_id = f"{self._session_id}-{start_ts}" (see plugins/memory/hindsight/__init__.py:931), which guarantees a new document per session, defeating dedup entirely.
  • The plugin never passes update_mode to the retain call — the parameter exists in Hindsight's MCP/API but is unreachable from the Hermes side.

Symptom

Same configuration change (e.g. "nudge_interval changed from 10 to 20") is retained as separate documents across every session that mentions it. Consolidation then produces conflicting facts because the input observations themselves contain stale duplicates. Mitigated by raising retain_every_n_turns from 1 to 3, but the root cause remains.

Example: In a 27-minute window on a single day, the same config change was retained 16+ times across sessions before throttling was applied.

Proposed Changes

1. Expose update_mode parameter in plugin retain call

  • Default to "replace" (Hindsight default).
  • Allow configuration via ~/.hermes/hindsight/config.json (e.g. "update_mode": "replace").

2. Add document_id_strategy config option

Strategydocument_id formatCostDedup scope
session (current){session_id}-{start_ts}ZeroNone — every session is a new doc
date{bank_id}-{date}Near-zeroSame-day merges; cross-day still duplicates
topic{bank_id}-{topic_hash}One extra lightweight LLM call per retain batchSame-topic convergence across all sessions

Recommended default: date — it eliminates the majority of same-day duplicates with zero additional cost and no LLM dependency. The topic strategy is for power users who want maximum dedup.

3. For "topic" strategy: lightweight topic extraction

  • Use the auxiliary model (or a dedicated cheap model) to extract a short topic key from the retained content.
  • Cache the topic→hash mapping to avoid redundant LLM calls.
  • Fall back to "date" on LLM failure.

Workaround Until Fix

No in-plugin solution. Manual cleanup via Hindsight API (DELETE /documents/{id}) is the only option.

Related

  • #19737 — Plugin missing retainRoles and retainOverlapTurns (other known gaps between Hindsight API and Hermes plugin)
  • Hindsight 0.5.0 changelog — update_mode introduction

extent analysis

TL;DR

To address the issue of duplicate documents, consider implementing a document_id_strategy config option, such as using a date or topic based strategy, and expose the update_mode parameter in the plugin retain call.

Guidance

  • Implement the proposed change to expose the update_mode parameter in the plugin retain call, defaulting to "replace", to allow for deduplication of documents.
  • Introduce a document_id_strategy config option to control the format of the document_id, with options such as session, date, or topic, to reduce duplicates.
  • Consider using the date strategy as the recommended default, as it eliminates same-day duplicates with zero additional cost and no LLM dependency.
  • For the topic strategy, implement lightweight topic extraction using an auxiliary model or a dedicated cheap model, with caching to avoid redundant LLM calls.

Example

No code snippet is provided as it is not clearly supported by the issue, but an example of the document_id_strategy config option in ~/.hermes/hindsight/config.json could be:

{
  "update_mode": "replace",
  "document_id_strategy": "date"
}

Notes

The topic strategy may introduce additional cost and dependency on LLM, and the date strategy may still produce duplicates across days. Manual cleanup via Hindsight API is the only workaround until the fix is implemented.

Recommendation

Apply the workaround by implementing the proposed changes, specifically exposing the update_mode parameter and introducing the document_id_strategy config option, to address the root cause of the issue.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#file not found#serialization error#model compatibility#GPU setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [hindsight plugin] Support update_mode + topic-aware document_id to deduplicate retained memories [1 pull requests, 1 participants]