hermes - 💡(How to fix) Fix Feature: Allow memory provider to serve as session search backend (bypass local FTS5)

Problem

Hermes Agent has two independent retrieval paths for past conversations, and they never talk to each other:

When a user configures an external memory provider like OpenViking, the provider already receives all session data via sync_turn() and on_session_end()/commit. The OpenViking plugin stores every conversation turn and runs VLM-based memory extraction on commit. This means the memory provider has richer, semantically searchable archives of the same conversations.

However, session_search always hits the local SQLite FTS5 — it has no awareness of the active memory provider. The agent system prompt (SESSION_SEARCH_GUIDANCE in prompt_builder.py) directs it to use session_search by default for cross-session recall, which completely bypasses the memory provider.

Why This Matters

1. Token efficiency: Semantic vector search retrieves relevant context in 1 query, while FTS5 keyword matching may need multiple attempts with different query phrasing
2. Result quality: FTS5 returns raw message snippets; memory providers like OpenViking return VLM-extracted, structured memories
3. Data duplication: Session data is stored in two places (local DB + provider) but only the local DB is searchable via session_search
4. Wasted potential: Memory providers like OpenViking already have prefetch() and queue_prefetch() methods in their plugin interface — suggesting someone already envisioned this integration, but it was never wired up

Proposed Solutions

Option A — Memory provider registers a session search backend

Extend the BaseMemoryProvider interface with an optional session_search(query, limit, ...) method. When the active memory provider implements it, the session_search tool delegates to the provider instead of hitting local FTS5. Backwards-compatible: providers without it fall back to current FTS5.

Option B — Config-level switch

Add a config key like memory.search_backend: auto | local | provider:

• auto (default): use provider if it supports search, fallback to local FTS5
• local: always use FTS5 (current behavior)
• provider: force delegate to memory provider

Option C — Unified search tool

Replace session_search + provider-specific search tools (e.g. viking_search) with a single unified search tool that knows which backend to query. Agent uses one tool for all cross-session retrieval regardless of backend.

Implementation Notes

• OpenViking plugin already exposes viking_search and has prefetch()/queue_prefetch() lifecycle hooks — well positioned for Option A
• session_search_tool.py operates purely on hermes_state.SessionDB (SQLite) — needs to accept a provider reference at construction or via a registry
• memory_manager.py already orchestrates provider lifecycle — could also manage search delegation
• prompt_builder.py SESSION_SEARCH_GUIDANCE would need updating

Related

• PR #10463 / Issue #8705: OpenViking session lifecycle fixes — baseline for making OpenViking a full search backend
• OpenViking plugin already has prefetch(query, session_id) — the hook point for session-aware retrieval