hermes - 💡(How to fix) Fix session_search: loads entire conversation before truncation, causing slow recall for large sessions [1 pull requests]

Problem

session_search loads and formats the entire conversation for every matched session before truncating to ~100K chars around matches. For large sessions, this causes significant latency in the "recall" phase — entirely outside the FTS5 index search and LLM summarization steps.

Real-world data

A session with 866 messages takes noticeably longer to "recall" than one with 50 messages, even when the FTS5 match is instantaneous and only 2-3 messages actually match the query.

Root cause

In tools/session_search_tool.py, the flow is:

FTS5 search → get_messages_as_conversation(session_id) → _format_conversation(all messages) → _truncate_around_matches(text, query)

The entire conversation is loaded and formatted (line 438: messages = db.get_messages_as_conversation(session_id), then line 442: conversation_text = _format_conversation(messages)), and only then truncated. The expensive work is done before the relevant portion is identified.

Suggested fix

FTS5 has a built-in snippet() function that can return matching context directly from the index, without loading full conversations. Alternatively, the match positions returned by db.search_messages() could be used to load only the N messages before and after each match, then format just that window.

The key change: move truncation before formatting, or use FTS5 snippet() to avoid loading the full conversation entirely.

Affected code

• tools/session_search_tool.py — session_search() function
• hermes_state.py — SessionDB.get_messages_as_conversation() (called with full session, no windowing support)