Bug: Bridge-worker assistant replies silently missing from state.db

Summary

When chatting through the TypeScript Hermes WebUI (Agent Bridge path), assistant (AI) replies are frequently missing from state.db (Hermes Agent's SQLite session store). The data IS correctly stored in the WebUI's own session-store (TypeScript SQLite), but the Hermes Agent's state.db only has the user message, not the corresponding assistant reply. This causes data inconsistency between the Python WebUI (8787) and TypeScript WebUI (5173).

Severity

HIGH — Silent data loss. No error messages, no exceptions logged. Affects all users of the TypeScript WebUI who rely on state.db for conversation history (e.g., Python WebUI, Hermes CLI, Insights).

Environment

• WebUI: TypeScript ~/hermes-web-ui/ (Vite dev server on port 8648, Socket.IO chat-run)
• Agent Bridge: hermes_bridge.py running as a child process managed by agent-bridge/manager.ts
• Hermes Agent: NousResearch/hermes-agent (via systemd gateway service + bridge worker)
• Session: mph7v2r9tdeata — a single session spanning ~900+ messages

Evidence

1. Database state comparison

After sending a message, the assistant reply exists in session-store but NOT in state.db:

session-store (dev):  491 messages, last at 10:11:43  ✅ assistant reply present
state.db:             819 messages, last at 10:11:34  ❌ assistant reply missing

2. Debug probe confirms nothing to flush

Added diagnostic probe at conversation_loop.py:3961 (after _persist_session()). Output from worker stderr:

[hermes-bridge-worker:default] [DBG-BUG2] NOTHING TO FLUSH! flush_from=490 msg_len=490 last_asst_role=assistant

This confirms:

• The messages list has len=490 (including the assistant reply at index 489)
• _last_flushed_db_idx=490 (set by bridge's _align_prepersist_flush_cursor)
• flush_from = max(history_len, _last_flushed_db_idx) = 490 ≥ len(messages) = 490
• _flush_messages_to_session_db() writes messages[490:] which is an empty list

3. No exceptions thrown at any catch point

Four except blocks were instrumented with [DBG-BUG] probes — none fired:

4. WAL checkpoint confirms data never written

PRAGMA wal_checkpoint(TRUNCATE)  → (0, 0, 0)  # SQLITE_OK, 0 pages

Zero uncheckpointed pages. The data was genuinely never written to state.db.

5. Streamed responses reach the frontend

The user SEES the assistant reply in the WebUI frontend. The reply is correctly streamed through the bridge's Socket.IO events and saved to the TypeScript session-store via flushResponseRunToDb(). The problem is ONLY in the state.db write path.

Root Cause

The conversation loop's streaming callback delivers the assistant response to the bridge/frontend without appending a structured assistant message dict to the AIAgent's internal messages list.

At the end of run_conversation() (in agent/conversation_loop.py):

# Line ~3960
agent._drop_trailing_empty_response_scaffolding(messages)
agent._persist_session(messages, conversation_history)  # Writes messages to state.db

The _drop_trailing_empty_response_scaffolding() removes flagged scaffolding messages from the tail. The resulting messages list only contains conversation_history + user_input (no assistant reply). Since the bridge's _align_prepersist_flush_cursor() set _last_flushed_db_idx = len(conversation_history) + 1, _flush_messages_to_session_db() finds flush_from = max(history_len, history_len+1) = len(messages) and writes nothing.

The assistant reply exists as the final_response string returned from run_conversation(), but not as a structured message dict in the messages list that _persist_session() writes.

Why this doesn't affect the Python WebUI (8787)

The Python WebUI handles sessions differently — it calls AIAgent in-process via streaming.py and uses state_sync.py to independently sync session metadata to state.db. It doesn't go through the bridge → worker architecture that has the _last_flushed_db_idx issue.

Data Flow

User sends message
  ↓
TypeScript backend (Socket.IO)
  ↓
handle-bridge-run.ts → addMessage() → session-store ✅ (immediate)
  ↓
hermes_bridge.py → _prepersist_user_message() → state.db ✅ (writes user msg)
  ↓
Worker process → AIAgent.run_conversation()
  ↓  (streams response to frontend via callback - user SEES it ✅)
  ↓  (but assistant NOT added to messages[] structured list ❌)
  ↓
conversation_loop.py:3961 → _persist_session(messages, ...)
  ↓
_flush_messages_to_session_db()
  ↓
flush_from=490, messages[490:] = [] → writes NOTHING to state.db ❌

Proposed Fix

In agent/conversation_loop.py, before _persist_session() at line ~3961, ensure final_response is injected into messages:

agent._drop_trailing_empty_response_scaffolding(messages)
# Inject streaming response into messages for persistence
if final_response and isinstance(final_response, str) and final_response.strip():
    if not (messages and isinstance(messages[-1], dict) and 
            messages[-1].get("role") == "assistant" and 
            messages[-1].get("content") == final_response):
        messages.append({"role": "assistant", "content": final_response})
agent._persist_session(messages, conversation_history)

This fix was tested and resolved the issue in a single-turn test.

Files Involved

Additional diagnostic probes added during debugging:

• [DBG-BUG] at 4 silent-catch except blocks
• [DBG-BUG2] at final _persist_session() exit
• [DBG-BUG3] at _session_db is None check
• [DBG-BUG4] at final_response injection point

Reproduction

1. Chat through TypeScript WebUI (port 5173 → backend 8648)
2. Send any message
3. Compare state.db vs session-store for the same session
4. Observe missing assistant replies in state.db
5. Session-store has complete data

Workaround

Use the Python WebUI (port 8787) or Hermes CLI for sessions where state.db data integrity is critical.