Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix [Bug]: TUI compression continuation creates ghost sessions with messages but incomplete metadata, polluting session_search results [2 pull requests, 1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#20001•Fetched 2026-05-06 06:39:21
View on GitHub
Comments
1
Participants
1
Timeline
10
Reactions
0
Author
lwj-9650lwj-9650
Participants
lwj-9650lwj-9650
Timeline (top)
labeled ×4cross-referenced ×2referenced ×2closed ×1

Root Cause

Context compression continuation creates new session rows, but the TUI session recovery path does not update api_call_count, title, or end_reason on these rows.

Evidence of duplication — same user message in 3 ghost + 1 normal session:

Ghost 20260504_223735 (api=0, msgs=139) → "签到读取记忆，然后查看一下T13..."
Ghost 20260504_222759 (api=0, msgs=108) → identical first message
Ghost 20260504_222140 (api=0, msgs=52)  → identical first message
Normal 20260504_215030 (api=45, msgs=226) → same first message, properly tracked

Another example — 6 ghost sessions all with the same first message:

Ghost 20260504_180530 (msgs=102) → "当前v0.11.0落后472个commit..."
Ghost 20260504_174410 (msgs=48)  → identical
Ghost 20260504_173546 (msgs=213) → identical
Ghost 20260504_172009 (msgs=178) → identical
Ghost 20260504_171153 (msgs=142) → identical
Ghost 20260504_165312 (msgs=94)  → identical

The pattern: each compression + session recovery creates a new root session row without updating metadata, leaving ghost records that look identical to normal sessions except for the missing api_call_count/title/end_reason.

Fix Action

Workaround

Manual SQL cleanup (safe because all ghost content is preserved in normal sessions):

DELETE FROM messages WHERE session_id IN (
    SELECT id FROM sessions
    WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL
);
-- Removed 43,505 duplicate messages

DELETE FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;
-- Removed 506 ghost sessions

Post-cleanup state: 374 sessions, 40,984 messages, 0 ghost sessions.

PR fix notes

PR #20009: fix(tui): reanchor sessions after compression rotation

Description (problem / solution / changelog)

Summary

  • re-anchor the TUI gateway session key after prompt-driven AIAgent session_id rotation from automatic compression
  • finalize the agent's effective session id instead of a stale pre-compression key
  • add regression coverage for prompt-submit rotation and close/finalize behavior

Fixes #20001

Tests

  • scripts/run_tests.sh tests/test_tui_gateway_server.py::test_session_close_finalizes_rotated_agent_session_id tests/test_tui_gateway_server.py::test_prompt_submit_syncs_session_key_after_agent_session_rotation tests/test_tui_gateway_server.py::test_prompt_submit_history_version_match_persists_normally tests/test_tui_gateway_server.py::test_session_compress_syncs_session_key_after_rotation -> 4 passed, 4 warnings
  • git diff --check -> passed

Notes

  • scripts/run_tests.sh tests/test_tui_gateway_server.py currently has unrelated existing failures in test_session_create_drops_pending_title_on_valueerror and test_browser_manage_connect_default_local_reports_launch_hint, both reproduced when run individually.

Changed files

  • tests/test_tui_gateway_server.py (modified, +90/-0)
  • tui_gateway/server.py (modified, +9/-2)

PR #20363: fix: resolve lazy session creation regressions (#18370 fallout)

Description (problem / solution / changelog)

Problem

PR #18370 (lazy session creation) introduced three regressions that accumulated over the past few days:

  1. Ghost sessions from compression (#20001) — _finalize_session() ends the stale session["session_key"] (the compression-ended parent) instead of agent.session_id (the live continuation). After auto-compression, continuation sessions are never finalized: ended_at=NULL, api_call_count=0, end_reason=NULL. One reporter accumulated 506 ghost sessions (57% of their DB) in 16 days.

  2. pending_title ValueError wedge (#19029) — The title flush moved from _start_agent_build to post-first-message but lost the except ValueError handler from #14334. ValueError (duplicate/invalid title) now leaves pending_title stuck forever — /title keeps showing the queued value, auto-title never fires.

  3. response=0 chars in gateway (#18765) — When the agent returns final_response=None with api_calls > 0 (partial failure, tool errors, etc.), the gateway does response = agent_result.get("final_response") or "" and silently sends nothing to the user.

Solution

Five focused production fixes:

Fix 1: _finalize_session() ends the correct session

Use session_id (computed from agent.session_id with fallback to session_key) for the end_session() call. One-line change.

Fix 2: Sync session_key after run_conversation() returns

Refactor _sync_session_key_after_compress() with policy flags (clear_pending_title, restart_slash_worker). Call it post-turn with clear_pending_title=False to preserve user title intent while updating session_key for downstream title/goal/finalize consumers. Restart the slash worker so /title etc. target the live session.

Fix 3: Clear pending_title on ValueError

Add except ValueError that drops the title (non-retryable) and logs. Transient exceptions keep pending_title for retry.

Fix 4: Gateway null-response normalization

Extract _normalize_empty_agent_response() helper that consolidates the existing failed handler and adds a catch-all for api_calls > 0 with no text. Interrupted turns stay empty (platform UX handles those).

Fix 5: SessionDB.finalize_orphaned_compression_sessions()

Non-destructive one-time migration: marks ghost continuation sessions as ended (end_reason="orphaned_compression") when their parent is confirmed ended with end_reason="compression". Preserves all messages. 7-day age cutoff.

Testing

  • 388 tests pass across test_tui_gateway_server.py + test_hermes_state.py + new regression tests
  • 9 pass in test_860_dedup.py (the original dedup test #18370 touched)
  • 7 pass in compression-related gateway tests
  • 14 new regression tests in test_lazy_session_regressions.py

Call order after fix

run_conversation()                    → agent may compress, rotating session_id
_sync_session_key_after_compress()    → session_key updated, slash worker restarted
                                        (pending_title preserved)
message.complete emit
goal continuation                     → uses correct session_key  
pending_title application             → applies to live continuation
auto-title                            → targets correct session_key

Refs

  • Fixes #20001 — TUI compression continuation creates ghost sessions
  • Fixes #19029 — pending_title ValueError wedge
  • Fixes #18765 — response=0 chars regression in gateway
  • Regression from #18370 — lazy session creation
  • Restores contract from #14334 — ValueError-drop on pending title
  • Supersedes #19029 (pending_title fix PR — incorporated here)

Changed files

  • cli.py (modified, +12/-0)
  • gateway/run.py (modified, +51/-27)
  • hermes_state.py (modified, +39/-0)
  • tests/test_lazy_session_regressions.py (added, +608/-0)
  • tests/test_tui_gateway_server.py (modified, +49/-35)
  • tui_gateway/server.py (modified, +50/-12)

Code Example

SELECT COUNT(*) FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;
-- Result: 506

---

Ghost 20260504_223735 (api=0, msgs=139) → "签到读取记忆，然后查看一下T13..."
Ghost 20260504_222759 (api=0, msgs=108) → identical first message
Ghost 20260504_222140 (api=0, msgs=52)  → identical first message
Normal 20260504_215030 (api=45, msgs=226) → same first message, properly tracked

---

Ghost 20260504_180530 (msgs=102) → "当前v0.11.0落后472个commit..."
Ghost 20260504_174410 (msgs=48)  → identical
Ghost 20260504_173546 (msgs=213) → identical
Ghost 20260504_172009 (msgs=178) → identical
Ghost 20260504_171153 (msgs=142) → identical
Ghost 20260504_165312 (msgs=94)  → identical

---

WHERE source = 'tui'
  AND title IS NULL
  AND ended_at IS NOT NULL      # Our ghosts have ended_at = NULL
  AND started_at < (now - 86400)
  AND NOT EXISTS (
      SELECT 1 FROM messages    # Our ghosts HAVE messages
      WHERE messages.session_id = sessions.id
  )

---

DELETE FROM messages WHERE session_id IN (
    SELECT id FROM sessions
    WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL
);
-- Removed 43,505 duplicate messages

DELETE FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;
-- Removed 506 ghost sessions
RAW_BUFFERClick to expand / collapse

Bug Description

Ghost sessions accumulate in state.db under a pattern not fully covered by the fix in #18370. These sessions contain real conversation messages but have incomplete metadata (api_call_count=0, title=NULL, end_reason=NULL), polluting session_search sort order.

Related to #12029 but with a different reproduction pattern — this happens in TUI/CLI-only setups (no gateway, no cron) and the ghost sessions contain messages (not empty stubs).

Environment

  • Hermes v0.12.0 (3 patches), installed from source
  • Backend: WSL2 on Windows, main model GLM-5.1 via zhiqi (智谱 Coding Plan)
  • Usage pattern: daily TUI sessions with frequent context compression
  • state.db size: ~960MB

Reproduction Steps

  1. Use Hermes TUI daily with context compression enabled
  2. Allow multiple compression cycles within a single session (compression continuation)
  3. Use /new to start new sessions or let TUI recover after interruptions
  4. After ~16 days, observe ghost sessions accumulating in state.db

Observed Behavior

After 16 days of TUI usage, state.db had 506 ghost sessions out of 880 total (57%):

SELECT COUNT(*) FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;
-- Result: 506
MetricGhost SessionsNormal Sessions
Count506374
Total messages43,50540,906
Avg msgs/session86.0109.4
api_call_count0>0
titleNULLSet
end_reasonNULLSet
parent_session_idNULL (all root)Mixed

Key observations:

  • All 506 ghost sessions are root sessions (parent_session_id IS NULL)
  • They contain real messages (not empty stubs) — avg 86 messages per session
  • Their content is duplicated from corresponding normal sessions (confirmed by first-user-message matching)
  • Daily accumulation rate: ~20-40 new ghost sessions per day

Root Cause

Context compression continuation creates new session rows, but the TUI session recovery path does not update api_call_count, title, or end_reason on these rows.

Evidence of duplication — same user message in 3 ghost + 1 normal session:

Ghost 20260504_223735 (api=0, msgs=139) → "签到读取记忆，然后查看一下T13..."
Ghost 20260504_222759 (api=0, msgs=108) → identical first message
Ghost 20260504_222140 (api=0, msgs=52)  → identical first message
Normal 20260504_215030 (api=45, msgs=226) → same first message, properly tracked

Another example — 6 ghost sessions all with the same first message:

Ghost 20260504_180530 (msgs=102) → "当前v0.11.0落后472个commit..."
Ghost 20260504_174410 (msgs=48)  → identical
Ghost 20260504_173546 (msgs=213) → identical
Ghost 20260504_172009 (msgs=178) → identical
Ghost 20260504_171153 (msgs=142) → identical
Ghost 20260504_165312 (msgs=94)  → identical

The pattern: each compression + session recovery creates a new root session row without updating metadata, leaving ghost records that look identical to normal sessions except for the missing api_call_count/title/end_reason.

Why #18370 Did Not Fix This

The prune_empty_ghost_sessions() from #18370 only matches:

WHERE source = 'tui'
  AND title IS NULL
  AND ended_at IS NOT NULL      # Our ghosts have ended_at = NULL
  AND started_at < (now - 86400)
  AND NOT EXISTS (
      SELECT 1 FROM messages    # Our ghosts HAVE messages
      WHERE messages.session_id = sessions.id
  )

Our ghost sessions fail both critical filters:

  1. ended_at IS NULL — sessions were never marked as ended
  2. Messages exist — these are not empty stubs

The lazy session creation fix prevents new empty stubs from being created on TUI open/close, but does not address the case where compression continuation creates sessions that receive messages but never get their metadata updated.

Impact

  1. session_search returns wrong results: The effective_last_active sort in list_sessions_rich() ranks ghost sessions high because their message timestamps are recent (from compression time), pushing actual latest sessions out of the top-N results.

  2. session_search auxiliary model timeouts: With 506 duplicate sessions, the auxiliary model has to process far more data than necessary, contributing to repeated timeouts observed in logs.

  3. State bloat: Ghost sessions consumed ~48% of total message storage (43,505 out of 84,489 messages were duplicates).

Workaround

Manual SQL cleanup (safe because all ghost content is preserved in normal sessions):

DELETE FROM messages WHERE session_id IN (
    SELECT id FROM sessions
    WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL
);
-- Removed 43,505 duplicate messages

DELETE FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;
-- Removed 506 ghost sessions

Post-cleanup state: 374 sessions, 40,984 messages, 0 ghost sessions.

Suggested Fix

Either:

  1. Broaden prune_empty_ghost_sessions() to also catch sessions where api_call_count = 0 AND title IS NULL AND ended_at IS NULL, even if they have messages (since their content is duplicated in the corresponding normal session).

  2. Fix the root cause: Ensure the TUI compression continuation path properly writes api_call_count, title, and end_reason to the session row when a session is finalized — similar to how #18370 added _ensure_db_session() but extended to also update metadata on session end.

extent analysis

TL;DR

The most likely fix involves updating the prune_empty_ghost_sessions() function to catch sessions with missing metadata or ensuring the TUI compression continuation path properly updates session metadata.

Guidance

  • Identify and remove duplicate ghost sessions using the provided manual SQL cleanup script as a temporary workaround.
  • Investigate broadening the prune_empty_ghost_sessions() function to catch sessions with api_call_count = 0, title IS NULL, and ended_at IS NULL, even if they contain messages.
  • Review the TUI compression continuation path to ensure it properly updates api_call_count, title, and end_reason when a session is finalized.
  • Verify the fix by checking the session_search results and auxiliary model timeouts after applying the changes.

Example

DELETE FROM messages WHERE session_id IN (
    SELECT id FROM sessions
    WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL
);

DELETE FROM sessions
WHERE api_call_count = 0 AND title IS NULL AND end_reason IS NULL;

Notes

The provided workaround is safe since all ghost content is preserved in normal sessions. However, a permanent fix requires addressing the root cause of the issue, which may involve updating the TUI compression continuation path or broadening the prune_empty_ghost_sessions() function.

Recommendation

Apply the workaround using the manual SQL cleanup script to remove duplicate ghost sessions, and then investigate updating the prune_empty_ghost_sessions() function or fixing the TUI compression continuation path to prevent future occurrences.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#tool integration#LLM response#prompt template#agent execution

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [Bug]: TUI compression continuation creates ghost sessions with messages but incomplete metadata, polluting session_search results [2 pull requests, 1 comments, 1 participants]