hermes - ✅(Solved) Fix [Bug]: Gateway can show a visible interim success before an invalid final-response failure [1 pull requests, 2 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#14154Fetched 2026-04-23 07:46:34
View on GitHub
Comments
2
Participants
1
Timeline
8
Reactions
0
Participants
Timeline (top)
labeled ×3commented ×2cross-referenced ×2closed ×1

Error Message

Relevant Logs / Traceback

The stream consumer sets already_sent = True as soon as any interim/streamed chunk is delivered to the user. Later, in gateway/run.py _handle_message_with_agent, when agent_result["failed"] is true and final_response is empty, a generic failure string is synthesised (around the "The request failed: {error}" / "⚠️ Session too large..." block). That failure string is then returned to the user without any contextualisation acknowledging that an earlier visible message was only intermediate — producing the contradictory UX.

Root Cause

Root Cause Analysis (optional)

Fix Action

Fixed

PR fix notes

PR #14155: fix(gateway): add failure context after visible interim replies

Description (problem / solution / changelog)

What does this PR do?

Narrow, messaging-only fix for the contradictory UX where the gateway shows a visible interim/streamed assistant message that looks like success, and then the run later fails validation — so the user sees an apparent "success" followed by an invalid-final-response failure with no acknowledgement that the earlier visible message was only intermediate.

When the run's final_response is empty and failed is true and an interim/streamed reply has already reached the user (already_sent=True), the final failure text is now prefixed with:

Note: the earlier message was an intermediate update, not a completed final answer.

The two messages then read coherently as "that was intermediate; here's the real status" instead of "it said it worked, then said it failed." Retry / validation logic is untouched; no adapter changes; no change to when messages are sent.

Related Issue

Fixes #14154

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • gateway/run.py: added pure helper _prefix_failure_with_interim_context(response, *, failed, interim_visible) and _INTERIM_FAILURE_PREFIX constant; called from _handle_message_with_agent immediately after the existing block that synthesises a failure string for failed && empty final_response. Idempotent (never double-prefixes).
  • tests/gateway/test_run_progress_topics.py: 4 unit tests on the helper — failure-after-visible-interim is prefixed, failure-without-interim stays plain, prefixing is idempotent, non-failure responses are never prefixed.

How to Test

source .venv/bin/activate
scripts/run_tests.sh tests/gateway/test_run_progress_topics.py -q

Expected output (just ran locally on macOS): 27 passed, 1 warning in 11.50s.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass — only the targeted file tests/gateway/test_run_progress_topics.py was run (27 passed); I have not run the full suite
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS (Darwin 25.4.0)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A (messaging-only change, helper is self-descriptive)
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure string formatting, platform-agnostic
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

Targeted test run:

▶ running pytest with 4 workers, hermetic env, in /…/hermes-agent
  (TZ=UTC LANG=C.UTF-8 PYTHONHASHSEED=0; all credential env vars unset)
...........................                                              [100%]
27 passed, 1 warning in 11.50s

Changed files

  • gateway/run.py (modified, +24/-0)
  • tests/gateway/test_run_progress_topics.py (modified, +58/-0)

Code Example

response.content is empty
⚠️ Max retries (3) for invalid responses — trying fallback...
Max retries (3) exceeded for invalid responses. Giving up.
RAW_BUFFERClick to expand / collapse

Bug Description

On Discord (and any platform where the gateway can deliver an interim/streamed assistant message before the run completes), users can see what looks like a successful reply, and then moments later see the run surface an invalid-final-response failure. The two messages read as a contradiction: the first appears to confirm success, the second says the request failed — with no acknowledgement that the earlier visible message was only intermediate.

Real-world symptom reported on Discord:

  • User: "Add 30 dollars to dave receipt for lunch today"
  • Bot (visible to user): Added. Dave's at $767.47 now (4/20 lunch, $30).
  • Bot shortly after: invalid-final-response failure path, e.g. ❌ Max retries (3) exceeded for invalid responses. Giving up.

Steps to Reproduce

  1. Run the gateway on Discord with streaming/preview delivery enabled.
  2. Send a prompt that causes the agent to emit a visible interim/streamed reply early in the run (tool result summary, progressive edit, etc.).
  3. Force (or encounter) an invalid final response so the agent's retry/fallback path eventually returns {"failed": True} with an empty final_response.
  4. Observe that the user sees a first, success-looking visible message, followed shortly by a generic failure string, with no indication the earlier message was intermediate.

Note: this is the exact public-facing sequence observed. Internal retry behaviour upstream of the gateway is not being changed by this report.

Expected Behavior

When a visible interim/streamed reply has already reached the user and the run subsequently fails, the failure message should clearly indicate that the earlier visible message was only an intermediate update — not a completed final answer — so the two messages read coherently instead of contradicting each other.

Actual Behavior

The user sees a first visible message that reads like a successful completion, then a second message from the gateway / agent that reports an invalid-response failure. Nothing in the second message tells the user the earlier visible message was intermediate, so the overall experience reads as "it said it worked, then said it failed."

Affected Component

Gateway (Telegram/Discord/Slack/WhatsApp)

Messaging Platform (if gateway-related)

Discord

Operating System

Reported on Discord; reporter's OS not captured at report time.

Python Version

Not captured at report time.

Hermes Version

Not captured at report time; reproducible on current main.

Relevant Logs / Traceback

Indicative strings from the invalid-final-response path (no user log dump was captured):

response.content is empty
⚠️ Max retries (3) for invalid responses — trying fallback...
❌ Max retries (3) exceeded for invalid responses. Giving up.

Root Cause Analysis (optional)

The stream consumer sets already_sent = True as soon as any interim/streamed chunk is delivered to the user. Later, in gateway/run.py _handle_message_with_agent, when agent_result["failed"] is true and final_response is empty, a generic failure string is synthesised (around the "The request failed: {error}" / "⚠️ Session too large..." block). That failure string is then returned to the user without any contextualisation acknowledging that an earlier visible message was only intermediate — producing the contradictory UX.

Retry / validation logic itself (the response.content is empty / max-retries invalid-response handling) is working as intended; the issue is purely the messaging seam at the gateway boundary.

Proposed Fix (optional)

Narrow, messaging-only fix in gateway/run.py: when failed is true and already_sent is true, prefix the final failure text with a short note, e.g.:

Note: the earlier message was an intermediate update, not a completed final answer.

No change to retry logic, no change to when messages are sent, no change to adapters.

Are you willing to submit a PR for this?

  • I'd like to fix this myself and submit a PR

extent analysis

TL;DR

The most likely fix is to modify the gateway's failure message to acknowledge the earlier intermediate message when a run fails after sending a visible interim reply.

Guidance

  • Review the _handle_message_with_agent function in gateway/run.py to understand how the failure string is synthesized when agent_result["failed"] is true and final_response is empty.
  • Consider adding a conditional check for already_sent being true, and if so, prefix the failure string with a note indicating that the earlier message was an intermediate update.
  • Verify that the modified failure message provides a coherent user experience by testing the scenario described in the steps to reproduce.
  • Ensure that the fix does not introduce any unintended changes to the retry logic or message delivery timing.

Example

if agent_result["failed"] and already_sent:
    failure_text = "Note: the earlier message was an intermediate update, not a completed final answer. " + failure_text

Notes

The proposed fix is a narrow, messaging-only change, and does not alter the underlying retry logic or message delivery timing. However, it is essential to test the modified failure message to ensure it provides a coherent user experience.

Recommendation

Apply the proposed fix in gateway/run.py to modify the failure message and provide a more coherent user experience. This change should help resolve the issue without introducing unintended side effects.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [Bug]: Gateway can show a visible interim success before an invalid final-response failure [1 pull requests, 2 comments, 1 participants]