[Bug]: Telegram streaming truncates at unclosed backtick during finalize when transient httpx hiccup occurs (regression edge case from #25710 / d44dafdb4)

TL;DR

After the fix for #25710 (commit d44dafdb4 — set REQUIRES_EDIT_FINALIZE so final MarkdownV2 edit is not skipped), Telegram streaming can still surface a truncated final message — but in a different way than #25710:

• the model returns a complete response (finish_reason=stop, full response_len in agent.log),
• the gateway logs Suppressing normal final send ... content_delivered=True,
• but the user-visible Telegram message is cut off at an unclosed backtick (a half code-span).

The truncation point is consistently at a single backtick where the next characters never appear. This looks like a missed corner of #25710 / d44dafdb4: the new REQUIRES_EDIT_FINALIZE=True path on Telegram still has a hole when a transient httpx.RemoteProtocolError overlaps the finalize=True MarkdownV2 edit while the streamed payload contains an unclosed inline-code marker.

Environment

• Hermes Agent: built from origin/main (after d44dafdb4 / PR for #25710 merged)
• Platform: Telegram gateway, long polling
• streaming.enabled: true, display.streaming: true, edit-based streaming
• Adapter: TelegramAdapter with REQUIRES_EDIT_FINALIZE: bool = True
• Model: claude-opus-4-7 (custom anthropic_messages provider)
• OS: macOS, python-telegram-bot via long polling

Steps to Reproduce

This is timing-sensitive but the conditions are:

1. Telegram edit-based streaming enabled (REQUIRES_EDIT_FINALIZE = True).

2. An assistant response that, in its final raw text, contains an inline-code marker that ends up unclosed at a streaming boundary — e.g. a message like:

   - **Honcho `… something here`

   where the final emitted token-stream pauses with the buffer ending at `… **Honcho `` and the closing backtick + tail is still pending.

3. While the consumer is in the middle of finalize=True MarkdownV2 edit, a transient httpx.RemoteProtocolError: Server disconnected without sending a response. occurs on the Telegram long-poll HTTP client (e.g. brief upstream blip — auto-recovers in ~5 s with Telegram polling resumed after network error).

4. The stream consumer reports _final_content_delivered = True because the mid-stream plaintext edit at that point returned success=True.

5. gateway/run.py sees content_delivered=True and emits Suppressing normal final send for session ...: final delivery already confirmed, so the normal fallback "full final send" never runs.

6. The visible Telegram message ends at the unclosed backtick. The remainder of the model's reply is lost from the user's view.

Observed Behavior

Logs (real capture, redacted):

agent.log:
  2026-05-20 15:36:49 ... API call #1: ... latency=11.3s
  2026-05-20 15:37:20 ... Turn ended: reason=text_response(finish_reason=stop) ... response_len=506
  2026-05-20 15:37:22 gateway.run: Suppressing normal final send for session ...:
       final delivery already confirmed (streamed=True previewed=False content_delivered=True).
  2026-05-20 15:38:06 ... inbound message: ... msg='那我们前几天弄过honcho，既然只读memory.md，那honcho的相关记忆又是啥时候注入的呢'
  2026-05-20 15:38:17 WARNING gateway.platforms.telegram:
       [Telegram] Telegram network error, scheduling reconnect:
       httpx.RemoteProtocolError: Server disconnected without sending a response.
  2026-05-20 15:38:23 INFO gateway.platforms.telegram:
       [Telegram] Telegram polling resumed after network error (attempt 1)
  2026-05-20 15:38:57 ... Turn ended: reason=text_response(finish_reason=stop) ... response_len=876
  2026-05-20 15:38:59 gateway.run: Suppressing normal final send for session ...:
       final delivery already confirmed (streamed=True previewed=False content_delivered=True).

  2026-05-20 15:44:21 ... API call #11: ... latency=17.0s
  2026-05-20 15:44:22 ... Turn ended: reason=text_response(finish_reason=stop) ... response_len=882
  2026-05-20 15:44:24 gateway.run: Suppressing normal final send for session ...:
       final delivery already confirmed (streamed=True previewed=False content_delivered=True).

Two consecutive responses (response_len=876, response_len=882) were both complete on the model side (finish_reason=stop) but landed on Telegram cut off at an unclosed backtick. The user followed up with "你是不是又被截断了，把刚刚的消息重新发我下" and the regenerated reply was truncated at the same place.

The httpx disconnect at 15:38:17 is the catalyst — when the same condition does not happen, finalize succeeds cleanly. But on the affected runs, finalize cannot deliver the full text and the gateway suppresses the fallback because it trusts the stream consumer's "content delivered" claim.

Expected Behavior

• Either the finalize=True MarkdownV2 edit succeeds end-to-end and Telegram shows the entire final text, or
• the stream consumer / adapter clearly reports final_content_delivered = False so gateway/run.py falls through to the normal final-send path and the user still receives the entire reply (possibly via re-send + best-effort delete of the partial).

The user must never see a silent truncation at an unclosed inline-code marker while the gateway log claims content_delivered=True.

Root Cause Analysis (hypothesis)

Three things compose into this regression:

1. stream_consumer.py short-circuits on identical text

   In _send_or_edit, the path for _message_id is not None includes:

   if text == self._last_sent_text and not (
       finalize and self._adapter_requires_finalize
   ):
       return True

   With REQUIRES_EDIT_FINALIZE = True (from #25710), the finalize=True edit now actually fires when text is unchanged. Good. But the next path passes the text through adapter.edit_message(..., finalize=True) which calls format_message(...) (plain → MarkdownV2). If that runs concurrently with a transient httpx disconnect, the await can raise / time out after the underlying network layer has already partially committed the edit, and:

2. _final_content_delivered is set from the mid-stream edit, not the finalize edit

   In stream_consumer.py at the end of the streaming loop:

   if got_done:
       if current_update_visible and self._accumulated:
           self._final_content_delivered = True

   current_update_visible is set by the mid-stream _send_or_edit(display_text, finalize=(got_done or got_segment_break)). When the finalize edit silently fails (or partially succeeds against a stale connection), _final_content_delivered is already True. The subsequent finalize-only edit failure is not propagated back.

3. Unclosed backtick + MarkdownV2 escape combine to confuse the visible payload

   TelegramAdapter.format_message()'s _protect_fenced / inline-code protection regex only matches paired backticks. An unclosed ` at the boundary of the streaming buffer slips through and is later escaped (or not), and depending on which side of _send_or_edit's short-circuit fires first, Telegram ends up with the plaintext-streamed prefix that visually ends at the unclosed backtick. The user perceives this as "truncation at a backtick" because everything after the lone backtick was supposed to arrive as part of the finalize MarkdownV2 edit that never landed.

The result: model output is complete, adapter believes it sent everything, gateway suppresses the safety-net resend, and the user sees a half-sentence ending in `.

This is consistent with the original #25710 design ("intermediate edits plaintext, final edit MarkdownV2") and with the d44dafdb4 fix ("don't skip the finalize edit"), but it shows the finalize edit still has a single point of failure when it's combined with httpx.RemoteProtocolError and the streamed prefix carries an unclosed backtick.

Suggested Fix Direction

Two layers, defense in depth:

1. Make _final_content_delivered only true when the finalize edit (or the truthy fresh-final / fallback path) returned success, not when an earlier mid-stream edit succeeded.

   In gateway/stream_consumer.py, separate _intermediate_content_visible from _final_content_delivered. Only set _final_content_delivered = True after the finalize edit / fresh-final send returns success.

2. Pre-flight the unclosed-backtick case in TelegramAdapter.edit_message(finalize=True).

   If the payload contains an odd number of un-escaped backticks (i.e. one unclosed inline-code marker), either:

   • close the span before format_message (append a synthetic closing backtick), or
   • skip MarkdownV2 and send plaintext (already the fallback path on parse error), but ensure the fallback truly delivers the complete payload, not the half that the previous plaintext edit already showed.

3. Add a regression test:

   Existing tests in tests/gateway/test_telegram_format.py and test_telegram_text_batching.py cover normal MarkdownV2 + chunking. Add a test that exercises:

   • REQUIRES_EDIT_FINALIZE=True,
   • a streamed payload whose last buffered chunk ends with an unclosed `,
   • an injected httpx.RemoteProtocolError during the finalize edit,
   • assert that either the full final text arrives or _final_content_delivered is False so the gateway final-send fallback runs.

Related

• #25710 — original "final MarkdownV2 edit skipped" bug, fixed in d44dafdb4.
• #22518 / #22264 — intermediate-plaintext / final-MarkdownV2 streaming design.

I'm happy to provide more log context (we have 4 consecutive runs that all reproduced) or test a candidate patch.