openclaw - 💡(How to fix) Fix Block-streaming channels deliver assembled final payload in addition to per-chunk streaming, causing full-reply duplication [1 participants]

openclaw2026-04-24 03:47:02

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

openclaw/openclaw#70921•Fetched 2026-04-24 10:37:51

View on GitHub

Comments

Participants

Timeline

Reactions

Author

joeykrug

Participants

joeykrug

When blockStreaming is enabled for a channel (Signal, WhatsApp, iMessage/BlueBubbles — default for most IM channels), assistant replies containing structured markdown (tables, lists, code blocks) are delivered twice in the same turn:

First as per-chunk streamed payloads during LLM generation (onBlockReply firing for each paragraph / newline-split chunk)
Then again as the assembled final payload via the normal reply-delivery path

The existing hasSentPayload content-key dedup in src/auto-reply/reply/block-reply-pipeline.ts only handles the exact-key case, so when the channel's block splitter produced chunks like "para1" + "para2" but the final assembled payload is "para1\n\npara2", the content keys differ and dedup falls through — delivering the full text a second time to the channel.

This appears to be the same class of bug as:

PR #61586 — fix(block-streaming): content-aware dedup prevents double sends when chunks don't match final payload. Closed without merging on 2026-04-11. Included 9 tests and a confirmed reproducer against BlueBubbles API showing two distinct GUIDs for the same turn.
PR #65457 — fix(gateway): stop dropping repeated markdown tokens in chat stream merge. Still open. Affects every streamed markdown table separator row (|---|---|---| collapses to single cell), every --- horizontal rule, every code fence.
PR #65541 — fix: deliver text blocks progressively when block streaming is disabled. Still open. Secondary but relevant.
#68056 — [Bug] Single WhatsApp media reply sent twice in 2026.4.15. CLOSED as bug.

Root Cause

From src/auto-reply/reply/block-reply-pipeline.ts:

return {
  // ...
  didStream: () => didStream,
  isAborted: () => aborted,
  hasSentPayload: (payload) => {
    const payloadKey = createBlockReplyContentKey(payload);
    return sentContentKeys.has(payloadKey);
  }
};

createBlockReplyContentKey hashes { text: trimmedText, mediaList }. Because the final assembled payload's text is para1 + "\n\n" + para2 but the individual chunks' text values are para1 and para2 separately, the hashes never collide.

The broader path in agent-runner-payloads.ts:

const shouldDropFinalPayloads =
  params.blockStreamingEnabled &&
  Boolean(params.blockReplyPipeline?.didStream()) &&
  !params.blockReplyPipeline?.isAborted();

This blanket drop works when streaming completes successfully. But in practice:

When the pipeline completes successfully, shouldDropFinalPayloads is true, and the final-payload path skips — no duplication. ✅
When the pipeline aborts (timeout, transport failure partway through), shouldDropFinalPayloads is false, so the final-payload path runs and falls through to per-payload hasSentPayload. Because the content keys mismatch, nothing is deduped, and the full reply is re-sent. ❌
On providers like Anthropic, the text_end event frequently arrives with a full-content snapshot that doesn't match the delta-accumulated buffer exactly (markdown token re-ordering), so resolveAssistantTextChunk in pi-embedded-subscribe.handlers.messages.ts:123 returns the full content as a fresh chunk, triggering an out-of-band final-payload emission even when the pipeline didn't abort. ❌

So the visible effect is "streaming did its job, then the full reply fires again," and users see 2-4 copies of the same text.

Fix Action

Fix / Workaround

Workaround we're running

As of 2026-04-23, we apply PR #61586 as a local runtime patch in the compiled dist after every openclaw update. The patch modifies dist/block-reply-pipeline-*.js to add the streamedTexts tracking and content-coverage dedup exactly as the PR proposed. Module loads cleanly, patch is idempotent, and duplication has stopped in manual verification.

Happy to supply the patch script / open a new PR if that helps move this forward.

Code Example

[2026-04-23T17:49:42.082Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.121Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.152Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.186Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:45.217Z] gateway/channels/signal delivered reply to +<phone>

---

return {
  // ...
  didStream: () => didStream,
  isAborted: () => aborted,
  hasSentPayload: (payload) => {
    const payloadKey = createBlockReplyContentKey(payload);
    return sentContentKeys.has(payloadKey);
  }
};

---

const shouldDropFinalPayloads =
  params.blockStreamingEnabled &&
  Boolean(params.blockReplyPipeline?.didStream()) &&
  !params.blockReplyPipeline?.isAborted();

RAW_BUFFERClick to expand / collapse

Summary

First as per-chunk streamed payloads during LLM generation (onBlockReply firing for each paragraph / newline-split chunk)
Then again as the assembled final payload via the normal reply-delivery path

This appears to be the same class of bug as:

PR #61586 — fix(block-streaming): content-aware dedup prevents double sends when chunks don't match final payload. Closed without merging on 2026-04-11. Included 9 tests and a confirmed reproducer against BlueBubbles API showing two distinct GUIDs for the same turn.
PR #65457 — fix(gateway): stop dropping repeated markdown tokens in chat stream merge. Still open. Affects every streamed markdown table separator row (|---|---|---| collapses to single cell), every --- horizontal rule, every code fence.
PR #65541 — fix: deliver text blocks progressively when block streaming is disabled. Still open. Secondary but relevant.
#68056 — [Bug] Single WhatsApp media reply sent twice in 2026.4.15. CLOSED as bug.

Reproduction

Environment

OpenClaw 2026.4.22 (reproduced daily through 2026.4.15 → 2026.4.22 on Signal)
Anthropic Claude Opus 4.7 provider via Anthropic Messages API
Signal channel plugin with default blockStreaming (enabled)
Default chunkMode = "length" ("newline" also reproduces, less visually obviously)

Steps

Send a prompt that elicits a reply containing a markdown table of 3+ rows AND at least 2 paragraphs (>1500 chars total).
Observe the Signal conversation.

Expected

Reply arrives once (chunked or not).

Actual

Reply body arrives 2× to 4× concatenated into the same Signal message (or split across 2-5 Signal messages when chunkMode = "length"), depending on how the paragraph boundaries fell. The full reply is structurally duplicated — same text, repeated.

Log evidence

[2026-04-23T17:49:42.082Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.121Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.152Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:42.186Z] gateway/channels/signal delivered reply to +<phone>
[2026-04-23T17:49:45.217Z] gateway/channels/signal delivered reply to +<phone>

5 deliveries, 3.1 seconds apart, for a single agent turn. The first 4 are the block-streaming chunks (one per paragraph); the 5th is the final assembled payload firing again because hasSentPayload didn't recognize the content coverage.

Root cause

From src/auto-reply/reply/block-reply-pipeline.ts:

return {
  // ...
  didStream: () => didStream,
  isAborted: () => aborted,
  hasSentPayload: (payload) => {
    const payloadKey = createBlockReplyContentKey(payload);
    return sentContentKeys.has(payloadKey);
  }
};

The broader path in agent-runner-payloads.ts:

const shouldDropFinalPayloads =
  params.blockStreamingEnabled &&
  Boolean(params.blockReplyPipeline?.didStream()) &&
  !params.blockReplyPipeline?.isAborted();

This blanket drop works when streaming completes successfully. But in practice:

When the pipeline completes successfully, shouldDropFinalPayloads is true, and the final-payload path skips — no duplication. ✅
When the pipeline aborts (timeout, transport failure partway through), shouldDropFinalPayloads is false, so the final-payload path runs and falls through to per-payload hasSentPayload. Because the content keys mismatch, nothing is deduped, and the full reply is re-sent. ❌
On providers like Anthropic, the text_end event frequently arrives with a full-content snapshot that doesn't match the delta-accumulated buffer exactly (markdown token re-ordering), so resolveAssistantTextChunk in pi-embedded-subscribe.handlers.messages.ts:123 returns the full content as a fresh chunk, triggering an out-of-band final-payload emission even when the pipeline didn't abort. ❌

So the visible effect is "streaming did its job, then the full reply fires again," and users see 2-4 copies of the same text.

Why PR #61586 should be reconsidered

Tyler Yust's PR #61586 addressed this exact scenario with:

A streamedTexts[] array in the pipeline, tracking trimmed text of every successfully delivered chunk.
A content-coverage check in hasSentPayload: if strip(streamedTexts.join("")) equals or contains strip(finalText), return true.
Media-safety: media payloads still require exact key match to avoid dropping unique attachments.
Diagnostic logVerbose for dedup decisions.
9 new unit tests covering all the edge cases.

It was closed without a merge explanation that I can find. Re-opening or re-landing this fix (or an equivalent) would resolve the duplication for Signal, WhatsApp, BlueBubbles, Telegram block-streaming replies, and any other channel that uses the block-reply pipeline.

Workaround we're running

Happy to supply the patch script / open a new PR if that helps move this forward.

PR #61586 (closed 2026-04-11) — the exact fix
PR #65457 (open 2026-04-12) — markdown token dropping in resolveMergedAssistantText, separate but compounding
PR #65541 (open 2026-04-12) — progressive text delivery when blockStreaming is off
#68056 (closed) — WhatsApp media sent twice 2026.4.15
#22258 (closed) — rapid-fire duplicate messages during tool execution
#40545 (closed) — duplicate proactive messages (Discord+Telegram)
#45134 (open) — Mattermost unthreaded message leaks
#49889 (open) — Telegram partial-stream finalization observability

Additional context

Reproduced reliably with Anthropic Claude Opus 4.7; likely also affects OpenAI streaming adapters since the root cause is in the generic block-reply pipeline, not provider-specific.
Channel-side mitigation of chunkMode = "newline" reduces the visual ugliness (no mid-table splits) but does NOT stop the underlying double-send.
Happy to provide more detailed logs / a minimal repro script if helpful.

extent analysis

TL;DR

The most likely fix for the issue of duplicate replies in block streaming is to implement a content-aware deduplication mechanism, such as the one proposed in PR #61586, to prevent the final assembled payload from being sent again when the individual chunks have already been delivered.

Guidance

Review and consider re-opening or re-landing PR #61586, which addresses the exact scenario of duplicate replies in block streaming.
Implement a content-coverage check in hasSentPayload to track trimmed text of every successfully delivered chunk and compare it with the final text.
Apply a local runtime patch to dist/block-reply-pipeline-*.js to add the streamedTexts tracking and content-coverage dedup, as a temporary workaround.
Verify the fix by testing with different chunk modes and providers to ensure the duplication issue is resolved.

Example

const streamedTexts = [];
// ...
hasSentPayload: (payload) => {
  const payloadKey = createBlockReplyContentKey(payload);
  const streamedText = streamedTexts.join("");
  if (strip(streamedText) === strip(payload.text)) {
    return true;
  }
  // ...
}

Notes

The issue is specific to block streaming and affects multiple channels, including Signal, WhatsApp, and BlueBubbles. The root cause is in the generic block-reply pipeline, not provider-specific. The proposed fix should be thoroughly tested to ensure it resolves the duplication issue without introducing new problems.

Recommendation

Apply the workaround by implementing the content-aware deduplication mechanism proposed in PR #61586, as it addresses the root cause of the issue and has been verified to resolve the duplication problem.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #conversation history #tool integration #LLM response #prompt template

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

openclaw - 💡(How to fix) Fix Block-streaming channels deliver assembled final payload in addition to per-chunk streaming, causing full-reply duplication [1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fix / Workaround

Workaround we're running

Code Example

Summary

Reproduction

Environment

Steps

Expected

Actual

Log evidence

Root cause

Why PR #61586 should be reconsidered

Workaround we're running

Related

Additional context

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

TRENDING

openclaw - 💡(How to fix) Fix Block-streaming channels deliver assembled final payload in addition to per-chunk streaming, causing full-reply duplication [1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fix / Workaround

Workaround we're running

Code Example

Summary

Reproduction

Environment

Steps

Expected

Actual

Log evidence

Root cause

Why PR #61586 should be reconsidered

Workaround we're running

Related

Additional context

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING