Issue Station
Sign In
Tools hereToolbox

openclaw - ✅(Solved) Fix test: add integration test for table-aware block streaming chunking [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#66605•Fetched 2026-04-15 06:25:27
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
zongqichenzongqichen
Participants
zongqichenzongqichen
Timeline (top)
cross-referenced ×2

#66568 added table-aware splitting to the block chunker (EmbeddedBlockChunker) and chunkByParagraph, preventing markdown tables from being split across messages during block streaming. The fix includes unit tests but no integration-level coverage through the subscribeEmbeddedPiSession pipeline.

The existing fence-awareness integration test in pi-embedded-subscribe.subscribe-embedded-pi-session.streams-soft-chunks-paragraph-preference.test.ts ("avoids splitting inside fenced code blocks") is the direct analog — a matching "avoids splitting inside markdown tables" case should be added for parity.

Root Cause

#66568 added table-aware splitting to the block chunker (EmbeddedBlockChunker) and chunkByParagraph, preventing markdown tables from being split across messages during block streaming. The fix includes unit tests but no integration-level coverage through the subscribeEmbeddedPiSession pipeline.

The existing fence-awareness integration test in pi-embedded-subscribe.subscribe-embedded-pi-session.streams-soft-chunks-paragraph-preference.test.ts ("avoids splitting inside fenced code blocks") is the direct analog — a matching "avoids splitting inside markdown tables" case should be added for parity.

Fix Action

Fixed

PR fix notes

PR #66568: fix(streaming): prevent block chunker from splitting markdown tables

Description (problem / solution / changelog)

Summary

  • Problem: Block streaming (flushOnParagraph) treats \n\n around markdown tables as valid split points, splitting tables across multiple Telegram messages.
  • Why it matters: Tables are unreadable when fragmented across messages.
  • What changed: Added parseTableSpans() / isSafeTableBreak() mirroring the existing FenceSpan pattern, wired into all break-finding paths in EmbeddedBlockChunker. Moved table span detection to standalone table-spans.ts to avoid import cycle.
  • What did NOT change: No changes to deliver.ts, config resolution, or non-table chunking behavior.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #66614
  • Related #47454
  • Related #17679
  • This PR fixes a bug or regression

Root Cause (if applicable)

  • Root cause: findNextParagraphBreak() and all findSafe*BreakIndex() functions only checked isSafeFenceBreak(). Tables have no equivalent protection.
  • Missing detection / guardrail: No table span detection existed.
  • Contributing context: The fence-protection pattern was correct and extensible — tables just were not covered.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: pi-embedded-block-chunker.test.ts, markdown/tables.test.ts
  • Scenario the test should lock in: text+table+text under maxChars → single chunk; table stays intact when flushOnParagraph splits around it; thematic breaks not falsely detected as tables
  • Why this is the smallest reliable guardrail: Unit tests on the chunker directly verify the break-finding logic without needing a full channel roundtrip.
  • Existing test that already covers this: None — new tests added in this PR.
  • If no new test is added, why not: N/A — new tests added.

User-visible / Behavior Changes

Markdown tables in streaming replies are no longer split into separate messages.

Diagram (if applicable)

Before:
[text] → msg1 | [table] → msg2 | [text] → msg3

After:
[text + table + text] → msg1

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Repro + Verification

Environment

  • OS: macOS 26.4.1 (arm64)
  • Runtime/container: Node 22+ / pnpm global
  • Model/provider: anthropic/claude-sonnet-4
  • Integration/channel: Telegram (block streaming + flushOnParagraph)
  • Relevant config: default streaming config

Steps

  1. Send a prompt that triggers a response containing text, a markdown table, then more text
  2. Observe the Telegram delivery

Expected

  • One message containing the full response with the table inline

Actual

  • Three separate messages: text before table, the table itself, text after table

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)
<img width="1126" height="565" alt="image" src="https://github.com/user-attachments/assets/8d2e115f-baf1-4f08-9fe0-2a20a8a3176f" />

Unit tests: 15+ new tests all passing. Live verified: applied patched build to local OpenClaw, sent table reply via Telegram, confirmed single message delivery.

Human Verification (required)

  • Verified scenarios: Unit tests + live Telegram end-to-end
  • Edge cases checked: Table inside code fence (ignored), thematic break --- (not falsely detected), table near maxChars
  • What you did not verify: Other channels (Slack, WhatsApp) — behavior should be identical since they share the same chunker code path.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Risks and Mitigations

  • Risk: Very large tables exceeding maxChars will force-split at maxChars instead of \n\n.
    • Mitigation: Matches existing code fence behavior — correct fallback.

Changed files

  • src/agents/pi-embedded-block-chunker.test.ts (modified, +85/-4)
  • src/agents/pi-embedded-block-chunker.ts (modified, +30/-13)
  • src/auto-reply/chunk.ts (modified, +3/-1)
  • src/markdown/table-spans.ts (added, +116/-0)
  • src/markdown/tables.test.ts (modified, +47/-0)
RAW_BUFFERClick to expand / collapse

Summary

#66568 added table-aware splitting to the block chunker (EmbeddedBlockChunker) and chunkByParagraph, preventing markdown tables from being split across messages during block streaming. The fix includes unit tests but no integration-level coverage through the subscribeEmbeddedPiSession pipeline.

The existing fence-awareness integration test in pi-embedded-subscribe.subscribe-embedded-pi-session.streams-soft-chunks-paragraph-preference.test.ts ("avoids splitting inside fenced code blocks") is the direct analog — a matching "avoids splitting inside markdown tables" case should be added for parity.

Suggested test

Add a case to the streams-soft-chunks-paragraph-preference test file (or a sibling file) using the existing createParagraphChunkedBlockReplyHarness:

  • Input: "Intro\n\n| A | B |\n|---|---|\n| 1 | 2 |\n\nOutro"
  • Expected: 3 block replies — "Intro", the full table as one chunk, "Outro"
  • Verify the table is never split across onBlockReply calls

Context

  • PR: #66568
  • Test harness: pi-embedded-subscribe.e2e-harness.ts
  • Reference case: "avoids splitting inside fenced code blocks" in the same file

extent analysis

TL;DR

Add an integration test case to verify that markdown tables are not split across messages during block streaming.

Guidance

  • Create a new test case in the streams-soft-chunks-paragraph-preference.test.ts file using the createParagraphChunkedBlockReplyHarness function.
  • Use the suggested input "Intro\n\n| A | B |\n|---|---|\n| 1 | 2 |\n\nOutro" and verify that it produces 3 block replies: "Intro", the full table as one chunk, and "Outro".
  • Ensure that the table is never split across onBlockReply calls by checking the output of the test harness.
  • Review the existing "avoids splitting inside fenced code blocks" test case for reference and parity.

Example

it('avoids splitting inside markdown tables', async () => {
  const input = 'Intro\n\n| A | B |\n|---|---|\n| 1 | 2 |\n\nOutro';
  const expectedReplies = ['Intro', '| A | B |\n|---|---|\n| 1 | 2 |', 'Outro'];
  const harness = createParagraphChunkedBlockReplyHarness(input);
  const replies = await harness.getBlockReplies();
  expect(replies).toEqual(expectedReplies);
});

Notes

The suggested test case is designed to ensure parity with the existing "avoids splitting inside fenced code blocks" test. The test harness and input are provided as a starting point, but the exact implementation may vary depending on the specific requirements of the project.

Recommendation

Apply workaround by adding the suggested test case to the streams-soft-chunks-paragraph-preference.test.ts file to ensure that markdown tables are not split across messages during block streaming. This will provide integration-level coverage for the fix introduced in PR #66568.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#database connection#vector store#embedding generation#cache error#pipeline error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - ✅(Solved) Fix test: add integration test for table-aware block streaming chunking [1 pull requests, 1 participants]