PR #75495: fix(memory-host-markdown): use global regex to replace all managed blocks (#75491)

• Repository: openclaw/openclaw
• Author: ottodeng
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75495

Description (problem / solution / changelog)

Fixes #75491.

Problem

replaceManagedMarkdownBlock used new RegExp(..., "m") (no g flag) to find and replace existing managed blocks. Two failure modes followed:

1. Whitespace / line-ending mismatch (e.g. CRLF) ⇒ append duplicate. The heading separator was \n, so a file with \r\n between heading and start marker did not match the existing block. The function then took the "no existing block" branch and appended a second copy.

2. Once duplicates exist, .replace() without g only rewrites one copy. Each subsequent dreaming run re-rewrote one duplicate while the others stayed, so files accumulated several copies of the block (the issue reports 5+ copies in a real file).

Fix

• Add g to the regex so .replace() rewrites every existing block in a single pass.
• Loosen the heading separator from \n to [\r\n]+ so CRLF and lone-CR files match the same managed block instead of silently appending.
• After the global replace, collapse adjacent identical managed blocks back to a single block. This lets affected files that already contain duplicates self-heal the next time the dreaming run executes — Otto reported 5 stale copies in 2026-04-30.md; with this fix the next run will leave exactly one.
• Reset lastIndex between .test() and .replace() to avoid the global-regex sticky-position hazard.

Tests

Existing tests still pass. Added three new cases:

• CRLF heading separator → matches and replaces in place.
• Pre-existing duplicate blocks → collapsed back to one canonical block (- old body fully removed).
• Repeated invocations with the same body are idempotent.

All 8 cases pass locally (verified via a standalone harness because vitest deps are not installed in this checkout; the test file itself uses the existing project test layout).

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/plugin-sdk/memory-host-markdown.test.ts (modified, +48/-0)
• src/plugin-sdk/memory-host-markdown.ts (modified, +25/-3)

---

PR #75810: fix(memory-markdown): collapse duplicate managed blocks, match CRLF

• Repository: openclaw/openclaw
• Author: everettjf
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75810

Description (problem / solution / changelog)

Summary

• Problem: replaceManagedMarkdownBlock in src/plugin-sdk/memory-host-markdown.ts accumulates duplicate managed blocks (e.g. <!-- openclaw:dreaming:light:start --> ... <!-- openclaw:dreaming:light:end -->) in daily memory and wiki files. The reporter observed up to 5 sequential copies of the same block in a single daily file (243 KB vs. an expected ~50 KB).
• Why it matters: every dreaming/wiki run that hits the existing block path keeps the duplicates around; injected context bloats and triggers truncation warnings, and the file size grows unboundedly until manual deduplication.
• What changed: the heading regex now uses \r?\n so a previously-written block in a CRLF file still matches and is replaced in place instead of being appended. The match path now uses the g flag and a replacement function: the first match is rewritten with the new managed block, and any subsequent duplicate occurrences are removed. Trailing blank-line runs left by removed duplicates are collapsed back to a single trailing newline so the result still matches the existing append-block layout.
• What did NOT change: the no-existing-block append path, the no-heading code path, the public function signature, and the layout of the rewritten block (heading + start marker + body + end marker on their existing lines).

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #75491
• This PR fixes a bug or regression

Root Cause

• Root cause: two compounding regressions in replaceManagedMarkdownBlock. First, the heading regex used a literal \n between heading and start marker, so any block written or hand-edited with CRLF line endings did not match — the function then took the no-existing-block path and appended a second managed block. Second, once duplicates existed the regex was used with String.prototype.replace and no g flag, so subsequent runs only rewrote the first occurrence; later duplicates were never cleaned up and continued to accumulate on every run that re-triggered the mismatch path.
• Missing detection / guardrail: no test exercised CRLF input or pre-existing duplicate input. The existing tests covered append-when-missing, replace-in-place with LF input, and headingless blocks — none of which surface either bug.
• Contributing context: callers (memory-core dreaming, memory-wiki compile/lint/vault/apply/chatgpt-import) wrap the result with withTrailingNewline, so trailing-whitespace details only matter for shape, not for layout drift.

Regression Test Plan

• Coverage level that should have caught this:
  • Unit test
• Target test or file: src/plugin-sdk/memory-host-markdown.test.ts
• Scenario the test should lock in:
  1. CRLF heading-to-start-marker spacing still matches the existing block and rewrites it in place (no append).
  2. A file already carrying multiple identical managed blocks collapses to exactly one updated block on rewrite.
• Why this is the smallest reliable guardrail: both regressions are deterministic in this single helper. A unit test against the helper is faster, more reliable, and more local than a memory-core or wiki integration test.

User-visible / Behavior Changes

• Daily dreaming files (~/.openclaw/workspace/memory/YYYY-MM-DD.md) and memory-wiki page files no longer accumulate duplicate <!-- openclaw:* --> blocks across runs.
• Existing files that already carry duplicates are deduplicated to a single updated block on the next dreaming/wiki rewrite that touches them. No data outside the managed block is touched (the function still only rewrites content between marker pairs).

Security Impact

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No
• Command/tool execution surface changed? No
• Data access scope changed? No

Repro + Verification

Environment

• OS: macOS 26.3 arm64 (per the issue); the helper itself is OS-agnostic
• Runtime/container: Node 22 (vitest)

Steps

1. Run pnpm test src/plugin-sdk/memory-host-markdown.test.ts — the two new cases (matches CRLF line endings between heading and start marker and collapses pre-existing duplicate managed blocks into one) cover both regressions; the original three cases still pass unchanged.
2. Run pnpm test extensions/memory-core/src/dreaming-markdown.test.ts extensions/memory-wiki/src/apply.test.ts extensions/memory-wiki/src/query.test.ts extensions/memory-wiki/src/cli.test.ts extensions/memory-core/src/dreaming-narrative.test.ts extensions/memory-core/src/dreaming-phases.test.ts extensions/memory-core/src/cli.test.ts to exercise the consumers of replaceManagedMarkdownBlock.

Expected

• All seven memory-host-markdown tests pass; consumer tests across memory-core and memory-wiki continue to pass (163 tests).

Actual

• 7/7 helper tests pass; 163/163 consumer tests pass locally.

Evidence

• Failing test/log before + passing after — added unit cases reproduce both the CRLF mismatch and the duplicate-collapse cases.

Human Verification

• Verified scenarios: CRLF heading match (in-place rewrite, no append); duplicate collapse (3 sequential blocks → 1 updated block); existing append-when-missing path; existing in-place rewrite path; headingless block path.
• Edge cases checked: trailing newline normalization (\n{3,} → \n\n, then trailing \n{2,} → \n) so the duplicate-removal output still matches the append-path layout the existing tests pin.
• What I did not verify: a real on-disk dreaming run on a CRLF host. Test coverage and the helper's pure-string contract should be sufficient.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? No
• Migration needed? No. Existing duplicated files will deduplicate in place the next time a dreaming/wiki rewrite touches the file; the issue notes manual deduplication as the prior workaround, which is no longer required after this fix takes effect.

Risks and Mitigations

• Risk: the g-flag/lastIndex usage on the shared RegExp between the .test() and .replace() calls could mis-iterate if it carries state.
  • Mitigation: existingPattern.lastIndex = 0 is reset between .test() and .replace(), so the global regex starts cleanly at index 0 for the replacement pass; the helper test covering 3 sequential duplicates locks this in.
• Risk: trimming trailing whitespace runs (\n{2,}$ → \n) on the result string could surprise callers.
  • Mitigation: every caller already runs withTrailingNewline(...) over the helper output, so a single-newline trailing shape matches the existing append-path output. The existing-block-replace test (replaces an existing managed block in place) still passes unchanged.

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/plugin-sdk/memory-host-markdown.test.ts (modified, +46/-0)
• src/plugin-sdk/memory-host-markdown.ts (modified, +18/-3)