openclaw - 💡(How to fix) Fix v2026.4.24: Stored snippet normalization mismatch with `relocateCandidateRange` causes silent rehydration failure for all post-Apr-23 promotion candidates [1 comments, 2 participants]

Summary

The dreaming-promotion --apply pipeline rehydrates each ranked candidate by re-locating the snippet inside the source file, then re-emitting a promotion section from the freshly-located range. The rehydration is gated by relocateCandidateRange → compareCandidateWindow, which compares the stored snippet against a normalized window of the source file using normalizeSnippet and three string-equality checks (===, windowSnippet.includes(targetSnippet), targetSnippet.includes(windowSnippet)).

The bug: the stored snippet and the live-window snippet pass through different normalizers. The stored side has rich normalization (markdown headings stripped, bullets joined with ;, truncated at ~280 chars, header-as-prefix conversion). The compare side does only whitespace collapse: trimmed.replace(/\s+/g, " "). The two outputs are not compatible — the stored form Key Decisions Made: **Architecture**: … cannot be matched by ===, includes, or reverse-includes against any whitespace-collapsed window of the source file, because the source file's window starts with ### Key Decisions Made (heading not stripped on the compare side) and contains raw markdown bullets.

relocateCandidateRange exhausts every starting line × span up to maxSpan = max(preferredSpan + 3, 8), never matches, returns null, and rehydratePromotionCandidate drops the candidate from rehydratedSelected. If every ranked candidate fails this way (and on this host, every one does), applyShortTermPromotions early-returns { applied: 0, appended: 0 } with no error logged.

The dry-run path (openclaw memory promote --dry-run) does not go through rehydratePromotionCandidate, so ranking output looks healthy: same candidates list, same scores, same gates passed. Only the --apply path silently does nothing. The daily managed dreaming cron uses the apply path, so every nightly run on a recall-store written by a recent dreaming version produces applied: 0 regardless of how many qualified candidates exist.

Environment

• OpenClaw: 2026.4.24 (cbcfdf6).
• Node: 22.22.2.
• OS: Linux 6.8.0-106-generic.
• Plugin: memory-core, dreaming enabled, frequency: "0 3 * * *", timezone: "Asia/Jakarta".
• Workspace inspected: ~/.openclaw/workspace-orchestrator/.
• Recall store: ~/.openclaw/workspace-orchestrator/memory/.dreams/short-term-recall.json — 948 entries, indexer-normalized snippets.

Steps to reproduce

1. On v2026.4.24, with memory-core dreaming enabled, allow at least one daily background scan to populate short-term-recall.json (one full day of operation, or trigger via openclaw memory scan if available).
2. Run the dry-run side first, to confirm ranking is healthy:

   openclaw memory promote --dry-run --json

   Expected output: a non-empty selected array with score-qualified candidates and audit.promotedCount: 0 (because dry-run doesn't write).
3. Run the apply side:

   openclaw memory promote --apply --json

   Observed output: applied: 0, appended: 0, despite the dry-run having shown qualified candidates.
4. No error or warning emitted. The cron-driven nightly run produces the same outcome silently every night.

Expected behavior

--apply should successfully rehydrate any candidate the dry-run path identified as qualified, and append the rehydrated promotion section to the agent's MEMORY.md. applied should match (or be very close to) the count of qualified candidates from the dry-run.

Actual behavior

relocateCandidateRange returns null for every candidate. rehydratePromotionCandidate drops each one. applyShortTermPromotions early-returns { applied: 0, appended: 0 }. No log entry attributes the failure — the only signal is the applied=0 count itself, which a casual reader would interpret as "no candidates today" rather than "every candidate silently dropped."

Diagnostic evidence

Code path

In /usr/lib/node_modules/openclaw/dist/short-term-promotion-CI1S22E7.js:

• applyShortTermPromotions(...) calls rehydratePromotionCandidate(...) for each ranked candidate.
• rehydratePromotionCandidate(...) calls relocateCandidateRange(...). Around line 1217:

  const relocated = relocateCandidateRange({ ... });
  if (!relocated) continue;

• relocateCandidateRange(...) walks starting line × span combinations and calls compareCandidateWindow(targetSnippet, windowSnippet).
• compareCandidateWindow(...) is a three-way string match: ===, windowSnippet.includes(targetSnippet), targetSnippet.includes(windowSnippet). Both arguments come through normalizeSnippet.
• normalizeSnippet(s) = s.trim().replace(/\s+/g, " ") — pure whitespace collapse, nothing else.

Normalization mismatch — concrete example

Anonymized example reproducing the exact failure shape on this host.

Candidate key: memory:memory/example-notes.md:458:461.

Stored snippet (from short-term-recall.json, written by the indexer):

Cache Eviction Policy: **Strategy**: LRU with per-key TTL overrides for hot entries that exceed default 5-minute window; max 1024 entries per shard, evict oldest first

Source file memory/example-notes.md lines 457–462:

### Cache Eviction Policy
- **Strategy**: LRU with per-key TTL overrides for hot entries that exceed default 5-minute window
- max 1024 entries per shard, evict oldest first
- Promotion threshold: 3 hits within 60 seconds
- Cold tier writes back to disk at 30-second checkpoints

The candidate range is 458–461 — bullets only, without the ### Cache Eviction Policy heading at line 457.

Window snippet for any 4-line span starting at 458, after normalizeSnippet:

- **Strategy**: LRU with per-key TTL overrides for hot entries that exceed default 5-minute window - max 1024 entries per shard, evict oldest first - Promotion threshold: 3 hits within 60 seconds - Cold tier writes back to disk at 30-second checkpoints

Note the differences:

• Stored side: heading ### Cache Eviction Policy is converted to a colonized prefix Cache Eviction Policy: and bold markers are kept (**Strategy**); the snippet is also truncated at the indexer's ~280-char limit.
• Window side: heading would be present if line 457 were included, but the candidate range starts at 458 so the heading is absent. Bullets are still raw - characters joined by spaces.

Neither ===, nor windowSnippet.includes(targetSnippet), nor targetSnippet.includes(windowSnippet) can succeed across this divergence. relocateCandidateRange exhausts all spans 4–8 from every starting line in a ±20-line neighborhood and returns null.

Verification on this host

Pre-patch:

$ openclaw memory promote --dry-run --json | jq '.selected | length, .audit.promotedCount'
12
0
$ openclaw memory promote --apply --json | jq '.applied, .audit.promotedCount'
0
0

After applying the workaround patch:

$ openclaw memory promote --apply --json | jq '.applied, .audit.promotedCount'
4
12

Same recall store, same ranked candidates. Only difference: the rehydration path no longer drops everything.

Workaround

One-line patch in short-term-promotion-CI1S22E7.js. Replace the early-continue after a failed relocation with a fallback to the stored candidate (its startLine/endLine/snippet are still good enough for buildPromotionSection to emit a usable section):

sudo cp /usr/lib/node_modules/openclaw/dist/short-term-promotion-CI1S22E7.js \
        /usr/lib/node_modules/openclaw/dist/short-term-promotion-CI1S22E7.js.bak-pre-2026-04-29

# Replace the early-skip with a stored-candidate fallback.
# Only one occurrence of the exact phrase exists in this file.
sudo sed -i 's/if (!relocated) continue;/if (!relocated) return { ...candidate };/' \
        /usr/lib/node_modules/openclaw/dist/short-term-promotion-CI1S22E7.js

sudo systemctl restart openclaw

Verify:

openclaw memory promote --apply --json | jq '.applied'
# expect a non-zero value matching dry-run's qualified-candidate count

The patch must be re-applied after every openclaw@<version> upgrade.

Suggested fix

The proper fix aligns the two normalizers. Two viable options:

1. Make normalizeSnippet apply the same markdown-stripping + bullet-joining + truncation that the indexer does. Means finding the indexer's snippet-emit code (lives in the dreaming-write path; not yet identified by file name in this report) and factoring out a shared richNormalize(s) helper called from both sides. Larger surface area, but it preserves the indexer's prettier display form.

2. Make the indexer store the raw whitespace-only normalized snippet (i.e., apply the same normalizeSnippet as the compare side). Smaller change, but loses the heading-as-prefix and bullet-join transforms that make stored snippets read nicely in short-term-recall.json and in --json output.

The fallback in the workaround (return the stored candidate verbatim) is a band-aid: it gets applied non-zero but doesn't actually re-locate anything, so any downstream consumer that relies on the rehydrated startLine/endLine accurately pointing into the current file is reading stale offsets. Aligning the normalizers is the right structural fix; option 1 is what I'd ship.

A defensive companion change either way: when relocateCandidateRange returns null for every candidate in a run, log a single WARN with a count and the first candidate's key. The current silence is what made this bug invisible for six consecutive nights.

Severity

High. This silently breaks the entire --apply pipeline — including the daily managed dreaming cron — for any deployment whose recall store has been written by a recent dreaming version (essentially: any production deployment older than a few days). The failure is invisible: no error log, no warning, no metric, no audit-row delta. Operators see "memory dreaming runs every night" as healthy because the cron fires and the gateway logs dreaming promotion complete; they have no way to know that applied=0 is a bug and not an empty input.

The bug interacts badly with the recall-tracker observability gap (filed separately as v4.24-dreaming-recall-tracker-missing.md): with recallCount=0 for all agent-driven recalls and applied=0 for all promotion runs, an operator looking at the system from the outside cannot distinguish "memory pipeline is healthy and there's nothing to promote" from "memory pipeline is silently broken." On this host the two bugs combined hid the breakage for six days before I caught it by hand-merging six days' worth of memory entries into MEMORY.md and noticing the applied=0 mismatch with the dry-run output.