openclaw - 💡(How to fix) Fix memory-core dreaming: narrative-timeout "fallback diary entry" empties DREAMS.md/MEMORY.md; promotion hard-fails at 16 MiB

Summary

memory-core's nightly dreaming promotion can destroy accumulated workspace memory files (DREAMS.md, MEMORY.md). Two distinct problems combine into data loss:

1. Timeout → destructive "fallback diary entry". When the dreaming narrative generation ends with status=timeout, memory-core writes a minimal "fallback diary entry". In our case the rich DREAMS.md (~130 KB of accumulated diary) ended up as a ~1 KB stub after the run. The fallback path appears to replace the file rather than append/preserve existing content.
2. Hard 16 MiB limit with no rotation. For a workspace whose diary had grown past 16 MiB, promotion fails outright with file exceeds limit of 16777216 bytes, so that workspace is silently skipped (applied=3, failed=1). There is no rotation/compaction/trim fallback — the file just becomes un-promotable.

Net effect for operators: agents "wake up" with empty/blank long-term memory and diary.

Environment

• OpenClaw 2026.5.26 (10ad3aa)
• Node 24.15.0, Linux
• Default memory-core dreaming cron (0 3 * * *, [managed-by=memory-core.short-term-promotion])
• Multiple agent workspaces

Evidence (verbatim, paths redacted)

Narrative timeout → fallback diary entry (repeats for light/rem/deep phases):

03:01:34 WARN  memory-core: narrative generation ended with status=timeout for light phase; writing fallback diary entry.
03:01:34 INFO  memory-core: narrative generation used fallback for light phase because the narrative run ended with status=timeout.
03:03:09 WARN  memory-core: narrative generation ended with status=timeout for rem phase; writing fallback diary entry.

Hard 16 MiB failure for one workspace:

03:02:33 ERROR memory-core: dreaming promotion failed for workspace <WORKSPACE>: file exceeds limit of 16777216 bytes (got 16786294)

Store rewrite + run summary:

03:02:07 INFO  memory-core: normalized recall artifacts before dreaming (rewrote recall store) [workspace=<WORKSPACE>].
03:04:07 INFO  memory-core: dreaming promotion complete (workspaces=7, candidates=20, applied=3, failed=1).

Also observed: narrative session cleanup warnings (Session dreaming-narrative-<phase>-… is still active; try again in a moment.) on every phase, and the embedded narrative runs taking 70 s+ against a slow provider, i.e. the timeout path is hit easily under load.

After the run, the affected DREAMS.md/MEMORY.md in the workspace root were generic stubs; the previously-rich content was gone.

Expected behavior

• A timeout / failed narrative must never replace an existing diary/memory file with a minimal fallback. It should append, write to a separate fallback location, or skip — never truncate accumulated content.
• The 16 MiB limit should trigger rotation/compaction (or read a bounded tail) instead of failing the whole promotion for that workspace.
• Destructive writes to canonical workspace docs should be guarded (e.g. refuse to shrink a file by >X% without an explicit flag) and/or take a timestamped backup first.

Suggested fix

• In the dreaming promotion path: make the fallback append-only / non-destructive; write fallback entries to a side file; snapshot before overwrite.
• Add a shrink/empty guard before writing DREAMS.md/MEMORY.md.
• Replace the hard 16 MiB exceeds limit failure with rotation or a bounded read.

Workaround (for other operators hitting this)

A */15 cron that snapshots the canonical workspace docs when healthy and auto-restores them if they collapse to a stub (size < 25 % of last-known-good). Happy to share the script if useful.