[Bug]: short-term-recall.json grows unboundedly because dreaming records its own session-corpus artifacts as recall candidates

Bug type

Behavior bug (memory store grows unbounded; impacts dreaming runtime cost)

Summary

memory/.dreams/short-term-recall.json accumulates entries over time without an effective retention path. Across 4 active workspaces I measured, 85–96% of entries point at memory/.dreams/session-corpus/* files — i.e. dreaming's own output, not user-authored daily memory. The deep-phase maxAgeDays config only gates promotion eligibility; it does not trim the store itself. Entries also lack a firstRecordedAt field, so even an external retention pass can't trim by age reliably.

This produces a self-feeding loop: dreaming reads memory/.dreams/session-corpus/*.txt, records each slice as a recall candidate, and the next dreaming run sees more candidates than the previous, forever.

Evidence

Snapshot from 4 workspaces, all updated at the same 03:00–03:05 dreaming cycle:

A representative entry key (real, from workspace-brain/memory/.dreams/short-term-recall.json):

"memory:memory/.dreams/session-corpus/2026-04-08.txt:1:1"

The path field in these entries is memory/.dreams/session-corpus/<date>.txt — generated by dreaming itself. Daily-memory MD entries (e.g. memory/2026-04-13.md) account for the remaining 4–15%.

Root cause analysis

Two independent gaps compound:

1. The store has no retention contract

The deep-phase config gates promotion candidacy:

"plugins.entries.memory-core.config.dreaming.phases.deep": {
  "minScore": 0.9,
  "minRecallCount": 5,
  "minUniqueQueries": 5,
  "maxAgeDays": 30
}

maxAgeDays answers "how old can an entry be when we consider it for promotion" — it does not answer "when do we drop entries from the store." There appears to be no equivalent retention pass that prunes aged or low-signal entries from short-term-recall.json after each dreaming run.

2. Entries lack firstRecordedAt

Inspecting entries directly (Python json.load → iterate entries), no entry carries a firstRecordedAt/created-at timestamp. The closest temporal field is lastRecalledAt, which ingestDailyMemorySignals and ingestSessionTranscriptSignals refresh daily as a side-effect (see #71976 for the same observation in the light sort path). This means even if a user wanted to trim "entries older than N days", there's no honest "creation time" to filter on.

3. Dreaming records its own corpus as a recall source

The memory/.dreams/session-corpus/*.txt files are dreaming's own intermediate artifact. Counting them as recall candidates means each dreaming run amplifies the prior run's output. The signal mixing concern in #72021 (raw recall vs. daily/session signal) is one face of this; the store-bloat concern documented here is the other.

Suggested fix directions

Listed in order of least-to-most invasive:

1. Exclude memory/.dreams/session-corpus/* from recall candidacy. Dreaming's own intermediate artifacts shouldn't seed future dreaming runs as if they were first-class memory.

2. Add a phases.deep.storeRetentionDays (or similar) that prunes the store after each run, separate from promotion maxAgeDays. Combine with #1 above.

3. Stamp firstRecordedAt on entries at insertion. Without it, no retention strategy that filters by age is honest — lastRecalledAt keeps refreshing under the daily-signal ingest. This is independent of #1 and #2 and would also help #71976's "real recall vs daily signal" untangling.

Related

• #71976 — light-dreaming sort prioritizes recency over recallCount, hiding real recalls
• #72021 — signalCount mixes daily/session signals with real recalls; minScore filters out legitimate entries
• #67363 — deep-phase promotes raw verbatim daily-log snippets (different surface, same general "dreaming feedback loop without curation" theme)

Environment

• OpenClaw: 2026.4.29
• OS: Linux 6.17.0-22-generic x86_64
• Channel: gateway runtime
• 14 workspaces, daily-memory layout (memory/YYYY-MM-DD.md), dreaming.enabled = true
• phases.deep.maxAgeDays = 30 (default-shape config, nested per the schema requirement)

Impact

Medium — dreaming runtime cost grows with workspace age (commander measured at 1.86k entries / 2 MB after weeks of operation; gateway has to load/parse this on every dreaming cycle). The store is not in the bootstrap-context scan path so prep time isn't directly affected, but it materially slows the 03:00 dreaming cycle and inflates promotion-candidate evaluation in the deep phase.