Bug type

Memory leak (process RSS grows until OOM or watchdog restart)

Summary

There's a slow leak hiding in the persistence layer that most people are working around without knowing what's causing it.

SessionManager.fileEntries is an in-memory array that mirrors every line written to the JSONL session transcript. Every message, tool call, tool result, and metadata entry gets pushed to it. Nothing ever removes them. Over hours of normal use, this array quietly accumulates hundreds of thousands of objects, eventually holding 1.4GB+ of retained heap. V8 can't collect any of it because SessionManager holds a live reference to the array for the lifetime of the process.

The tricky part: the LLM context window stays bounded. replaceMessages() keeps Agent._state.messages at a manageable size during compaction. So from the model's perspective, everything looks fine. But underneath, fileEntries is a parallel data structure in the persistence layer that compaction never touches. Two layers, only one with a ceiling.

This is likely the root cause behind multiple reported memory and session-bloat issues, including #6190, #13758, #2254, and #20910, which all share the same symptom pattern. It's a contributing factor in several others (see Related Issues below).

Environment

• OpenClaw v2026.3.24
• Node v24.14.0
• Ubuntu 24.04, 16GB RAM
• Telegram channel, 7 agents (1 main + 5 social + 1 coding subagent)
• 5 custom plugins (mem0 fork, LCM fork, sticky-context, privacy-guardrail, read-guardrail)
• Context engine plugin with ownsCompaction: true (LCM/lossless-claw fork)

Steps to reproduce

1. Run OpenClaw gateway with any channel for 6+ hours with moderate tool use (web_search, web_fetch, exec)
2. Monitor the gateway child process RSS: cat /proc/$(pgrep -f openclaw-gateway)/status | grep VmRSS
3. Observe RSS growing from ~300-400MB baseline to 1.5-2.5GB+ over the session
4. Growth is activity-correlated: heavy tool-call bursts cause rapid spikes, quiet periods show minimal (but nonzero) growth

Note: systemctl show openclaw --property=MainPID returns the parent process (~124MB, stable). The actual gateway child process must be found via pgrep -f openclaw-gateway.

Evidence: V8 heap snapshots with retainer analysis

We chased this across three debugging sessions over several days before taking heap snapshots. The usual suspects (plugin re-registration, hook runners, registry caches) all came back clean. What finally told the story was comparing two V8 heap snapshots of the same gateway process (PID 4158913) using --heapsnapshot-signal=SIGUSR2:

Delta: +9,083,104 nodes, +1,429 MB self-size growth.

What's growing

Large strings include raw web_search HTML results (40+ MB in 102 strings), base64 encoded images (25+ MB), and LLM response content. Separately, the session store duplicates system prompt/skills text per session entry (47 MB across 4,495 copies in skillsSnapshot fields). That's a distinct waste from the fileEntries issue but compounds the overall footprint.

What's NOT growing

We checked every suspect we could think of. None of them are the cause.

Plugin re-registration (+68 instances over 7 hours) contributes less than 20 KB total, 0.001% of the 1.4 GB growth.

The retainer chain

This is what made the root cause unambiguous. We sampled 200 instances across message, anthropic-messages, and toolResult from different positions in the heap. 99.5% share the identical retainer pattern:

string:"message" (or "toolResult", "anthropic-messages")
  → Object (message entry with .type, .role, .api properties)
    → Array[N] (fileEntries array, indices up to 12,000+)
      → SessionManager.fileEntries
        → AgentSession.sessionManager

Every retained object traces back through fileEntries to SessionManager to AgentSession. One array. One field. That's the leak.

Scale

• 73 SessionManager instances in the grown snapshot (7 agents, but each accumulates multiple sessions: cron jobs with isolatedSession: true create unique session keys, plus direct/group/subagent sessions)
• 20 largest fileEntries arrays hold ~11,500 entries each
• Estimated 100-400 MB in the top 20 sessions alone
• Combined with string content referenced by the entries: ~1.4 GB total

Root cause

SessionManager.fileEntries (source: badlogic/pi-mono, packages/coding-agent/src/core/session-manager.ts) is a parallel data structure to Agent._state.messages. While Agent._state.messages is bounded by replaceMessages() (called during compaction, keeping the LLM context window at a manageable size), SessionManager.fileEntries is an unbounded in-memory mirror of the JSONL transcript that is never pruned. Neither native compaction nor context engine compaction touches it. This affects all OpenClaw deployments with long-running sessions, regardless of configuration.

Every appendMessage(), appendAssistantMessage(), tool result write, and metadata entry pushes to this array. The data is simultaneously persisted to the JSONL file on disk, but the in-memory array is never pruned and is retained for the lifetime of the SessionManager. For permanent sessions (main DM, social agent channels), the SessionManager lives for the lifetime of the process, so fileEntries grows without bound until the gateway restarts.

Why existing mitigations don't help

OpenClaw v2026.2.23 added disk-side session maintenance (session.maintenance), and pi-coding-agent has context compaction (Dec 2025). These address disk storage and LLM context respectively. Neither addresses the in-memory fileEntries array:

• replaceMessages(): Bounds Agent._state.messages (the LLM context window). Does not touch SessionManager.fileEntries (the persistence layer).
• contextPruning (cache-ttl mode): Trims tool results from the in-memory context before LLM calls. Does not rewrite JSONL history. Does not touch fileEntries.
• session.maintenance (pruneAfter, maxEntries): Prunes the sessions.json store (metadata). Does not prune in-memory arrays of active sessions.
• --max-old-space-size: Forces V8 to GC more aggressively, but these objects are reachable (live references from SessionManager). GC cannot collect them regardless of pressure.
• session?.dispose() in cron/subagent teardown: Works correctly for short-lived sessions. But for permanent sessions (main DM, social agent channels), dispose() is never called because the session runs for the lifetime of the process.

Every mitigation that exists targets a different layer. The persistence layer's in-memory mirror falls through all of them.

Additional amplifier: ownsCompaction disables native compaction globally

When a context engine registers with ownsCompaction: true, the gateway disables Pi's internal auto-compaction trigger (proactive mid-run compaction) via setCompactionEnabled(false). This is global, not per-session. The overflow recovery path still calls contextEngine.compact(), but for sessions the context engine ignores (cron sessions, social agent sessions via ignoreSessionPatterns), the engine returns { compacted: false, reason: "session excluded" } and the gateway does not fall back to native compaction. Both compaction paths (proactive and overflow) end up doing nothing for ignored sessions, which means their fileEntries arrays grow without any check.

Suggested fix

Cap or eliminate the in-memory JSONL mirror in SessionManager (badlogic/pi-mono, packages/coding-agent/src/core/session-manager.ts). The data is already on disk in the JSONL file. Options:

1. Don't hold fileEntries in memory after write. If no code path reads from the in-memory array after appending (needs verification), switch to append-only disk writes with on-demand JSONL parsing when reads are needed.

2. Cap fileEntries with a sliding window. When the array exceeds N entries (e.g., 200), shift from the front. Older entries remain on disk in the JSONL file. This is the least disruptive change if reads from fileEntries do occur.

3. Periodic trim to match context window size. After each turn, trim fileEntries to match the agent's bounded message count (e.g., the last 200 entries). This keeps the in-memory mirror useful for recent lookups without unbounded growth.

Additionally: when a context engine declines to compact an ignored session (compacted: false), the gateway should fall back to native compaction rather than giving up. This prevents unbounded growth in sessions the context engine doesn't manage.

Related issues

Master/parent issues

• #6190: "Session Log growing and bot hanging up" (common ancestor for session bloat reports; #2254 and #6650 dup to this)

Same symptom (RSS growth from session accumulation), no prior heap analysis

• #13758: 1.9GB + 69% CPU after 13h. Note: commenter echoVic identified session-manager-cache.ts as the likely primary cause, which is the right neighborhood. Our contribution is pinpointing the specific field (fileEntries) and mechanism (unbounded in-memory JSONL mirror) via heap snapshot retainer analysis. Closed as dup of #6413.
• #6413: "Gateway process massive virtual memory leak (22GB+)." Parent of #13758 via dup chain, but was closed by stale bot without a fix. Reported VIRT growth, not RSS.
• #24689: 6GB peak in 26 min (mixed causes reported, including Docker health check misconfiguration in one case; closed by stale bot)
• #2254: Large session files, 396KB single tool result in JSONL (disk-side of the same problem; closed as dup of #6190)
• #20910: 7,069 messages, 20MB transcript, death spiral (extreme case of unbounded growth; open/stale)

Same class of bug (unbounded in-memory structures)

• #4948: Multiple in-memory caches grow unbounded (fixed via PR #11093; same CWE-400 pattern)
• #17820: Cron runs never clean up agent-events Maps (~68 MB/hr, reaching 1-2GB over 10-12 hours; fixed)

Related (address content size or compaction gaps, not fileEntries directly)

• #6650: Tool result session bloat proposal (addresses content size but not array growth; closed as dup of #6190)
• #24800: Auto-compaction not triggered in tool-use loops (fixed via PR #29371; explains why compaction fails to bound growth in tool-heavy sessions)
• #43767: Heartbeat loads unbounded session history (open; another entry point to same problem)

Not fileEntries (different root causes, included for completeness)

• #42662: 3GB in 7 min crash loop. Root cause confirmed as Node.js v24 regression, resolved by downgrading to Node v22. Shares OOM symptoms but is a different bug.
• #43193: Cron/subagent session accumulation. Documented expected behavior (audit trail), not a bug. Addressed by --delete-after-run flag and sessionRetention config.

Workaround

Periodic gateway restarts clear all SessionManager instances and their fileEntries arrays. A watchdog cron that kills the gateway child process at a configurable RSS threshold (e.g., 2GB) provides automated mitigation:

RSS=$(awk '/VmRSS/{print $2}' /proc/$(pgrep -f openclaw-gateway)/status 2>/dev/null)
[ "${RSS:-0}" -gt 2097152 ] && sudo systemctl restart openclaw

Reducing --max-old-space-size from the default to match the watchdog threshold (e.g., 2048) forces V8 to run more aggressive GC on the genuinely temporary allocations (per-turn context assembly, HTTP response buffers), which slows the growth rate but does not stop it.