PR #84220: fix(agents): abandon hung in-flight write lock on attempt cleanup (#84193)

• Repository: openclaw/openclaw
• Author: yetval
• State: closed | merged: False
• Link: https://github.com/openclaw/openclaw/pull/84220

Description (problem / solution / changelog)

Summary

Fixes #84193 — post-run Pi auto-compaction can hang while holding the session JSONL write lock. After the run's outer abort/timeout fires the wrapped await run() body never settles, so the lock-controller's finally never runs, so the lock keeps living in-process for the entire maxHoldMs window (defaults to runTimeout + 900s compaction grace = many minutes). Every later turn on the same Discord channel then bounces off SessionWriteLockTimeoutError: session file locked (timeout 60000ms) until a Gateway restart.

This change tracks every lock reacquired through withSessionWriteLock and abandons the still-held entries from acquireForCleanup, so the next turn can acquire the session file immediately. The existing transcript fence still rejects later writes that would have raced a partial-compaction mutation.

What changed

• src/agents/pi-embedded-runner/run/attempt.session-lock.ts
  • Wrap each reacquired write lock in a small InFlightLockEntry and add it to a per-controller Set.
  • New abandonInFlightWriteLocks() flips the controller into takeover state and awaits force-release of every still-held entry (idempotent via the entry's released flag).
  • acquireForCleanup() calls it before re-acquiring, so a hung compact() can never keep the JSONL lock alive past attempt teardown. Also releases the lingering heldLock (initial coarse lock) if it was still held when takeover was detected.
  • Emit a stderr diagnostic line on abandon so maintainers can correlate journalctl -u openclaw traces with the cleanup path: [session-write-lock] abandoned N in-flight lock(s) on attempt cleanup: sessionFile=... owner=pid=... reason=stuck-compaction-or-hook.
• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.test.ts (new, unit)
  • Reproduces the leak with a stuck withSessionWriteLock(() => never-resolve) and proves cleanup releases it.
  • Asserts further writes after the abandon are rejected via the existing EmbeddedAttemptSessionTakeoverError fence path.
• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.integration.test.ts (new, real fs)
  • Uses the real acquireSessionWriteLock against a real tmp session.jsonl (no mocks). Holds the lock through a stuck run, verifies a competing acquireSessionWriteLock from a separate caller fails with SessionWriteLockTimeoutError BEFORE cleanup, then runs cleanup and verifies the same competing acquire succeeds AFTER cleanup. Captures the diagnostic line and confirms the on-disk .jsonl bytes are not torn.

Real behavior proof

Behavior addressed: post-run Pi auto-compaction holds the session JSONL write lock past the run's outer timeout, blocking every later turn for the same session with SessionWriteLockTimeoutError until Gateway restart. (#84193)

Real environment tested: macOS arm64, Node v25.9.0, vitest 4.1.6 from node scripts/run-vitest.mjs. Real node:fs lock file + real acquireSessionWriteLock + real createEmbeddedAttemptSessionLockController — no mocks for the lock subsystem.

Exact steps or command run after this patch:

node scripts/run-vitest.mjs \
  src/agents/pi-embedded-runner/run/attempt.session-lock.test.ts \
  src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.test.ts \
  src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.integration.test.ts \
  src/agents/pi-embedded-runner/run/compaction-retry-aggregate-timeout.test.ts \
  src/agents/session-write-lock.test.ts

Evidence after fix (vitest summary):

 Test Files  10 passed (10)
      Tests  114 passed (114)
   Duration  1.11s

Real-fs trace from a standalone repro script (real acquireSessionWriteLock, no vitest, no mocks — captured locally with tmp paths redacted to $TMP):

[trace] sessionFile=$TMP/repro-84193-pVdd3y/session.jsonl
[trace] process.pid=72484
[trace] BEFORE cleanup: lock file owner=pid=72484
[trace] BEFORE cleanup: competing acquire failed after 1503ms with SessionWriteLockTimeoutError: session file locked (timeout 1500ms): pid=72484 $TMP/repro-84193-pVdd3y/session.jsonl.lock
[session-write-lock] abandoned 1 in-flight lock(s) on attempt cleanup: sessionFile=$TMP/repro-84193-pVdd3y/session.jsonl owner=pid=72484 reason=stuck-compaction-or-hook
[trace] AFTER cleanup:  competing acquire succeeded in 0ms
[trace] session.jsonl bytes intact: "{\"type\":\"session\"}\n"

This is the same shape of evidence the issue reporter would see: lock file on disk with the live Gateway PID before cleanup, the SessionWriteLockTimeoutError matching the user-reported error name and pid format (pid=N), the stderr diagnostic line on abandon, then immediate acquire success on the same session file and a verified non-torn session.jsonl. Same regression suite on upstream/main without this patch fails at:

embedded attempt session lock — stuck auto-compaction (#84193)
  ✗ abandons in-flight write lock when cleanup runs while a hung compact() still owns it
    expected "vi.fn()" to be called 1 times, but got 0 times
  ✗ rejects further withSessionWriteLock calls after abandoning a hung in-flight lock
    expected [Function] to throw error matching /session file changed/ but got
    'Cannot read properties of undefined (reading 'release')'

Observed result after fix: cleanup releases the stuck reacquired lock immediately. The 60s acquire timeout no longer blocks the next turn. Transcript fencing still rejects further writes after abandon via the existing EmbeddedAttemptSessionTakeoverError. Diagnostic stderr line is observable so a maintainer running journalctl -u openclaw after the failure will see which session file/pid was abandoned.

What was not tested: live Discord + Anthropic Opus session against a running Gateway. The integration test covers the file-lock subsystem semantics that the bug actually leaks (on-disk .jsonl.lock, the controller's reacquire path, the cleanup hook called from attempt.ts:4832); a Discord/Opus run would only add transport plumbing on top of the same locking code, so the lock-side behavior is fully covered here.

Risk addressed

ClawSweeper flagged two session-state-sensitive risks. Addressed:

• Force-releasing in-flight compaction lock. Abandon only fires from acquireForCleanup(), which the embedded attempt runner only calls in its outer finally block (attempt.ts:4832). At that point the run's outer abort/timeout has already fired, so any late compaction write is already on an error path. The transcript fence (assertSessionFileFence) on the next attempt's controller catches any post-abandon byte-level divergence on its first withSessionWriteLock call, so a late compaction write cannot silently corrupt a future turn.
• Race with concurrent in-process acquire. The abandon now awaits each lock.release() before returning, so the .jsonl.lock file is gone on disk before cleanup returns. The integration test exercises exactly this race (competing acquire from the same process fired immediately after cleanup) and confirms it succeeds without bouncing off the file-lock-manager's stale-lock detection.

Test plan

• node scripts/run-vitest.mjs src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.integration.test.ts — real-fs proof, 2/2 pass
• node scripts/run-vitest.mjs src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.test.ts — unit regression, 4/4 pass
• node scripts/run-vitest.mjs src/agents/pi-embedded-runner/run/attempt.session-lock.test.ts — pre-existing suite, 28/28 pass (unchanged)
• node scripts/run-vitest.mjs src/agents/pi-embedded-runner/run/compaction-retry-aggregate-timeout.test.ts — pass
• node scripts/run-vitest.mjs src/agents/session-write-lock.test.ts — pass
• node scripts/run-oxlint.mjs on touched files — 0 errors / 0 warnings
• pnpm exec oxfmt --check on touched files — formatted
• Live Discord/Opus auto-compaction reproduction against a running Gateway (left for maintainer / Crabbox / Mantis)

Changed files

• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.integration.test.ts (added, +116/-0)
• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.test.ts (added, +107/-0)
• src/agents/pi-embedded-runner/run/attempt.session-lock.ts (modified, +79/-5)

PR #84353: fix(agents): abandon hung in-flight write lock on attempt cleanup (#84193)

• Repository: openclaw/openclaw
• Author: clawsweeper[bot]
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/84353

Description (problem / solution / changelog)

Makes https://github.com/openclaw/openclaw/pull/84220 merge-ready for the ClawSweeper automerge loop. The edit pass should inspect the live PR diff, review comments, and failing checks; rebase if needed; keep the contributor branch credited; and stop only when validation is green or an external blocker is proven.

ClawSweeper 🐠 replacement reef notes:

• Cluster: automerge-openclaw-openclaw-84220
• Source PRs: https://github.com/openclaw/openclaw/pull/84220
• Credit: Source PR: https://github.com/openclaw/openclaw/pull/84220
• Validation: pnpm check:changed
• Replacement reason: ClawSweeper could not update the source PR branch directly, so it opened a writable replacement PR instead.
• Automerge requested by: @Takhoffman

• Repair fallback: GitHub rejected the repair branch push because it updates workflow files and the ClawSweeper app token does not have workflows permission

Inherited issue-closing references from the source PR: Fixes #84193

Co-author credit kept:

• @yetval: Co-authored-by: Yuval Dinodia [email protected]

fish notes: model gpt-5.5, reasoning high; reviewed against c1b31439982f.

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.integration.test.ts (added, +116/-0)
• src/agents/pi-embedded-runner/run/attempt.session-lock.compaction-leak.test.ts (added, +109/-0)
• src/agents/pi-embedded-runner/run/attempt.session-lock.ts (modified, +79/-5)