PR #72251: fix(backup): retry on tar EOF race and skip known volatile files

• Repository: openclaw/openclaw
• Author: abnershang
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/72251

Description (problem / solution / changelog)

Problem

openclaw backup create fails with Error: did not encounter expected EOF on any live installation. Root cause: node-tar's WriteEntry records size from the initial lstat(); if the file grows between lstat() and the end of fs.read(), the header no longer matches the consumed bytes, the stream errors, and tar.c() aborts the whole archive. Common culprits are append-only JSONL transcripts and .log files under the state directory.

Details and generic reproducer: #72249. Related but distinct races: #67417 (ENOENT when a session file is deleted mid-backup). Broader exclude-rule proposal: #67990.

Impact

Affects any live install where backup source files are written concurrently — session transcripts, cron run logs, gateway logs. Today the workaround is to stop the gateway before backing up, which is not viable for scheduled backups.

Fix

1. Default volatile-path filter in the backup archiver — no user config required:

   • {stateDir}/sessions/**/*.{jsonl,log}
   • {stateDir}/cron/runs/**/*.log
   • {stateDir}/logs/**/*.{jsonl,log}
   • *.{sock,pid,tmp,lock} anywhere

   Rationale: these files are either actively appended to (logs), transient (sockets/pid/lock), or have no restoration value from a partial tail. Snapshotting them racily has no upside.

2. Retry on EOF-class errors (up to 3 attempts, 10s / 20s backoff) for residual races on non-filtered files. The partial temp archive is removed between attempts, so no half-written artifact escapes.

3. Error enrichment: on final failure, surface err.path (the offending file) and attempt count. Turns an opaque error into an actionable one.

4. skippedVolatileCount observability: extend BackupCreateResult with the skip count; surface it in both the stdout summary and the --json output.

Backwards compatibility

• Filter is additive; no previously-included path stops being included unless it matches a volatile rule. The volatile rules cover files that already produced errors or had no snapshot value.
• --json output gains a new optional field skippedVolatileCount. Existing consumers that ignore unknown fields are unaffected.
• Error message format changes only on failure, and only to add diagnostic context.

Testing

Added:

• src/infra/backup-volatile-filter.test.ts — predicate unit tests for each rule and the anywhere-patterns, plus negative cases.
• src/infra/backup-create.test.ts — retry tests using a mocked tar runner:
  • Throws EOF twice then succeeds → 3 calls, 10s + 20s backoff, logs emitted.
  • Throws EOF on every attempt → final error includes offending path + attempt count.
  • Throws non-EOF → no retry, no sleep.
• formatBackupCreateSummary updated to include the skip-count line when > 0.

Verified locally: a previously-failing live-install backup (large state tree with actively written session and cron logs) now completes cleanly; the skipped-volatile count is reported and the archive is usable.

Future work (not in this PR)

A --quiesce flag that pauses cron and drains in-flight work before archival, for users who want stronger consistency guarantees than "retry-plus-filter".

Fixes #72249 See also #67417, #67990

Changed files

• src/commands/backup.ts (modified, +4/-1)
• src/infra/backup-create.test.ts (modified, +160/-2)
• src/infra/backup-create.ts (modified, +137/-15)
• src/infra/backup-volatile-filter.test.ts (added, +49/-0)
• src/infra/backup-volatile-filter.ts (added, +81/-0)