Issue Station
Sign In
Tools hereToolbox

openclaw - ✅(Solved) Fix Cron jobs with sessionTarget=isolated produce false `lost` errors after successful runs [1 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#71963•Fetched 2026-04-27 05:36:46
View on GitHub
Comments
2
Participants
3
Timeline
5
Reactions
0
Author
Tommy3808Tommy3808
Participants
clawsweeper[bot]clawsweeper[bot]Sanjays2402Sanjays2402Tommy3808Tommy3808
Timeline (top)
commented ×2closed ×1cross-referenced ×1referenced ×1

Cron jobs with sessionTarget: "isolated" are flagged as lost errors in openclaw tasks audit even after the cron execution completes successfully (lastRunStatus: ok). This causes monotonically growing lost error counts on long-running deployments and creates false alarms in audit reports.

Error Message

Cron jobs with sessionTarget: "isolated" are flagged as lost errors in openclaw tasks audit even after the cron execution completes successfully (lastRunStatus: ok). This causes monotonically growing lost error counts on long-running deployments and creates false alarms in audit reports. 4. Each completed run leaves a Task error lost ... backing session missing record. Task error lost <task-id> lost 6d1h backing session missing Task error lost <task-id> lost 5d23h backing session missing

Root Cause

Root cause hypothesis

The isolated session is cleaned up before the task tracker writes the terminal state. Race between session reaper and task state finalizer.

Fix Action

Fix / Workaround

Workaround (if any)

None known. openclaw tasks maintenance --apply only prunes orphaned task-flows, not these cron-spawned isolated session records.

PR fix notes

PR #71968: fix(cron): finalize task ledger before clearing active-jobs flag (#71963)

Description (problem / solution / changelog)

Fixes part of #71963.

What

applyOutcomeToStoredJob currently clears the in-memory active-jobs flag before finalizing the cron run's task-ledger record. The registry maintenance sweep (60s timer) treats cron tasks as lost / backing session missing when they are still running AND isCronJobActive(jobId) returns false. The clear-then-finalize order opens a sub-millisecond window where an in-flight sweep can observe exactly that state and mark a successful cron run lost.

Swap the order: finalize the task ledger first, then clear the active-jobs flag. The active flag now vouches for the task right up until it has reached a terminal status, so the sweep can never observe the false-lost state in-process.

Why this matches the report

The reporter (#71963) sees Task error lost ... backing session missing errors accumulating against sessionTarget: "isolated" cron jobs whose lastRunStatus is ok. Their proposal #2 — "write the terminal task state before the isolated session is reaped" — is exactly this change at the cron-side bookkeeping layer.

Scope

  • 7 lines, comment included.
  • Existing cron timer/ops tests still pass (timer.test.ts, timer.regression.test.ts, ops.test.ts, task-registry.maintenance.issue-60299.test.ts).
  • No public API change.

What this does NOT fix

A second cause in the same report — cron tasks left in running state across gateway restarts — is not addressed here. After a restart the in-memory active-jobs set is empty, so any pre-existing running cron task records hit the same condition on the first sweep and get marked lost. Reconciling those needs a startup-time pass (e.g. cancel-with-reason or promote-to-succeeded based on cronJob.lastRunAtMs) and intentionally lives in a follow-up PR to keep this change reviewable.

The 883 inconsistent_timestamps warnings the reporter mentions are also a separate issue.

Test

pnpm test src/cron/service/timer.test.ts src/cron/service/ops.test.ts \
  src/cron/service/timer.regression.test.ts \
  src/tasks/task-registry.maintenance.issue-60299.test.ts -- --run
# Test Files  4 passed (4)
# Tests       40 passed (40)

Changed files

  • src/cron/service/timer.ts (modified, +6/-1)

Code Example

Tasks audit: 896 findings · 11 errors · 885 warnings
Task error lost <task-id> lost 6d1h backing session missing
Task error lost <task-id> lost 5d23h backing session missing
...
RAW_BUFFERClick to expand / collapse

Summary

Cron jobs with sessionTarget: "isolated" are flagged as lost errors in openclaw tasks audit even after the cron execution completes successfully (lastRunStatus: ok). This causes monotonically growing lost error counts on long-running deployments and creates false alarms in audit reports.

Repro

  1. Create any cron with sessionTarget: "isolated" and payload.kind: "agentTurn" running on an interval (e.g. hourly).
  2. Let it execute one or more times — the cron's lastRunStatus reports ok and the agent turn produces correct output.
  3. Run openclaw tasks audit after a few executions.
  4. Each completed run leaves a Task error lost ... backing session missing record.

Observed

On version 2026.4.24, after roughly 24h of normal cron operation:

Tasks audit: 896 findings · 11 errors · 885 warnings
Task error lost <task-id> lost 6d1h backing session missing
Task error lost <task-id> lost 5d23h backing session missing
...

Cron jobs that produced these errors (all sessionTarget: isolated, all lastRunStatus: ok):

  • social-sentiment-fetcher (8 lost records in last 3 minutes alone — runs hourly)
  • auth-state-watchdog (10-minute interval)
  • whitepaper-hunter, position-monitor

The work itself succeeded. The bookkeeping is wrong.

Expected

For sessionTarget: isolated crons whose lastRunStatus === "ok", the task tracker should:

  • Mark the task succeeded (not lost) when the isolated session terminates normally.
  • Or: write the terminal task state before the isolated session is reaped.
  • Or: treat "backing session missing AND lastRunStatus=ok" as succeeded during audit, not lost.

Root cause hypothesis

The isolated session is cleaned up before the task tracker writes the terminal state. Race between session reaper and task state finalizer.

Impact

  • openclaw tasks audit accumulates false-positive errors indefinitely on long-running deployments.
  • openclaw tasks maintenance --apply cannot prune them (they're attached to active cron jobs).
  • Real lost tasks become harder to spot among the noise.

Secondary issue (might be separate)

Across the same audit, 883 warnings are inconsistent_timestamps (startedAt is earlier than createdAt), all with Age: fresh. This looks like a separate clock/race issue at task creation.

Environment

  • Version: 2026.4.24 (cbcfdf6) — also reproduced on 2026.4.23
  • Linux 6.17.0-14-generic
  • 14 cron jobs total, 4-5 of which use sessionTarget: isolated
  • ~2,156 tasks tracked

Workaround (if any)

None known. openclaw tasks maintenance --apply only prunes orphaned task-flows, not these cron-spawned isolated session records.

Happy to share full anonymized audit output or a minimal repro cron config.

extent analysis

TL;DR

The most likely fix is to modify the task tracker to treat "backing session missing AND lastRunStatus=ok" as succeeded during audit, not lost, to prevent false-positive errors.

Guidance

  • Investigate the race condition between the session reaper and task state finalizer to determine the root cause of the issue.
  • Consider adding a delay or synchronization mechanism to ensure the task tracker writes the terminal state before the isolated session is reaped.
  • Review the openclaw tasks audit logic to handle the case where a task has a lastRunStatus of ok but the backing session is missing.
  • Verify that the issue is not related to the inconsistent_timestamps warnings, which may be a separate clock/race issue.

Example

No code snippet is provided as the issue does not contain sufficient information to create a specific example.

Notes

The issue may be related to a race condition, and resolving it may require modifications to the task tracker or session reaper. The inconsistent_timestamps warnings may be a separate issue that needs to be addressed independently.

Recommendation

Apply a workaround to modify the openclaw tasks audit logic to treat "backing session missing AND lastRunStatus=ok" as succeeded during audit, not lost, to prevent false-positive errors. This will help reduce the noise in the audit reports and make it easier to spot real lost tasks.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#runtime error#dependency conflict#environment setup#docker error#permission error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - ✅(Solved) Fix Cron jobs with sessionTarget=isolated produce false `lost` errors after successful runs [1 pull requests, 2 comments, 3 participants]