openclaw - 💡(How to fix) Fix feat: configurable CommandLane.Nested concurrency — unblock parallel hook/cron/sessions_send execution [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#57952Fetched 2026-04-08 01:55:46
View on GitHub
Comments
0
Participants
1
Timeline
0
Reactions
0

The CommandLane.Nested lane is hardcoded to maxConcurrent: 1 with no configuration surface. All webhook hook sessions (/hooks/agent), cron isolated agent runs, and sessions_send inter-agent calls route through this lane, meaning only one can execute at a time across the entire gateway.

This creates severe serialization bottlenecks for any deployment using webhooks or cron jobs at scale — queue backlogs grow linearly with the number of concurrent events, and sessions_send calls from cron/hook context deadlock because the caller holds the single Nested slot while the target session waits for it.

Root Cause

applyGatewayLaneConcurrency() configures concurrency for Main, Cron, and Subagent lanes but omits Nested entirely:

function applyGatewayLaneConcurrency(cfg) {
  setCommandLaneConcurrency(CommandLane.Cron, cfg.cron?.maxConcurrentRuns ?? 1);
  setCommandLaneConcurrency(CommandLane.Main, resolveAgentMaxConcurrent(cfg));
  setCommandLaneConcurrency(CommandLane.Subagent, resolveSubagentMaxConcurrent(cfg));
  // Missing: setCommandLaneConcurrency(CommandLane.Nested, ???);
}

Additionally, resolveNestedAgentLane() routes cron and hook work to CommandLane.Nested regardless of the lane parameter sent by the caller. Even when a webhook connector sends lane: "main", the gateway's normalizeAgentPayload() extracts only 11 fields and drops the lane field, so hook sessions always fall back to Nested.

Fix Action

Workaround

Patching the bundled JS to add the missing setCommandLaneConcurrency(CommandLane.Nested, N) call. Must be re-applied after every update.

Code Example

function applyGatewayLaneConcurrency(cfg) {
  setCommandLaneConcurrency(CommandLane.Cron, cfg.cron?.maxConcurrentRuns ?? 1);
  setCommandLaneConcurrency(CommandLane.Main, resolveAgentMaxConcurrent(cfg));
  setCommandLaneConcurrency(CommandLane.Subagent, resolveSubagentMaxConcurrent(cfg));
  // Missing: setCommandLaneConcurrency(CommandLane.Nested, ???);
}

---

# Create multiple cron jobs that fire simultaneously
for i in 1 2 3 4 5 6 7 8; do
  openclaw cron create \
    --name "test-lane-$i" \
    --cron "30 18 * * *" \
    --agent main \
    --session isolated \
    --message "Write a short response." \
    --no-deliver --exact --timeout-seconds 120
done

# Trigger all at once
for id in $(openclaw cron list | grep "test-lane" | awk '{print $1}'); do
  openclaw cron run "$id" &
done
wait

---

{
  "agents": {
    "defaults": {
      "nestedMaxConcurrent": 8
    }
  }
}

---

setCommandLaneConcurrency(
    CommandLane.Nested,
    cfg.agents?.defaults?.nestedMaxConcurrent ?? 4
);
RAW_BUFFERClick to expand / collapse

Summary

The CommandLane.Nested lane is hardcoded to maxConcurrent: 1 with no configuration surface. All webhook hook sessions (/hooks/agent), cron isolated agent runs, and sessions_send inter-agent calls route through this lane, meaning only one can execute at a time across the entire gateway.

This creates severe serialization bottlenecks for any deployment using webhooks or cron jobs at scale — queue backlogs grow linearly with the number of concurrent events, and sessions_send calls from cron/hook context deadlock because the caller holds the single Nested slot while the target session waits for it.

Root Cause

applyGatewayLaneConcurrency() configures concurrency for Main, Cron, and Subagent lanes but omits Nested entirely:

function applyGatewayLaneConcurrency(cfg) {
  setCommandLaneConcurrency(CommandLane.Cron, cfg.cron?.maxConcurrentRuns ?? 1);
  setCommandLaneConcurrency(CommandLane.Main, resolveAgentMaxConcurrent(cfg));
  setCommandLaneConcurrency(CommandLane.Subagent, resolveSubagentMaxConcurrent(cfg));
  // Missing: setCommandLaneConcurrency(CommandLane.Nested, ???);
}

Additionally, resolveNestedAgentLane() routes cron and hook work to CommandLane.Nested regardless of the lane parameter sent by the caller. Even when a webhook connector sends lane: "main", the gateway's normalizeAgentPayload() extracts only 11 fields and drops the lane field, so hook sessions always fall back to Nested.

Steps to Reproduce

# Create multiple cron jobs that fire simultaneously
for i in 1 2 3 4 5 6 7 8; do
  openclaw cron create \
    --name "test-lane-$i" \
    --cron "30 18 * * *" \
    --agent main \
    --session isolated \
    --message "Write a short response." \
    --no-deliver --exact --timeout-seconds 120
done

# Trigger all at once
for id in $(openclaw cron list | grep "test-lane" | awk '{print $1}'); do
  openclaw cron run "$id" &
done
wait

All 8 jobs serialize through Nested lane — total wall time ~120s instead of ~15s.

For sessions_send deadlock: have a cron job call sessions_send to another session. The target can never execute because the caller holds the only Nested slot.

Suggested Fix

Make nested lane concurrency configurable, consistent with existing lane config patterns:

{
  "agents": {
    "defaults": {
      "nestedMaxConcurrent": 8
    }
  }
}

And in applyGatewayLaneConcurrency():

setCommandLaneConcurrency(
    CommandLane.Nested,
    cfg.agents?.defaults?.nestedMaxConcurrent ?? 4
);

Session-level serialization (enqueueSession) already prevents concurrent access to the same session, so increasing nested lane concurrency is safe for independent jobs.

Also: normalizeAgentPayload() should preserve the lane field from webhook payloads so callers can target the intended lane.

Workaround

Patching the bundled JS to add the missing setCommandLaneConcurrency(CommandLane.Nested, N) call. Must be re-applied after every update.

Related Issues

  • #52682 — cron isolated agent runs serialize on nested lane
  • #52271 — sessions_send from cron/heartbeat deadlocks on nested lane
  • #55516 — hooks.maxConcurrentRuns config request
  • #14214 — original nested lane maxConcurrent: 1 report (closed)
  • #22167 — configurable nested lane concurrency feature request (closed)
  • #45165 — sessions_send nested lane serialization

Environment

  • OpenClaw: latest (2026.3.x)
  • OS: Ubuntu 24.04 ARM64
  • Node.js: v22.22.x

extent analysis

Fix Plan

To resolve the issue, we need to make the following changes:

  • Configure the nestedMaxConcurrent setting in the configuration file:
{
  "agents": {
    "defaults": {
      "nestedMaxConcurrent": 8
    }
  }
}
  • Update the applyGatewayLaneConcurrency() function to use the new configuration setting:
setCommandLaneConcurrency(
  CommandLane.Nested,
  cfg.agents?.defaults?.nestedMaxConcurrent ?? 4
);
  • Modify the normalizeAgentPayload() function to preserve the lane field from webhook payloads:
function normalizeAgentPayload(payload) {
  // ... existing code ...
  const normalizedPayload = {
    // ... existing fields ...
    lane: payload.lane, // add this line
  };
  return normalizedPayload;
}

Verification

To verify that the fix worked, you can:

  • Run the cron jobs simultaneously using the provided script and measure the total wall time.
  • Check that the jobs are executing concurrently by monitoring the system resources and logs.
  • Test the sessions_send functionality to ensure that it no longer deadlocks.

Extra Tips

  • Make sure to update the configuration file and restart the service after applying the changes.
  • Monitor the system performance and adjust the nestedMaxConcurrent setting as needed to avoid overloading the system.
  • Consider implementing additional logging and monitoring to detect any potential issues with the concurrent execution of cron jobs and webhook sessions.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING