openclaw - 💡(How to fix) Fix Gateway: redundant per-plugin dep-resolution saturates event loop in embedded-fallback path [1 comments, 2 participants]

Summary

• Every openclaw agent --local (embedded-fallback) invocation triggers a full bundled-runtime-dep resolution pass for every loaded plugin — 33 passes against an identical 41-spec list per run.
• All 33 plugins share the same spec list (~/.openclaw/plugin-runtime-deps/openclaw-<version>/); no per-plugin subset diffing occurs. The deps are already installed on disk — the work is pure re-verification with no actual installation.
• Each pass takes 5–13 ms synchronously on the event loop. At 33 plugins × up to 46 embedded runs per minute, the loop never clears between requests.
• Observed peak: eventLoopUtilization=0.997, eventLoopDelayP99=47,882 ms, runtime-plugins stage consuming 21,860 ms of a single embedded run's 49,248 ms startup time.
• The fix is a run-scoped Map<specHash, ResolvedSet> allocated once per registry-load call and checked inside the installDeps callback before installBundledRuntimeDeps is invoked. First plugin resolves; the remaining 32 get a cache hit and skip the span entirely.

Environment

• OpenClaw version: 2026.4.29 (commit a448042)
• Platform: macOS (darwin 25.3.0, arm64)
• Node version: 24.x
• Install method: Homebrew (npm -g via brew install openclaw)
• Dist bundle: dist/loader-CLyHx60E.js (content-hashed, changes each release)
• Plugin count: 33 loaded plugins

Reproduction

openclaw agent --local --agent <any-configured-agent>

Or trigger any model-provider rate-limit so the gateway falls over to the embedded path. With 33 plugins loaded, saturation is immediate.

To confirm before/after a patch:

grep "staging bundled runtime deps" ~/.openclaw/logs/gateway.log \
  | grep "$(date +%Y-%m-%d)" \
  | wc -l

Each distinct runId should produce exactly 1 staging line post-fix, not 33.

Observed Metrics (2026-05-01, production gateway)

Root Cause

File (dist): dist/loader-CLyHx60E.js, lines 4078–4128 File (source): src/plugins/loader.ts Function: prepareBundledPluginRuntimeLoadRoot Inner callback: installDeps

The installDeps callback is invoked once per plugin during the registry-load for-loop. Each invocation runs measureDiagnosticsTimelineSpanSync("runtimeDeps.stage", ...) then installBundledRuntimeDeps(...), which stat/require.resolve()s all 41 specs in the shared runtime-deps root.

Critical issue: installBundledRuntimeDeps receives the same installSpecs array for every plugin — all 33 plugins share an identical 41-spec list pointing at the same versioned runtime-deps root. No memoization or guard checks whether a prior plugin in the same run already resolved an identical spec list. 32 of 33 resolution passes are functionally redundant.

The work is not I/O-heavy (deps already present, "installed in 5–13ms" confirms no extraction), but the synchronous span accumulation across all 33 plugins in a tight for-loop prevents the loop from yielding, producing the observed saturation.

Proposed Fix

Add a run-scoped Map<specHash, ResolvedSet> cache to the plan-build closure that contains the candidates for-loop. Allocated once per registry-load call, doesn't outlive the run.

// ─── within the plan-build closure containing the for-loop over `candidates` ───
const _runSpecCache = new Map();

function _hashSpecs(specs) {
  return specs.slice().sort().join('\0');
}

// ─── inside the installDeps callback, before calling installBundledRuntimeDeps ───
async function installDeps(record, installSpecs, installBundledRuntimeDeps, logInstalled, measureDiagnosticsTimelineSpanSync) {
  const specHash = _hashSpecs(installSpecs);

  if (_runSpecCache.has(specHash)) {
    const cached = _runSpecCache.get(specHash);
    console.debug(`[plugins] ${record.id} skipped dep resolution (run cache hit, ${installSpecs.length} specs already resolved this run)`);
    logInstalled(record.id, installSpecs, cached);
    return cached;
  }

  let resolved;
  measureDiagnosticsTimelineSpanSync('runtimeDeps.stage', () => {
    resolved = installBundledRuntimeDeps(record, installSpecs);
  });

  _runSpecCache.set(specHash, resolved);
  return resolved;
}

Expected Impact

Compatibility

• No changes to any public API, CLI interface, or plugin SDK contract
• Cache is strictly internal to the plan-build closure
• Safe to gate behind a feature flag (e.g., OPENCLAW_PLUGIN_SPEC_CACHE=1) during rollout
• Behavior under cache miss is byte-for-byte identical to current code path

Test Plan

1. Log-line count: grep staging bundled runtime deps for one embedded-fallback run → expect 1 line (was 33).
2. Loop utilization: trigger 10 consecutive embedded-fallback runs → expect eventLoopUtilization < 0.40.
3. Startup time: runtime-plugins stage time should drop from ~6,400 ms average to <300 ms.
4. Correctness: openclaw plugins deps --check resolves all 41 specs cleanly.

Happy to send a PR with the patch if maintainers want.