PR #75048: fix(plugins): canonicalize packageRoot before hashing runtime-deps stage key

• Repository: openclaw/openclaw
• Author: openperf
• State: closed | merged: True
• Link: https://github.com/openclaw/openclaw/pull/75048

Description (problem / solution / changelog)

Summary

• Problem: On Windows multi-instance deployments (5x PM2 workers, OpenClaw 2026.4.27, Node v22.x), bundled channels fail to load with ENOENT on shared dist chunks under ~/.openclaw/plugin-runtime-deps/openclaw-2026.4.27-<hash>/dist/*.js (e.g. channel-reply-pipeline-DIVeGRyD.js, channel-options-UH47ik5d.js, cache-controls-DyMccij0.js). Different processes — and even different code paths inside the same process — derive different <hash> values for the same physical install, so loaders look up files in a stage directory that was populated under a different key. Site of the divergence: src/plugins/bundled-runtime-deps-roots.ts:55-57, createPathHash, which hashes path.resolve(value) instead of the canonical filesystem path.
• Root Cause: createPathHash did not canonicalize the packageRoot to its OS-real path before hashing — it only normalized lexically via path.resolve. The two consumers of this hash arrive with different lexical forms of the same physical directory:
  • Loader path: src/plugins/loader.ts:1419 produces pluginRoot = safeRealpathOrResolve(candidate.rootDir) which already resolves symlinks via fs.realpathSync. Walking up to packageRoot yields the realpath form (e.g. C:\Users\<u>\.npm\node_modules\openclaw).
  • Bundled-channel path: src/channels/plugins/bundled.ts:206-211 calls resolveBundledChannelBoundaryRoot with params.rootScope.packageRoot, which originates from OPENCLAW_PACKAGE_ROOT in src/channels/plugins/bundled-root.ts and is not realpath-normalized. Walking up yields the symlinked form (e.g. C:\Users\<u>\AppData\Roaming\npm\node_modules\openclaw). Because path.resolve only normalizes path components — it does not resolve symlinks, junctions, or drive-letter casing — the same physical install hashes to two different 12-char keys (matching the two values reported in #74963: 5f4e2e59ed9a and 456555aaef5c). The loader stages dist chunks into hash-A's directory; the channel loader then opens hash-B and gets ENOENT. PM2 multi-instance amplifies the symptom because each worker can independently produce a third lexical form via its argv1/cwd context, but the bug also reproduces single-instance whenever an npm symlink, Windows junction, or drive-letter casing variant is in play.
• Fix: Hash the OS-canonical (realpath) form of packageRoot instead of the lexical form. createPathHash now delegates path normalization to the file-local realpathOrResolve helper (src/plugins/bundled-runtime-deps-roots.ts:240), which calls fs.realpathSync.native(value) and falls back to path.resolve(value) only when the path does not yet exist. This is the same canonicalization already used by resolveExistingExternalBundledRuntimeDepsRoots to detect already-staged package roots, so the file's two normalization sites are now consistent. Side-effect surface stays small: createPathHash is reachable from exactly one caller (resolveExternalBundledRuntimeDepsInstallRoots, line 213); no other call site re-implements the hash format. Existing 4.27 stage directories created with the old (lexical) hash become orphaned on disk after upgrade — they are not referenced by the new key but cause no functional regression; first start after upgrade re-stages once into the canonical directory and all subsequent starts (including all PM2 workers) share it. The fallback to path.resolve preserves the old behavior in the degenerate case where the package root has been deleted between resolution and hashing.
• What changed:
  • src/plugins/bundled-runtime-deps-roots.ts: createPathHash now hashes realpathOrResolve(value) instead of path.resolve(value); one-line behavioral change plus a two-line comment explaining why.
  • src/plugins/bundled-runtime-deps.test.ts: new regression test stages bundled runtime deps to the same root for symlinked packageRoot views (issue #74963) — creates a real package root and a sibling symlink to it, calls resolveBundledRuntimeDependencyInstallRoot via both views, and asserts the two install roots are byte-equal and match the openclaw-<version>-<12-hex> shape. Skipped on win32 to follow the existing itSupportsSymlinks precedent at line 2938 (Windows symlink creation requires admin in CI).
• What did NOT change (scope boundary):
  • No CHANGELOG.md entry — left for maintainers to place under the active release section per their preferred phrasing/credit format.
  • No change to realpathOrResolve itself, the file lock implementation (bundled-runtime-deps-lock.ts), the dist-mirror cache (bundled-runtime-dist-mirror-cache.ts), the channel loader (src/channels/plugins/bundled.ts), or the manifest discovery flow (src/plugins/loader.ts). The fix is intentionally confined to the hashing seam; it does not also "fix" the lexical inconsistency between the loader and channel call sites because doing so would broaden the surface (hot import paths per src/channels/CLAUDE.md) without changing the observable outcome — converging the hash already converges the stage directory.
  • No change to the existing path.resolve(installRoot) hash construction inside the test at bundled-runtime-deps.test.ts:2444. That value is computed only as the right-hand side of a .not.toBe assertion (line 2450), so the assertion still holds: the resolver returns the canonical installRoot, which is not equal to any hypothetical fallback root — regardless of whether the fallback is computed lexically or canonically.
  • No new env vars, no new public API, no new types, no any, no behavioral change for source-checkout layouts (the source-checkout branch in resolveBundledRuntimeDependencyInstallRootPlan/resolveBundledRuntimeDependencyPackageInstallRootPlan short-circuits before reaching createPathHash).
  • No deprecation, no migration shim. Existing stage dirs from the buggy hash become disk-resident orphans; they are version-prefixed (openclaw-2026.4.27-*) so the existing prune logic at pruneUnknownBundledRuntimeDepsRoots (which targets openclaw-unknown-*) leaves them alone — same as any prior version's stage directory. Disk cost is bounded and one-time.

Reproduction

1. On Windows, run 5 OpenClaw 2026.4.27 gateway instances under PM2 against the same install (or, simpler, expose the install via two paths — one canonical and one through an npm/Windows junction).
2. Boot the gateway; observe [channels] failed to load bundled channel <id>: ENOENT for several channels (Discord, Feishu, Google Chat, iMessage, LINE, IRC).
3. Inspect ~/.openclaw/plugin-runtime-deps/: two or more directories named openclaw-2026.4.27-<12-hex> exist with the same version prefix but different hashes. The chunks named in the ENOENT messages exist under one hash but the failing process is reading from the other.

After applying this PR, only one such directory exists per version and all instances/call sites resolve to it.

Equivalent unit-level reproduction (already wired in this PR): pnpm test src/plugins/bundled-runtime-deps.test.ts -t "issue #74963" — fails on main (two install roots), passes after the fix.

Risk / Mitigation

• Risk — Stage-directory churn on upgrade: existing 2026.4.27 deployments will create one new realpath-keyed stage directory on first run and leave the old lexical-keyed directory(ies) behind. This is a one-time disk cost (single-digit GB at worst, dependent on the bundled plugin runtime-deps tree size); subsequent starts share the new directory. Mitigation: behavior is identical to any major version's stage directory rotation, no operator action is required. The orphaned directory is version-prefixed so it does not collide with any future hash and does not get pruned by the openclaw-unknown-* cleanup; operators who care about disk usage can rm -rf it manually or wait for the next major version's natural rotation.
• Risk — realpathSync.native failing on a path that does exist: would silently fall back to path.resolve, reproducing the old (buggy) hash. In practice realpathSync.native only fails on ENOENT/EACCES/loop-detection; the packageRoot here is the parent of the dist tree we just enumerated, so it is guaranteed to exist at this point in the flow. Mitigation: the fallback preserves the pre-fix behavior, so failure mode is "no improvement", not "new regression". The added regression test exercises the realpath path on Linux/macOS; the Windows-skipped path still benefits from realpath's native libuv resolver in production where realpathSync.native is the canonical Windows realpath.
• Risk — Test flakiness on macOS due to /tmp → /private/tmp symlinking: the new test creates its own intra-tempdir symlink (linkedPackageRoot → realPackageRoot), so it does not rely on /tmp being or not being a symlink — both sides go through the same realpathSync.native. Mitigation: tested mentally against macOS /var/folders/.../T (os.tmpdir() default) and Linux /tmp; the test asserts equality between two resolver calls, both of which canonicalize identically.
• Risk — Performance regression: one fs.realpathSync.native stat per createPathHash call. Mitigation: createPathHash is invoked only inside resolveExternalBundledRuntimeDepsInstallRoots, which runs at plugin discovery time (cold start), not on the hot request path. Negligible measured impact.

Change Type (select all)

• Bug fix

Scope (select all touched areas)

• Plugins
• Bundled runtime deps
• Channels (downstream consumer of the converged stage directory)

Linked Issue/PR

Fixes #74963

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/plugins/bundled-runtime-deps-roots.ts (modified, +3/-1)
• src/plugins/bundled-runtime-deps.test.ts (modified, +35/-0)
• test/release-check.test.ts (modified, +13/-1)

---

PR #75183: fix: simplify bundled runtime dependency repair

• Repository: openclaw/openclaw
• Author: steipete
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75183

Description (problem / solution / changelog)

Summary

This PR unifies bundled plugin runtime-dependency repair around the package-level plan and lets npm/pnpm own dependency convergence once OpenClaw has decided a repair is required.

What changed:

• Build the active bundled plugin dependency plan before startup/runtime imports, then repair the selected package-level install root once instead of per-plugin ad hoc scans.
• Treat an existing node_modules tree without complete generated materialization as incomplete, even when package sentinels such as node_modules/<dep>/package.json exist.
• Force the package-manager repair once the planner has marked a tree incomplete, so writing the generated install manifest cannot accidentally turn the repair into a no-op.
• Keep post-install verification on requested packages and declared entry files, so package-manager success is not trusted until the staged tree is actually usable.
• Accept generated manifest supersets for narrower plugin loads so a complete package-level stage is reused rather than pruned/reinstalled.
• Preserve config/doctor/hot-reload behavior: config determines the plugin plan; doctor/config edits enter plan mode; startup only repairs when the plan or materialization requires it.

Fixes / related reports

Fixes #75309.

Supersedes the implementation approach in #75310 by keeping the same narrow idea but closing the reviewed hole where a generated manifest plus a no-main package sentinel could still look materialized.

Also hardens the recovery side of the already-addressed reports #75296 and #75304:

• #75296: post-install verification already fails closed when npm/pnpm reports success but requested packages are missing; this PR additionally makes later doctor/startup repair recover from the leftover partial tree.
• #75304: mirrored root dependencies and manifest-superset reuse already address the json5/prune crash loop; this PR prevents interrupted runtime-deps stages from staying stuck behind false package sentinels.

Not claimed: #73520, #74948, #74963, #75071, and #75288 are nearby runtime-deps lifecycle issues with different root causes.

Verification

Local:

• pnpm docs:list
• pnpm exec oxfmt --check --threads=1 src/plugins/bundled-runtime-deps-install.ts src/plugins/bundled-runtime-deps.ts src/plugins/bundled-runtime-deps-materialization.ts src/plugins/bundled-runtime-deps.test.ts src/commands/doctor-bundled-plugin-runtime-deps.test.ts
• pnpm exec oxfmt --check --threads=1 CHANGELOG.md
• pnpm check:changelog-attributions
• git diff --check
• pnpm test src/plugins/bundled-runtime-deps.test.ts src/commands/doctor-bundled-plugin-runtime-deps.test.ts src/gateway/server-startup-plugins.test.ts src/gateway/server.reload.test.ts -- --reporter=verbose

Blacksmith/Testbox:

• tbx_01kqgsehj3tf33k2dmzy35ds7j: pnpm test src/plugins/bundled-runtime-deps.test.ts src/commands/doctor-bundled-plugin-runtime-deps.test.ts src/gateway/server-startup-plugins.test.ts src/gateway/server.reload.test.ts -- --reporter=verbose passed, 3 Vitest shards / 177 tests.

Broad pnpm check:changed was attempted on tbx_01kqgrw1phqcm20c9w06n1w4wx, but the Testbox full-sync omitted the tracked-but-gitignored pnpm-lock.yaml; the workaround polluted remote node_modules into the changed-file scan and tsgolint was later SIGKILLed. I do not count that polluted run as product signal.

Changed files

• CHANGELOG.md (modified, +4/-0)
• docs/cli/channels.md (modified, +3/-0)
• docs/cli/configure.md (modified, +1/-0)
• docs/cli/gateway.md (modified, +1/-1)
• docs/cli/onboard.md (modified, +2/-0)
• docs/cli/plugins.md (modified, +6/-2)
• docs/docs.json (modified, +1/-0)
• docs/gateway/doctor.md (modified, +1/-1)
• docs/plugins/dependency-resolution.md (added, +214/-0)
• docs/tools/acp-agents.md (modified, +2/-1)
• docs/tools/plugin.md (modified, +6/-3)
• scripts/lib/bundled-runtime-deps-install.mjs (modified, +18/-3)
• scripts/postinstall-bundled-plugins.mjs (modified, +4/-146)
• scripts/release-check.ts (modified, +1/-1)
• src/channels/plugins/bundled.shape-guard.test.ts (modified, +2/-2)
• src/channels/plugins/read-only.test.ts (modified, +14/-14)
• src/channels/plugins/read-only.ts (modified, +3/-3)
• src/cli/command-bootstrap.test.ts (modified, +28/-3)
• src/cli/command-bootstrap.ts (modified, +12/-5)
• src/cli/command-catalog.ts (modified, +28/-4)
• src/cli/command-execution-startup.test.ts (modified, +7/-0)
• src/cli/command-execution-startup.ts (modified, +1/-0)
• src/cli/command-path-policy.test.ts (modified, +59/-68)
• src/cli/command-path-policy.ts (modified, +1/-0)
• src/cli/command-startup-policy.test.ts (modified, +2/-0)
• src/cli/command-startup-policy.ts (modified, +23/-7)
• src/cli/plugin-registry-loader.test.ts (modified, +11/-7)
• src/cli/plugin-registry-loader.ts (modified, +10/-7)
• src/cli/plugins-cli.list.test.ts (modified, +41/-3)
• src/cli/plugins-cli.ts (modified, +8/-456)
• src/cli/plugins-command-helpers.ts (modified, +8/-0)
• src/cli/plugins-deps-command.test.ts (modified, +47/-32)
• src/cli/plugins-deps-command.ts (modified, +40/-42)
• src/cli/plugins-inspect-command.ts (added, +361/-0)
• src/cli/plugins-list-command.ts (added, +114/-0)
• src/cli/program/preaction.test.ts (modified, +9/-3)
• src/commands/agents.providers.test.ts (modified, +1/-1)
• src/commands/agents.providers.ts (modified, +2/-2)
• src/commands/channels.list.auth-profiles.test.ts (modified, +2/-2)
• src/commands/channels.remove.test.ts (modified, +45/-20)
• src/commands/channels.resolve.test.ts (modified, +30/-24)
• src/commands/channels/capabilities.ts (modified, +1/-1)
• src/commands/channels/list.ts (modified, +1/-1)
• src/commands/channels/remove.ts (modified, +9/-2)
• src/commands/channels/resolve.ts (modified, +6/-1)
• src/commands/channels/status-config-format.ts (modified, +1/-1)
• src/commands/configure.wizard.test.ts (modified, +31/-1)
• src/commands/configure.wizard.ts (modified, +2/-0)
• src/commands/doctor-bundled-plugin-runtime-deps.test.ts (modified, +50/-18)
• src/commands/doctor-bundled-plugin-runtime-deps.ts (modified, +25/-24)
• src/commands/doctor-security.test.ts (modified, +2/-2)
• src/commands/doctor-security.ts (modified, +1/-1)
• src/commands/doctor/shared/channel-doctor.test.ts (modified, +1/-1)
• src/commands/doctor/shared/channel-doctor.ts (modified, +1/-1)
• src/commands/health.snapshot.test.ts (modified, +2/-2)
• src/commands/health.ts (modified, +2/-2)
• src/commands/onboard-non-interactive.gateway.test.ts (modified, +15/-0)
• src/commands/onboard-non-interactive/local.ts (modified, +2/-0)
• src/commands/post-config-runtime-deps.test.ts (added, +164/-0)
• src/commands/post-config-runtime-deps.ts (added, +133/-0)
• src/commands/status-all/channels.ts (modified, +3/-3)
• src/commands/status-runtime-shared.test.ts (modified, +1/-1)
• src/commands/status-runtime-shared.ts (modified, +1/-1)
• src/commands/status.link-channel.ts (modified, +1/-1)
• src/commands/status.scan-overview.test.ts (modified, +2/-2)
• src/commands/status.scan-overview.ts (modified, +1/-1)
• src/commands/status.scan.test.ts (modified, +3/-3)
• src/gateway/config-reload-plan.ts (modified, +29/-0)
• src/gateway/config-reload.test.ts (modified, +6/-0)
• src/gateway/config-reload.ts (modified, +1/-0)
• src/gateway/server-aux-handlers.test.ts (modified, +1/-0)
• src/gateway/server-plugin-bootstrap.ts (modified, +5/-1)
• src/gateway/server-plugins.ts (modified, +5/-1)
• src/gateway/server-reload-handlers.ts (modified, +52/-0)
• src/gateway/server-runtime-state.test.ts (added, +66/-0)
• src/gateway/server-runtime-state.ts (modified, +5/-2)
• src/gateway/server-startup-plugins.test.ts (modified, +130/-55)
• src/gateway/server-startup-plugins.ts (modified, +145/-89)
• src/gateway/server-startup-post-attach.test.ts (modified, +56/-0)
• src/gateway/server-startup-post-attach.ts (modified, +26/-6)
• src/gateway/server.impl.ts (modified, +66/-15)
• src/gateway/server.reload.test.ts (modified, +92/-0)
• src/gateway/server/readiness.test.ts (modified, +18/-0)
• src/gateway/server/readiness.ts (modified, +3/-1)
• src/infra/channel-summary.ts (modified, +1/-1)
• src/infra/npm-install-env.ts (modified, +4/-0)
• src/infra/safe-package-install.test.ts (modified, +10/-0)
• src/infra/safe-package-install.ts (modified, +5/-0)
• src/plugin-sdk/facade-loader.test.ts (modified, +1/-1)
• src/plugins/bundled-runtime-deps-activity.ts (modified, +1/-5)
• src/plugins/bundled-runtime-deps-install.ts (modified, +9/-47)
• src/plugins/bundled-runtime-deps-lock.ts (modified, +7/-0)
• src/plugins/bundled-runtime-deps-materialization.ts (modified, +76/-31)
• src/plugins/bundled-runtime-deps-package-manager.ts (modified, +7/-1)
• src/plugins/bundled-runtime-deps-roots.ts (modified, +66/-47)
• src/plugins/bundled-runtime-deps-selection.ts (modified, +5/-5)
• src/plugins/bundled-runtime-deps.test.ts (modified, +683/-69)
• src/plugins/bundled-runtime-deps.ts (modified, +249/-80)
• src/plugins/bundled-runtime-root.test.ts (modified, +109/-3)
• src/plugins/bundled-runtime-root.ts (modified, +141/-24)