openclaw - 💡(How to fix) Fix [Bug]: bundled runtime deps install lands in $HOME/node_modules instead of install root (npm walkup), causing infinite re-staging [1 comments, 2 participants]

Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

installBundledRuntimeDeps runs npm install in the cache install root with no package.json present, so npm walks up to the nearest ancestor with a package.json (e.g. $HOME/package.json) and installs there instead. The cache install root's node_modules/ never gets populated, so hasDependencySentinel returns false on every subsequent invocation and re-runs the full bundled-deps install in a loop. With Node 25 + native deps like better-sqlite3 (no upstream prebuilds yet), each loop reinstalls the package without its compiled binding, breaking subprocesses such as qmd.

Steps to reproduce

1. npm install -g [email protected] on Node 25.5.0.
2. Have any package.json at $HOME (e.g. for an unrelated user-installed dep — in my case @tobilu/qmd).
3. Run openclaw (or restart the gateway) so the bundled-plugin runtime-deps stager runs.
4. Observe: the cache install root ~/.openclaw/plugin-runtime-deps/openclaw-<ver>-<hash>/ contains dist/ and .openclaw-runtime-deps.json but no node_modules/. The 67 install specs land in ~/node_modules/ instead.
5. Run openclaw again — full bundled-deps install fires again. Repeat indefinitely.

Minimal repro of the underlying npm behavior (no openclaw involved):

mkdir -p ~/.openclaw/walkup-test && cd ~/.openclaw/walkup-test
npm install --no-save --no-package-lock left-pad@^1
ls ~/.openclaw/walkup-test/node_modules  # missing
ls ~/node_modules/left-pad               # got installed here instead

This reproduces only when $HOME/package.json exists; in /tmp (no ancestor package.json) the install behaves correctly.

Expected behavior

hasDependencySentinel([installRoot], dep) returns true after a successful first install so subsequent invocations are no-ops. node_modules/ lives under the install root, not in $HOME.

Actual behavior

node_modules/ is created under the nearest ancestor of installExecutionRoot that has a package.json (typically $HOME). The sentinel check at the install root therefore always fails, and every CLI invocation / gateway boot retriggers the full 67-spec install.

Side effect on Node ≥25: better-sqlite3 doesn't ship prebuilds for v141; each install reinstalls the package without its .node binding, and qmd subprocesses (e.g. qmd update, qmd collection remove) crash with:

Error: Could not locate the bindings file. Tried:
  → /home/<user>/node_modules/better-sqlite3/build/Release/better_sqlite3.node
  ... (14 paths) ...
  at new Database (/home/<user>/node_modules/better-sqlite3/lib/database.js:48:64)
  at openDatabase (file:///home/<user>/node_modules/@tobilu/qmd/dist/db.js:27:12)
  at createStore (file:///home/<user>/node_modules/@tobilu/qmd/dist/store.js:632:16)
  at getStore (file:///home/<user>/node_modules/@tobilu/qmd/dist/qmd.js:23:17)
  at getDb  (file:///home/<user>/node_modules/@tobilu/qmd/dist/qmd.js:28:12)
  at collectionRemove (file:///home/<user>/node_modules/@tobilu/qmd/dist/qmd.js:1161:16)

This shows up in the gateway journal as [memory] qmd boot update failed / qmd collection remove failed and unhandled rejections.

OpenClaw version

2026.4.24

Operating system

Ubuntu 24.04 / Linux 6.17.0-20-generic, Node v25.5.0

Install method

npm install -g openclaw (global), gateway run via systemd --user service.

Model

Not relevant to this defect.

Provider / routing chain

Not relevant to this defect.

Logs, screenshots, and evidence

Cache install root after gateway boot, showing it stays bare:

$ ls -la ~/.openclaw/plugin-runtime-deps/openclaw-2026.4.24-5857d97fadfb/
drwxr-xr-x 3 user user   dist/
drwxrwxr-x 4 user user   .openclaw-npm-cache/
-rw-rw-r-- 1 user user   .openclaw-runtime-deps.json
# no node_modules/

cwd of the running npm install (parent: openclaw CLI):

$ readlink /proc/<npm_pid>/cwd
/home/<user>/.openclaw/plugin-runtime-deps/openclaw-2026.4.24-5857d97fadfb

…confirms the cwd is the install root, but the deps land in $HOME/node_modules because npm walks up to find ~/package.json.

Impact and severity

• Every CLI invocation and every gateway boot re-runs a ~5–10s npm install per enabled bundled plugin (4 plugins → 20–40s of churn).
• better-sqlite3 binding is destroyed on each loop on Node ≥25, breaking qmd (memory subsystem) until manually rebuilt.
• Disk thrash and confusing "staging bundled runtime deps" log spam.
• Worth noting #72315 reports a Windows-side symptom (popup shells every ~15s) that is consistent with the same loop.

Additional information

Root cause is in dist/.../bundled-runtime-deps-*.js, installBundledRuntimeDeps:

fs.mkdirSync(params.installRoot, { recursive: true });
fs.mkdirSync(installExecutionRoot, { recursive: true });
if (isolatedExecutionRoot) fs.writeFileSync(path.join(installExecutionRoot, "package.json"), `${JSON.stringify({
  name: "openclaw-runtime-deps-install",
  private: true
}, null, 2)}\n`, "utf8");

The stub package.json is only written when isolatedExecutionRoot === true. For the bundled-plugin packaged-install path, installRoot === installExecutionRoot, so the guard is false and no package.json is written. npm therefore walks up.

Suggested fix: drop the if (isolatedExecutionRoot) guard and always write the stub package.json at installExecutionRoot before spawnSync — it costs nothing and prevents npm from escaping the install root. (Idempotent rewrite is fine; the file is only there to anchor npm.)

Workaround verified locally: writing {"private":true,"name":"openclaw-runtime-deps"} at the cache install root is sufficient. After applying it:

• node_modules/ populates under the install root as intended.
• Subsequent openclaw invocations skip staging (sentinel check passes).
• ~/node_modules/better-sqlite3 is no longer reinstalled, so a one-time npm rebuild better-sqlite3 sticks.
• Gateway boot drops from ~120s to ~30s.