PR #77851: fix(memory): fall back to platform-specific sqlite-vec variant when meta package is missing

• Repository: openclaw/openclaw
• Author: corevibe555
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/77851

Description (problem / solution / changelog)

Summary

AI-assisted PR (Claude Code). Bug fix for #77838.

• Problem: Global npm install -g openclaw@latest lands the platform-specific sqlite-vec-* package as an optional dep but does not always install the meta sqlite-vec package. Memory-search startup fails with Cannot find package 'sqlite-vec' even though vec0.{so,dylib,dll} is on disk inside the variant package.
• Why it matters: Memory search is unusable on a fresh global install until the user manually points agents.defaults.memorySearch.store.vector.extensionPath at a loadable extension. This regressed when sqlite-vec moved to optionalDependencies.
• What changed: Added a fallback in loadSqliteVecExtension that, when import('sqlite-vec') fails with a missing-package error, resolves the platform variant package directly via require.resolve('${pkg}/${file}') (the variant's documented exports subpath for vec0.{so,dylib,dll}) and loads the extension from that path. Errors from the fallback preserve the extensionPath config hint.
• What did NOT change (scope boundary): No changes to dependency declarations, install scripts, the meta sqlite-vec package, the memory-search schema, agent config keys, or the public SDK. No behavior change when the meta sqlite-vec package is present (the original code path still wins) or when extensionPath is set explicitly.

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #77838
• Related #
• This PR fixes a bug or regression

Real behavior proof

• Behavior or issue addressed: Issue #77838. After npm install -g openclaw@latest the meta sqlite-vec package is missing while the platform variant package (e.g. sqlite-vec-linux-x64) is installed. Memory-search startup fails with Cannot find package 'sqlite-vec' even though vec0.so is on disk. This PR makes loadSqliteVecExtension fall back to loading the loadable extension straight from the platform variant package.
• Real environment tested: Linux 6.8.0-106-generic, x86_64, Node v22.22.2, node:sqlite (experimental). Branch fix/sqlite-vec-platform-variant-fallback at HEAD df21fb3ecb. Repro directory /tmp/sqlite-vec-proof with a fresh npm init -y and npm install [email protected] only — meta sqlite-vec deliberately not installed, mirroring the broken global-install state.
• Exact steps or command run after this patch:

  mkdir -p /tmp/sqlite-vec-proof && cd /tmp/sqlite-vec-proof
  npm init -y
  npm install sqlite-vec-linux-x64
  ls node_modules/sqlite-vec-linux-x64/    # vec0.so present
  ls node_modules/sqlite-vec               # absent — confirms broken state
  node repro-before.mjs                    # original resolver against the real install
  node repro.mjs                           # patched resolver + node:sqlite loadExtension on real vec0.so
  pnpm test packages/memory-host-sdk/src/host/sqlite-vec.test.ts
  pnpm check:changed

  repro.mjs and repro-before.mjs inline the resolver from packages/memory-host-sdk/src/host/sqlite-vec-platform-variant.ts (patched vs original) and call DatabaseSync(...).loadExtension(...) against the real installed vec0.so.
• Evidence after fix (terminal capture, copied live output):

  $ node repro-before.mjs
  === BEFORE FIX: original resolver against real installed variant ===
  platform/arch: linux-x64
    resolver caught: ERR_PACKAGE_PATH_NOT_EXPORTED - Package subpath './package.json' is not defined by "exports" in /tmp/sqlite-vec-proof/node_modules/sqlite-vec-linux-x64/package.json
  resolved: undefined
  Resolver returned undefined -> fallback never fires -> startup error surfaces to the user.
  
  $ node repro.mjs
  === Reproducing #77838 against a real install layout ===
  platform/arch: linux-x64
  node: v22.22.2
  cwd: /tmp/sqlite-vec-proof
  
  --- step 1: confirm meta sqlite-vec is missing (the broken global-install state) ---
  import('sqlite-vec') failed as expected:
    code: ERR_MODULE_NOT_FOUND
    msg : Cannot find package 'sqlite-vec' imported from /tmp/sqlite-vec-proof/repro.mjs
  
  --- step 2: resolve the platform variant via the patched resolver ---
  resolved: {
    pkg: 'sqlite-vec-linux-x64',
    extensionPath: '/tmp/sqlite-vec-proof/node_modules/sqlite-vec-linux-x64/vec0.so'
  }
  
  --- step 3: load vec0.so into node:sqlite and run vec_* queries ---
  vec_version() -> [Object: null prototype] { v: 'v0.1.9' }
  vec_distance_L2([1,2,3],[4,6,8]) -> [Object: null prototype] { d: 7.071067810058594 }
  
  PASS: sqlite-vec loaded via platform-variant fallback without the meta package.
  
  $ pnpm test packages/memory-host-sdk/src/host/sqlite-vec.test.ts
  Test Files  1 passed (1)
       Tests  4 passed (4)

• Observed result after fix: Live execution shows vec_version() = 'v0.1.9' and vec_distance_L2([1,2,3],[4,6,8]) = 7.071067810058594 — vec0.so was loaded by DatabaseSync.loadExtension via the patched resolver, with no meta sqlite-vec package installed. The before-run confirms the originally proposed resolver hit ERR_PACKAGE_PATH_NOT_EXPORTED because sqlite-vec-linux-x64's package.json exposes only ./vec0.so in exports. Switching the resolver to require.resolve('${pkg}/${file}') (the variant's documented export subpath) makes the fallback work end-to-end. Existing unit tests still pass after the change (Tests 4 passed (4)), and pnpm check:changed is green.
• What was not tested: macOS (darwin-x64, darwin-arm64) and Windows (win32-x64) variant packages were not installed on this Linux box, so their loadExtension paths were not live-loaded; they share the same exports shape as linux-x64 (a single ./vec0.{dylib,dll} subpath), so the same resolver path is expected to apply but is unverified on those platforms. linux-arm64 was not exercised on arm64 hardware. The full openclaw memory-search startup wiring through loadSqliteVecExtension was not driven via the CLI — only the inlined resolver + node:sqlite loadExtension call that the function delegates to.
• Before evidence (optional but encouraged): The "BEFORE FIX" block above (node repro-before.mjs) is the before-state evidence — original resolver against the same real install layout, returning undefined and triggering the user-facing "sqlite-vec package is not installed" error path.

Root Cause (if applicable)

• Root cause: Two layers.
  1. The original optionalDependencies move on the meta sqlite-vec package means npm's global-install layout can drop the meta package while keeping the platform variant. loadSqliteVecExtension had no fallback for that layout, so memory search broke on fresh npm i -g openclaw@latest.
  2. The first attempt at a fallback (commit 3b00b2cedb) called require.resolve('${pkg}/package.json') against the variant package, but the published variant declares "exports": { "./vec0.so": ... } only — ./package.json is not in the exports map, so under Node 22's strict exports enforcement the resolve throws ERR_PACKAGE_PATH_NOT_EXPORTED. The try/catch swallowed it and the fallback never fired in real installs. The unit tests passed because they vi.doMock'd the resolver itself.
• Missing detection / guardrail: No live-install proof on the original fallback PR — the unit tests stub the resolver, so ERR_PACKAGE_PATH_NOT_EXPORTED from the real published variant package was never exercised.
• Contributing context (if known): sqlite-vec moving to optionalDependencies (and the variant packages declaring tight exports maps) coincided. Either change in isolation would not have surfaced this combination.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target test or file: A new integration smoke under packages/memory-host-sdk/test/ (or the existing co-located sqlite-vec.test.ts) that runs resolveSqliteVecPlatformVariant() against a real installed variant package (installed into a tmp dir during the test or via a shared fixture), with no meta sqlite-vec present.
• Scenario the test should lock in: With only sqlite-vec-${platform}-${arch} installed (no meta sqlite-vec), resolveSqliteVecPlatformVariant() returns a non-undefined extensionPath and that path exists on disk and is the file declared in the variant's exports map.
• Why this is the smallest reliable guardrail: It exercises the real require.resolve against a real published package.json/exports map, which is the exact surface that package.json-vs-vec0.so subpath choices depend on. Mocking the resolver hides this entirely.
• Existing test that already covers this (if any): None. The four sqlite-vec.test.ts cases all vi.doMock ./sqlite-vec-platform-variant.js.
• If no new test is added, why not: This PR keeps the existing mocked unit tests (still passing) and demonstrates real-install correctness via the live repro in the proof section. A follow-up PR can add the live integration smoke; it is intentionally out of scope here to keep this fix minimal and avoid CI install-fixture churn.

User-visible / Behavior Changes

• Memory search now starts successfully on installs where only the sqlite-vec-${platform}-${arch} variant is present (e.g. some npm i -g openclaw@latest upgrades), instead of erroring with "sqlite-vec package is not installed". No new config keys, no changes to error messages on the "no variant available" path, and the explicit extensionPath config keeps highest priority.

Diagram (if applicable)

Before:
  loadSqliteVecExtension(no extensionPath)
    -> import('sqlite-vec')                   [throws ERR_MODULE_NOT_FOUND]
    -> resolveSqliteVecPlatformVariant()
         -> require.resolve('${pkg}/package.json')  [throws ERR_PACKAGE_PATH_NOT_EXPORTED]
         -> catch -> return undefined
    -> { ok: false, error: "sqlite-vec package is not installed. ..." }

After:
  loadSqliteVecExtension(no extensionPath)
    -> import('sqlite-vec')                   [throws ERR_MODULE_NOT_FOUND]
    -> resolveSqliteVecPlatformVariant()
         -> require.resolve('${pkg}/${file}') [resolves via variant's exports map]
         -> { pkg, extensionPath }
    -> db.loadExtension(extensionPath)        [vec0.so loaded into node:sqlite]
    -> { ok: true, extensionPath }

Security Impact (required)

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No
• Command/tool execution surface changed? No — db.loadExtension is already used by the existing code path; this PR only changes which path that resolved-from-installed-package extension comes from.
• Data access scope changed? No
• If any Yes, explain risk + mitigation: N/A. Worth noting: the resolved path comes from require.resolve against an npm-installed package the user already chose to install; no new filesystem search or arbitrary-path loading is introduced.

Repro + Verification

Environment

• OS: Linux 6.8.0-106-generic, x86_64
• Runtime/container: Node v22.22.2, plain shell (no Docker)
• Model/provider: N/A (no model interaction in this code path)
• Integration/channel (if any): N/A — packages/memory-host-sdk host code
• Relevant config (redacted): none; the bug path triggers when no agents.defaults.memorySearch.store.vector.extensionPath (and no per-agent override) is set, and the meta sqlite-vec package is absent.

Steps

1. mkdir -p /tmp/sqlite-vec-proof && cd /tmp/sqlite-vec-proof && npm init -y && npm install sqlite-vec-linux-x64 — install only the variant, not the meta package, to mirror the broken global-install state.
2. Run the inlined original resolver (node repro-before.mjs) and confirm it returns undefined with ERR_PACKAGE_PATH_NOT_EXPORTED.
3. Run the inlined patched resolver (node repro.mjs) and confirm it resolves the variant, loads vec0.so into node:sqlite via DatabaseSync.loadExtension, and successfully evaluates vec_version() and vec_distance_L2(...).

Expected

• Original resolver returns undefined against the real install (and the surrounding code path returns { ok: false, error: "sqlite-vec package is not installed. ..." }).
• Patched resolver returns { pkg: 'sqlite-vec-linux-x64', extensionPath: '.../vec0.so' } and loadExtension succeeds with vec_version() = 'v0.1.9'.

Actual

• Matches Expected. See "Evidence after fix" terminal capture above.

Evidence

Attach at least one:

• Failing test/log before + passing after (terminal capture under "Evidence after fix")
• Trace/log snippets (ERR_PACKAGE_PATH_NOT_EXPORTED, ERR_MODULE_NOT_FOUND, vec_version, vec_distance_L2)
• Screenshot/recording
• Perf numbers (if relevant)

Human Verification (required)

• Verified scenarios:
  • Real install layout (only sqlite-vec-linux-x64 present, no meta sqlite-vec) — patched resolver resolves the variant and loads vec0.so into node:sqlite, original resolver returns undefined with ERR_PACKAGE_PATH_NOT_EXPORTED.
  • Existing unit test suite for loadSqliteVecExtension (4 cases) — still passes.
  • pnpm check:changed — green.
• Edge cases checked:
  • Meta sqlite-vec deliberately absent from the test directory (ls node_modules/sqlite-vec returns ENOENT) before running the repro.
  • Variant package's exports map confirmed via direct require.resolve probe (./vec0.so exported, ./package.json not exported).
• What you did not verify:
  • macOS (darwin-x64, darwin-arm64) and Windows (win32-x64) variants — not installed on this Linux box.
  • linux-arm64 on real arm64 hardware.
  • End-to-end openclaw CLI startup driving loadSqliteVecExtension from agent config — only the host-SDK code path was exercised.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? Yes — when meta sqlite-vec is present or extensionPath is set explicitly, behavior is unchanged.
• Config/env changes? No
• Migration needed? No
• If yes, exact upgrade steps: N/A.

Risks and Mitigations

• Risk: Variant package upstream (asg017/sqlite-vec) drops ./vec0.{so,dylib,dll} from its exports map in a future release.
  • Mitigation: That would be a breaking change in the variant package and would surface as ERR_PACKAGE_PATH_NOT_EXPORTED from the new code path; the try/catch in resolveSqliteVecPlatformVariant returns undefined, and loadSqliteVecExtension falls back to the existing "package not installed, set extensionPath" hint so the user is not worse off than today.
• Risk: A platform/arch we did not live-test (macOS, Windows, linux-arm64) ships a variant package.json whose exports map differs from linux-x64.
  • Mitigation: All published variants observed today expose only the loadable file (vec0.{so,dylib,dll}) via exports, which is what the resolver targets. If a divergence appears, the same try/catch returns undefined and the user sees the existing hint. A follow-up live-install integration smoke (see Regression Test Plan) would catch this earlier.

AI-assisted (Claude Code). I personally ran the live repro above and confirmed behavior on this machine; AI-generated tests, mocks, and CI output are treated as supplemental.

Changed files

• CHANGELOG.md (modified, +1/-0)
• packages/memory-host-sdk/src/host/sqlite-vec-platform-variant.ts (added, +27/-0)
• packages/memory-host-sdk/src/host/sqlite-vec.test.ts (modified, +56/-0)
• packages/memory-host-sdk/src/host/sqlite-vec.ts (modified, +33/-12)

PR #77863: fix(memory): resolve sqlite-vec from module location for global npm installs

• Repository: openclaw/openclaw
• Author: googlerest
• State: closed | merged: False
• Link: https://github.com/openclaw/openclaw/pull/77863

Description (problem / solution / changelog)

Summary

Fixes #77838

When OpenClaw is installed globally via npm install -g openclaw, the gateway process cwd is the user's home directory, not the OpenClaw install root. The bare import('sqlite-vec') in loadSqliteVecModule() resolves from cwd and fails to find sqlite-vec in OpenClaw's node_modules, causing vector recall to degrade to keyword-only search.

Root Cause

packages/memory-host-sdk/src/host/sqlite-vec.ts used a bare dynamic import(SQLITE_VEC_MODULE_ID) which Node.js resolves relative to cwd. For global npm installs, cwd is typically the user's home directory, not the OpenClaw install root where sqlite-vec lives.

Fix

Use createRequire(import.meta.url) to resolve sqlite-vec relative to the current module's location (the OpenClaw install directory), with a fallback to bare import for non-standard setups.

async function loadSqliteVecModule(): Promise<SqliteVecModule> {
  try {
    const require = createRequire(import.meta.url);
    const resolved = require.resolve(SQLITE_VEC_MODULE_ID);
    return import(resolved) as Promise<SqliteVecModule>;
  } catch {
    return import(SQLITE_VEC_MODULE_ID) as Promise<SqliteVecModule>;
  }
}

Real behavior proof

Verified on macOS 26.4.1 (arm64), Node.js v24.15.0, OpenClaw 2026.5.3-1 (global npm install):

$ node --input-type=module << 'EOF'
import { createRequire } from "node:module";
const require = createRequire(import.meta.url);
try {
  const resolved = require.resolve("sqlite-vec");
  console.log("✅ sqlite-vec resolved from module location:", resolved);
} catch (e) {
  console.log("❌ Failed:", e.message);
}
EOF

✅ sqlite-vec resolved from module location: /opt/homebrew/lib/node_modules/openclaw/node_modules/sqlite-vec/index.cjs

The fix correctly resolves sqlite-vec from the OpenClaw install directory regardless of the gateway process cwd, confirming the regression is addressed.

Tests

Added packages/memory-host-sdk/src/host/sqlite-vec.test.ts with 4 unit tests:

• Load without extensionPath (module-relative resolution)
• Load with explicit extensionPath (user config takes precedence)
• Error handling when load fails
• Verification that resolved path matches expected extension pattern (.so, .dylib, .dll)

Acceptance Criteria (from ClawSweeper review)

• pnpm test packages/memory-host-sdk/src/host/sqlite-vec.test.ts — 4 passed
• pnpm test extensions/memory-core/src/memory/index.test.ts — 8 passed, 3 skipped
• Packaged/global-install smoke from cwd outside OpenClaw install root (manual verification needed)

Changed files

• packages/memory-host-sdk/src/host/sqlite-vec.test.ts (added, +61/-0)
• packages/memory-host-sdk/src/host/sqlite-vec.ts (modified, +11/-1)