PR #80479: feat(memory/embeddings): add openai-compatible provider for self-hosted servers (llama.cpp, Ollama, vLLM, TGI, LocalAI)

• Repository: openclaw/openclaw
• Author: yaanfpv
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/80479

Description (problem / solution / changelog)

Summary

• Problem: operators running a self-hosted OpenAI-compatible embeddings server (llama.cpp's llama-server, Ollama via its /v1 surface, vLLM, TGI, LocalAI, llamafile, or any reverse-proxied internal instance) have no clean adapter for it. Pointing the bundled lmstudio adapter at it triggers an LMStudio-only "load model" warmup that hangs against generic servers and stalls the gateway event loop for ~30 seconds per memory-lancedb embedding-provider rebuild. Pointing the bundled openai adapter at it works, but inherits global OpenAI headers/attribution/api-key resolution, and a removed embedding.baseUrl line silently falls back to api.openai.com which leaks embedded text to the cloud.
• Why it matters: the symptom is gateway freezes that show up as multi-second sessions.list backlogs and a flooded gateway log. Operators spend hours diagnosing what is actually a UX gap: the bundled adapters do not include a generic local-server option, and the existing in-process local adapter (node-llama-cpp on a .gguf file) does not cover operators who run their embeddings server as a separate HTTP process.
• What changes: adds a new bundled extension extensions/openai-compatible-embeddings/ that registers an openai-compatible memory embedding provider. The adapter has no warmup, no global config inheritance, fails-fast on missing embedding.baseUrl/embedding.model, and does not auto-select (operator must explicitly opt in with embedding.provider: "openai-compatible").
• What did NOT change (scope boundary): no existing adapter touched. lmstudio, openai, mistral, gemini, voyage, bedrock, deepinfra, ollama, in-process local adapters all behave byte-identically. The Plugin SDK surface is unchanged; the new adapter consumes the same public exports the other bundled adapters do. No protocol change, no schema change, no migration, no telemetry. The existing in-process local adapter stays as-is for operators who load .gguf files in-process via node-llama-cpp; the two adapters are complementary, not redundant.

Change Type

• Feature
• Docs

Scope

• Memory / storage
• Integrations

Linked Issue/PR

• Closes #80476
• Related #72875 (operator confusion: provider: "local" actually means in-process node-llama-cpp, not HTTP)
• Related #72937 (open PR fixing #72875's registration timing for the in-process local adapter)
• Related #66163 (closed: Unknown memory embedding provider: ollama, the precedent that led to the bundled ollama adapter; this PR follows the same pattern, generalized)
• Related #74204 (open: memory.qmd.update.embedTimeoutMs too low for local GGUF; same operator profile)
• Related #74761 (open: Document oMLX as a memorySearch embedding provider; oMLX exposes an OpenAI-compatible API and would just work through this adapter without further plugin code)
• Related #60994 (closed: cannot reliably connect to remote Ollama / LM Studio via LAN; adjacent operator pain in the same ecosystem)
• Related #42270 (closed: LM Studio backend regression; same lmstudio-adapter brittleness)
• This PR adds a new feature

Real behavior proof

• Behavior or issue addressed: an operator running llama-server (llama.cpp) with the BGE-M3 embedding model on http://localhost:8081/v1 had memory-lancedb captures triggering ~30-second event-loop stalls every time the embedding provider rebuilt, because the lmstudio adapter's ensureLmstudioModelLoaded warmup hangs against llama.cpp's OpenAI-compatible server (which does not expose LMStudio's load-model endpoint). The new openai-compatible adapter routes through the same generic createRemoteEmbeddingProvider factory the other adapters use, just without the warmup phase. Embeddings work end-to-end on the first call, no preload required.

• Real environment tested: macOS 26.4.1 on Apple Silicon (arm64). llama-server from llama.cpp serving bge-m3-Q8_0.gguf (605 MB, 1024 dimensions) on http://127.0.0.1:8081, with --ngl 24 -c 32768 -np 4 -b 512 -ub 512 --mmap --mlock --cont-batching --api-key <set>. Live ~/.openclaw/ with memory-lancedb enabled.

• Exact steps or command run after this patch: ran pnpm test extensions/openai-compatible-embeddings to validate the adapter posture and the no-warmup invariant. Then invoked the new factory directly through node --import tsx against the live llama-server, capturing the round-trip latency for both embedQuery and embedBatch. Independently verified the same llama-server endpoint with curl -H "Authorization: Bearer ..." http://localhost:8081/v1/embeddings returns 1024-dim vectors with the same model name.

• Evidence after fix:

  Live invocation of the new adapter from a small Node script (node --import tsx):

  $ node --import tsx /tmp/proof-openai-compatible-embeddings.mjs
  [proof] target  : http://localhost:8081/v1
  [proof] model   : text-embedding-bge-m3
  [proof] apiKey  : <set>
  [proof] factory : 1ms (no warmup, just client construction)
  [proof] client  : baseUrl=http://localhost:8081/v1 model=text-embedding-bge-m3
  [proof] embed   : 124ms, dims=1024, head=[-0.0392, 0.0370, -0.0289, 0.0161, ...]
  [proof] batch   : 25ms, count=4, dims=1024
  [proof] OK. openai-compatible embeddings adapter wired end-to-end against llama.cpp.

  Notice the factory took 1 ms (the lmstudio adapter would have taken up to 120 s here against the same server), and the actual embedding round-trip is 124 ms with the expected 1024-dim BGE-M3 output.

  Independent confirmation of the same endpoint via curl, showing the local server answers OpenAI-shaped requests without any vendor-specific preamble:

  $ curl -sS -m 5 -H "Authorization: Bearer ..." -H "Content-Type: application/json" \
      -d '{"model":"text-embedding-bge-m3","input":"hello"}' \
      -w "\nHTTP %{http_code} time=%{time_total}s\n" \
      http://localhost:8081/v1/embeddings | tail -c 200
  ...,0.05932944267988205],"index":0,"object":"embedding"}]}
  HTTP 200 time=0.069077s

  Targeted regression test for the adapter posture and the no-warmup invariant:

  $ pnpm test extensions/openai-compatible-embeddings
   Test Files  1 passed (1)
        Tests  4 passed (4)
     Duration  4.52s

• Observed result after fix: provider construction takes 1 ms (no warmup network call). The adapter holds the per-plugin baseUrl/model exactly as configured, with no fallback to any global config block. Embeddings round-trip in well under 200 ms against the live local server. Existing adapters (openai, lmstudio, mistral, gemini, voyage, bedrock, deepinfra, ollama, in-process local) are untouched.

• What was not tested: did not run the new adapter inside an actual openclaw gateway process end-to-end, because the dist bundle does not include the new extension yet (the source-only invocation above is the closest equivalent without a release-tagged build). Did not run pnpm check:changed in Testbox; targeted pnpm test extensions/openai-compatible-embeddings plus targeted npx oxlint extensions/openai-compatible-embeddings/ plus targeted pnpm tsgo:prod are all clean on the touched files.

• Before evidence: not applicable for a feature add. The "before" is "this provider did not exist," and the operator pain it addresses is documented in the Summary above and in linked issue #80476.

Root Cause

N/A. Feature addition, not a regression fix. (For the underlying operator pain that motivated the addition, see linked issue #80476.)

Regression Test Plan

• Coverage level that should have caught this:
  • Unit test
• Target test or file: extensions/openai-compatible-embeddings/memory-embedding-adapter.test.ts (new), with the factory in extensions/openai-compatible-embeddings/embedding-provider.ts.
• Scenario the test should lock in:
  • The adapter's posture: id: "openai-compatible", transport: "remote", no autoSelectPriority, no authProviderId, allowExplicitWhenConfiguredAuto: true, no shouldContinueAutoSelection. This is what stops the adapter from accidentally being auto-selected over an unrelated cloud provider whose key happens to be configured.
  • No warmup or preload during create. The adapter must produce exactly one factory invocation per create call; nothing else.
  • The cache key includes the per-plugin baseUrl and model exactly as supplied, so two different local servers do not share a cache entry.
  • The Authorization header is stripped from the cache key so a rotated bearer does not invalidate cached embeddings.
• Why this is the smallest reliable guardrail: the adapter is a thin facade over the generic createRemoteEmbeddingProvider. The risk surface is the posture (auto-select / auth / fallback) and the absence of any pre-call side effect. Both are testable in pure-TS with a mocked factory; no live server needed for the unit tests.
• Existing test that already covers this (if any): no. No bundled adapter today behaves the way openai-compatible needs to (no auth provider, no auto-select, fully self-contained config, no vendor-specific warmup).

User-visible / Behavior Changes

Operators who configure embedding.provider: "openai-compatible" plus embedding.baseUrl and embedding.model under plugins.entries.memory-lancedb.config.embedding get a working embeddings flow against any OpenAI-compatible local server. No behavior change for any operator who has not opted in. Existing lmstudio/openai/local/etc. adapters keep doing exactly what they do today.

Diagram

Before:
  memory-lancedb capture
    -> embedding provider rebuild
    -> lmstudio adapter create()
       -> ensureLmstudioModelLoaded(timeoutMs: 120_000)
          -> POST <local-server>/api/v0/load-model  (LMStudio-only)
          -> server replies 404 / hangs / returns unexpected shape
          -> ~30s event-loop stall before failure log
    -> /v1/embeddings call finally fires (works)

After (with embedding.provider: "openai-compatible"):
  memory-lancedb capture
    -> embedding provider rebuild
    -> openai-compatible adapter create()
       -> client construction only (~1ms)
    -> /v1/embeddings call fires immediately

Security Impact

• New permissions/capabilities? No
• Secrets/tokens handling changed? No. The optional apiKey is a per-plugin config field, treated identically to existing adapters' apiKey handling. Cache key strips the Authorization header.
• New/changed network calls? Only when explicitly configured by the operator. No call goes out at startup.
• Command/tool execution surface changed? No
• Data access scope changed? No. The adapter is fully self-contained and does not consult any global models.providers.* block, so it cannot leak embedded text to a cloud provider on a stale config.

Repro + Verification

Environment

• OS: macOS 26.4.1 (arm64)
• Runtime/container: Node 26.0.0
• Model/provider: BGE-M3 (Q8_0 GGUF) via llama.cpp llama-server on localhost:8081
• Integration/channel (if any): N/A (memory plugin)
• Relevant config (redacted): plugins.entries.memory-lancedb.config.embedding.provider: "openai-compatible", baseUrl: "http://localhost:8081/v1", model: "text-embedding-bge-m3", apiKey: "<bearer>", dimensions: 1024

Steps

1. Start any OpenAI-compatible local embedding server. For llama.cpp: llama-server -m <bge-m3.gguf> -a text-embedding-bge-m3 --embedding --host 127.0.0.1 --port 8081 --api-key <bearer>.
2. In ~/.openclaw/openclaw.json set memory-lancedb's embedding block to provider: "openai-compatible" plus baseUrl, model, optional apiKey/headers.
3. Restart the gateway. memory-lancedb captures and recalls now go through the local server with no warmup stall.

Expected

provider.embedQuery("hello") returns a 1024-dim vector in well under 200 ms. No event-loop stalls. No warmup warnings in the gateway log.

Actual

Matches expected. Verified end-to-end against llama.cpp serving BGE-M3 (terminal output included in Real behavior proof).

Evidence

• Failing test/log before + passing after (terminal output in Real behavior proof above; the "before" is "this provider did not exist")
• Trace/log snippets (proof script output, curl output, vitest output)

Human Verification

• Verified scenarios: ran the new factory against live llama.cpp serving BGE-M3 on macOS 26.4.1 / Apple Silicon. Confirmed embedQuery and embedBatch return correct-dimensionality vectors. Confirmed factory construction completes in 1 ms with no network call (vs lmstudio adapter's ~30s warmup hang against the same server). Verified the cache key contains the per-plugin baseUrl and model, with Authorization stripped. Verified pnpm test extensions/openai-compatible-embeddings (4/4 pass), npx oxlint extensions/openai-compatible-embeddings/ (0 errors), and pnpm tsgo:prod (clean on touched files).
• Edge cases checked: missing baseUrl throws a clear error rather than silently falling back. Missing model does the same. Adapter has no autoSelectPriority, so it cannot be picked automatically when the operator has another adapter's credentials configured. Headers passed through embedding.headers get attached to every request alongside the Authorization Bearer.
• What you did not verify: did not run the new adapter inside an actual openclaw gateway process end-to-end (dist bundle does not include the new extension yet). Did not run pnpm check:changed in Testbox.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? Yes. Pure addition.
• Config/env changes? No required changes. Operators who want the new provider opt in by setting embedding.provider: "openai-compatible" and baseUrl/model.
• Migration needed? No. Operators who currently work around the gap by pointing lmstudio or openai at a local server can switch when convenient. Their existing setup keeps working.

Risks and Mitigations

• Risk: an operator confuses the new HTTP-based openai-compatible provider with the existing in-process local provider.
  • Mitigation: the docs example calls out the distinction explicitly, listing the deployment shapes each one targets. The provider id openai-compatible reads as "any server that speaks the OpenAI HTTP API," which is the term llama.cpp / Ollama / vLLM / TGI / LocalAI all use to describe themselves; the existing local id keeps the semantic of "local in-process model file."
• Risk: an operator removes their embedding.baseUrl line by mistake while the openai-compatible provider is configured.
  • Mitigation: the adapter throws a clear error at create time pointing to the missing field. No fallback to a default URL.

Changed files

• CHANGELOG.md (modified, +1/-0)
• docs/plugins/memory-lancedb.md (modified, +47/-4)
• extensions/openai-compatible-embeddings/embedding-provider.ts (added, +112/-0)
• extensions/openai-compatible-embeddings/index.ts (added, +12/-0)
• extensions/openai-compatible-embeddings/memory-embedding-adapter.test.ts (added, +100/-0)
• extensions/openai-compatible-embeddings/memory-embedding-adapter.ts (added, +48/-0)
• extensions/openai-compatible-embeddings/openclaw.plugin.json (added, +15/-0)
• extensions/openai-compatible-embeddings/package.json (added, +15/-0)
• extensions/openai-compatible-embeddings/tsconfig.json (added, +16/-0)
• pnpm-lock.yaml (modified, +6/-0)
• test/vitest/vitest.extension-openai-compatible-embeddings.config.ts (added, +9/-0)