openclaw - 💡(How to fix) Fix LCM semantic search: Gemini Embedding 001 integration [1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#57319Fetched 2026-04-08 01:51:05
View on GitHub
Comments
0
Participants
1
Timeline
0
Reactions
0
Author
Participants

Implement semantic vector search over the LCM summary DAG using Google's Gemini Embedding 001 (3072-dim, Matryoshka, $0.15/M tokens).

Spec: docs/specs/lcm-semantic-search-gemini-embeddings.md

Root Cause

Implement semantic vector search over the LCM summary DAG using Google's Gemini Embedding 001 (3072-dim, Matryoshka, $0.15/M tokens).

Spec: docs/specs/lcm-semantic-search-gemini-embeddings.md

RAW_BUFFERClick to expand / collapse

Summary

Implement semantic vector search over the LCM summary DAG using Google's Gemini Embedding 001 (3072-dim, Matryoshka, $0.15/M tokens).

Spec: docs/specs/lcm-semantic-search-gemini-embeddings.md

Phases

  1. Schema + backfillsummary_vectors table, embed 852 existing summaries (~$0.08)
  2. Embed at compaction — hook into compaction.ts, fire-and-forget after summary creation
  3. Hybrid search — add semantic and hybrid modes to lcm_grep, upgrade lcm_expand_query to use hybrid by default (RRF fusion of vector + FTS5)
  4. Polish + testing — doctor check, embedding stats, model migration handling

Key Numbers

  • 852 summaries today, ~150/day growth
  • $0.33/month ongoing cost
  • 5-9 hours estimated implementation
  • Pure TypeScript cosine similarity (no sqlite-vec needed at current scale)

Future

When corpus exceeds ~50K entries, evaluate TurboQuant/PolarQuant vector compression (ICLR 2026) before reaching for external vector DBs. Reference added to spec.

extent analysis

Fix Plan

To implement semantic vector search, we will focus on the following steps:

  • Create a summary_vectors table to store the embedded vectors
  • Integrate Google's Gemini Embedding into the compaction process
  • Develop a hybrid search function that combines vector search and FTS5

Code Changes

We will use the following code snippets as examples:

// Import necessary libraries
import { GeminiEmbedding } from '@google/gemini-embedding';

// Create a function to embed a summary using Gemini Embedding
async function embedSummary(summary: string): Promise<number[]> {
  const embedding = new GeminiEmbedding();
  const vector = await embedding.embed(summary);
  return vector;
}

// Hook into compaction.ts to embed summaries after creation
async function compactAndEmbed(summary: string): Promise<void> {
  // Compact the summary
  // ...
  // Embed the summary
  const vector = await embedSummary(summary);
  // Store the vector in the summary_vectors table
  // ...
}

// Develop a hybrid search function that combines vector search and FTS5
async function hybridSearch(query: string): Promise<string[]> {
  // Perform FTS5 search
  const fts5Results = await fts5Search(query);
  // Perform vector search
  const vectorResults = await vectorSearch(query);
  // Combine the results using RRF fusion
  const results = combineResults(fts5Results, vectorResults);
  return results;
}

// Combine the results using RRF fusion
function combineResults(fts5Results: string[], vectorResults: string[]): string[] {
  // Implement RRF fusion logic here
  // ...
  return combinedResults;
}

Verification

To verify that the fix worked, we can test the hybrid search function with sample queries and verify that the results are accurate and relevant.

Extra Tips

  • Monitor the performance and cost of the implementation to ensure it meets the estimated costs and implementation time.
  • Evaluate the need for vector compression techniques like TurboQuant/PolarQuant when the corpus exceeds ~50K entries.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - 💡(How to fix) Fix LCM semantic search: Gemini Embedding 001 integration [1 participants]