openclaw - ✅(Solved) Fix [Bug]: corpus=sessions memory_search returns 0 results for many keywords despite data being indexed [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#74036Fetched 2026-04-30 06:29:33
View on GitHub
Comments
1
Participants
2
Timeline
8
Reactions
0
Timeline (top)
cross-referenced ×2closed ×1commented ×1mentioned ×1

When using memory_search with corpus: "sessions" on the builtin memory backend, many search queries return 0 results even though:

  • The keywords exist in the indexed session data (verified via direct SQL)
  • The same keywords return results with corpus: "memory" (memory files work fine)
  • FTS5 direct SQL queries return correct results from Python

Error Message

.catch(() => []) // ← Error silently swallowed

  • The silent error masking makes diagnosis extremely difficult
  1. Remove silent .catch(() => []) in searchKeyword — log the actual error so users can diagnose

Root Cause

The searchKeyword function (in dist/manager-*.js) calls FTS5 MATCH queries but errors are silently swallowed by .catch(() => []):

// Line ~2829
const keywordResults = hybrid.enabled && this.fts.enabled && this.fts.available 
    ? await this.searchKeyword(cleaned, candidates, void 0, sourceFilterList)
        .catch(() => [])  // ← Error silently swallowed
    : [];

Direct SQL confirmation: All failing keywords return correct results when queried directly against chunks_fts. The FTS index is correct and complete. The problem is in the runtime call path.

Fix Action

Workaround

Currently there is no reliable workaround. Users must either:

  • Avoid corpus: "sessions" entirely (lose ability to search past conversations)
  • Use corpus: "memory" only (only searches MEMORY.md and memory/*.md files)

PR fix notes

PR #74175: fix(memory): add LIKE fallback when FTS5 MATCH throws and log silent search errors

Description (problem / solution / changelog)

Summary

  • Adds a LIKE-based fallback in searchKeyword() when FTS5 MATCH throws (e.g. unicode61 tokenizer rejects certain query patterns), so searches return results instead of silently yielding 0 hits
  • Replaces 4 silent .catch(() => []) patterns in the search orchestrator (manager.ts) with .catch((err) => { log.warn(...); return []; }) so failures are visible in diagnostics
  • Adds 3 new tests for the LIKE fallback path, BM25 scoring path, and source filter in fallback

Problem

memory_search with corpus: "sessions" returns 0 results for ~50% of keywords. The root cause is that FTS5 MATCH queries fail silently when the unicode61 tokenizer rejects certain query patterns (special characters, CJK edge cases, etc.), and the errors are swallowed by .catch(() => []).

Changes

extensions/memory-core/src/memory/manager-search.ts

  • searchKeyword() now wraps the FTS5 MATCH query in a try/catch
  • On MATCH failure, falls back to SELECT ... WHERE text LIKE ? with the same source/model filters
  • Fallback results get textScore=1 (signals no BM25 ranking available)
  • The snippet column uses a simple substr() instead of FTS5's snippet() function

extensions/memory-core/src/memory/manager.ts

  • 4 .catch(() => []) sites now log log.warn(...) with the error before returning empty arrays
  • Covers: FTS keyword search, trigram keyword search, vector search, and embedding-augmented keyword search

extensions/memory-core/src/memory/manager-search.test.ts

  • 3 new tests behind supportsFts() guard (skipped when FTS5 unavailable):
    • Falls back to LIKE search when FTS MATCH throws
    • Returns BM25-scored results when FTS MATCH succeeds
    • Applies source filter in LIKE fallback

Fixes #74036

Changed files

  • extensions/memory-core/src/memory/manager-search.test.ts (modified, +241/-0)
  • extensions/memory-core/src/memory/manager-search.ts (modified, +68/-22)
  • extensions/memory-core/src/memory/manager.ts (modified, +22/-4)

Code Example

// Line ~2829
const keywordResults = hybrid.enabled && this.fts.enabled && this.fts.available 
    ? await this.searchKeyword(cleaned, candidates, void 0, sourceFilterList)
        .catch(() => [])  // ← Error silently swallowed
    : [];
RAW_BUFFERClick to expand / collapse

Summary

When using memory_search with corpus: "sessions" on the builtin memory backend, many search queries return 0 results even though:

  • The keywords exist in the indexed session data (verified via direct SQL)
  • The same keywords return results with corpus: "memory" (memory files work fine)
  • FTS5 direct SQL queries return correct results from Python

Steps to reproduce

  1. Have session transcripts indexed (builtin backend, memorySearch.experimental.sessionMemory: true)
  2. Run memory_search with corpus: "sessions" for various keywords
  3. Compare results between corpus: "sessions" and corpus: "memory"

Expected behavior

Keywords present in session transcripts should return matching chunks.

Actual behavior

50% failure rate across tested keywords. Some words work, others return 0 results.

Querycorpus=sessionsFTS SQL verified
大语言模型✅ 3 results✅ present
知识库✅ 2 results✅ present
RAG✅ 2 results✅ present
向量数据库✅ 2 results✅ present
Transformer✅ 3 results✅ present
注意力机制✅ 2 results✅ present
Dify❌ 0 results✅ present
OpenClaw❌ 0 results✅ present
数据库❌ 0 results✅ present
部署❌ 0 results✅ present
Agent❌ 0 results✅ present
cron❌ 0 results✅ present
汇报❌ 0 results✅ present
PPT❌ 0 results✅ present
插件❌ 0 results✅ present
API❌ 0 results✅ present

Root Cause Analysis

The searchKeyword function (in dist/manager-*.js) calls FTS5 MATCH queries but errors are silently swallowed by .catch(() => []):

// Line ~2829
const keywordResults = hybrid.enabled && this.fts.enabled && this.fts.available 
    ? await this.searchKeyword(cleaned, candidates, void 0, sourceFilterList)
        .catch(() => [])  // ← Error silently swallowed
    : [];

Direct SQL confirmation: All failing keywords return correct results when queried directly against chunks_fts. The FTS index is correct and complete. The problem is in the runtime call path.

Pattern observed

Working keywords: Chinese compound words (大语言模型, 向量数据库, 注意力机制) Failing keywords: Short English words (Dify, cron, PPT, API, Agent), common Chinese terms (数据库, 部署, 汇报, 插件)

This suggests a potential issue with how Node.js node:sqlite FTS5 trigram tokenizer handles certain character patterns, which works correctly in Python sqlite3 but fails in the Node.js runtime.

Environment

  • OpenClaw version: 2026.4.26
  • Operating system: Ubuntu 24.04 (Linux 6.8.0-110-generic, x64)
  • Install method: npm global
  • Memory backend: builtin (SQLite)
  • Embedding provider: openai-compatible (SiliconFlow)
  • Embedding model: BAAI/bge-m3 (1024 dims)
  • Session chunks indexed: 1312 (source=sessions)
  • Memory chunks indexed: 838 (source=memory)
  • FTS: ready
  • Vector: ready (sqlite-vec)

Impact

  • High severity — 50% of session search queries return empty results
  • Affects all users with corpus: "sessions" or memorySearch.sources: ["sessions"]
  • Session transcripts are effectively partially invisible to memory recall
  • The silent error masking makes diagnosis extremely difficult

Suggested fixes

  1. Remove silent .catch(() => []) in searchKeyword — log the actual error so users can diagnose
  2. Test Node.js sqlite FTS5 trigram tokenizer against CJK + mixed-language queries
  3. Add fallback: when FTS MATCH query fails, fall back to LIKE-based substring search (as the code already partially supports via substringTerms)
  4. Add diagnostic: openclaw memory status --deep should verify FTS queries work, not just report "FTS: ready"

Workaround

Currently there is no reliable workaround. Users must either:

  • Avoid corpus: "sessions" entirely (lose ability to search past conversations)
  • Use corpus: "memory" only (only searches MEMORY.md and memory/*.md files)

extent analysis

TL;DR

Remove the silent .catch(() => []) in the searchKeyword function to log actual errors and diagnose the issue.

Guidance

  • Identify and log errors in the searchKeyword function to understand why FTS MATCH queries are failing for certain keywords.
  • Test the Node.js sqlite FTS5 trigram tokenizer with CJK and mixed-language queries to reproduce the issue.
  • Consider adding a fallback to LIKE-based substring search when FTS MATCH queries fail.
  • Enhance diagnostic tools, such as openclaw memory status --deep, to verify FTS queries work as expected.

Example

No code snippet is provided as the issue is more related to understanding and debugging the existing code rather than introducing new code.

Notes

The issue seems to be related to how the Node.js node:sqlite FTS5 trigram tokenizer handles certain character patterns, which may not be immediately solvable without further investigation into the tokenizer's behavior.

Recommendation

Apply the suggested fix of removing the silent .catch(() => []) to log actual errors and diagnose the issue, as this will provide more insight into why the FTS MATCH queries are failing for certain keywords.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Keywords present in session transcripts should return matching chunks.

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - ✅(Solved) Fix [Bug]: corpus=sessions memory_search returns 0 results for many keywords despite data being indexed [1 pull requests, 1 comments, 2 participants]