openclaw - 💡(How to fix) Fix [Feature Request] Multi-Model Support for Memory Search Embeddings with Fallback [1 comments, 2 participants]

Problem Statement

Currently, memorySearch only supports a single embedding model configuration without fallback options:

"memorySearch": {
  "provider": "openai",
  "model": "BAAI/bge-m3",
  "remote": {
    "apiKey": "sk-xxx",
    "baseUrl": "https://api.siliconflow.cn/v1"
  }
}

Unlike the main LLM model which supports primary + fallbacks configuration, memory search lacks:

• Fallback models when primary embedding service fails
• Runtime model switching based on query complexity
• Per-agent embedding model customization

Use Cases

1. Cost Optimization

Use lightweight model (e.g., BAAI/bge-m3) for routine queries, fallback to powerful model (e.g., Qwen/Qwen2.5-7B-Instruct) for complex semantic understanding.

2. Multi-Agent Scenarios

Different agents need different embedding capabilities:

• General agent: Short context (bge-m3, 8K)
• Research agent: Long documents (Qwen2.5, 32K)
• Multimodal agent: Image+text (Qwen2-VL)

3. Provider Redundancy

Switch to backup provider when primary embedding service is down or rate-limited.

Proposed Solutions

Option A: Primary + Fallbacks (Preferred)

Similar to main model configuration:

"memorySearch": {
  "primary": {
    "provider": "openai",
    "model": "BAAI/bge-m3",
    "remote": { "apiKey": "${KEY1}", "baseUrl": "..." }
  },
  "fallbacks": [
    {
      "provider": "openai", 
      "model": "Qwen/Qwen2.5-7B-Instruct",
      "remote": { "apiKey": "${KEY2}", "baseUrl": "..." }
    }
  ]
}

Option B: Per-Agent Configuration

"agents": {
  "list": [
    {
      "id": "main",
      "memorySearch": {
        "model": "BAAI/bge-m3"
      }
    },
    {
      "id": "research",
      "memorySearch": {
        "model": "Qwen/Qwen2.5-7B-Instruct"
      }
    }
  ]
}

Option C: Dynamic Model Selection (Advanced)

"memorySearch": {
  "strategy": "adaptive",
  "models": [
    { "name": "bge-m3", "threshold": 4096 },
    { "name": "Qwen2.5", "threshold": 32768 }
  ]
}

Current Workaround

Manually edit openclaw.json and restart Gateway to switch models.

Environment

• OpenClaw Version: 2026.03.24
• Current Config: Using Siliconflow BAAI/bge-m3 via OpenAI-compatible API

Additional Context

With providers like Siliconflow offering multiple embedding models (bge-m3, Qwen series, etc.), users need flexibility to optimize cost vs quality trade-offs and handle different context length requirements.