openclaw - 💡(How to fix) Fix [Feature]: Support local/self-hosted STT engine for Talk Mode realtime voice [1 comments, 2 participants]

Summary

Add support for local or self-hosted Speech-to-Text (STT) engines in Talk Mode, so users without cloud API keys can still use high-quality realtime voice conversations.

Problem to Solve

Talk Mode currently has two STT paths, both with significant limitations:

No path supports a local or self-hosted STT server. Users without cloud API access (regions with network restrictions, privacy-conscious users, zero API cost users) are locked into macOS native speech recognition with limited quality.

Proposed Solution

Allow talk.realtime.providers.<provider> to accept a baseUrl configuration, similar to how messages.tts.providers.openai.baseUrl already works:

{
  talk: {
    realtime: {
      provider: "openai",
      providers: {
        openai: {
          baseUrl: "http://localhost:8080",  // proposed new field
          apiKey: "not-needed",
          model: "whisper-large-v3",
        },
      },
    },
  },
}

This would enable pointing Talk Mode to:

• whisper.cpp server (OpenAI-compatible WebSocket)
• faster-whisper streaming endpoint
• SenseVoice or other local ASR engines
• Any self-hosted OpenAI Realtime API-compatible STT service

For Gateway relay, similarly allow streaming.providers.<provider> to accept custom endpoints.

Combined with existing messages.tts.providers.openai.baseUrl (local TTS) and talk.providers.mlx (local TTS), this enables fully local, high-quality realtime voice.

Use Cases

1. Users in regions where OpenAI/Google APIs are inaccessible (GFW, etc.)
2. Privacy-first users — voice data never leaves local network
3. Zero API cost — self-host on Apple Silicon
4. Custom language support — local models for regional languages

Environment

• OpenClaw version: 2026.5.18
• Hardware: Mac Mini M4 Pro (Gateway) + Mac Studio (model server)
• Talk mode tested: macOS native (quality limited) and WebChat (API key required, unavailable)