openclaw - 💡(How to fix) Fix [Bug]: channels.bluebubbles.start-account hangs after plugin config hot-reload — every webhook silently 404s [1 comments, 2 participants]

Summary

After a hot-reload of plugins.entries.* config (in our case, fleet-wide edit to plugins.entries.google.config.webSearch.model), the bluebubbles channel gets stuck in the channels.bluebubbles.start-account startup phase indefinitely. The gateway's HTTP listener stays responsive (returns HTTP 200 on POST /bluebubbles-webhook), but the plugin's actual message-handling pipeline never receives the inbound — so the webhook source (BB :1234/1237) sees "delivered" and never retries, and every customer message is silently dropped.

This pattern appears already reported for WhatsApp (#78165, #78690), Slack (#78435), Telegram (#78352), and Discord-adjacent channels. Filing the bluebubbles instance separately for grep-ability.

Reproduction

1. OpenClaw v2026.5.7 (caught on multiple tenants, including a native install and 14 Docker tenants on the same image)
2. Tenant configs with channels.bluebubbles.enabled: true, groupPolicy: open or allowlist
3. Apply a hot-reloadable plugin config edit across the fleet (we edited plugins.entries.google.config.webSearch.model via direct JSON write, no restart). Gateway detects + reloads.
4. ~Some fraction of tenants enter the stuck state. In our incident: 15 of 27 tenants on bluebubbles hit it simultaneously.

Symptoms

Externally healthy

• systemctl is-active → active
• TCP listener on gateway port → bound
• POST /bluebubbles-webhook (or POST /bluebubbles-webhook via mux forward) → HTTP 200 in 2-5ms (suspiciously fast — handler isn't actually processing)
• /healthz → 200

Internally stuck

Gateway log entries cease entirely after startup. The last entries you see:

[default] starting provider (webhook=/bluebubbles-webhook)
[default] BlueBubbles server macOS 26.3.1
[default] BlueBubbles Private API enabled
[default] BlueBubbles webhook listening on /bluebubbles-webhook
[default] BlueBubbles catchup: replayed=N fetched=N window_ms=NNNN
cron: started

Then silence — for HOURS. Heartbeat output (at DEBUG level) shows webhooks=0/0/0 despite traffic clearly arriving (mux-side logs confirm forwards land successful).

Diagnostic liveness signature (same as #78165 / #78690 but for BB)

liveness warning: reasons=event_loop_delay interval=30s
  eventLoopDelayP99Ms=21.3 eventLoopDelayMaxMs=5964.3
  cpuCoreRatio=0.094 active=1 waiting=0 queued=1
  phase=channels.bluebubbles.start-account
  recentPhases=sidecars.restart-sentinel:0ms,sidecars.subagent-recovery:13ms,
    sidecars.main-session-recovery:8ms,post-attach.update-sentinel:0ms,
    sidecars.session-locks:61235ms,post-ready.maintenance:759ms

The 60+ seconds spent in sidecars.session-locks is consistent across affected tenants. Once the phase enters start-account, it never transitions to a completed state.

Confirmation it's NOT firing webhooks

Inside the container, with debug logging enabled (logging.level: debug):

• [bluebubbles] webhook received path=/bluebubbles-webhook — does NOT appear despite mux confirming forward
• [bluebubbles] webhook accepted ... — never logged

So either the HTTP listener IS the BB plugin's listener but the request body is being discarded before reaching the handler, or there's a separate intercept layer that accepts + returns 200 without dispatching.

Trigger

In our case the trigger was hot-reload of plugins.entries.google.config.webSearch.model (a string change inside config). Plugin reload kicked in, attempted to re-init the BB plugin, and the re-init blocked on session locks.

This matches the WhatsApp/Slack pattern reported in #78165 and #78435: ANY hot-reload that touches plugin entries seems to trigger the stuck-start-account state on a fraction of the channels.

Workaround

systemctl restart openclaw-gateway (or docker compose up -d for Docker tenants — full recreation, not just restart) clears the state. Webhook starts logging immediately. BB plugin processes the catchup queue normally.

Impact for us

• 15 of 27 tenants hit it in one incident (2026-05-10)
• 8 of those were paying customers
• Customers' bots silently 404'd every inbound message for 12+ hours
• Symptom isn't visible from external monitoring because port responds and HTTP returns 200
• We caught it only after a customer complained to us out-of-band

Suggested fix direction

Same as #78690's recommendation: the BB plugin's start-account phase shouldn't hold session-locks across hot-reload, OR the hot-reload path should force a clean tear-down + cold start instead of attempting an in-place re-init. Either approach would unblock the channel.

If a session-lock acquisition timeout is the right answer (vs blocking indefinitely), 30-60s is probably reasonable based on observed sidecars.session-locks:61235ms durations.

Env

• OpenClaw 2026.5.7 (native install at /opt/badclaw/openclaw/)
• Docker tenants on badclaw-openclaw:2026.5.7-patched-claude (same upstream base + 1 unrelated patch for BB Private API)
• BB v1.9.9, macOS 26.3.1 host
• Node 24.15.0
• Linux host (Ubuntu 24.04 + Hetzner CPX41)

Happy to provide additional log dumps / stack traces if useful.