Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

On a multi-tenant gateway (12 WhatsApp accounts), all 12 channels enter a perpetual health-monitor: restarting (reason: stopped) loop after their underlying WhatsApp Web sessions are invalidated. The health monitor restarts channels regardless of why they stopped, so channels stopped by terminal DisconnectReasons (loggedOut, connectionReplaced, badSession) get re-started indefinitely with no chance of success — they need a fresh QR pairing, not a restart. Over ~20 hours this drove gateway RSS from <500 MB to 12.9 GB peak, CPU to 16 h of 20 h wall time, and RPC agents.files.get latency from <200 ms to 24-25 s, blocking unrelated tenants on the same gateway.

Steps to reproduce

1. Run an OpenClaw gateway with multiple WhatsApp Web accounts (channels.whatsapp.accounts.*) on 2026.5.3-1.
2. Wait for one or more accounts to receive a terminal disconnect from WhatsApp servers — easiest reproductions:
   • User logs out the linked device from the phone (Settings → Linked Devices → unlink) → DisconnectReason.loggedOut (401).
   • User links a new device that replaces the OpenClaw session → DisconnectReason.connectionReplaced (440).
   • Auth state corruption → DisconnectReason.badSession.
3. Observe that the channel transitions to running=false after the close handler.
4. Watch ~/.openclaw/logs/gateway.log (or the systemd-user journal) for ~30 min:

   [health-monitor] [whatsapp:cp-tenantN] health-monitor: restarting (reason: stopped)

   firing every health-check interval, per channel, with each restart immediately failing the same way.
5. With 12 stuck channels, observe gateway RSS climb steadily (Baileys WebSocket + Noise handshake state per attempt), event-loop delay rise, and other RPCs degrade.

Expected behavior

• Channels stopped by terminal DisconnectReasons are marked in a state distinct from "stopped" (e.g. reauth_required) and not restarted by the health-monitor.
• A channel.reauth_required event (or equivalent) is emitted on the gateway event stream so control planes can surface a re-pairing prompt to the operator.
• channels.status exposes a reauthRequired: boolean field so downstream consumers can distinguish "needs human action" from "transient outage".
• Channels that exit for retryable reasons (connectionClosed, connectionLost, restartRequired, timedOut) continue to be restarted as today.
• A bounded restart budget exists for any restart loop that escapes the DisconnectReason classification (defense-in-depth — see Additional information).

Actual behavior

• Channel exits with connection: 'close' and a terminal DisconnectReason from Baileys.
• Gateway sets the channel to running=false (status: stopped).
• Health monitor wakes on its interval, sees running=false, and calls startChannel().
• New connection immediately fails with the same terminal reason (or a connectionReplaced 440 because the previous Noise key is now stale on WhatsApp's side).
• Loop continues indefinitely. Per-attempt cost (Baileys auth state + protobuf parsing + WebSocket setup) accumulates: in our case 12 channels × ~one attempt every health-check interval × no upper bound on consecutive failures.

Observed metrics (gateway PID, single instance, 20h window):

• RSS: <500 MB → 12.9 GB peak (~26× growth).
• CPU: 16 h of 20 h (~80%).
• agents.files.get RPC p50/p99: ~150 ms / ~24-25 s (normal <200 ms).
• Other tenants' control-plane RPCs queued behind the saturated event loop.

OpenClaw version

2026.5.3-1

Operating system

Ubuntu 24.04 (Linux 6.8.0-110-generic), VPS

Install method

npm global, gateway as systemd --user unit (openclaw-gateway.service)

Model

Not relevant to this bug — failure is upstream of any model invocation. (Mixed: Anthropic Claude Opus 4.7, Sonnet 4.6 for tenant agents.)

Provider / routing chain

Not relevant — failure is in the WhatsApp transport before any agent runs.

Additional provider/model setup details

Multi-tenant SaaS control plane: ~12 active WhatsApp accounts on a single gateway, each tied to a distinct tenant. This is the relevant difference from prior single-user reports — the blast radius spans tenants.

Logs, screenshots, and evidence

# 12 channels entering health-monitor restart on the same global sweep
[health-monitor] [whatsapp:cp-tenant1]   health-monitor: restarting (reason: stopped)
[health-monitor] [whatsapp:cp-tenant2]   health-monitor: restarting (reason: stopped)
[health-monitor] [whatsapp:cp-tenant3]   health-monitor: restarting (reason: stopped)
... (12 lines, all within the same second) ...

# RPC latency during saturation vs. recovery
[ws] ⇄ res ✓ agents.files.get  24913ms  conn=<redacted>  id=<redacted>
[ws] ⇄ res ✓ agents.files.get  24241ms  conn=<redacted>  id=<redacted>
# After the offending channel was removed from config:
[ws] ⇄ res ✓ agents.files.get     36ms  conn=<redacted>  id=<redacted>

# Gateway RSS over 20h (sampled via systemd-cgtop / RSS column of /proc/<pid>/status)
00:00 RSS=  462 MB
04:00 RSS=  3.1 GB
08:00 RSS=  6.4 GB
12:00 RSS=  9.8 GB
16:00 RSS= 11.7 GB
20:00 RSS= 12.9 GB  ← peak before manual intervention

Impact and severity

• Affected: any operator running >1 WhatsApp account on a single gateway where one or more accounts is invalidated upstream (logout from phone, account ban, device replaced). Severity scales with channel count.
• Severity: high — degrades the entire gateway, not just the affected channel. In multi-tenant deployments, one tenant's logout can cause cross-tenant outage.
• Frequency: 100% reproducible whenever a WhatsApp Web session ends with a terminal DisconnectReason. Frequency in the wild depends on user behavior; in our population we see it weekly.
• Consequence: gateway memory exhaustion, event-loop saturation, missed cross-channel replies, RPC latency that cascades into agent-side timeouts. Recovery currently requires manual gateway restart + per-channel re-pairing.

Additional information

Relation to prior closed issues

This is the same problem family as #48390, #49305, #51342, #54614, #70463, but the reason logged is now stopped (not stale-socket), and the symptom is observed on 2026.5.3-1 after those fixes shipped. Reading the closed-issue history, the post-fix gateway appears to have moved the health-monitor trigger from "stale event timestamp" to "channel running=false" — which closes the stale-socket false positive, but reintroduces the same loop because the health-monitor still cannot tell why a channel is stopped.

The architectural gap: the health-monitor and the close-reason classifier are decoupled. The close handler knows the channel was logged out; that information does not reach the health-monitor.

Proposed fix #1 — terminal DisconnectReason classification (root cause)

Capture connection.update from Baileys and, when connection === 'close', read lastDisconnect.error.output.statusCode (or lastDisconnect.error.data.reason) and map to DisconnectReason:

Additions to public surface:

• channels.status RPC: add optional reauthRequired: boolean (and ideally reauthReason: 'logged_out' | 'connection_replaced' | 'bad_session' | ...). Optional → does not break existing clients.
• New gateway event: channel.reauth_required on the event stream, with { channelId, accountId, reason } payload, so control planes can proactively show a re-pair / new-QR prompt.
• When a fresh QR scan completes successfully, clear reauthRequired and resume normal health-monitor behavior automatically.

Proposed fix #2 — bounded restart budget (defense-in-depth)

For close reasons that fall outside the mapping (new Baileys constants, unmapped 4xx, undocumented disconnects, or close-before-connection.update-fires races):

• After N consecutive restart attempts (suggested default 5) within a rolling window of X minutes (suggested 10 min), stop restarting.
• Mark the channel disconnected (max_retries_exceeded); surface the same channel.reauth_required event so operators see the failure.
• Reset the counter when the channel stays connected for Y minutes (suggested 5 min) without a close.
• Make N, X, Y configurable under gateway.channelHealthMonitor.{maxConsecutiveRestarts, restartWindowMinutes, healthyResetMinutes}.

Why both fixes, not just one:

• Fix #1 alone fails on classes the gateway doesn't yet know about (new Baileys releases, undocumented WhatsApp server codes). Fix #2 catches that.
• Fix #2 alone preserves the bad UX of silent failure: operators only learn after the budget exhausts (5 attempts × ~30 s ≈ 2-3 min lag) and lose the strong "logged out → please re-pair" signal that Fix #1 provides.

Last known good / first known bad

We have not bisected — this is structural rather than a regression. Prior closed issues in the same family suggest the loop has existed across several versions with different surface symptoms (stale-socket → stopped).