PR #74152: feat(gateway): make chat-delta broadcast throttle configurable via OPENCLAW_CHAT_DELTA_THROTTLE_MS

• Repository: openclaw/openclaw
• Author: chphch
• State: closed | merged: False
• Link: https://github.com/openclaw/openclaw/pull/74152

Description (problem / solution / changelog)

Summary

Make the gateway chat-delta broadcast throttle configurable via OPENCLAW_CHAT_DELTA_THROTTLE_MS. Default stays at the existing 150 ms, so behavior is unchanged for everyone who does not opt in.

No linked issue — operator observability/UX seam, filed in response to a real deployment where the hardcoded value did not fit.

Problem

emitChatDelta in src/gateway/server-chat.ts rate-limits assistant text broadcasts to one frame per 150 ms per run via a hardcoded threshold. With fast LLMs (Gemma 31B at OpenRouter, Sonnet 4.6, etc. emitting 30–40 ms/token), the visible streaming pattern lands at ~6 frames/second with ~5 tokens batched per visible chunk, plus a residual flush at the final event. On small mobile screens this reads as "silent then chunks" rather than the smoother per-token rendering that providers' SSE streams already make available.

The 150 ms value is a sensible default but it is the wrong default for some deployments:

• Local/loopback gateways where bandwidth is free and operators want maximum smoothness.
• Mobile-only deployments where operators are willing to trade extra WebSocket frames for a more readable stream.
• Bandwidth-constrained / metered links where operators want to raise the throttle.

Today there is no knob to tune this without a recompile.

For comparison, channel-side throttles like Telegram's existing streamThrottleMs request (#38066) cover their own surface — the gateway broadcast path that feeds Control UI / mobile / 3rd-party WebSocket clients has no equivalent.

Change

src/gateway/chat-delta-throttle.ts (new, 22 LOC):

• Exports DEFAULT_CHAT_DELTA_THROTTLE_MS = 150 and resolveChatDeltaThrottleMs().
• Reads OPENCLAW_CHAT_DELTA_THROTTLE_MS and falls back to the default on unset / empty / non-numeric / NaN / negative.
• 0 is honored (disables the throttle).

src/gateway/server-chat.ts:

• One-line import of the resolver.
• Replaces the hardcoded < 150 in emitChatDelta with < resolveChatDeltaThrottleMs().
• Updates the existing comment near flushBufferedChatDeltaIfNeeded so it stays accurate ("default 150 ms, overridable via env var").

No public API change. No new dependency. No changelog entry (operator-facing diagnostic / tuning knob, off by default — same convention as OPENCLAW_RAW_STREAM).

Why this shape

• Default unchanged: existing 150 ms preserved; users who do not set the env var see zero behavior change.
• Resolver called per emit, not memoized at module load — keeps env mutation honored under tests and avoids a restart-only contract for an operator knob.
• Single env var, no schema change: matches OPENCLAW_RAW_STREAM precedent for operator-only diagnostic/tuning knobs and is automatically blocked from workspace .env by the existing OPENCLAW_* prefix policy in src/infra/dotenv.ts.
• Strict fallback: Number.isFinite + >= 0 keeps a typo (abc, -50, NaN) from silently breaking streaming in a hard-to-debug way.

Alternatives considered

• Config-file knob (agents.defaults.streaming.chatDeltaThrottleMs): would be more discoverable but introduces a config-schema migration for a setting with narrow operator audience. Easy to add later if there is demand.
• Per-channel override: out of scope — this PR targets the gateway broadcast path that all WebSocket clients share.
• Memoized at module load: rejected; tests need to mutate env between assertions, and operator restarts are an unnecessary tax on a knob this small.

Tests

src/gateway/chat-delta-throttle.test.ts (new, 8 cases):

• Default returned when env var is unset / empty.
• Honors a positive integer override.
• Honors 0 (disables throttle).
• Honors fractional values (e.g. 75.5).
• Falls back to default on non-numeric / NaN / negative.

$ pnpm test src/gateway/chat-delta-throttle.test.ts
Test Files  1 passed (1)
     Tests  8 passed (8)
$ pnpm test src/gateway/server-chat.agent-events.test.ts
Test Files  1 passed (1)
     Tests  42 passed (42)

pnpm build clean; the resulting server-chat-*.js bundle contains the new env-var lookup.

Notes

• OPENCLAW_CHAT_DELTA_THROTTLE_MS is operator-only by design; workspace .env already cannot set it.
• A config-file alias under agents.defaults.streaming is a possible follow-up if maintainers prefer that surface. Happy to roll it into this PR if requested — kept minimal here so the contract is one knob, one place.

Changed files

• src/gateway/chat-delta-throttle.test.ts (added, +57/-0)
• src/gateway/chat-delta-throttle.ts (added, +21/-0)
• src/gateway/server-chat.ts (modified, +6/-4)