Summary

In non-streaming mode (display.streaming: false, or any platform that doesn't register a stream consumer), every message processed by the gateway pays an unnecessary ~5s tax on the response path.

Observed end-to-end latency on Telegram with streaming disabled + Haiku 4.5 via OpenRouter:

• Direct OpenRouter curl: ~1.5s
• Hermes gateway: 8.6–10s (consistent, regardless of payload size or model)
• After fix: 1.5s (-83%)

Root cause

gateway/run.py:13352:

# Wait for stream consumer to finish its final edit
if stream_task:
    try:
        await asyncio.wait_for(stream_task, timeout=5.0)
    except (asyncio.TimeoutError, asyncio.CancelledError):
        stream_task.cancel()
        try:
            await stream_task
        except asyncio.CancelledError:
            pass

stream_task is created at line ~12815 from _start_stream_consumer():

async def _start_stream_consumer():
    for _ in range(200):  # Up to 10s wait
        if stream_consumer_holder[0] is not None:
            await stream_consumer_holder[0].run()
            return
        await asyncio.sleep(0.05)

When no consumer is ever registered (non-streaming path), the task never returns naturally — it polls for the full 10s. The cleanup code at 13352 then waits the full timeout=5.0 before cancelling. The 5s is the dominant component of the wall-clock response time for short messages.

py-spy profile confirms: 5s of post-run_conversation time is spent in _run_in_executor_with_context waiting on this task, while CPU is idle.

Repro

1. streaming.enabled: false in config.yaml
2. Send any short Telegram message ("ping")
3. Time inbound→response: ~8s constant, regardless of model / tools / history size

Diagnostic logs around agent.run_conversation confirm:

• run_conversation itself: 1.6s (matches OpenRouter direct curl)
• run_sync executor wrapper: 1.6s
• Time between executor task completion and gateway returning: 5.0s

Fix

Skip the wait entirely when no consumer was ever registered — that's the non-streaming case:

if stream_task:
    if stream_consumer_holder[0] is None:
        # No consumer was registered — cancel immediately.
        stream_task.cancel()
        try:
            await stream_task
        except asyncio.CancelledError:
            pass
    else:
        try:
            await asyncio.wait_for(stream_task, timeout=5.0)
        except (asyncio.TimeoutError, asyncio.CancelledError):
            stream_task.cancel()
            try:
                await stream_task
            except asyncio.CancelledError:
                pass

Same pattern applies to the two other call sites: gateway/run.py:11559 (proxy mode) and gateway/run.py:13233 (queued-message branch).

Side findings during diagnosis (separate, lesser issues)

• agent.reasoning_effort: medium is the effective default for Haiku 4.5 even though reasoning_effort: none is set at the top level of config.yaml. The two keys aren't unified — only the nested one is read. Costs ~1.5s per message.
• _POLL_INTERVAL = 5.0 in the executor poll loop (gateway/run.py:12965) is benign in steady state because asyncio.wait returns as soon as the task finishes, but reducing it gave a marginal improvement under contention.
• Background memory/skill review (_run_review) correctly runs in a daemon thread after the user response — confirmed NOT in the critical path. Misleading initially because it appears in py-spy profiles overlapping the next user message.

Environment

• hermes-agent: v0.12.0
• Provider: OpenRouter
• Model: anthropic/claude-haiku-4.5
• Platform: Telegram (polling)
• streaming.enabled: false