PR #29004: fix(run_agent): unfreeze first tool call after idle on macOS (#28834)

• Repository: NousResearch/hermes-agent
• Author: xxxigm
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/29004

Description (problem / solution / changelog)

What does this PR do?

Fixes #28834 — first tool call after a 2-3 minute idle pause freezes at [Calling tool: ...] until the user kicks the loop (e.g. with /goal).

Root cause is in AIAgent._build_keepalive_http_client (run_agent.py), which configures the TCP keepalive socket options applied to every provider connection. Two adjacent bugs combine to produce the freeze:

1. The Linux branch sets TCP_KEEPIDLE / TCP_KEEPINTVL / TCP_KEEPCNT together → dead peer detected in ~60 s. The macOS branch only sets TCP_KEEPALIVE (the idle knob's macOS name) and falls through, leaving KEEPINTVL and KEEPCNT at kernel defaults of 75 s × 8 ≈ 10 minutes. After a 2-3 min idle, the provider socket is silently dropped by intermediate NAT/firewall but macOS doesn't notice for nearly 10 more minutes.
2. Even with the keepalive fix, there's still a narrow window where httpx's keepalive pool hands out a zombie connection to the next request before the keepalive timer has had a chance to mark it dead. Without a connection-level retry, that request hangs / errors with no automatic recovery.

The fix is two small changes inside _build_keepalive_http_client, each in its own commit:

• Split TCP_KEEPINTVL / TCP_KEEPCNT out of the TCP_KEEPIDLE branch and gate them on their own hasattr checks — both are exposed on macOS in Python ≥ 3.10. macOS now matches Linux's ~60 s detection budget.
• Pass retries=1 to httpx.HTTPTransport so a stale-pool connection that beats the keepalive timer triggers a single transparent re-dial. httpx only retries connection-establishment failures, so this can't double-submit a half-sent request.

Related Issue

Fixes #28834.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• run_agent.py (+27/-3) — _build_keepalive_http_client:
  • Set TCP_KEEPINTVL=10 and TCP_KEEPCNT=3 on both the Linux and macOS branches via independent hasattr checks (was Linux-only).
  • Pass retries=1 to httpx.HTTPTransport so connection-establishment failures (stale pool connections) re-dial transparently.
• tests/run_agent/test_keepalive_socket_options.py (+179, new) — three test classes covering the new contract:
  • TestKeepaliveSharedKnobs — 4 cases running against the real host's socket module: SO_KEEPALIVE on, TCP_KEEPINTVL=10, TCP_KEEPCNT=3, and a 30 s idle warm-up under whichever of TCP_KEEPIDLE / TCP_KEEPALIVE the platform exposes.
  • TestMacOSKeepaliveParity — 1 case stubbing sys.modules['socket'] with a macOS-flavored facade (no TCP_KEEPIDLE) so the test exercises the macOS branch even on Linux CI runners. Pins all three values (30 / 10 / 3) to lock the budget.
  • TestStalePoolRetry — 1 case asserting the constructed httpx.HTTPTransport carries retries=1 via the underlying httpcore pool's _retries attribute.

No other production files touched. No config schema changes, no new env vars, no public-API surface change.

How to Test

1. Check out this branch and ensure .venv is set up: python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[all,dev]"
2. Run the new tests on their own:

   scripts/run_tests.sh tests/run_agent/test_keepalive_socket_options.py -v

   Expected: 6 passed.
3. Run the wider OpenAI-client transport suite to confirm no cross-file regressions:

   scripts/run_tests.sh tests/run_agent/test_keepalive_socket_options.py \
     tests/run_agent/test_create_openai_client_reuse.py \
     tests/run_agent/test_create_openai_client_proxy_env.py \
     tests/run_agent/test_create_openai_client_kwargs_isolation.py \
     tests/run_agent/test_async_httpx_del_neuter.py \
     tests/run_agent/test_sequential_chats_live.py

   Expected: 27 passed, 1 skipped.
4. (Optional, on macOS only — reproduces the original issue) Open hermes in interactive mode, run any tool call, idle 3 minutes, then send a request that requires another tool call. Before the fix: hangs at [Calling tool: ...]. After the fix: completes within the usual latency.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(run_agent): ... × 2, test(run_agent): ... × 1)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix (no unrelated commits)
• I've run scripts/run_tests.sh tests/run_agent/test_keepalive_socket_options.py and all tests pass
• I've added tests for my changes
• I've tested on my platform: macOS 15.2 (Darwin 24.6.0), Python 3.12

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — N/A (no public-API change; inline comments updated to call out the macOS gap + #28834)
• I've updated cli-config.yaml.example if I added/changed config keys — N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — macOS branch fixed to match Linux; Windows path is unchanged (the helper short-circuits via try/except, and Windows has no TCP_KEEPINTVL); fix is the documented behaviour on both supported platforms
• I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

$ scripts/run_tests.sh tests/run_agent/test_keepalive_socket_options.py -v
4 workers [6 items]
============================== 6 passed in 1.59s ===============================

$ scripts/run_tests.sh tests/run_agent/test_keepalive_socket_options.py \
    tests/run_agent/test_create_openai_client_reuse.py \
    tests/run_agent/test_create_openai_client_proxy_env.py \
    tests/run_agent/test_create_openai_client_kwargs_isolation.py \
    tests/run_agent/test_async_httpx_del_neuter.py \
    tests/run_agent/test_sequential_chats_live.py
4 workers [28 items]
======================== 27 passed, 1 skipped in 3.94s =========================

Changed files

• run_agent.py (modified, +27/-3)
• tests/run_agent/test_keepalive_socket_options.py (added, +179/-0)