hermes - 💡(How to fix) Fix HTTP/StreamableHTTP OAuth MCP servers connect via `hermes mcp test` but register 0 tools in the agent (stdio works)

Summary

A remote HTTP / StreamableHTTP MCP server that uses OAuth 2.1 PKCE connects and lists its tools fine via hermes mcp test <name> (29 tools discovered), but the agent loop registers 0 tools from it — in standalone (hermes -z), gateway, and cron sessions alike. stdio MCP servers on the same install work perfectly (postiz + vugola → 30 tools). So the agent never gets the remote server's tools even though the connection itself is healthy.

Repro server: the official Higgsfield MCP (https://mcp.higgsfield.ai/mcp, OAuth 2.1 PKCE, streamable-http).

Environment

• Hermes Agent v0.15.1, git HEAD d29caf382 (2026-06-04) — reproduced on the latest code, not just the tagged release
• Windows 11, Python 3.13.2
• Transport: streamable-http (_MCP_NEW_HTTP path, streamable_http_client)
• Other MCP servers configured: postiz, vugola (both stdio) — these work
• config.yaml:

  mcp_servers:
    higgsfield:
      url: https://mcp.higgsfield.ai/mcp
      auth: oauth

• OAuth completed successfully (mcp-tokens/higgsfield.{json,client.json,meta.json} present)

Repro

1. hermes mcp add higgsfield --url https://mcp.higgsfield.ai/mcp --auth oauth and complete the browser OAuth.
2. hermes mcp test higgsfield → ✓ Connected, 29 tools discovered.
3. hermes mcp list → higgsfield shows ✓ enabled, Tools=all.
4. Ask the agent (any of: hermes -z "...", gateway/Telegram, or a cron job) to use a higgsfield tool (e.g. balance).
5. The agent reports it has no higgsfield tools (only the stdio servers' tools are present). The MCP summary logs registered 30 tool(s) from 2 server(s) — higgsfield is absent.

Observed logs

On a fresh gateway start, the higgsfield connection is attempted and gets through protocol negotiation, then dies:

mcp.client.streamable_http: Negotiated protocol version: 2025-11-25
tools.mcp_oauth_manager: MCP OAuth 'higgsfield': tokens file changed (mtime 0 -> 1780611392450751600), forcing reload
ERROR asyncio: Task was destroyed but it is pending!

No MCP server 'higgsfield' (HTTP): registered N tool(s) line, and no Failed to connect to MCP server 'higgsfield' line — the connection task is silently destroyed before tool registration.

Root cause #1 (confirmed + patched locally)

MCPOAuthManager.invalidate_if_disk_changed (tools/mcp_oauth_manager.py) is called as a pre-flow hook inside HermesMCPOAuthProvider.async_auth_flow (run by httpx on every auth flow, including the very first connection handshake).

On the first observation, entry.last_mtime_ns == 0, while the tokens file on disk has a nonzero mtime, so it is treated as an external change:

if mtime_ns != entry.last_mtime_ns:      # 0 != <real mtime> -> always true on first read
    ...
    entry.provider._initialized = False  # resets OAuth init DURING the first handshake

Resetting _initialized = False in the middle of the first auth flow races with the SDK's initial connection handshake and tears the streamable-http connection task down (Task was destroyed but it is pending) before tools are registered. hermes mcp test doesn't hit this because it warms the provider first, so by the time the real auth flow runs the baseline mtime is already recorded.

Local patch that removes the forcing reload + Task was destroyed symptoms (establish the baseline on first read instead of forcing a reload — a reload is only meaningful for catching external refreshes after the baseline exists):

if mtime_ns != entry.last_mtime_ns:
    old = entry.last_mtime_ns
    entry.last_mtime_ns = mtime_ns
    if old == 0:
        # First observation: record baseline, do NOT force a reload.
        return False
    if hasattr(entry.provider, "_initialized"):
        entry.provider._initialized = False
    logger.info("MCP OAuth '%s': tokens file changed (mtime %d -> %d), forcing reload",
                server_name, old, mtime_ns)
    return True
return False

After this patch the forcing reload and Task was destroyed log lines no longer appear.

Remaining issue #2 (needs maintainer insight)

Even with patch #1 applied, the agent still registers 0 tools from the HTTP/OAuth server. Post-patch, agent/gateway/cron sessions log no higgsfield connection attempt at all (no success line, no Failed to connect line), while hermes mcp test higgsfield keeps working (29 tools). It looks like there is a second problem in the HTTP-MCP registration/retry/lifecycle path — e.g. the agent/gateway not (re)attempting the server in agent sessions, or a registration-cache interaction — that is independent of the OAuth reload race. Pointers on how register_mcp_servers / _discover_and_register_server load HTTP servers in the agent loop vs the mcp test path would help pin this down.

Why this matters

Remote OAuth MCP servers (Higgsfield, and any streamable-http + OAuth 2.1 server) are effectively unusable from the agent today even though mcp test says they're connected — a confusing failure mode because everything looks wired up. Happy to test patches.