PR #15171: fix(api-server): persist response snapshot on client disconnect when store=True

• Repository: NousResearch/hermes-agent
• Author: UgwujaGeorge
• State: closed | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/15171

Description (problem / solution / changelog)

Summary

• persist an initial in_progress response snapshot as soon as the streaming /v1/responses path emits response.created
• update stored snapshots on terminal completed and failed outcomes before writing the final SSE event
• write an incomplete snapshot when the SSE client disconnects before completion so stored responses remain retrievable

Root Cause

The streaming /v1/responses implementation only wrote to ResponseStore on the clean response.completed path. If the client disconnected mid-stream, the handler interrupted and cancelled the agent task without persisting the response state, so GET /v1/responses/{id} returned 404 even after a valid response.created event had already exposed the response ID.

Impact

Clients using store=true can now recover streamed responses after disconnects because the response ID always has at least an initial stored record, and disconnects update that record to an incomplete snapshot instead of dropping it entirely.

Validation

• scripts/run_tests.sh tests/gateway/test_api_server.py -q

Changed files

• gateway/platforms/api_server.py (modified, +74/-22)

PR #15492: fix(api-server): Implement background response run recovery

• Repository: NousResearch/hermes-agent
• Author: zhboner
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/15492

Description (problem / solution / changelog)

What does this PR do?

This PR implements true server-side background execution for streaming POST /v1/responses requests.

Previously, a stream=true Responses API request was tightly coupled to the SSE client connection. If the client disconnected, the agent execution could be interrupted and the response could not be reliably recovered. This PR decouples response execution from the SSE subscriber lifecycle so the server continues running the response in the background and allows clients to recover state through GET /v1/responses/{response_id}.

Current limitations

This PR does not attempt to make stream and store fully orthogonal across all /v1/responses modes.

The issue addressed here is specific to stream=true: durable streaming responses should continue running after SSE disconnect and should be recoverable by response ID. This PR therefore unifies the two streaming modes around ResponseRun:

• stream=true + store=true: durable/background/recoverable
• stream=true + store=false: ephemeral/connection-owned, cancelled on disconnect

The existing stream=false synchronous path is left unchanged.

Making all four combinations fully orthogonal would require moving non-streaming execution onto ResponseRun as well. That is a larger lifecycle refactor touching cancellation semantics, conversation active-state tracking, idempotency, error handling, tests, and client expectations. Since that is outside the scope of the reported background streaming issue, it is deferred to a future PR.

Related Issue

Fixes #15026

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• Added ResponseRun / ResponseRunManager to own background /v1/responses execution lifecycle.
• Changed streaming /v1/responses behavior so SSE client disconnects only detach the subscriber and do not cancel the underlying agent run.
• Added active response recovery through GET /v1/responses/{response_id}.
• Required store=true for stream=true Responses API requests.
  • stream=true + store=false now returns 400 store_required.
• Added streaming Idempotency-Key support.
  • Matching retries can reattach to active runs or replay stored responses.
  • Conflicting retries return 409 idempotency_key_conflict.
• Added bounded queues and concurrent active response run limits.
• Added response.snapshot SSE event for active-run reattach/idempotent retry.
• Added SSE keepalive comments for response streams.
• Added CORS headers for stored streaming response replay.
• Updated previous_response_id and conversation chaining so only completed responses are accepted.
• Reworked conversation tracking to separate:
  • latest_completed_response_id
  • active_response_id
• Added migration support from the legacy conversations.response_id schema.
• Ensured DELETE and LRU eviction clear conversation references and idempotency mappings.
• Added shutdown/delete cancellation handling for active background response runs.
• Removed the legacy private _write_sse_responses(...) path in favor of ResponseRun.

How to Test

1. Run the targeted test suite

~/.hermes/hermes-agent/venv/bin/python -m pytest -o addopts= tests/gateway/test_api_server.py tests/gateway/test_sse_agent_cancel.py -q

Expected result:

140 passed

2. Verify stream=true + store=false works as ephemeral streaming

Run:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "Reply with exactly: ephemeral-ok",
    "stream": true,
    "store": false
  }'

Expected result:

• HTTP 200
• SSE stream includes response.created
• SSE stream includes response.completed
• stream ends with:

data: [DONE]

Copy the response.id from the stream, then run:

curl -i http://127.0.0.1:8642/v1/responses/<response_id> \
  -H "Authorization: Bearer <API_KEY>"

Expected result:

• HTTP 404
• because store=false responses are ephemeral and not recoverable

3. Verify stream=true + store=false rejects Idempotency-Key

Run:

curl -i http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: ephemeral-test-key" \
  -d '{
    "model": "hermes-agent",
    "input": "hello",
    "stream": true,
    "store": false
  }'

Expected result:

• HTTP 400
• response error code:

"idempotency_requires_store"

This confirms that idempotent streaming replay/reattach is only available for durable store=true streams.

4. Verify stream=true + store=true survives client disconnect

Start a durable streaming response:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "Take a little time, then reply with exactly: background-recovered-ok",
    "stream": true,
    "store": true
  }'

After receiving the first response.created event, stop the client with Ctrl+C.

Expected server behavior:

• SSE subscriber disconnects
• agent run continues in the background
• run is not interrupted just because the client disconnected

Then recover the response:

curl http://127.0.0.1:8642/v1/responses/<response_id> \
  -H "Authorization: Bearer <API_KEY>"

Expected result:

• while still running, response may show status: "in_progress"
• after completion, response shows:

"status": "completed"

5. Verify durable streaming idempotency retry

Start a durable streaming request with an idempotency key:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: durable-stream-test-1" \
  -d '{
    "model": "hermes-agent",
    "input": "Reply with exactly: durable-idempotency-ok",
    "stream": true,
    "store": true
  }'

Retry the exact same request with the same Idempotency-Key:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: durable-stream-test-1" \
  -d '{
    "model": "hermes-agent",
    "input": "Reply with exactly: durable-idempotency-ok",
    "stream": true,
    "store": true
  }'

Expected result:

• retry returns the same response.id
• if the original run is still active, retry starts with:

event: response.snapshot

• if the original run already completed, retry streams/replays the stored response

6. Verify idempotency conflict handling

Reuse the same idempotency key with a different request body:

curl -i http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: durable-stream-test-1" \
  -d '{
    "model": "hermes-agent",
    "input": "This is a different request body.",
    "stream": true,
    "store": true
  }'

Expected result:

• HTTP 409
• response error code:

"idempotency_key_conflict"

7. Verify conversation behavior for store=false streaming

First create a stored checkpoint in a conversation:

curl http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "Remember the codeword pineapple. Reply exactly: checkpoint-ok",
    "conversation": "pr-test-conversation",
    "store": true
  }'

Then send an ephemeral streaming response in the same conversation:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "This is a temporary streamed turn. Reply exactly: ephemeral-conversation-ok",
    "conversation": "pr-test-conversation",
    "stream": true,
    "store": false
  }'

Expected result:

• request succeeds
• it may read existing conversation history
• it does not update latest_completed_response_id
• it does not set active_response_id
• it is not recoverable via GET /v1/responses/{response_id}

Then send another stored request in the same conversation:

curl http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "What codeword did I ask you to remember? Reply with only the codeword.",
    "conversation": "pr-test-conversation",
    "store": true
  }'

Expected result:

• request succeeds
• conversation continues from the last stored checkpoint, not from the ephemeral streamed turn

8. Verify active conversation protection

Start a durable background stream in a conversation:

curl -N http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "Take a little time before answering.",
    "conversation": "active-conversation-test",
    "stream": true,
    "store": true
  }'

While it is still running, send another request to the same conversation:

curl -i http://127.0.0.1:8642/v1/responses \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hermes-agent",
    "input": "Follow-up while previous response is active.",
    "conversation": "active-conversation-test",
    "store": true
  }'

Expected result:

• HTTP 409
• response error code:

"conversation_response_not_completed"

This confirms that conversation ordering is protected while a durable response is active.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Ubuntu 26.04

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Changed files

• gateway/platforms/api_server.py (modified, +856/-553)
• tests/gateway/test_api_server.py (modified, +559/-98)