PR #25206: fix(pool): re-select from credential pool on primary runtime restore

• Repository: NousResearch/hermes-agent
• Author: jmmaloney4
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/25206

Description (problem / solution / changelog)

Summary

Fixes #25205.

_restore_primary_runtime restores api_key from a construction-time snapshot. When the credential pool has rotated (exhaustion, revocation, rate-limit cooldown), the snapshot key is stale. The next turn immediately hits the same error, re-exhausts entries, and falls through to cross-provider fallback.

After restoring the snapshot, re-select from the credential pool if one exists. Falls back gracefully to the snapshot key when the pool is absent or empty.

Test plan

• 7 new test cases in test_restore_primary_pool_reselect.py
  • Rotation picks new entry (not stale snapshot key)
  • Pool absent → snapshot key used (existing behavior)
  • Pool empty (all exhausted) → snapshot key used
  • Entry with empty key → snapshot key used
  • Base URL updated from pool entry
  • Client rebuild triggered after reselect
• 53 existing credential pool tests pass
• Manual: run multi-turn session with codex pool, verify restore log shows "Restore re-selected pool entry" and fallback does NOT activate on next turn

Risks

• Low: pool.select() is idempotent and already used at session init. The restore path just calls it again.
• If select() fails or returns None, we fall back to the snapshot key (same as pre-fix behavior).

Changed files

• run_agent.py (modified, +60/-0)
• tests/agent/test_restore_primary_pool_reselect.py (added, +222/-0)

---

PR #25277: fix: load credential pool in switch_model for new provider (#25273)

• Repository: NousResearch/hermes-agent
• Author: jmmaloney4
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/25277

Description (problem / solution / changelog)

Problem

When switching providers mid-session via /model (e.g., from the default zai provider to openai-codex), switch_model() updates model, provider, api_key, base_url, api_mode, and rebuilds the client — but never loads the credential pool for the new provider.

The agent's _credential_pool stays as whatever was loaded at construction time (typically None for providers like zai that use a single API key from env). This silently disables pool rotation on the new provider:

1. All 429/401/402 errors burn the same credential with no rotation
2. After exhausting retries, the agent falls through to cross-provider fallback
3. Zero pool rotation log entries appear (pool is None, recovery bails immediately)

This is why a user with 3 openai-codex OAuth credentials (where one has usage available) still hits rate limits — the pool never rotates because it was never loaded.

Closes #25273

Fix

run_agent.py — switch_model() (line ~2659)

• Call load_pool(new_provider) after swapping runtime fields, before client rebuild
• Set self._credential_pool to the loaded pool (or None if loading fails or pool has no credentials)
• Wrapped in try/except to avoid breaking model switch if pool loading fails

run_agent.py — _primary_runtime snapshot in switch_model() (line ~2770)

• Include credential_pool in the snapshot dict so it survives _restore_primary_runtime() across turns

run_agent.py — _restore_primary_runtime() (line ~8895)

• Restore credential_pool from snapshot when present and not None
• Older snapshots that predate this field are handled gracefully (pool unchanged)

Test plan

8 new tests in tests/agent/test_switch_model_credential_pool.py:

• test_switch_loads_pool_for_new_provider — verifies load_pool is called
• test_switch_sets_pool_none_when_no_credentials — pool=None when no creds
• test_switch_handles_load_pool_exception — pool=None on load error
• test_switch_includes_pool_in_primary_runtime — snapshot has pool
• test_switch_same_provider_keeps_pool — re-load on same provider
• test_restore_primary_preserves_pool_from_snapshot — restore path works
• test_restore_primary_keeps_pool_when_snapshot_has_none — None guard
• test_restore_primary_keeps_pool_when_snapshot_predates_field — backward compat

All 53 existing credential pool tests still pass.

Risks

• Low: load_pool is called once during switch_model. If it fails (e.g., corrupted auth.json), the pool is set to None and the agent operates without pool rotation — same behavior as before the fix.
• The _primary_runtime snapshot now holds a reference to the pool object. Since the pool is a singleton per provider (loaded from auth.json), this is a shared reference, not a copy. State changes to the pool (e.g., exhaustion) are reflected in both the agent and the snapshot.

Related

• #24173 — Codex soft-failure pool rotation (response_invalid block)
• #25205 — _restore_primary_runtime uses stale snapshot key
• #24159 — Codex Responses API soft failures bypass pool rotation

Changed files

• run_agent.py (modified, +24/-0)
• tests/agent/test_switch_model_credential_pool.py (added, +302/-0)