Bug Description

After switching models inside an existing HermesAgent session, the session becomes unusable.

gpt-5.4 works normally before switching models. My local OpenAI-compatible AIAPI aggregation endpoint has also been tested separately and can chat with the models normally, so this does not appear to be an endpoint connectivity issue.

The issue starts after switching from gpt-5.4 to another test model. After that, HermesAgent begins failing with a retry loop. Switching back to gpt-5.4 does not recover the session.

I am using Hindsight as memory.

The visible error is:

⚠️  API call failed (attempt 6/100): IndexError
   🔌 Provider: custom  Model: gpt-5.4
   🌐 Endpoint: local AIAPI aggregator
   📝 Error: list index out of range
⏳ Retrying in 69.0s (attempt 6/100)...

Relevant environment:

HermesAgent version: Hermes Agent v0.13.0 (2026.5.7)
Project: /opt/hermes
Python: 3.13.5
OpenAI SDK: 2.33.0
Runtime: Docker container
Working directory: /opt/hermes
Provider: custom:aiapi
Endpoint: local OpenAI-compatible AIAPI aggregation service
Default model: gpt-5.4
Memory: Hindsight
Interface: CLI / gateway

Relevant config output:

◆ Model
  Model:        {'provider': 'custom:aiapi', 'default': 'gpt-5.4'}
  Reasoning:    off
  Model:        gpt-5.4
  Provider:     custom:aiapi

One suspicious detail is that config says reasoning is off, but the actual request payload still appears to contain reasoning-related fields:

"reasoning": {
  "effort": "xhigh",
  "summary": "auto"
},
"include": [
  "reasoning.encrypted_content"
],
"tool_choice": "auto",
"parallel_tool_calls": true

The request also contains Hindsight memory tools:

hindsight_retain
hindsight_recall
hindsight_reflect

This looks like a model-switching/session-state/request-construction issue rather than a raw model or endpoint failure.

Steps to Reproduce

1. Run HermesAgent v0.13.0 in Docker.

2. Configure a custom OpenAI-compatible provider pointing to a local AIAPI aggregation service:

   Provider: custom:aiapi
   Endpoint: local AIAPI aggregation service
   Default model: gpt-5.4

3. Enable/use Hindsight memory.

4. Start a Hermes session using gpt-5.4.

5. Confirm that gpt-5.4 can chat normally.

6. Switch from gpt-5.4 to another test model inside the existing session.

7. Send a normal message.

8. HermesAgent starts failing/retrying.

9. Switch back to gpt-5.4.

10. Send another normal message.

11. HermesAgent still fails and retries instead of recovering.

Expected Behavior

After switching models, HermesAgent should either:

1. continue the conversation normally with the selected model,
2. clear/sanitize model-specific session state before sending the next request, or
3. return a clear compatibility/configuration error.

Switching back to a previously working model such as gpt-5.4 should recover the session if the endpoint and model are working.

HermesAgent should not retry the same failing request up to 100 times with only:

IndexError: list index out of range

Actual Behavior

HermesAgent enters a retry loop:

⚠️  API call failed (attempt 6/100): IndexError
   🔌 Provider: custom  Model: gpt-5.4
   🌐 Endpoint: local AIAPI aggregator
   📝 Error: list index out of range
⏳ Retrying in 69.0s (attempt 6/100)...

After interrupting:

⚡ Interrupting agent... (press Ctrl+C again to force exit)
⚡ Interrupt detected during retry wait, aborting.

⚕ gpt-5.4 │ 0/1.1M │ [░░░░░░░░░░] 0% │ 3m │ ⏱ 2m 52s

After force exit:

Resume this session with:
  hermes --resume 20260514_083429_b52751

Session:        20260514_083429_b52751
Duration:       3m 20s
Messages:       1 (1 user, 0 tool calls)

There are also unrelated Telegram gateway warnings in the logs, such as:

telegram.error.NetworkError: httpx.ConnectError
[Telegram] Telegram polling reconnect failed: httpx.ConnectError

Those appear separate from the agent API failure. The model failure happens in the agent API call path and shows:

API call failed
Provider: custom
Model: gpt-5.4
Error: list index out of range

Affected Component

Agent Core (conversation loop, context compression, memory)

Messaging Platform (if gateway-related)

Telegram

Debug Report

Report       https://paste.rs/3b5kB
  agent.log    https://paste.rs/gTHkw
  gateway.log  https://paste.rs/9xqUA

Operating System

Ubuntu 24.04 in Docker

Python Version

3.13.5

Hermes Version

Hermes Agent v0.13.0 (2026.5.7)

Additional Logs / Traceback (optional)

Additional visible runtime error:


⚠️  API call failed (attempt 6/100): IndexError
   🔌 Provider: custom  Model: gpt-5.4
   🌐 Endpoint: local AIAPI aggregator
   📝 Error: list index out of range
⏳ Retrying in 69.0s (attempt 6/100)...


The local AIAPI aggregation endpoint was tested separately and works normally. `gpt-5.4` also works before switching models. The failure starts after switching from `gpt-5.4` to another test model, and switching back to `gpt-5.4` does not recover the session.

Root Cause Analysis (optional)

I have not identified the exact source line.

My current hypothesis is that HermesAgent keeps stale model-specific session/request state after model switching. This may interact with Hindsight memory tools and reasoning/encrypted reasoning fields.

One suspicious detail is that config reports:

Reasoning: off

but the actual request payload appears to include:

"reasoning": {
  "effort": "xhigh",
  "summary": "auto"
},
"include": [
  "reasoning.encrypted_content"
]

So the issue may be related to stale reasoning state, request construction, or response parsing after a model switch.

Proposed Fix (optional)

Possible areas to check:

1. Clear or sanitize model-specific session state when switching models.
2. Avoid sending reasoning / include: ["reasoning.encrypted_content"] when config says reasoning is off.
3. Avoid carrying stale encrypted reasoning / thinking state across model switches.
4. Surface the raw provider response or malformed response instead of only IndexError: list index out of range.
5. Avoid retrying the same malformed request up to 100 times.
6. Check whether Hindsight memory tools interact badly with model switching and custom OpenAI-compatible providers.
7. Make switching back to a previously working model recover the session, or clearly explain that the existing session must be reset.

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR