hermes - ✅(Solved) Fix [Bug]: openai-codex headers from PR #12664 are read off wrong SDK attribute (_default_headers vs _custom_headers), silently dropped before request [1 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#19981Fetched 2026-05-05 06:04:08
View on GitHub
Comments
2
Participants
2
Timeline
7
Reactions
0
Timeline (top)
labeled ×4commented ×2cross-referenced ×1

PR #12664 (merged 2026-04-19) added _codex_cloudflare_headers to agent/auxiliary_client.py and wired it into resolve_provider_client's raw_codex branch (line 2161 in current main). The headers are correctly attached to the OpenAI client returned by the resolver.

However the hand-off in run_agent.py reads them off the wrong attribute:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L1459-L1460

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

OpenAI Python SDK ≥ 1.x stores user-supplied default_headers=... under _custom_headers, not _default_headers. Verified live with openai==2.32.0:

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

The same wrong attribute is used at run_agent.py:1504-1505 for the fallback path, while a sibling code path at line 7662 already uses _custom_headers correctly:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L7662

So the codex headers are silently dropped between the resolver and the new client. Downstream the request goes out with only Authorization + Content-Type, the Codex backend can't determine the plan, and rejects every model with:

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

Error Message

$ jq '.error.message' ~/.hermes/sessions/request_dump_*.json | tail -1 "Error code: 400 - {'detail': "The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}"

Root Cause

PR #12664 (merged 2026-04-19) added _codex_cloudflare_headers to agent/auxiliary_client.py and wired it into resolve_provider_client's raw_codex branch (line 2161 in current main). The headers are correctly attached to the OpenAI client returned by the resolver.

However the hand-off in run_agent.py reads them off the wrong attribute:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L1459-L1460

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

OpenAI Python SDK ≥ 1.x stores user-supplied default_headers=... under _custom_headers, not _default_headers. Verified live with openai==2.32.0:

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

The same wrong attribute is used at run_agent.py:1504-1505 for the fallback path, while a sibling code path at line 7662 already uses _custom_headers correctly:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L7662

So the codex headers are silently dropped between the resolver and the new client. Downstream the request goes out with only Authorization + Content-Type, the Codex backend can't determine the plan, and rejects every model with:

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

Fix Action

Fix / Workaround

After the patch below is applied, the request goes out with originator: codex_cli_rs, User-Agent: codex_cli_rs/0.0.0 (Hermes Agent), and ChatGPT-Account-ID: ... — and the same prompt returns "PONG" cleanly.

PR fix notes

PR #19985: fix(codex): copy headers from _custom_headers (SDK >= 1.x) not _default_headers

Description (problem / solution / changelog)

Summary

PR #12664 wires _codex_cloudflare_headers into resolve_provider_client's raw_codex branch so the returned OpenAI client carries originator: codex_cli_rs, the codex_cli_rs-shaped User-Agent, and ChatGPT-Account-ID — the headers Cloudflare expects on the Codex backend. ✓

run_agent.AIAgent.__init__ then tries to copy those headers into the new client's client_kwargs via:

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

But OpenAI Python SDK 1.x+ stores user-supplied default_headers= under _custom_headers, not _default_headers. Verified live with openai==2.32.0:

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

So the lookup misses, the headers silently drop, and the request goes out with only Authorization + Content-Type. The Codex backend can't determine the plan and rejects every model:

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

A sibling code path in the same file at line 7662 already uses _custom_headers correctly — so this is just a transcription mismatch in two call sites.

Fix

Two-line replacement at run_agent.py:1459 (the openai-codex hand-off) and :1504 (the fallback path). Read _custom_headers first, fall back to _default_headers to stay compatible with older SDK versions.

Verification

Before:

$ hermes --provider openai-codex -m gpt-5.5 -z "Reply with only the word PONG."
[no output, request rejected with 400 model-not-supported]

After (same machine, same OAuth, same prompt):

$ hermes --provider openai-codex -m gpt-5.5 -z "Reply with only the word PONG."
PONG

Request_dump shows the headers are now on the wire after the patch.

Closes #19981

Changed files

  • run_agent.py (modified, +15/-5)

Code Example

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

---

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

---

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

---

$ hermes --provider openai-codex -m gpt-5.5 -z "Reply with only the word PONG."
[no output]
$ jq '.error.message' ~/.hermes/sessions/request_dump_*.json | tail -1
"Error code: 400 - {'detail': \"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account.\"}"
$ jq '.request.headers | keys' ~/.hermes/sessions/request_dump_*.json | tail -10
[
  "Authorization",
  "Content-Type"
]

---

- if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
-     client_kwargs["default_headers"] = dict(_routed_client._default_headers)
+ _routed_headers = getattr(_routed_client, '_custom_headers', None) \
+     or getattr(_routed_client, '_default_headers', None)
+ if _routed_headers:
+     client_kwargs["default_headers"] = dict(_routed_headers)
RAW_BUFFERClick to expand / collapse

Summary

PR #12664 (merged 2026-04-19) added _codex_cloudflare_headers to agent/auxiliary_client.py and wired it into resolve_provider_client's raw_codex branch (line 2161 in current main). The headers are correctly attached to the OpenAI client returned by the resolver.

However the hand-off in run_agent.py reads them off the wrong attribute:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L1459-L1460

if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
    client_kwargs["default_headers"] = dict(_routed_client._default_headers)

OpenAI Python SDK ≥ 1.x stores user-supplied default_headers=... under _custom_headers, not _default_headers. Verified live with openai==2.32.0:

>>> from openai import OpenAI
>>> c = OpenAI(api_key="x", default_headers={"A": "b"})
>>> getattr(c, "_default_headers", "<missing>")
'<missing>'
>>> c._custom_headers
{'A': 'b'}

The same wrong attribute is used at run_agent.py:1504-1505 for the fallback path, while a sibling code path at line 7662 already uses _custom_headers correctly:

https://github.com/NousResearch/hermes-agent/blob/main/run_agent.py#L7662

So the codex headers are silently dropped between the resolver and the new client. Downstream the request goes out with only Authorization + Content-Type, the Codex backend can't determine the plan, and rejects every model with:

{"detail":"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account."}

Reproduction

  • Hermes 0.12.0 (2026.4.30)
  • ChatGPT subscription via hermes auth add openai-codex --type oauth
  • Linux (Hetzner Ubuntu 24.04, x86_64), Python 3.11.15, OpenAI SDK 2.32.0
$ hermes --provider openai-codex -m gpt-5.5 -z "Reply with only the word PONG."
[no output]
$ jq '.error.message' ~/.hermes/sessions/request_dump_*.json | tail -1
"Error code: 400 - {'detail': \"The 'gpt-5.5' model is not supported when using Codex with a ChatGPT account.\"}"
$ jq '.request.headers | keys' ~/.hermes/sessions/request_dump_*.json | tail -10
[
  "Authorization",
  "Content-Type"
]

The same account works fine with the official codex CLI on the same host (gpt-5.5 returns "PONG" immediately), so the account, network, and OAuth flow are not at fault.

After the patch below is applied, the request goes out with originator: codex_cli_rs, User-Agent: codex_cli_rs/0.0.0 (Hermes Agent), and ChatGPT-Account-ID: ... — and the same prompt returns "PONG" cleanly.

Suggested fix

One-line replacement at both call sites (preserving _default_headers as a fallback for OpenAI SDK versions where the attribute name differs):

- if hasattr(_routed_client, '_default_headers') and _routed_client._default_headers:
-     client_kwargs["default_headers"] = dict(_routed_client._default_headers)
+ _routed_headers = getattr(_routed_client, '_custom_headers', None) \
+     or getattr(_routed_client, '_default_headers', None)
+ if _routed_headers:
+     client_kwargs["default_headers"] = dict(_routed_headers)

Same shape for the fallback site at run_agent.py:1504-1505.

I'm happy to open a PR if that helps; otherwise the diff is above for any maintainer who'd like to land it.

Why this didn't surface in the test suite

tests/agent/test_codex_cloudflare_headers.py checks that _codex_cloudflare_headers() returns the right dict and that it's wired into resolve_provider_client. It doesn't assert that the kwargs passed to _create_openai_client actually carry those headers (i.e. it doesn't cover the cross-module hand-off). Adding an integration test that constructs _routed_client then asserts client_kwargs["default_headers"] contains originator would prevent regressions like this from being silent.

Side-effects I noticed while debugging

  • The _codex_cloudflare_headers helper exists in auxiliary_client.py only. Inlining or making it a public helper in a shared module would discourage future hand-off mismatches.
  • The cli flag --provider <p> requires --model <m> (or HERMES_INFERENCE_MODEL). If only --provider openai-codex is passed without a model, Hermes errors out before hitting any Codex code, so the bug is invisible from a one-line provider-only smoke test.

Environment

  • hermes --version: 0.12.0 (2026.4.30)
  • Project: /home/kabooy/.hermes/hermes-agent
  • Python: 3.11.15
  • OpenAI SDK: 2.32.0
  • OS: Ubuntu 24.04 (Hetzner VPS)
  • Auth: ChatGPT subscription OAuth via hermes auth add openai-codex --type oauth

extent analysis

TL;DR

The issue can be fixed by updating the attribute name used to access the default headers in the run_agent.py file to _custom_headers for OpenAI Python SDK versions ≥ 1.x.

Guidance

  • Verify that the OpenAI Python SDK version is ≥ 1.x, as the attribute name change applies to these versions.
  • Update the run_agent.py file to use _custom_headers instead of _default_headers to access the default headers, as shown in the suggested fix.
  • Apply the same change to the fallback path at run_agent.py:1504-1505.
  • Test the updated code to ensure that the Codex headers are correctly passed to the OpenAI client.

Example

The suggested fix provides a one-line replacement for the affected code:

_routed_headers = getattr(_routed_client, '_custom_headers', None) \
    or getattr(_routed_client, '_default_headers', None)
if _routed_headers:
    client_kwargs["default_headers"] = dict(_routed_headers)

Notes

The issue is specific to OpenAI Python SDK versions ≥ 1.x, where the attribute name for default headers was changed to _custom_headers. The suggested fix provides a fallback to _default_headers for older SDK versions.

Recommendation

Apply the suggested fix to update the attribute name used to access the default headers in the run_agent.py file. This should resolve the issue and allow the Codex headers to be correctly passed to the OpenAI client.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING