hermes - ✅(Solved) Fix openai-codex fallback sends 'gpt-5-4' instead of 'gpt-5.4' to Codex backend, returning HTTP 404 [2 pull requests, 2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#17171Fetched 2026-04-29 06:36:55
View on GitHub
Comments
2
Participants
2
Timeline
8
Reactions
0
Timeline (top)
labeled ×4commented ×2cross-referenced ×2

When the Hermes fallback chain activates openai-codex with model gpt-5.4 (the documented Codex chat model), the request reaches the ChatGPT Codex backend with model name gpt-5-4 (dots replaced with hyphens) and gets HTTP 404. The Codex /responses endpoint accepts only gpt-5.4 (with the dot).

The conversion is not in hermes_cli/model_normalize.pynormalize_model_for_provider("gpt-5.4", "openai-codex") returns "gpt-5.4" correctly. The dot→hyphen rewrite is happening downstream of normalization, somewhere in the codex_responses request-build / fallback-activation path.

Error Message

🔄 Primary model failed — switching to fallback: gpt-5.4 via openai-codex ⚠️ API call failed (attempt 1/3): NotFoundError [HTTP 404] 🔌 Provider: openai-codex Model: gpt-5.4 🌐 Endpoint: https://chatgpt.com/backend-api/codex 📝 Error: HTTP 404: model: gpt-5-4

Root Cause

If a single root cause lurks, fixing one might fix all four.

Fix Action

Workaround

None on the user side. The model name gpt-5-4 doesn't exist in Codex; gpt-5.4 is the canonical value. Direct API calls work; only Hermes-routed ones fail.

PR fix notes

PR #17282: fix(anthropic_adapter): only mangle dots in Claude-shaped model IDs (#17171)

Description (problem / solution / changelog)

Summary

  • normalize_model_name blindly rewrote dots to hyphens for every non-Bedrock model, corrupting non-Anthropic vendor IDs whose dots are version separators (gpt-5.4gpt-5-4, glm-4.7glm-4-7, z-ai/glm-5.1z-ai/glm-5-1, minimax-m2.5-freeminimax-m2-5-free).
  • After the prefix-strip and Bedrock-detection guards, the rewrite is now gated on the model containing claude (case-insensitive). Claude callers are unaffected; non-Claude IDs survive intact even when preserve_dots=False.

The bug

The dot→hyphen rewrite is an Anthropic-Messages-API quirk: OpenRouter form claude-opus-4.6 must become Anthropic form claude-opus-4-6. The function applied it unconditionally to every model except Bedrock IDs, so a non-Claude model accidentally routed through the Anthropic adapter — typically because the caller defaulted to provider=anthropic after failing to infer the right provider — got its name mangled into a non-existent ID and 404'd at the API. The user-facing failure named a model they never typed (gpt-5-4), making the misconfiguration hard to recognize.

The reporter's repro (#17171):

  1. webui session created with {"model": "gpt-5.4"} and no provider hint.
  2. resolve_model_provider("gpt-5.4") returns (gpt-5.4, None, None) — no inference from the gpt-* shape.
  3. resolve_runtime_provider(requested=None) falls back to the default-detected anthropic provider.
  4. AIAgent is built with provider="anthropic", api_mode="anthropic_messages", model="gpt-5.4".
  5. normalize_model_name("gpt-5.4", preserve_dots=False) returns "gpt-5-4".
  6. Request goes to api.anthropic.com/v1/messages with model: gpt-5-4 → HTTP 404.

Same root cause underlies #16417 (custom anthropic_messages providers) and #13061 (zenmux + custom anthropic_messages). The opencode-zen subset (#7421) was previously mitigated by adding the provider to the _anthropic_preserve_dots() allowlist, but the underlying invariant — only Anthropic-shaped names need Anthropic-shaped normalization — was never enforced at the function itself.

The fix

Narrow the rewrite to Claude-shaped IDs:

if not preserve_dots:
    if _is_bedrock_model_id(model):
        return model
    if "claude" not in lower:        # ← new
        return model
    model = model.replace(".", "-")
  • Claude users are unaffected (claude-opus-4.6claude-opus-4-6 still works, mixed case Claude-Opus-4.6 too).
  • The explicit preserve_dots=True flag still short-circuits before the new check, so the documented contract for the alibaba / minimax / opencode-zen / zai / bedrock allowlist is preserved.
  • The default preserve_dots=False path becomes safe for non-Claude IDs that get routed through the Anthropic adapter — they round-trip verbatim instead of being silently corrupted.

Contract Protected

Invariant: the dot→hyphen rewrite in normalize_model_name only fires when both (a) the caller has not opted into preserve_dots=True and (b) the model name shape is Claude-like (contains claude after the optional anthropic/ prefix strip).

Input classExampleBeforeAfter
Claude bareclaude-opus-4.6claude-opus-4-6claude-opus-4-6
Claude OpenRouteranthropic/claude-opus-4.6claude-opus-4-6claude-opus-4-6
Claude mixed caseClaude-Opus-4.6Claude-Opus-4-6Claude-Opus-4-6
Bedrock inference profileglobal.anthropic.claude-opus-4-7global.anthropic.claude-opus-4-7global.anthropic.claude-opus-4-7
OpenAI Codexgpt-5.4gpt-5-4gpt-5.4 ✅ (#17171)
ZAIglm-4.7glm-4-7glm-4.7 ✅ (#16417)
zenmux/GLMz-ai/glm-5.1z-ai/glm-5-1z-ai/glm-5.1 ✅ (#13061)
MiniMaxMiniMax-M2.7MiniMax-M2-7MiniMax-M2.7
Geminigoogle/gemini-2.5-progoogle/gemini-2-5-progoogle/gemini-2.5-pro

Test plan

  • Parametrized regression TestNormalizeModelNameNonClaudeInvariant::test_non_claude_model_ids_preserve_dots[…] for nine known-bad IDs from #17171, #16417, #13061, #7421.
  • Negative invariant test_negative_claude_dotted_still_mangled: Claude IDs continue to be rewritten; mixed-case detection works.
  • Reporter repro test_repro_17171_default_anthropic_provider_with_gpt_model: gpt-5.4 and anthropic/gpt-5.4 both round-trip.
  • preserve_dots=True still authoritative for non-Claude callers (test_preserve_dots_true_remains_authoritative_for_non_claude).
  • Updated test_minimax_provider.py::test_normalize_converts_without_preserve — its docstring previously labelled the behavior "broken for MiniMax"; it now documents the fix.
  • tests/agent/test_bedrock_integration.py (55 tests) all pass — Bedrock detection still wins before the new Claude check.
  • tests/agent/test_minimax_provider.py (41 tests) all pass.
  • Regression guard: reverting the production change made 10/12 invariant tests fail with the expected gpt-5-4/MiniMax-M2-7/etc. outputs; restoring the fix flipped them back to green.

Related

Fixes #17171 Related: #16417, #13061, #7421 (mitigated earlier via _anthropic_preserve_dots() allowlist; this PR enforces the same outcome at the normalization layer for vendors not on that allowlist)

Changed files

  • agent/anthropic_adapter.py (modified, +17/-0)
  • tests/agent/test_anthropic_adapter.py (modified, +74/-0)
  • tests/agent/test_minimax_provider.py (modified, +9/-3)

PR #17290: fix: narrow Anthropic adapter dot-mangling to Claude models only (#17171)

Description (problem / solution / changelog)

Problem

normalize_model_name() in agent/anthropic_adapter.py unconditionally converts dots to hyphens in all model names (line 1117). This causes non-Anthropic models to be mangled when routed through the Anthropic adapter path:

  • gpt-5.4gpt-5-4 → HTTP 404 from Codex backend
  • gemini-2.5-progemini-2-5-pro → would fail similarly

This affects the webui session route when resolve_model_provider() doesn't infer the provider from the model name, and the Anthropic adapter is auto-selected as default.

Fix

Added a guard in normalize_model_name() to only apply dot→hyphen conversion for models that look like Anthropic models (claude-* or anthropic/*). Non-Anthropic models pass through unchanged.

Before:

model = model.replace(".", "-")  # ALL models

After:

_lower = model.lower()
if _lower.startswith("claude-") or _lower.startswith("anthropic/"):
    model = model.replace(".", "-")  # Only Anthropic models

Before vs After

ModelBeforeAfter
claude-opus-4.6claude-opus-4-6claude-opus-4-6
anthropic/claude-sonnet-4.5claude-sonnet-4-5claude-sonnet-4-5
gpt-5.4gpt-5-4gpt-5.4
gemini-2.5-progemini-2-5-progemini-2.5-pro
Bedrock IDspreserved (existing guard)preserved (unchanged)

Tests

All existing tests pass — they all use Claude model names which still get dot→hyphen conversion.

Fixes #17171 Related: #7421, #13061, #16417

Changed files

  • agent/anthropic_adapter.py (modified, +6/-3)

Code Example

model: claude-haiku-4-5-20251001
  fallback_providers:
    - provider: anthropic
      model: claude-sonnet-4-6
    - provider: openai-codex
      model: gpt-5.4
    - provider: openrouter
      model: meta-llama/llama-3.3-70b-instruct:free

---

🔄 Primary model failed — switching to fallback: gpt-5.4 via openai-codex
⚠️  API call failed (attempt 1/3): NotFoundError [HTTP 404]
   🔌 Provider: openai-codex  Model: gpt-5.4
   🌐 Endpoint: https://chatgpt.com/backend-api/codex
   📝 Error: HTTP 404: model: gpt-5-4

---

TOKEN=$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('~/.hermes/auth.json'))); print(d['credential_pool']['openai-codex'][0]['access_token'])")

curl -sN -X POST https://chatgpt.com/backend-api/codex/responses \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "OpenAI-Beta: responses=experimental" \
  -d '{"model":"gpt-5.4","input":[{"role":"user","content":"Reply: WORKING"}],"instructions":"helpful","stream":true,"store":false}'
# → SSE stream, response.created with model: "gpt-5.4", deltas "WORK" + "ING"

---

{"detail":"The 'gpt-5-codex' model is not supported when using Codex with a ChatGPT account."}
RAW_BUFFERClick to expand / collapse

Summary

When the Hermes fallback chain activates openai-codex with model gpt-5.4 (the documented Codex chat model), the request reaches the ChatGPT Codex backend with model name gpt-5-4 (dots replaced with hyphens) and gets HTTP 404. The Codex /responses endpoint accepts only gpt-5.4 (with the dot).

The conversion is not in hermes_cli/model_normalize.pynormalize_model_for_provider("gpt-5.4", "openai-codex") returns "gpt-5.4" correctly. The dot→hyphen rewrite is happening downstream of normalization, somewhere in the codex_responses request-build / fallback-activation path.

Environment

  • Hermes Agent: latest (post hermes update 2026-04-28)
  • macOS 15.x arm64
  • Provider chain (config.yaml):
    model: claude-haiku-4-5-20251001
    fallback_providers:
      - provider: anthropic
        model: claude-sonnet-4-6
      - provider: openai-codex
        model: gpt-5.4
      - provider: openrouter
        model: meta-llama/llama-3.3-70b-instruct:free
  • Codex auth: working (token freshly refreshed via Codex CLI, pool entry last_status: ok)
  • Codex backend: https://chatgpt.com/backend-api/codex (default)

Reproduction

  1. Configure openai-codex with model gpt-5.4 anywhere in the chain.
  2. Trigger fallback (e.g. by sending a request when the primary is rate-limited, or by setting primary to gpt-5.4 directly).
  3. Observe gateway log:
🔄 Primary model failed — switching to fallback: gpt-5.4 via openai-codex
⚠️  API call failed (attempt 1/3): NotFoundError [HTTP 404]
   🔌 Provider: openai-codex  Model: gpt-5.4
   🌐 Endpoint: https://chatgpt.com/backend-api/codex
   📝 Error: HTTP 404: model: gpt-5-4

Note the discrepancy: log line Model: gpt-5.4 (the configured value), but error message from the backend says gpt-5-4 — the wire-level model name was rewritten.

Confirmation that Codex backend itself works

Same machine, same OAuth token, same minute — direct curl to the Codex backend with gpt-5.4 returns 200:

TOKEN=$(python3 -c "import json,os; d=json.load(open(os.path.expanduser('~/.hermes/auth.json'))); print(d['credential_pool']['openai-codex'][0]['access_token'])")

curl -sN -X POST https://chatgpt.com/backend-api/codex/responses \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "OpenAI-Beta: responses=experimental" \
  -d '{"model":"gpt-5.4","input":[{"role":"user","content":"Reply: WORKING"}],"instructions":"helpful","stream":true,"store":false}'
# → SSE stream, response.created with model: "gpt-5.4", deltas "WORK" + "ING"

Same payload with gpt-5 or gpt-5-codex:

{"detail":"The 'gpt-5-codex' model is not supported when using Codex with a ChatGPT account."}

So the backend strictly accepts gpt-5.4 (with the dot) and rejects the dash form.

What I've ruled out

  • model_normalize.py: I verified normalize_model_for_provider("gpt-5.4", "openai-codex") returns "gpt-5.4". openai-codex is in _STRIP_VENDOR_ONLY_PROVIDERS (dots preserved), not _DOT_TO_HYPHEN_PROVIDERS.
  • agent/anthropic_adapter.py: contains model.replace(".", "-") calls but only on the Anthropic path.
  • agent/model_metadata.py:1057: contains return model.replace(".", "-") but it's in a comparison helper, not the wire path.

Suspected location

Somewhere in the codex_responses fallback path:

  • run_agent.py _try_activate_fallback → constructs the responses request body
  • agent/auxiliary_client.py CodexAuxiliaryClient / resolve_provider_client Codex branch
  • Or a model-comparison helper that normalizes both sides for matching but accidentally returns the normalized form to the wire layer

I don't have a definitive line number. Adding print statements just before the client.responses.stream() call would localize it quickly.

Related issues

These are not duplicates but share the same dot→hyphen rewrite pattern affecting different providers / paths:

  • #7421 — opencode-zen incorrectly converts dots to hyphens
  • #13061 — normalize_model_name rewrites custom provider model IDs with dots
  • #16417 — custom anthropic_messages providers rewrite model names (.-)

If a single root cause lurks, fixing one might fix all four.

Impact

Anyone configuring openai-codex with gpt-5.4 (or any GPT-5.x with a dot) as a primary or fallback gets a 100% failure rate on that hop. Especially painful right now because Anthropic Sonnet/Opus on subscription OAuth is currently being gated by Anthropic (#17169), making openai-codex the natural backstop — which then 404s.

Workaround

None on the user side. The model name gpt-5-4 doesn't exist in Codex; gpt-5.4 is the canonical value. Direct API calls work; only Hermes-routed ones fail.

What might fix it

  • Find the offending .replace(".", "-") and gate it behind a provider check.
  • Or replace it with the centralized normalize_model_for_provider so the "preserve dots for _STRIP_VENDOR_ONLY_PROVIDERS" rule applies.

Happy to provide gateway logs, full request dumps, or run a focused trace.

extent analysis

TL;DR

The issue can be fixed by finding and modifying the code that incorrectly replaces dots with hyphens in the model name for the openai-codex provider.

Guidance

  1. Investigate the codex_responses fallback path: Look into run_agent.py _try_activate_fallback, agent/auxiliary_client.py CodexAuxiliaryClient / resolve_provider_client Codex branch, and model-comparison helpers to find where the dot-to-hyphen replacement is happening.
  2. Add debug logging: Insert print statements before the client.responses.stream() call to localize the issue quickly.
  3. Replace or gate the offending replacement: Once found, replace the .replace(".", "-") call with the centralized normalize_model_for_provider function or gate it behind a provider check to preserve dots for _STRIP_VENDOR_ONLY_PROVIDERS.
  4. Verify the fix: Test the openai-codex provider with the gpt-5.4 model to ensure it no longer returns a 404 error.

Example

No code example is provided as the exact location of the issue is still unknown. However, the fix will likely involve modifying a line similar to model = model.replace(".", "-") to use the normalize_model_for_provider function or adding a conditional check for the openai-codex provider.

Notes

The root cause of the issue is not yet identified, but it is suspected to be in the codex_responses fallback path. The fix will require finding and modifying the specific line of code that is causing the dot-to-hyphen replacement.

Recommendation

Apply a workaround by finding and modifying the offending code to correctly handle the openai-codex provider model name. This will require investigating the codex_responses fallback path and replacing or gating the dot-to-hyphen replacement.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING