Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix [Bug] _PROVIDER_MODELS["xai"] is stale — grok-4.20-reasoning renamed; broader fragility from hardcoded provider lists [2 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#16699•Fetched 2026-04-28 06:51:30
View on GitHub
Comments
1
Participants
2
Timeline
7
Reactions
0
Author
tyler-b4innovatyler-b4innova
Participants
tyler-b4innovatyler-b4innovavominh1919vominh1919
Timeline (top)
labeled ×4cross-referenced ×2commented ×1

The hardcoded _PROVIDER_MODELS["xai"] list in hermes_cli/models.py is stale, and the pattern of hardcoded per-provider model lists is structurally fragile across the file.

Concretely:

# hermes_cli/models.py (current main: 4a9ac5c3)
_PROVIDER_MODELS: dict[str, list[str]] = {
    ...
    "xai": [
        "grok-4.20-reasoning",        # ← xAI no longer accepts this name
        "grok-4-1-fast-reasoning",    # ← never existed in xAI's catalog
    ],
    ...
}

xAI's actual current model IDs (per models.dev cache, fetched today) use date-suffixed names like grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning, grok-4.20-multi-agent-0309. None of those match the hardcoded list.

Error Message

except Exception:

Root Cause

_PROVIDER_MODELS hardcodes per-provider lists that don't track upstream catalogs. Of the ~15 providers in this dict, only openai-codex is currently sourced dynamically (_codex_curated_models(), added in PR #7844 to fix the analogous bug #6595). Every other provider — xai, nous, openai, copilot, gemini, zai, nvidia, kimi-coding, stepfun, moonshot, minimax, minimax-cn, etc. — keeps drifting silently.

The drift surfaces in two places:

  1. The /model picker (this issue, also #14057, #16161)
  2. detect_provider_for_model() validation, which uses _PROVIDER_MODELS membership and emits "Could not reach … to validate" warnings for any model not in the curated list (same false-warning bug as #6595).

Fix Action

Workaround

Patch locally by replacing the xai literal with a _xai_curated_models() helper modeled on _codex_curated_models(). We're maintaining a small Ansible playbook that re-applies this on each managed-host install; happy to share if helpful.

PR fix notes

PR #16734: fix(models): add xai to _MODELS_DEV_PREFERRED; refresh stale curated IDs (#16699)

Description (problem / solution / changelog)

Summary

  • Add "xai" to _MODELS_DEV_PREFERRED so the /model picker merges fresh models.dev entries on top of curated, matching the existing pattern for opencode-go, deepseek, zai, gemini, etc.
  • Replace stale curated entries (grok-4.20-reasoning, grok-4-1-fast-reasoning — both 400 on selection) with current valid IDs taken from the live models.dev xai catalog (filtered to tool_call=True).
  • Add regression tests against provider_model_ids("xai") for both the live-merge and offline-fallback paths.

The bug

_PROVIDER_MODELS["xai"] in hermes_cli/models.py contains two slugs that no longer match xAI's catalog:

"xai": [
    "grok-4.20-reasoning",       # ← xAI no longer accepts this name
    "grok-4-1-fast-reasoning",   # ← never existed in xAI's catalog
],

Reproduction (from the issue, verified against live xAI catalog):

  1. Configure xAI provider via XAI_API_KEY and base https://api.x.ai/v1.

  2. Run hermes, open /model, select xAI, pick either entry.

  3. Result:

    HTTP 400: Unknown Model, please check the model code.

Verified against a fresh https://models.dev/api.json fetch — the xai provider's tool_call=True IDs are date-suffixed: grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning, plus grok-4-fast, grok-4-fast-non-reasoning, grok-code-fast-1, etc. None of those match the curated list.

The fix

Two changes — each is incomplete on its own:

  1. Add "xai" to _MODELS_DEV_PREFERRED. xAI's catalog drifts (the grok-4.20 family was renamed to date-suffixed IDs upstream). Adding xai to the preferred set causes provider_model_ids and the gateway /model picker to merge fresh models.dev entries on top of the curated list — the exact pattern already in place for opencode-go, opencode-zen, deepseek, zai, gemini, etc. Without this, refreshing the curated list just locks us into another stale snapshot the next time xAI renames a model.

  2. Refresh the curated fallback to current valid IDs. This list is the offline / restricted-network fallback (when models.dev is unreachable, this is the only catalog users see), so it has to contain real current IDs that don't 400 on selection. New entries are pulled from the live models.dev xai catalog filtered to tool_call=True:

    "xai": [
    "grok-4.20-0309-reasoning",
    "grok-4.20-0309-non-reasoning",
    "grok-4-fast",
    "grok-4-fast-non-reasoning",
    "grok-code-fast-1",
],

xAI doesn't fit either documented exclusion (openrouter — too many models to dump; nous — Portal /models endpoint is the source of truth), so adding it is consistent with the existing block comment.

Test plan

  • tests/hermes_cli/test_models_dev_preferred_merge.py — 14 passed (3 new):

    • test_xai_is_preferred — xai is in _MODELS_DEV_PREFERRED.
    • test_xai_includes_fresh_models_dev_entries — fresh models.dev entries surface (the picker bug from #16699).
    • test_xai_offline_falls_back_to_curated — curated floor contains current IDs, excludes the two stale ones.

  • Adjacent suite: test_model_catalog, test_user_providers_model_switch, test_api_key_providers, test_model_validation, test_model_metadata — 354 passed.

  • Regression guard: ran the 3 new tests against unmodified hermes_cli/models.py (only test file diffed) — all 3 fail with the exact reporter symptom:

    AssertionError: assert 'grok-4.20-0309-reasoning' in ['grok-4.20-reasoning', 'grok-4-1-fast-reasoning']

    After applying the source fix, all 3 pass.

Notes

  • Doc reference to /model grok-4-1-fast-reasoning in website/docs/integrations/providers.md and the test fixture in tests/agent/test_model_metadata.py (substring-matching context-length lookup) are intentionally not touched here — both are independent of the picker behaviour and would broaden scope. Worth a small follow-up.
  • The VERCEL_AI_GATEWAY_MODELS and OpenRouter slug entries (xai/grok-4.20-reasoning, x-ai/grok-4.20) are gateway-specific aliases, not direct xAI API IDs, so they are left alone here.

Related

  • Fixes #16699
  • Same shape as the closed openai-codex precedent #6595 → PR #7844.

Changed files

  • hermes_cli/models.py (modified, +9/-2)
  • tests/hermes_cli/test_models_dev_preferred_merge.py (modified, +38/-0)

PR #16818: fix(models): update stale xAI model list (#16699)

Description (problem / solution / changelog)

Summary

The hardcoded _PROVIDER_MODELS["xai"] list in hermes_cli/models.py is stale. The previous entries (grok-4.20-reasoning, grok-4-1-fast-reasoning) no longer exist in xAI's catalog, causing HTTP 400 "Unknown Model" errors when users select them from the /model picker.

Root Cause

xAI renamed their models. The old names (grok-4.20-reasoning, grok-4-1-fast-reasoning) are no longer accepted.

Fix

Updated to current xAI model IDs (verified via OpenRouter catalog):

OldNewContext
grok-4.20-reasoninggrok-4.202M
grok-4-1-fast-reasoninggrok-4.1-fast2M
—grok-4.20-multi-agent2M
—grok-4-fast2M
—grok-4256K
—grok-code-fast-1256K

File changed: hermes_cli/models.py (6 lines changed in _PROVIDER_MODELS["xai"])

Testing

Verified model IDs against OpenRouter's live catalog (/api/v1/models).

Fixes #16699

Changed files

  • hermes_cli/models.py (modified, +6/-2)

Code Example

# hermes_cli/models.py (current main: 4a9ac5c3)
_PROVIDER_MODELS: dict[str, list[str]] = {
    ...
    "xai": [
        "grok-4.20-reasoning",        # ← xAI no longer accepts this name
        "grok-4-1-fast-reasoning",    # ← never existed in xAI's catalog
    ],
    ...
}

---

HTTP 400: Unknown Model, please check the model code.
RAW_BUFFERClick to expand / collapse

Summary

The hardcoded _PROVIDER_MODELS["xai"] list in hermes_cli/models.py is stale, and the pattern of hardcoded per-provider model lists is structurally fragile across the file.

Concretely:

# hermes_cli/models.py (current main: 4a9ac5c3)
_PROVIDER_MODELS: dict[str, list[str]] = {
    ...
    "xai": [
        "grok-4.20-reasoning",        # ← xAI no longer accepts this name
        "grok-4-1-fast-reasoning",    # ← never existed in xAI's catalog
    ],
    ...
}

xAI's actual current model IDs (per models.dev cache, fetched today) use date-suffixed names like grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning, grok-4.20-multi-agent-0309. None of those match the hardcoded list.

Reproduction

  1. Configure xAI provider via env: XAI_API_KEY=…
  2. Run hermes and select xAI from /model picker
  3. Picker shows only the 2 stale entries; selecting either yields:
HTTP 400: Unknown Model, please check the model code.

Same shape as the open Z.AI bug #7922.

Root cause

_PROVIDER_MODELS hardcodes per-provider lists that don't track upstream catalogs. Of the ~15 providers in this dict, only openai-codex is currently sourced dynamically (_codex_curated_models(), added in PR #7844 to fix the analogous bug #6595). Every other provider — xai, nous, openai, copilot, gemini, zai, nvidia, kimi-coding, stepfun, moonshot, minimax, minimax-cn, etc. — keeps drifting silently.

The drift surfaces in two places:

  1. The /model picker (this issue, also #14057, #16161)
  2. detect_provider_for_model() validation, which uses _PROVIDER_MODELS membership and emits "Could not reach … to validate" warnings for any model not in the curated list (same false-warning bug as #6595).

Suggested fix

Short-term: apply the _codex_curated_models() pattern to xai (and the other most-active providers). For xai specifically, derive from the existing models_dev_cache.json that hermes already maintains:

```python def _xai_curated_models() -> list[str]: import json, os cache = os.path.expanduser("~/.hermes/models_dev_cache.json") try: if os.path.isfile(cache): with open(cache) as fh: data = json.load(fh) ids = list(data.get("xai", {}).get("models", {}).keys()) if ids: return sorted(ids) except Exception: pass return [...static fallback...]

_PROVIDER_MODELS["xai"] = _xai_curated_models() ```

Longer-term: the pattern of per-provider helpers will keep growing. Worth considering a single _provider_models(provider: str) resolver that:

  • Reads models_dev_cache.json for any provider available there (covers most),
  • Falls back to provider-specific helpers (e.g. _codex_curated_models) where the cache isn't authoritative,
  • Falls back to a small per-provider static list only as a last resort.

Related: the model-allowlist feature request #16608 would let users opt into a known-good subset rather than relying on the curation embedded in source.

Workaround

Patch locally by replacing the xai literal with a _xai_curated_models() helper modeled on _codex_curated_models(). We're maintaining a small Ansible playbook that re-applies this on each managed-host install; happy to share if helpful.

Environment

  • Hermes Agent main @ 4a9ac5c3 (2026-04-27)
  • Python 3.12, Ubuntu 24.04 LTS
  • Provider: xAI direct (XAI_API_KEY, https://api.x.ai/v1)

References

  • #6595 (closed → PR #7844, the openai-codex precedent)
  • #7922 (open, same shape for Z.AI: stored slug → HTTP 400 Unknown Model)
  • #14057 (open, duplicate provider entries in picker)
  • #16161 (open, missing GPT-5.5 across catalog)
  • #16608 (open, model allowlist feature request)

extent analysis

TL;DR

Apply the _codex_curated_models() pattern to xai by deriving model IDs from the existing models_dev_cache.json to fix the stale model list issue.

Guidance

  • Identify the models_dev_cache.json file and verify it contains the current model IDs for xai.
  • Implement the _xai_curated_models() function to read model IDs from the cache file and update the _PROVIDER_MODELS dictionary accordingly.
  • Consider implementing a single _provider_models(provider: str) resolver to handle model IDs for all providers, reducing code duplication and improving maintainability.
  • Test the updated code to ensure the /model picker displays the correct model IDs and selection yields the expected results.

Example

def _xai_curated_models() -> list[str]:
    import json, os
    cache = os.path.expanduser("~/.hermes/models_dev_cache.json")
    try:
        if os.path.isfile(cache):
            with open(cache) as fh:
                data = json.load(fh)
            ids = list(data.get("xai", {}).get("models", {}).keys())
            if ids:
                return sorted(ids)
    except Exception:
        pass
    return [...]  # static fallback

Notes

This solution assumes the models_dev_cache.json file is up-to-date and contains the correct model IDs for xai. If the cache file is not available or outdated, the static fallback will be used.

Recommendation

Apply the workaround by implementing the _xai_curated_models() function to fix the immediate issue, and consider implementing the longer-term solution of a single _provider_models(provider: str) resolver to improve code maintainability and reduce duplication.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#docker error#permission error#memory optimization#batch processing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [Bug] _PROVIDER_MODELS["xai"] is stale — grok-4.20-reasoning renamed; broader fragility from hardcoded provider lists [2 pull requests, 1 comments, 2 participants]