Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix Clarify that model.default requires a concrete model ID [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#14963•Fetched 2026-04-24 10:43:58
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
aoife-brennan-bfaoife-brennan-bf
Participants
aoife-brennan-bfaoife-brennan-bf
Timeline (top)
labeled ×4cross-referenced ×1

During a support thread about switching Hermes from gpt-5.4 to gpt-5.5, I incorrectly suggested model.default: latest as if it were a plausible Hermes config value.

That was wrong. latest is not a standard Hermes model setting. Hermes expects a concrete model ID such as gpt-5.5, and setting model.default: latest would likely just pass latest through as a literal model name to the provider.

Error Message

This is exactly the kind of support failure that can cause users to put invalid values into config.yaml and then waste time debugging a provider error that Hermes could have helped prevent.

Root Cause

This is exactly the kind of support failure that can cause users to put invalid values into config.yaml and then waste time debugging a provider error that Hermes could have helped prevent.

The fix may not need to be complicated. A few possible improvements:

  1. Document clearly that model.default must be a concrete provider model ID, not aliases like latest.
  2. Add config validation that warns when model.default is set to known unsupported pseudo-values such as latest, auto, or similar.
  3. If Hermes ever wants a latest-model behavior, implement it explicitly with a documented setting rather than letting users infer it from provider model discovery.

Fix Action

Fixed

PR fix notes

PR #14972: fix(config): warn when model.default is set to a pseudo-alias like 'latest' (#14963)

Description (problem / solution / changelog)

Problem

A support incident (#14963) surfaced a real user pain point: model.default: latest looks like a valid config value but Hermes does not implement a 'latest' alias. Hermes passes the value literally to the provider, which returns an opaque API error rather than a clear config diagnostic.

The fix was requested in the issue: add config-time validation that warns when model.default is set to a known pseudo-alias.

Changes

hermes_cli/config.py

Added _PSEUDO_MODEL_VALUES constant — a set of strings users commonly try as 'magic' model selectors:

_PSEUDO_MODEL_VALUES = {
    "latest", "auto", "default", "current", "best",
    "newest", "recommended", "stable", "preview",
}

Added a check in validate_config_structure() that detects these values in model.default and emits a WARNING-level ConfigIssue with a clear hint:

⚠ model.default is set to 'latest', which is not a valid model ID
  Hermes requires a concrete provider model ID such as 'gpt-5.5' or
  'anthropic/claude-sonnet-4'. Run 'hermes setup' or set model.default
  to a real model ID.

Detection is case-insensitive. Empty and null values are safely handled.

Tests

Extended tests/hermes_cli/test_config_validation.py with a new TestModelDefaultPseudoValue class (9 tests):

TestVerifies
test_latest_warns'latest' produces a WARNING
test_auto_warns'auto' produces a WARNING
test_default_warns'default' produces a WARNING
test_case_insensitive'Latest', 'LATEST', 'Auto' all warn
test_concrete_model_id_no_warningReal IDs (gpt-5.5, claude-sonnet-4) don't trigger
test_hint_mentions_concrete_exampleHint guides user to correct action
test_missing_model_section_no_crashNo model section → no crash
test_model_default_empty_string_no_warningEmpty string → no warning
test_model_default_none_no_warningnull value → no warning

All 26 tests in the file pass ✅ (all prior tests unaffected)

Closes #14963

Changed files

  • hermes_cli/config.py (modified, +21/-0)
  • tests/hermes_cli/test_config_validation.py (modified, +86/-0)

Code Example

model:
  default: gpt-5.4
  provider: openai-codex
  base_url: https://chatgpt.com/backend-api/codex
RAW_BUFFERClick to expand / collapse

Summary

During a support thread about switching Hermes from gpt-5.4 to gpt-5.5, I incorrectly suggested model.default: latest as if it were a plausible Hermes config value.

That was wrong. latest is not a standard Hermes model setting. Hermes expects a concrete model ID such as gpt-5.5, and setting model.default: latest would likely just pass latest through as a literal model name to the provider.

What happened

The conversation was about this existing config shape:

model:
  default: gpt-5.4
  provider: openai-codex
  base_url: https://chatgpt.com/backend-api/codex

When asked why Hermes could not always use the latest model, I floated model.default: latest as a possible approach. The user challenged that, and after checking the repo I confirmed there is no implemented or documented latest alias.

Relevant code paths checked:

  • hermes_cli/auth.py saves the selected model as a concrete string in model.default.
  • gateway/run.py only falls back to a provider default when no model is configured.
  • hermes_cli/models.py:get_default_model_for_provider() returns the first concrete model in the provider list.
  • hermes_cli/codex_models.py has live Codex model discovery and forward-compatible model IDs, but not a latest config alias.

Why this matters

This is exactly the kind of support failure that can cause users to put invalid values into config.yaml and then waste time debugging a provider error that Hermes could have helped prevent.

The fix may not need to be complicated. A few possible improvements:

  1. Document clearly that model.default must be a concrete provider model ID, not aliases like latest.
  2. Add config validation that warns when model.default is set to known unsupported pseudo-values such as latest, auto, or similar.
  3. If Hermes ever wants a latest-model behavior, implement it explicitly with a documented setting rather than letting users infer it from provider model discovery.

Expected behavior

Hermes docs and validation should make it hard to mistake a hypothetical alias for a supported config value.

Actual behavior

The config format is permissive enough that a bad suggestion like model.default: latest can sound plausible, even though Hermes does not currently implement that behavior.

extent analysis

TL;DR

To prevent configuration errors, Hermes should validate and document that model.default requires a concrete provider model ID, rejecting unsupported pseudo-values like latest.

Guidance

  • Document the required format for model.default to avoid user confusion.
  • Implement config validation to warn against known unsupported values like latest or auto.
  • Consider adding a feature to support a latest model behavior with explicit documentation.
  • Review the hermes_cli code paths to ensure consistent handling of model configuration.

Example

No code snippet is provided as the issue focuses on configuration and documentation rather than code implementation.

Notes

The solution involves improving documentation and validation rather than changing the underlying model selection logic. This approach ensures that users are aware of the required configuration format and prevents potential errors.

Recommendation

Apply workaround: Improve documentation and validation to prevent invalid configurations, as implementing a latest model feature may require additional development and testing.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Hermes docs and validation should make it hard to mistake a hypothetical alias for a supported config value.

#api#training loop#device allocation#model download#tokenizer error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix Clarify that model.default requires a concrete model ID [1 pull requests, 1 participants]