hermes - 💡(How to fix) Fix follow-up: hermes models status --probe (reachability check, extracted from #23369) [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

The hermes models status --probe flag — opt-in network call to /v1/models per provider that reports {status: "ok"|"skipped"|"error", model_count, took_ms} per row.

  • --probe and --offline are mutually exclusive (rejected with parser.error() exit 2).
  • Probe field shape: {status, model_count, took_ms, error?, reason?}.

Root Cause

  • --probe and --offline are mutually exclusive (rejected with parser.error() exit 2).
  • Probe skip predicate (_probe_can_skip in hermes_cli/inventory.py:340) is already in the codebase — preserved from the consolidation PR because _models_for_canonical_slug reuses it for "skip live HTTP for unconfigured api-key providers". Just needs to be wired up to a per-provider provider_model_ids(slug, force_refresh=True) call.
  • Probe field shape: {status, model_count, took_ms, error?, reason?}.
  • No timeout knob (relies on caller's subprocess.run(timeout=...) or per-provider HTTP timeouts) — flagged for a future "real reachability check" iteration.

Fix Action

Fixed

Code Example

hermes models status --probe [--provider NAME] [--json]
RAW_BUFFERClick to expand / collapse

Follow-up to #23359

PR #23369 (the inventory CLI consolidation) intentionally extracted the reachability-probe surface to keep that PR focused on consolidation rather than scope creep. This issue tracks adding it back as a small focused follow-up.

What's missing

The hermes models status --probe flag — opt-in network call to /v1/models per provider that reports {status: "ok"|"skipped"|"error", model_count, took_ms} per row.

Why it's useful

  • Cron health checks: "of my N configured providers, which are actually reachable right now?" — currently requires parsing hermes doctor's free-form output.
  • Adjacent feature to #21785: hermes doctor already wants this skip-by-endpoint logic. A consolidated probe surface lets doctor consume inventory.build_status_payload(probe=True) instead of duplicating it.
  • Pre-batch verification: "is my Anthropic key still valid before I kick off this 4-hour run?" — single command, exit code reflects health.

Proposed scope

hermes models status --probe [--provider NAME] [--json]
  • --probe and --offline are mutually exclusive (rejected with parser.error() exit 2).
  • Probe skip predicate (_probe_can_skip in hermes_cli/inventory.py:340) is already in the codebase — preserved from the consolidation PR because _models_for_canonical_slug reuses it for "skip live HTTP for unconfigured api-key providers". Just needs to be wired up to a per-provider provider_model_ids(slug, force_refresh=True) call.
  • Probe field shape: {status, model_count, took_ms, error?, reason?}.
  • No timeout knob (relies on caller's subprocess.run(timeout=...) or per-provider HTTP timeouts) — flagged for a future "real reachability check" iteration.

Implementation notes

  • The original implementation lived in PR #23369 commit history before extraction. ~85 LOC: a _probe_provider(slug) function in inventory.py, a probe: bool = False param on build_status_payload, the mutex check, and --probe argparse wiring in hermes_cli/main.py.
  • Tests covered: H2 regression (oauth_external providers report skipped not ok because of the curated-fallback masking), the offline/probe mutex, gemini-cli probe-skipped E2E.
  • The _OAUTH_EXTERNAL_HAS_MODELS_ENDPOINT = frozenset({"openai-codex"}) allowlist is also already in the codebase (used by _probe_can_skip).

Out of scope

  • True reachability (HTTP HEAD to /v1/models without falling through to curated). provider_model_ids falls back to curated catalogs on probe failure, so a successful return means "we got a model list" — not "the endpoint responded." Worth a separate feature once probe is back.
  • hermes doctor migration to consume this surface — that's #21785.

References

  • Parent issue: #23359
  • Predecessor: PR #23369 (consolidation; this surface was extracted in the cut)
  • Adjacent: #21785 (hermes doctor skip-by-endpoint)

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING