hermes - ✅(Solved) Fix /model picker uses static Copilot list, ignores live catalog — newer/tier-gated models invisible [2 pull requests, 1 comments, 2 participants]

idanshimon · 2026-05-10T05:23:31Z

[hermes] The /model picker and hermes model shows a hardcoded list of 17 Copilot models from PROVIDER MODELS "copilot" in hermes cli/models.py . It does not ca… The `/model` picker (and `hermes model`) shows a hardcoded list of 17 Copilot models from `_PROVIDER_MODELS["copilot"]` in `hermes_cli/models.py`. It **does not** call the live GitHub Copilot catalog (`/models` on the configured `base_url`), so any model the user's account is entitled to but that isn't in the static list is invisible from the picker. This matters because the Copilot model catalog is **tier-dependent**: different subscription/SKU tiers (Pro, Business, Enterprise) and different base URLs (`api.githubcopilot.com`, `api.business.githubcopilot.com`, `api.enterprise.githubcopilot.com`) return different model sets, and the catalog evolves continuously as GitHub ships new models without a Hermes release. # PR #23085: fix(model-picker): use live Copilot catalog in /model - Repository: NousResearch/hermes-agent - Author: LeonSGP43 - State: open | merged: False - Link: https://github.com/NousResearch/hermes-agent/pull/23085 ## Description (problem / solution / changelog) ## What does this PR do? Fixes the `/model` provider picker so GitHub Copilot entries use the live catalog when credentials are available, instead of getting stuck on the static curated list during `list_authenticated_providers()`. The root cause is that section 1 of `list_authenticated_providers()` consumes `copilot` / `copilot-acp` before section 2 can replace the curated models with `provider_model_ids(...)`. This change applies the live-catalog path in section 1 as well, while still preserving the existing fallback behavior inside `provider_model_ids()`. ## Related Issue Fixes #22990 ## Type of Change - [x] 🐛 Bug fix (non-breaking change that fixes an issue) - [ ] ✨ New feature (non-breaking change that adds functionality) - [ ] 🔒 Security fix - [ ] 📝 Documentation update - [ ] ✅ Tests (adding or improving test coverage) - [ ] ♻️ Refactor (no behavior change) - [ ] 🎯 New skill (bundled or hub) ## Changes Made - Updated `hermes_cli/model_switch.py` so `copilot` and `copilot-acp` use `provider_model_ids(...)` in section 1 of `list_authenticated_providers()`. - Kept the existing curated fallback behavior by relying on `provider_model_ids(...)` to fall back when the live GitHub catalog is unavailable. - Verified the existing Copilot picker regression tests now pass against this path. ## How to Test 1. Run `uv run --frozen pytest -q -o addopts='' tests/hermes_cli/test_copilot_in_model_list.py` 2. Run `uv run --frozen ruff check hermes_cli/model_switch.py tests/hermes_cli/test_copilot_in_model_list.py` 3. With GitHub Copilot credentials available, open the `/model` picker and confirm the Copilot entry reflects the live catalog instead of only the static curated list. ## Checklist ### Code - [x] I've read the [Contributing Guide](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md) - [x] My commit messages follow [Conventional Commits](https://www.conventionalcommits.org/) (`fix(scope):`, `feat(scope):`, etc.) - [x] I searched for [existing PRs](https://github.com/NousResearch/hermes-agent/pulls) to make sure this isn't a duplicate - [x] My PR contains **only** changes related to this fix/feature (no unrelated commits) - [ ] I've run `pytest tests/ -q` and all tests pass - [ ] I've added tests for my changes (required for bug fixes, strongly encouraged for features) - [x] I've tested on my platform: macOS 15.x ### Documentation & Housekeeping - [x] I've updated relevant documentation (README, `docs/`, docstrings) — or N/A - [x] I've updated `cli-config.yaml.example` if I added/changed config keys — or N/A - [x] I've updated `CONTRIBUTING.md` or `AGENTS.md` if I changed architecture or workflows — or N/A - [x] I've considered cross-platform impact (Windows, macOS) per the [compatibility guide](https://github.com/NousResearch/hermes-agent/blob/main/CONTRIBUTING.md#cross-platform-compatibility) — or N/A - [x] I've updated tool descriptions/schemas if I changed tool behavior — or N/A ## Screenshots / Logs - `uv run --frozen pytest -q -o addopts='' tests/hermes_cli/test_copilot_in_model_list.py` -> `2 passed` - `env -i ... UV_PYTHON=3.11 uv run --frozen --extra all pytest tests/ --collect-only -q -o addopts=''` -> `21827 tests collected` ## Changed files - `hermes_cli/model_switch.py` (modified, +14/-7) --- # PR #23101: fix(model-picker): use live Copilot catalog instead of static 17-model list (#22990) - Repository: NousResearch/hermes-agent - Author: KhanCold - State: closed | merged: False - Link: https://github.com/NousResearch/hermes-agent/pull/23101 ## Description (problem / solution / changelog) ## Bug Description The `/model` picker showed a hardcoded 17-model list for Copilot, ignoring the live GitHub Copilot catalog. Newer models like `gpt-5.5`, `claude-opus-4.7` were invisible to users even though their accounts had access. ## Root Cause Copilot has `

hermes2026-05-10 05:23:31

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

NousResearch/hermes-agent#22990•Fetched 2026-05-11 03:31:53

View on GitHub

Comments

Participants

Timeline

Reactions

Author

idanshimon

Participants

alt-glitch

idanshimon

Timeline (top)

labeled ×4cross-referenced ×3referenced ×2commented ×1

The /model picker (and hermes model) shows a hardcoded list of 17 Copilot models from _PROVIDER_MODELS["copilot"] in hermes_cli/models.py. It does not call the live GitHub Copilot catalog (/models on the configured base_url), so any model the user's account is entitled to but that isn't in the static list is invisible from the picker.

This matters because the Copilot model catalog is tier-dependent: different subscription/SKU tiers (Pro, Business, Enterprise) and different base URLs (api.githubcopilot.com, api.business.githubcopilot.com, api.enterprise.githubcopilot.com) return different model sets, and the catalog evolves continuously as GitHub ships new models without a Hermes release.

Error Message

model_ids = provider_model_ids(hermes_id) # falls back to static on network error

Root Cause

hermes_cli/model_switch.py::list_authenticated_providers() has two sections:

Section 1 (lines ~1200-1264): handles auth_type == "api_key" providers via PROVIDER_TO_MODELS_DEV. Uses the static curated list.
Section 2 (lines ~1266-1377): handles HERMES_OVERLAYS. Has a Copilot special-case at line 1346 that calls provider_model_ids("copilot") → _fetch_github_models() → live catalog.

Copilot has auth_type == "api_key" (env vars COPILOT_GITHUB_TOKEN, GH_TOKEN, GITHUB_TOKEN), so Section 1 always wins and adds copilot to seen_slugs. Section 2's dedup then skips it — the live-catalog code path is dead for Copilot.

Code Example

>>> from hermes_cli.models import provider_model_ids
>>> ids = provider_model_ids("copilot")        # live catalog
>>> any("opus-4.7" in m for m in ids)
True

>>> from hermes_cli.model_switch import list_authenticated_providers
>>> picker = next(p for p in list_authenticated_providers(current_provider="copilot")
...               if p["slug"] == "copilot")
>>> picker["total_models"]
17
>>> any("opus-4.7" in m for m in picker["models"])
False                                          # bug

---

# inside the section-1 loop, after has_creds check:
if hermes_id in {"copilot", "copilot-acp"}:
    model_ids = provider_model_ids(hermes_id)   # falls back to static on network error
else:
    model_ids = curated.get(hermes_id, [])
    if hermes_id in _MODELS_DEV_PREFERRED:
        model_ids = _merge_with_models_dev(hermes_id, model_ids)

RAW_BUFFERClick to expand / collapse

Summary

Concrete examples missing from the picker

Confirmed returned by _fetch_github_models() against a configured Copilot base URL but absent from _PROVIDER_MODELS["copilot"]:

claude-opus-4.7, claude-opus-4.7-high, claude-opus-4.7-xhigh
claude-opus-4.5, claude-opus-4.6
gpt-5.5, gpt-5.2

These are valid, working Copilot models — the API serves them at runtime when set as model.default, but they cannot be selected via the picker.

Why the static list can't fix this long-term

Catalog membership is a function of the user's Copilot tier × base URL × GitHub's current rollout. Examples of tier/endpoint differences a user can hit today:

A Copilot Pro user on api.githubcopilot.com sees one catalog.
A Copilot Business org on api.business.githubcopilot.com may see additional models (e.g. higher-effort Claude variants) that aren't on Pro.
A Copilot Enterprise org on api.enterprise.githubcopilot.com may see yet another superset (long-context variants, additional reasoning tiers).

Hermes already supports configuring base_url per provider, so users on all three tiers actively run the agent — but the picker shows the same 17 models to all of them, regardless of what their account actually has.

Root cause

hermes_cli/model_switch.py::list_authenticated_providers() has two sections:

Section 1 (lines ~1200-1264): handles auth_type == "api_key" providers via PROVIDER_TO_MODELS_DEV. Uses the static curated list.
Section 2 (lines ~1266-1377): handles HERMES_OVERLAYS. Has a Copilot special-case at line 1346 that calls provider_model_ids("copilot") → _fetch_github_models() → live catalog.

Reproduction

>>> from hermes_cli.models import provider_model_ids
>>> ids = provider_model_ids("copilot")        # live catalog
>>> any("opus-4.7" in m for m in ids)
True

>>> from hermes_cli.model_switch import list_authenticated_providers
>>> picker = next(p for p in list_authenticated_providers(current_provider="copilot")
...               if p["slug"] == "copilot")
>>> picker["total_models"]
17
>>> any("opus-4.7" in m for m in picker["models"])
False                                          # bug

Suggested fix

Special-case copilot / copilot-acp in Section 1 to use the live catalog the same way Section 2 already does:

# inside the section-1 loop, after has_creds check:
if hermes_id in {"copilot", "copilot-acp"}:
    model_ids = provider_model_ids(hermes_id)   # falls back to static on network error
else:
    model_ids = curated.get(hermes_id, [])
    if hermes_id in _MODELS_DEV_PREFERRED:
        model_ids = _merge_with_models_dev(hermes_id, model_ids)

provider_model_ids("copilot") already gracefully falls back to the static curated list on network failure, so offline behavior is preserved.

Environment

Hermes version: current main (behind by ~145 commits at time of writing — happy to verify on tip)
Provider: copilot
Python: 3.9
macOS 26.4

Happy to send a PR if a maintainer confirms the proposed direction.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #generation error #database connection #vector store #embedding generation

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

hermes - ✅(Solved) Fix /model picker uses static Copilot list, ignores live catalog — newer/tier-gated models invisible [2 pull requests, 1 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Fix Action

Fixed

PR fix notes

PR #23085: fix(model-picker): use live Copilot catalog in /model

Description (problem / solution / changelog)

What does this PR do?

Related Issue

Type of Change

Changes Made

How to Test

Checklist

Code

Documentation & Housekeeping

Screenshots / Logs

Changed files

PR #23101: fix(model-picker): use live Copilot catalog instead of static 17-model list (#22990)

Description (problem / solution / changelog)

Bug Description

Root Cause

Fix

Tests

Changed files

Code Example

Summary

Concrete examples missing from the picker

Why the static list can't fix this long-term

Root cause

Reproduction

Suggested fix

Environment

Still need to ship something?

RELATED_DISCOVERY

TRENDING