Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix [Bug]: copilot_auth.py fallback to gh auth token causes false positive for Copilot availability [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#25246•Fetched 2026-05-14 03:47:52
View on GitHub
Comments
0
Participants
1
Timeline
7
Reactions
0
Author
servitor-daemonservitor-daemon
Participants
servitor-daemonservitor-daemon
Timeline (top)
labeled ×5cross-referenced ×2

Error Message

A user who selects a Copilot model gets a runtime auth error from the Copilot API — confusing and frustrating. 5. Select any Copilot model — fails at runtime with auth error

Root Cause

resolve_copilot_token() in hermes_cli/copilot_auth.py:85-93 falls back to gh auth token after checking env vars:

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    if not valid:
        raise ValueError(...)
    return token, "gh auth token"

gh auth token returns a GitHub API OAuth token — valid for repos, workflows, gists, etc. — not a Copilot API token. validate_copilot_token() only checks the token prefix (accepts gho_, github_pat_, ghu_; rejects ghp_). It does not verify the token against the Copilot API.

list_available_providers() in hermes_cli/models.py:1531 calls get_auth_status("copilot") → get_api_key_provider_status("copilot") → _resolve_api_key_provider_secret() → resolve_copilot_token(). The gho_ token passes validation, so Copilot is marked authenticated: true.

Fix Action

Fixed

PR fix notes

PR #25254: fix(copilot): remove gh auth token fallback from Copilot auth resolution

Description (problem / solution / changelog)

Summary

Removes the gh auth token CLI fallback from resolve_copilot_token() in hermes_cli/copilot_auth.py.

Problem

hermes model shows GitHub Copilot as an authenticated provider for any user with gh CLI installed and logged in — even without a Copilot subscription. Selecting a Copilot model then fails at runtime with an auth error.

Root cause: gh auth token returns a GitHub API OAuth token (gho_*) that passes prefix-based validation but does NOT grant Copilot API access.

Fix

Remove the gh auth token fallback (step 4 in the credential search order). Copilot now only appears as authenticated when explicitly configured via:

  • COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN env vars
  • The OAuth device code flow (hermes model → Copilot)

The _try_gh_cli_token() and _gh_cli_candidates() helpers are preserved as dead code for potential future opt-in use.

Testing

  • Updated 2 tests in test_copilot_auth.py to verify gh CLI is no longer consulted
  • All 25 tests in test_copilot_auth.py pass

Files Changed

FileChange
hermes_cli/copilot_auth.pyRemoved gh auth token fallback; updated docstring
tests/hermes_cli/test_copilot_auth.pyUpdated 2 tests for new no-fallback behavior

Closes #25246

Changed files

  • hermes_cli/copilot_auth.py (modified, +13/-12)
  • tests/hermes_cli/test_copilot_auth.py (modified, +11/-6)

PR #25252: fix(copilot): remove gh auth token fallback to prevent false positive availability

Description (problem / solution / changelog)

Summary

Remove the gh auth token fallback from resolve_copilot_token() so Copilot only appears available when the user has explicitly configured it.

Root Cause

resolve_copilot_token() in hermes_cli/copilot_auth.py fell back to gh auth token after checking env vars. gh auth token returns a GitHub API OAuth token (gho_*) — valid for repos, workflows, gists — not a Copilot API token. validate_copilot_token() only checks the token prefix (accepts gho_*, github_pat_*, ghu_*), so the gho_* token passed validation and Copilot was marked as authenticated: true.

Any user with gh installed and logged in (via gh auth login) would see Copilot as an available provider with a full model list, even without a Copilot subscription. Selecting a Copilot model would fail at runtime with an auth error.

Fix

Remove the gh auth token fallback from resolve_copilot_token(). Users who want Copilot must explicitly set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN, or use the OAuth device code flow.

The _try_gh_cli_token() and _gh_cli_candidates() helpers are preserved (they may be used by other code paths or future features) but are no longer called from the token resolution path.

Code Intelligence

  • Analyzed: resolve_copilot_token (callers: 2, callees: 2, flows: 1)
  • Blast radius: LOW — direct callers are _resolve_api_key_provider_secret (auth.py) and _seed_from_singletons (credential_pool.py); both handle empty token gracefully
  • Related patterns: PR #24781 adds COPILOT_GH_USER selector and .env fallback to the same file (different concern, no conflict)

Regression Coverage

  • Updated test_gh_cli_fallback → test_gh_cli_fallback_removed: verifies gh auth token is ignored even when it returns a valid token
  • Removed test_gh_cli_classic_pat_raises: no longer applicable (fallback removed)
  • Updated test_no_token_returns_empty: simplified (no mock needed)
  • Updated test_copilot_status_uses_gh_cli_token → test_copilot_status_ignores_gh_cli_token: verifies Copilot is not configured from gh auth token alone
  • Updated test_resolve_copilot_with_gh_cli_fallback → test_resolve_copilot_ignores_gh_cli_fallback: verifies empty credentials returned
  • Updated test_runtime_copilot_uses_gh_cli_token → test_runtime_copilot_requires_explicit_token: verifies empty api_key from runtime
  • Updated test_runtime_copilot_uses_responses_for_gpt_5_4: uses env var instead of mock
  • Updated test_gh_cli_token_counts → test_gh_cli_token_no_longer_counts: verifies _has_any_provider_configured returns False

Testing

  • tests/hermes_cli/test_copilot_auth.py: 24 passed
  • tests/hermes_cli/test_api_key_providers.py: 157 passed

Changed files

  • hermes_cli/copilot_auth.py (modified, +7/-9)
  • tests/hermes_cli/test_api_key_providers.py (modified, +28/-16)
  • tests/hermes_cli/test_copilot_auth.py (modified, +6/-14)

Code Example

Report       https://paste.rs/SKOu5
agent.log    https://paste.rs/wEY0A
gateway.log  https://paste.rs/10Y6V

---

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    if not valid:
        raise ValueError(...)
    return token, "gh auth token"

---

-    # 2. Fall back to gh auth token
-    token = _try_gh_cli_token()
-    if token:
-        valid, msg = validate_copilot_token(token)
-        if not valid:
-            raise ValueError(
-                f"Token from `gh auth token` is a classic PAT (ghp_*). {msg}"
-            )
-        return token, "gh auth token"
-
+    # 2. Do NOT fall back to gh auth token.
+    # gh auth token provides a GitHub API token (repo access), NOT a Copilot
+    # API token. Users with gh installed but without an active Copilot
+    # subscription were getting Copilot models listed as "available" when
+    # they weren't actually usable.
     return "", ""
RAW_BUFFERClick to expand / collapse

Bug Description

hermes model shows GitHub Copilot as an authenticated provider with a full model list, even when the user has no Copilot subscription. The only Copilot-related credential present is a standard gh auth login token (gho_*) used for Git operations.

A user who selects a Copilot model gets a runtime auth error from the Copilot API — confusing and frustrating.

Steps to Reproduce

  1. Have gh authenticated (gh auth login) producing a gho_ token — no Copilot subscription
  2. Ensure no COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN env var is set
  3. Run hermes model
  4. GitHub Copilot and Copilot ACP appear as authenticated providers with curated model lists
  5. Select any Copilot model — fails at runtime with auth error

Expected Behavior

Copilot should only appear as available when the user has explicitly configured it — via COPILOT_GITHUB_TOKEN env var, or via the hermes model → Copilot OAuth device code flow. A generic gh auth token should not be treated as proof of Copilot access.

Actual Behavior

Copilot shows up as authenticated: true for any user with gh installed and logged in, regardless of Copilot subscription status.

Debug Report

Report       https://paste.rs/SKOu5
agent.log    https://paste.rs/wEY0A
gateway.log  https://paste.rs/10Y6V

Operating System

Linux (Ubuntu 24.04, 6.17.0-23-generic)

Python Version

3.11.14

Hermes Version

v0.13.0 (2026.5.7)

Root Cause Analysis

resolve_copilot_token() in hermes_cli/copilot_auth.py:85-93 falls back to gh auth token after checking env vars:

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    if not valid:
        raise ValueError(...)
    return token, "gh auth token"

gh auth token returns a GitHub API OAuth token — valid for repos, workflows, gists, etc. — not a Copilot API token. validate_copilot_token() only checks the token prefix (accepts gho_, github_pat_, ghu_; rejects ghp_). It does not verify the token against the Copilot API.

list_available_providers() in hermes_cli/models.py:1531 calls get_auth_status("copilot") → get_api_key_provider_status("copilot") → _resolve_api_key_provider_secret() → resolve_copilot_token(). The gho_ token passes validation, so Copilot is marked authenticated: true.

Proposed Fix

Remove the gh auth token fallback. Users who want Copilot should explicitly set COPILOT_GITHUB_TOKEN or use the OAuth flow. The fallback was always a guess — a GitHub token ≠ Copilot access.

-    # 2. Fall back to gh auth token
-    token = _try_gh_cli_token()
-    if token:
-        valid, msg = validate_copilot_token(token)
-        if not valid:
-            raise ValueError(
-                f"Token from `gh auth token` is a classic PAT (ghp_*). {msg}"
-            )
-        return token, "gh auth token"
-
+    # 2. Do NOT fall back to gh auth token.
+    # gh auth token provides a GitHub API token (repo access), NOT a Copilot
+    # API token. Users with gh installed but without an active Copilot
+    # subscription were getting Copilot models listed as "available" when
+    # they weren't actually usable.
     return "", ""

After this change, the _try_gh_cli_token() and _gh_cli_candidates() helpers become dead code — they can be removed or preserved behind an opt-in toggle.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#pipeline error#runtime error#dependency conflict#environment setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [Bug]: copilot_auth.py fallback to gh auth token causes false positive for Copilot availability [2 pull requests, 1 participants]