hermes - ✅(Solved) Fix copilot_auth.py fallback to gh auth token causes false positive for Copilot availability [1 pull requests, 1 comments, 1 participants]

hermes2026-05-13 21:40:04

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

NousResearch/hermes-agent#25245•Fetched 2026-05-14 03:47:54

View on GitHub

Comments

Participants

Timeline

Reactions

Author

servitor-daemon

Participants

servitor-daemon

Timeline (top)

closed ×1commented ×1cross-referenced ×1

resolve_copilot_token() in hermes_cli/copilot_auth.py falls back to gh auth token when no explicit Copilot env var is set. This causes users who have gh authenticated for Git operations (but no Copilot subscription) to see GitHub Copilot models listed as available in hermes model — when they aren't actually usable.

Root Cause

copilot_auth.py:85-93:

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    ...
    return token, "gh auth token"

gh auth token returns a GitHub API token (for repos, workflows, etc.) — not a Copilot API token. The presence of a valid gho_ token does not imply an active Copilot subscription.

list_available_providers() in hermes_cli/models.py calls get_auth_status("copilot"), which resolves the token and marks Copilot as authenticated: true.

Fix Action

Fixed

Fixed by PR: fix(copilot): stop using gh auth as token fallback (https://github.com/NousResearch/hermes-agent/pull/25250)

PR fix notes

PR #25250: fix(copilot): stop using gh auth as token fallback

Repository: NousResearch/hermes-agent
Author: he-yufeng
State: open | merged: False
Link: https://github.com/NousResearch/hermes-agent/pull/25250

Description (problem / solution / changelog)

Summary

Closes #25245.

resolve_copilot_token() no longer falls back to gh auth token. A GitHub CLI token proves the user is logged in to GitHub, but it does not prove that the token can be exchanged for GitHub Copilot API access. This was making hermes model and provider status paths show Copilot as authenticated for users who only had gh logged in.

This keeps the explicit token paths:

COPILOT_GITHUB_TOKEN
GH_TOKEN
GITHUB_TOKEN
stored Copilot credential-pool entries used by the existing catalog/runtime fallback paths

I also removed the now-unused gh CLI token lookup helper and adjusted the credential-pool source tagging so GH_TOKEN remains an explicit env source instead of being grouped with the old gh_cli source.

Tests

python -m py_compile hermes_cli\copilot_auth.py hermes_cli\models.py hermes_cli\main.py agent\credential_pool.py agent\auxiliary_client.py agent\credential_sources.py tests\hermes_cli\test_copilot_auth.py tests\hermes_cli\test_api_key_providers.py tests\agent\test_credential_pool.py tests\hermes_cli\test_copilot_catalog_oauth_fallback.py tests\run_agent\test_run_agent_codex_responses.py
python -m ruff check hermes_cli\copilot_auth.py hermes_cli\models.py hermes_cli\main.py agent\credential_pool.py agent\auxiliary_client.py agent\credential_sources.py tests\hermes_cli\test_copilot_auth.py tests\hermes_cli\test_api_key_providers.py tests\agent\test_credential_pool.py tests\hermes_cli\test_copilot_catalog_oauth_fallback.py tests\run_agent\test_run_agent_codex_responses.py
python -m pytest -o addopts='' tests\hermes_cli\test_copilot_auth.py tests\hermes_cli\test_api_key_providers.py tests\agent\test_credential_pool.py tests\hermes_cli\test_copilot_catalog_oauth_fallback.py tests\run_agent\test_run_agent_codex_responses.py -q --basetemp .tmp\pytest -p no:cacheprovider

Result: 291 passed.

I used -o addopts='' because this Windows environment does not have the repo's default pytest-xdist plugin installed.

Changed files

agent/auxiliary_client.py (modified, +0/-2)
agent/credential_pool.py (modified, +3/-3)
agent/credential_sources.py (modified, +4/-4)
hermes_cli/copilot_auth.py (modified, +3/-70)
hermes_cli/main.py (modified, +2/-4)
hermes_cli/models.py (modified, +2/-3)
tests/agent/test_credential_pool.py (modified, +20/-4)
tests/hermes_cli/test_api_key_providers.py (modified, +12/-43)
tests/hermes_cli/test_copilot_auth.py (modified, +1/-26)
tests/hermes_cli/test_copilot_catalog_oauth_fallback.py (modified, +1/-2)
tests/run_agent/test_run_agent_codex_responses.py (modified, +1/-1)

Code Example

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    ...
    return token, "gh auth token"

---

-    # 2. Fall back to gh auth token
-    token = _try_gh_cli_token()
-    if token:
-        valid, msg = validate_copilot_token(token)
-        if not valid:
-            raise ValueError(...)
-        return token, "gh auth token"
-
+    # 2. Do NOT fall back to gh auth token.
+    # gh auth token provides a GitHub API token (repo access), NOT a Copilot
+    # API token. Users with gh installed but without an active Copilot
+    # subscription get a false positive.
     return "", ""

RAW_BUFFERClick to expand / collapse

Summary

Steps to reproduce

Have gh authenticated (gh auth login) — produces a valid gho_ token
Set no COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN env var
Run hermes model
GitHub Copilot appears as an authenticated provider with models listed

Root cause

copilot_auth.py:85-93:

# 2. Fall back to gh auth token
token = _try_gh_cli_token()
if token:
    valid, msg = validate_copilot_token(token)
    ...
    return token, "gh auth token"

gh auth token returns a GitHub API token (for repos, workflows, etc.) — not a Copilot API token. The presence of a valid gho_ token does not imply an active Copilot subscription.

list_available_providers() in hermes_cli/models.py calls get_auth_status("copilot"), which resolves the token and marks Copilot as authenticated: true.

Proposed fix

Remove the gh auth token fallback from resolve_copilot_token(). Copilot should only show as available when the user has explicitly set COPILOT_GITHUB_TOKEN, GH_TOKEN, or GITHUB_TOKEN — or has completed the hermes model → Copilot OAuth flow.

-    # 2. Fall back to gh auth token
-    token = _try_gh_cli_token()
-    if token:
-        valid, msg = validate_copilot_token(token)
-        if not valid:
-            raise ValueError(...)
-        return token, "gh auth token"
-
+    # 2. Do NOT fall back to gh auth token.
+    # gh auth token provides a GitHub API token (repo access), NOT a Copilot
+    # API token. Users with gh installed but without an active Copilot
+    # subscription get a false positive.
     return "", ""

The _try_gh_cli_token() and _gh_cli_candidates() helpers become dead code and can be removed or preserved for potential opt-in use behind an explicit config/env toggle.

Impact

Users with gh installed see Copilot models they can't use. Selecting one fails at runtime with auth errors from the Copilot API — confusing and frustrating.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #runtime error #dependency conflict #environment setup #docker error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

hermes - ✅(Solved) Fix copilot_auth.py fallback to gh auth token causes false positive for Copilot availability [1 pull requests, 1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #25250: fix(copilot): stop using gh auth as token fallback

Description (problem / solution / changelog)

Summary

Tests

Changed files

Code Example

Summary

Steps to reproduce

Root cause

Proposed fix

Impact

Still need to ship something?

TRENDING

hermes - ✅(Solved) Fix copilot_auth.py fallback to gh auth token causes false positive for Copilot availability [1 pull requests, 1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #25250: fix(copilot): stop using gh auth as token fallback

Description (problem / solution / changelog)

Summary

Tests

Changed files

Code Example

Summary

Steps to reproduce

Root cause

Proposed fix

Impact

Still need to ship something?

RELATED_DISCOVERY

TRENDING