hermes - 💡(How to fix) Fix [Bug]: model_aliases provider: custom:<subname> normalized to bare 'custom', wrong API key picked from pool [2 pull requests]

Bug Description

When a model_aliases entry declares provider: custom:<subname> (e.g. provider: custom:bobapi-deepseek), invoking that alias via hermes chat -m <alias> resolves the request with provider=custom (bare) — the :<subname> half is stripped. As a result, the credential pool falls back to the first entry under any custom* pool that shares the same base_url, often picking the wrong API key.

For multi-key setups where several custom_providers share the same base_url (very common with Chinese aggregators like Bob API where each key is bound to a different "分组"/group on the same domain), this means a request fired with the wrong key — and the upstream returns errors that look like model availability problems but are actually authorization-misroute problems.

This appears to be in hermes_cli/runtime_provider.py:resolve_runtime_provider() — the alias resolver in hermes_cli/model_switch.py:_load_direct_aliases correctly stores the full custom:bobapi-deepseek provider name into DIRECT_ALIASES, but downstream that string gets normalized down to bare custom, dropping the sub-provider name.

Steps to Reproduce

1. Configure two or more custom_providers sharing the same base_url, each with a distinct key_env:

   custom_providers:
     bobapi-deepseek:
       base_url: https://bobdong.cn
       key_env: BOBAPI_DEEPSEEK_KEY    # bound to group A
       api_mode: anthropic_messages
     bobapi-claude:
       base_url: https://bobdong.cn
       key_env: BOBAPI_CLAUDE_KEY      # bound to group B
       api_mode: anthropic_messages

2. Add aliases that reference each by sub-name:

   model_aliases:
     deepseek-v4-pro:
       provider: custom:bobapi-deepseek
       model: deepseek-v4-pro
       key_env: BOBAPI_DEEPSEEK_KEY
     claude-opus-4-7:
       provider: custom:bobapi-claude
       model: claude-opus-4-7
       key_env: BOBAPI_CLAUDE_KEY

3. Set model.default + model.provider to one of them (e.g. claude-opus-4-7 + custom:bobapi-claude).

4. Run the OTHER alias:

   hermes chat -m deepseek-v4-pro -q "OK"

Expected Behavior

The alias's provider: custom:bobapi-deepseek should be honored end-to-end, the credential pool should select the entry under custom:bobapi-deepseek (i.e. use BOBAPI_DEEPSEEK_KEY), and the request should succeed.

Actual Behavior

Request fires with BOBAPI_CLAUDE_KEY instead. With Bob API this returns:

HTTP 503 - {"error": {"code": "model_not_found",
  "message": "当前分组 ClaudeCode Kiro-1 下模型 deepseek-v4-pro 无可用管道(distributor)",
  "type": "new_api_error"}}

— the upstream's way of saying "this key is bound to the Claude group, you can't call deepseek-v4-pro on it."

Adding --provider custom:bobapi-deepseek explicitly at the CLI works around the bug:

hermes chat -m deepseek-v4-pro --provider custom:bobapi-deepseek -q "OK"
# → 200 OK, correct response

Affected Component

Configuration (config.yaml, .env, hermes setup), Agent Core (conversation loop, context compression, memory)

Debug Report

Cannot run hermes debug share against production because the system is in active multi-user use; pasting the verbose-log fingerprint instead. The smoking gun is two consecutive lines from HERMES_LOG_HTTP=1 hermes chat -m deepseek-v4-pro -q OK -v:

agent.context_compressor - INFO - Context compressor initialized:
  model=deepseek-v4-pro context_length=256000 threshold=230400 (90%)
  target_ratio=20% tail_budget=46080
  provider=custom base_url=https://bobdong.cn

anthropic._base_client - DEBUG - Sending HTTP Request: POST https://bobdong.cn/v1/messages
anthropic._base_client - DEBUG - HTTP Response: POST https://bobdong.cn/v1/messages "503 Service Unavailable"

agent.chat_completion_helpers - INFO - Streaming failed before delivery:
  Error code: 503 - {'error': {'code': 'model_not_found',
  'message': '当前分组 ClaudeCode Kiro-1 下模型 deepseek-v4-pro 无可用管道(distributor)',
  'type': 'new_api_error'}}

Note provider=custom (bare), not provider=custom:bobapi-deepseek — that's the moment the sub-name disappears.

Operating System

Ubuntu 24.04 (Linux 6.8.0-106-generic)

Python Version

3.11.15

Hermes Version

v0.14.0 (2026.5.16)

Additional Logs / Traceback (optional)

Direct curl against the same Bob API endpoint succeeds with the correct key and fails with the identical 503 message when using the wrong key, confirming the upstream is fine and Hermes is the one picking the wrong credential:

# WORKS: deepseek-v4-pro with BOBAPI_DEEPSEEK_KEY
$ curl -s -X POST https://bobdong.cn/v1/messages \
    -H "Authorization: Bearer $BOBAPI_DEEPSEEK_KEY" \
    -H "anthropic-version: 2023-06-01" \
    -H "Content-Type: application/json" \
    -d '{"model":"deepseek-v4-pro","max_tokens":50,"messages":[{"role":"user","content":"OK"}]}'
# → 200 OK, content "OK"

# FAILS (identical to Hermes): deepseek-v4-pro with BOBAPI_CLAUDE_KEY
$ curl -s -X POST https://bobdong.cn/v1/messages \
    -H "Authorization: Bearer $BOBAPI_CLAUDE_KEY" \
    ...
# → 503: 当前分组 ClaudeCode Kiro-1 下模型 deepseek-v4-pro 无可用管道(distributor)

hermes auth list was clean during the test (1 entry per provider, no zombies, every entry's access_token matched the expected key_env). So this is not pool drift — it's the resolver dropping the sub-provider name regardless of pool state.

Root Cause Analysis (optional)

Trace through the code:

1. hermes_cli/model_switch.py:179-243 (_load_direct_aliases) — correctly reads provider: custom:bobapi-deepseek from model_aliases and stores it into DIRECT_ALIASES as DirectAlias(model=..., provider='custom:bobapi-deepseek', base_url=...). ✅

2. hermes_cli/oneshot.py:268-276 — looks up the alias via DIRECT_ALIASES.get(...), sets effective_provider = direct.provider (still 'custom:bobapi-deepseek'). ✅

3. hermes_cli/oneshot.py:290-294 (resolve_runtime_provider) — somewhere in the runtime resolver chain, 'custom:bobapi-deepseek' gets reduced to 'custom'. The final provider=custom base_url=https://bobdong.cn log line confirms this happened. ❌

4. Credential pool lookup — uses bare custom + https://bobdong.cn to find a token. With multiple custom:* pools sharing that base_url, the pool that "wins" depends on iteration order, not on the alias's sub-provider name.

I haven't traced the exact line where the normalization happens (the resolver chain goes through runtime_provider.py:resolve_runtime_provider and various helpers in hermes_cli/model_normalize.py like _normalize_provider_alias), but the symptom is unambiguous: the sub-provider name is preserved into DirectAlias and then lost before the HTTP request fires.

Proposed Fix (optional)

resolve_runtime_provider() should preserve custom:<subname> form throughout the chain when an alias supplies it explicitly, and the credential pool lookup should select the pool whose name matches custom:<subname> exactly rather than collapsing to bare custom.

Two failure modes to handle:

• Forward path: when effective_provider == 'custom:bobapi-deepseek', downstream code that expects provider == 'custom' needs to accept the longer form (or split it: provider family + sub-name) without losing the sub-name.

• Pool lookup: today the pool lookup appears to do something like "find any pool whose base_url matches"; it should be "find the pool named custom:bobapi-deepseek first, fall back to base-url matching only if no exact name match exists."

The same problem will affect Web UI and any automation that relies on model_aliases.provider instead of passing --provider at every call site, so a fix benefits more than just the CLI.

Are you willing to submit a PR for this?

Not at this time, but happy to test a fix on real Bob API hardware.