Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix Fallback provider defined in custom_providers ignores its own base_url, sends request to primary provider's endpoint instead [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#15743•Fetched 2026-04-26 05:25:23
View on GitHub
Comments
1
Participants
2
Timeline
6
Reactions
0
Author
stanleybluestanleyblue
Participants
alt-glitchalt-glitchstanleybluestanleyblue
Timeline (top)
labeled ×4commented ×1cross-referenced ×1

Error Message

⚠️ API call failed (attempt 1/3): NotFoundError [HTTP 404] 🔌 Provider: aliyun-singapore Model: qwen3.6-plus 🌐 Endpoint: https://inference-api.nousresearch.com/v1 📝 Error: HTTP 404: Model 'qwen3.6-plus' not found. The requested model does not exist in our configuration or OpenRouter catalog.

Root Cause

Root cause hypothesis

Fix Action

Fix / Workaround

Workaround

But this is16 only a workaround. The root issue remains.

PR fix notes

PR #15756: fix(fallback): resolve base_url from custom_providers when fallback entry has none (#15743)

Description (problem / solution / changelog)

Problem

When fallback_providers references a provider defined in custom_providers but does not supply an inline base_url, _try_activate_fallback() passed explicit_base_url=None to resolve_provider_client(). The resolution then fell through to the primary provider's endpoint, silently misdirecting the fallback request.

Reported in #15743.

Root cause

_try_activate_fallback() only sourced fb_base_url_hint from the fallback entry's own base_url field and from KNOWN_PROVIDERS. Named custom_providers entries were never consulted, so any provider whose connectivity lives entirely in custom_providers (no built-in entry) produced a None hint.

Fix

Before calling resolve_provider_client(), when neither fb_base_url_hint nor fb_api_key_hint is set, attempt a lookup via _get_named_custom_provider(fb_provider) and propagate base_url, api_key, and key_env as explicit hints:

if not fb_base_url_hint and not fb_api_key_hint:
    try:
        from hermes_cli.runtime_provider import _get_named_custom_provider
        _cp = _get_named_custom_provider(fb_provider)
        if _cp:
            fb_base_url_hint = (_cp.get("base_url") or "").strip() or None
            _cp_key = (_cp.get("api_key") or "").strip()
            _cp_key_env = (_cp.get("key_env") or "").strip()
            if not fb_api_key_hint:
                if _cp_key:
                    fb_api_key_hint = _cp_key or None
                elif _cp_key_env:
                    fb_api_key_hint = os.getenv(_cp_key_env, "").strip() or None
    except Exception:
        pass

The entire block is wrapped in except Exception: pass so a config-load failure degrades gracefully — resolve_provider_client() still runs with whatever it can derive from the provider name alone, exactly as before.

Precedence is preserved: the lookup only runs when both hints are empty. An inline base_url in the fallback entry always takes priority.

Tests

Three new tests in TestCustomProviderFallbackBaseUrl (appended to tests/run_agent/test_provider_fallback.py):

TestAssertion
test_custom_provider_base_url_used_when_not_in_fallback_entryexplicit_base_url is the custom provider's URL; agent.base_url is updated; agent.provider matches
test_inline_base_url_takes_precedence_over_custom_providers_lookup_get_named_custom_provider is not called when inline base_url is present
test_custom_provider_lookup_failure_does_not_crash_fallbackA RuntimeError from the lookup does not abort fallback activation

All 22 tests in the file pass (22 passed).

Checklist

  • Fixes the reported symptom end-to-end (wrong endpoint → correct endpoint)
  • Inline base_url continues to take precedence (no behaviour change for existing configs)
  • Lookup failure is silently swallowed — no regression for providers that aren't in custom_providers
  • Three targeted tests cover the happy path, the no-op path, and the error path
  • No new dependencies introduced

Closes #15743

Changed files

  • run_agent.py (modified, +22/-0)
  • tests/run_agent/test_provider_fallback.py (modified, +92/-0)

Code Example

model:
     provider: nous
     model: deepseek/deepseek-v4-pro
     base_url: https://inference-api.nousresearch.com/v1

---

fallback_model:
     provider: aliyun-singapore
     model: qwen3.6-plus

   custom_providers:
     - name: aliyun-singapore
       base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
       api_key: ${ALIYUN_SINGAPORE_API_KEY}
       models:
         qwen3.6-plus:
           context_length: 1000000

---

⚠️  API call failed (attempt 1/3): NotFoundError [HTTP 404]
   🔌 Provider: aliyun-singapore  Model: qwen3.6-plus
   🌐 Endpoint: https://inference-api.nousresearch.com/v1
   📝 Error: HTTP 404: Model 'qwen3.6-plus' not found. The requested model does not exist in our configuration or OpenRouter catalog.

---

fallback_model:
  provider: custom_openrouter
  model: deepseek/deepseek-v4-flash
RAW_BUFFERClick to expand / collapse

Describe the bug

When the primary provider hits a rate limit, Hermes falls back to a model whose provider is defined in custom_providers. However, the fallback request is sent to the primary provider's base_url instead of the fallback provider's own base_url, causing errors.

To Reproduce

  1. Configure a tentative primary provider that you know will hit rate limiting (e.g., Nous Portal with rate_limit: True or in a busy account):

    model:
  provider: nous
  model: deepseek/deepseek-v4-pro
  base_url: https://inference-api.nousresearch.com/v1

  2. Define a fallback model whose provider lives in custom_providers:

    fallback_model:
  provider: aliyun-singapore
  model: qwen3.6-plus

custom_providers:
  - name: aliyun-singapore
    base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
    api_key: ${ALIYUN_SINGAPORE_API_KEY}
    models:
      qwen3.6-plus:
        context_length: 1000000

  3. Trigger a rate limit on the primary provider.

  4. Observe the fallback request.

Expected behavior

The fallback request should be sent to https://dashscope.aliyuncs.com/compatible-mode/v1 (the aliyun-singapore provider's own base_url), using Authorization: Bearer <aliyun-singapore-api-key>.

Actual behavior

The fallback request is sent to https://inference-api.nousresearch.com/v1 (the primary nous provider's base_url), which naturally doesn't have the model qwen3.6-plus, resulting in:

⚠️  API call failed (attempt 1/3): NotFoundError [HTTP 404]
   🔌 Provider: aliyun-singapore  Model: qwen3.6-plus
   🌐 Endpoint: https://inference-api.nousresearch.com/v1
   📝 Error: HTTP 404: Model 'qwen3.6-plus' not found. The requested model does not exist in our configuration or OpenRouter catalog.

Root cause hypothesis

The fallback resolution logic correctly identifies the provider name (aliyun-singapore), but fails to look up its base_url from the custom_providers list. Instead it inherits the primary provider's base_url. This likely affects all fallback providers defined in custom_providers — they all end up hitting the primary provider's endpoint with an incompatible model name.

Workaround

Use custom_openrouter as the fallback provider (OpenRouter resolves model routing itself, so the wrong base_url to Nous is harmless for OpenRouter models):

fallback_model:
  provider: custom_openrouter
  model: deepseek/deepseek-v4-flash

But this is16 only a workaround. The root issue remains.

Environment

  • Hermes Agent: git HEAD
  • Config version: _config_version: 22 -alan macOS + CLI/TUI mode

extent analysis

TL;DR

The fallback request is sent to the primary provider's base_url instead of the fallback provider's own base_url, causing errors, and a fix would involve correctly looking up the base_url from the custom_providers list.

Guidance

  • Verify that the custom_providers list is correctly populated and the base_url for the fallback provider is set.
  • Check the fallback resolution logic to ensure it correctly identifies the provider name and looks up its base_url from the custom_providers list.
  • Consider modifying the fallback resolution logic to use the base_url from the custom_providers list instead of inheriting the primary provider's base_url.
  • Test the fallback request with a different fallback provider to see if the issue is specific to the aliyun-singapore provider.

Example

No code snippet is provided as the issue is more related to the logic and configuration rather than a specific code block.

Notes

The provided workaround using custom_openrouter as the fallback provider may not be suitable for all use cases, and the root issue remains. The fix would require modifying the fallback resolution logic to correctly look up the base_url from the custom_providers list.

Recommendation

Apply a workaround by using custom_openrouter as the fallback provider, as it resolves model routing itself and the wrong base_url to Nous is harmless for OpenRouter models, but note that this is only a temporary solution and the root issue should be addressed.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#generation error#database connection#vector store#embedding generation

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix Fallback provider defined in custom_providers ignores its own base_url, sends request to primary provider's endpoint instead [1 pull requests, 1 comments, 2 participants]