Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix WS2 item 4: wire run_agent.py provider profile activation (incremental) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#14515•Fetched 2026-04-24 06:16:51
View on GitHub
Comments
0
Participants
1
Timeline
11
Reactions
0
Author
kshitijk4poorkshitijk4poor
Participants
kshitijk4poorkshitijk4poor
Timeline (top)
labeled ×9cross-referenced ×2

Root Cause

PR #14424 adds providers/ package with 8 provider profiles and a transport single-path (_build_kwargs_from_profile). The profiles are proven correct via 73 parity tests. However, wiring run_agent.py to activate profiles caused 50 test regressions because of a fundamental detection mismatch.

Fix Action

Fixed

PR fix notes

PR #14424: feat: add provider modules — ProviderProfile ABC + 7 providers

Description (problem / solution / changelog)

Summary

Cycle 2 PR 1 (#14418). Introduces providers/ package with ProviderProfile ABC and wires first two providers live.

What this PR delivers

  1. ProviderProfile ABC (providers/base.py) — dataclass declaring auth, endpoints, quirks, hooks
  2. Provider registry (providers/__init__.py) — auto-discovery via pkgutil.iter_modules, get_provider_profile() with alias resolution
  3. 8 provider modules: NVIDIA, Kimi + Kimi-CN, OpenRouter, Nous, DeepSeek, Qwen
  4. Transport single-path (ChatCompletionsTransport._build_kwargs_from_profile()) — replaces 28 flag params with one profile object
  5. run_agent.py wiring — NVIDIA and DeepSeek activated via _PROFILE_ACTIVE_PROVIDERS allowlist
  6. 78 tests: 30 profile declarations + 19 transport parity + 22 profile wiring + 2 override parity + 5 E2E wiring

Provider activation strategy

Providers are activated incrementally via an explicit allowlist in _build_api_kwargs:

_PROFILE_ACTIVE_PROVIDERS = frozenset({
    "nvidia", "nvidia-nim",
    "deepseek", "deepseek-chat",
})

These two have zero special params — just default_max_tokens (NVIDIA) and baseline (DeepSeek). Remaining providers (OpenRouter, Nous, Kimi, Qwen) need additional agent-level params (anthropic_max_output, supports_reasoning, qwen_session_metadata) before activation. See #14515.

Test plan

  • 78 provider tests pass (profiles + parity + wiring + E2E)
  • Full agent/run_agent suite: 2683 passed, 3 pre-existing failures

Changed files

  • agent/transports/__init__.py (modified, +15/-5)
  • agent/transports/anthropic.py (modified, +10/-10)
  • agent/transports/base.py (modified, +7/-7)
  • agent/transports/bedrock.py (modified, +11/-7)
  • agent/transports/chat_completions.py (modified, +146/-29)
  • agent/transports/codex.py (modified, +26/-21)
  • agent/transports/types.py (modified, +16/-15)
  • providers/__init__.py (added, +66/-0)
  • providers/base.py (added, +77/-0)
  • providers/deepseek.py (added, +13/-0)
  • providers/kimi.py (added, +69/-0)
  • providers/nous.py (added, +46/-0)
  • providers/nvidia.py (added, +14/-0)
  • providers/openrouter.py (added, +45/-0)
  • providers/qwen.py (added, +80/-0)
  • pyproject.toml (modified, +44/-5)
  • run_agent.py (modified, +33/-1)
  • tests/providers/__init__.py (added, +0/-0)
  • tests/providers/test_e2e_wiring.py (added, +87/-0)
  • tests/providers/test_profile_wiring.py (added, +293/-0)
  • tests/providers/test_provider_profiles.py (added, +203/-0)
  • tests/providers/test_transport_parity.py (added, +250/-0)

Code Example

_is_nvidia = "integrate.api.nvidia.com" in self._base_url_lower
_is_kimi = base_url_host_matches(self.base_url, "api.kimi.com") or ...
_is_qwen = self._is_qwen_portal()  # checks base_url

---

_profile = get_provider_profile(self.provider)  # "nvidia", "openrouter", etc.
RAW_BUFFERClick to expand / collapse

Problem

PR #14424 adds providers/ package with 8 provider profiles and a transport single-path (_build_kwargs_from_profile). The profiles are proven correct via 73 parity tests. However, wiring run_agent.py to activate profiles caused 50 test regressions because of a fundamental detection mismatch.

The mismatch

Legacy detection: URL pattern matching

_is_nvidia = "integrate.api.nvidia.com" in self._base_url_lower
_is_kimi = base_url_host_matches(self.base_url, "api.kimi.com") or ...
_is_qwen = self._is_qwen_portal()  # checks base_url

Profile resolution: Provider name matching

_profile = get_provider_profile(self.provider)  # "nvidia", "openrouter", etc.

A user with provider="custom" and base_url="https://integrate.api.nvidia.com/v1" hits the NVIDIA path in legacy but gets NO profile. Conversely, tests that set provider="openrouter" without a matching base_url now hit the profile path when they expected legacy.

Missing params in profile path

The legacy chat_completions _build_api_kwargs computes these agent-level params that the profile path doesn't:

  1. anthropic_max_output — Claude max output for OpenRouter/Nous (calls _get_anthropic_max_output)
  2. qwen_session_metadata — Random UUID per call
  3. supports_reasoning — Model-family-gated reasoning safety check
  4. github_reasoning_extra — GitHub Models reasoning config
  5. _fixed_temperature_for_model — Temperature override from auxiliary_client
  6. provider_preferences — OpenRouter provider routing preferences

Proposed approach

Wire profiles one provider at a time, simplest first:

  1. NVIDIA (only default_max_tokens, zero missing params)
  2. DeepSeek (minimal profile, zero extras)
  3. Kimi (OMIT_TEMPERATURE + reasoning_effort — profile handles both)
  4. Qwen (needs session_metadata generation in profile)
  5. OpenRouter (needs anthropic_max_output + supports_reasoning + provider_preferences)
  6. Nous (similar to OpenRouter)

Each wiring step: add the missing params to the profile, write an E2E test that calls _build_api_kwargs with real agent state, verify identical output, then enable.

Acceptance criteria

  • Each provider profile produces identical build_kwargs output to the legacy flag path
  • Tests that create agents with provider="X" work correctly on the profile path
  • provider="custom" with provider-specific base_url still hits legacy path (profile returns None)

extent analysis

TL;DR

Wire provider profiles one at a time, starting with the simplest, and add missing parameters to each profile to ensure identical output to the legacy flag path.

Guidance

  • Begin with the NVIDIA provider, as it has zero missing parameters, and write an E2E test to verify identical output.
  • For each subsequent provider (e.g., DeepSeek, Kimi, Qwen, OpenRouter, Nous), add the required missing parameters to the profile and write corresponding E2E tests.
  • Verify that tests creating agents with provider="X" work correctly on the profile path and that provider="custom" with a provider-specific base URL still hits the legacy path.
  • Ensure that the acceptance criteria are met for each provider before moving on to the next one.

Example

# Example of adding missing parameters to a profile (e.g., Qwen)
qwen_profile = {
    # ... existing profile config ...
    'qwen_session_metadata': lambda: uuid.uuid4(),  # generate random UUID per call
    # ... other missing parameters ...
}

Notes

The proposed approach of wiring profiles one provider at a time, simplest first, seems reasonable. However, it is crucial to thoroughly test each provider to ensure that the profile path produces identical output to the legacy flag path.

Recommendation

Apply the proposed workaround of wiring provider profiles one at a time, starting with the simplest, to ensure a gradual and controlled rollout of the new profile-based system. This approach allows for thorough testing and verification of each provider, reducing the risk of introducing regressions.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#model loading#dependency error#configuration error#environment variable

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING