hermes - 💡(How to fix) Fix Feature: native Google Cloud Vertex AI provider support [2 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#13484Fetched 2026-04-22 08:06:09
View on GitHub
Comments
2
Participants
2
Timeline
8
Reactions
0
Timeline (top)
labeled ×5commented ×2subscribed ×1

Hermes currently has no working path for Google Cloud Vertex AI. The entry in HERMES_OVERLAYS for google-vertex exists, but there's no auth machinery: Vertex uses short-lived OAuth access tokens derived from a service-account JSON, not a static API key. As a result, setting fallback_model.provider: google-vertex silently fails with "provider not configured" on every cron tick. Happy to share log excerpts.

This came up in a solo-founder setup where the goal was to use GCP credits to cover primary Gemini plus fallback inference (burning credits instead of paying cash), but the provider plumbing blocks that path today.

Root Cause

Hermes currently has no working path for Google Cloud Vertex AI. The entry in HERMES_OVERLAYS for google-vertex exists, but there's no auth machinery: Vertex uses short-lived OAuth access tokens derived from a service-account JSON, not a static API key. As a result, setting fallback_model.provider: google-vertex silently fails with "provider not configured" on every cron tick. Happy to share log excerpts.

This came up in a solo-founder setup where the goal was to use GCP credits to cover primary Gemini plus fallback inference (burning credits instead of paying cash), but the provider plumbing blocks that path today.

Fix Action

Fix / Workaround

"google-vertex": HermesOverlay(
    transport="anthropic_messages",   # or dispatch by model prefix
    auth_type="gcp_service_account",
    base_url_override="https://{region}-aiplatform.googleapis.com/...",
),

Code Example

custom_providers:
  - name: vertex-gemini
    base_url: http://127.0.0.1:8787/gemini
    transport: openai_chat
  - name: vertex-anthropic
    base_url: http://127.0.0.1:8787/anthropic
    transport: anthropic_messages

fallback_model:
  provider: vertex-gemini
  model: gemini-2.5-pro

---

"google-vertex": HermesOverlay(
    transport="anthropic_messages",   # or dispatch by model prefix
    auth_type="gcp_service_account",
    base_url_override="https://{region}-aiplatform.googleapis.com/...",
),
RAW_BUFFERClick to expand / collapse

Feature: native Google Cloud Vertex AI provider support

Context

Hermes currently has no working path for Google Cloud Vertex AI. The entry in HERMES_OVERLAYS for google-vertex exists, but there's no auth machinery: Vertex uses short-lived OAuth access tokens derived from a service-account JSON, not a static API key. As a result, setting fallback_model.provider: google-vertex silently fails with "provider not configured" on every cron tick. Happy to share log excerpts.

This came up in a solo-founder setup where the goal was to use GCP credits to cover primary Gemini plus fallback inference (burning credits instead of paying cash), but the provider plumbing blocks that path today.

What I built in the meantime (happy to upstream)

I wrote a standalone proxy that sits on 127.0.0.1:8787, handles GCP service-account auth plus 50-minute token refresh, and exposes Anthropic Messages API, Gemini generateContent API, and OpenAI Chat Completions endpoints backed by Vertex AI.

Repo: https://github.com/prasadus92/vertex-proxy

The Hermes integration is zero-code. It works through the existing custom_providers mechanism:

custom_providers:
  - name: vertex-gemini
    base_url: http://127.0.0.1:8787/gemini
    transport: openai_chat
  - name: vertex-anthropic
    base_url: http://127.0.0.1:8787/anthropic
    transport: anthropic_messages

fallback_model:
  provider: vertex-gemini
  model: gemini-2.5-pro

So for users like me, the external shim is already a complete solution. The open question is whether Hermes wants a native Vertex path that avoids the localhost hop.

Two possible Hermes changes (if any)

Option 1: Add a gcp_service_account auth type

Teach hermes_cli/auth.py to load a GCP service-account JSON, refresh access tokens in the background, and present them as bearer tokens to the normal anthropic_messages and openai_chat transports. Then add overlays:

"google-vertex": HermesOverlay(
    transport="anthropic_messages",   # or dispatch by model prefix
    auth_type="gcp_service_account",
    base_url_override="https://{region}-aiplatform.googleapis.com/...",
),

Roughly 150-200 lines of code plus credential pool changes. I have a reference implementation in vertex-proxy that I can port.

Option 2: Document the custom_providers pattern

Leave Hermes core unchanged, publish an example in docs/ showing the shim pattern. Lower maintenance burden, works for everyone today, no credential-type surgery in Hermes core.

What I'd appreciate feedback on

  1. Do you want native Vertex AI support in Hermes core, or is the custom_providers plus external shim pattern the preferred long-term answer?
  2. If native support is welcome, I'm happy to open a PR. Preferred scope? (Just Gemini? Gemini plus Claude? Full Vertex MaaS partner models: Kimi, GLM, MiniMax, Qwen, Grok?)
  3. Any constraints on new dependencies? google-auth is the cleanest way to handle service-account token refresh, but it's a ~2MB transitive dep chain. Alternative is hand-rolling JWT to token exchange in ~50 lines of pure stdlib.

Happy to scope a PR either way. Thanks for the excellent tool. The custom_providers abstraction made the external shim path clean.

extent analysis

TL;DR

Implement native Google Cloud Vertex AI support in Hermes core by adding a gcp_service_account auth type or document the custom_providers pattern as a long-term solution.

Guidance

  • Evaluate the trade-offs between implementing native Vertex AI support in Hermes core versus documenting the custom_providers pattern, considering maintenance burden and flexibility.
  • If native support is preferred, determine the scope of the implementation, such as supporting only Gemini or including other Vertex MaaS partner models.
  • Assess the impact of adding new dependencies, like google-auth, on the project's overall size and complexity.
  • Consider the potential benefits of using an external shim, like the provided vertex-proxy, as a complete solution for users.

Example

No code example is provided, as the issue is focused on design and implementation decisions rather than specific code changes.

Notes

The decision to implement native Vertex AI support or document the custom_providers pattern depends on the project's goals, maintenance priorities, and user needs. The custom_providers pattern offers a flexible solution, while native support may provide a more seamless user experience.

Recommendation

Apply the custom_providers pattern as a long-term solution, as it offers flexibility and works for all users today, with minimal maintenance burden and no credential-type changes in Hermes core.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - 💡(How to fix) Fix Feature: native Google Cloud Vertex AI provider support [2 comments, 2 participants]