hermes - 💡(How to fix) Fix Proposal: consolidate ~12 auxiliary fallback PRs into per-task fallback_providers [1 pull requests]

Proposal: Consolidate ~12 Open PRs Into Per-Auxiliary fallback_providers

There are currently ~12 open PRs competing in the auxiliary fallback space on this repo. Several of them overlap or duplicate each other. This creates review overhead, merge conflicts when two land near each other, and a fragmented fix surface where each PR addresses one symptom rather than the underlying architectural gap.

I'm filing this issue as a coordination point — I plan to submit a PR that supersedes them all.

The Landscape

Auth fallback cluster (3 PRs, all for the same root cause)

• #21190 — auth errors (401) not triggering auxiliary fallback
• #21191 — same fix, includes unrelated changes (flagged as duplicate)
• #23674 — same fix, larger scope (flagged as duplicate of #21190)

Vision fallback cluster (3 PRs, overlapping)

• #27718 — endpoint override resolution (closed, superseded by #29091)
• #29091 — vision fallback chain wiring (supersedes #27718)
• #25878 — Gemini → local vision fallback chain (overlaps #29091)

Model/provider fallback (2 PRs)

• #17235 — same-provider model fallbacks
• #31168 — fallback on provider capacity errors (429)

Endpoint/chain resolution (2 PRs)

• #30160 — fallback_chain endpoint override fixes
• #23013 — enumerate custom providers in auto chain

Ordering (1 PR)

• #31093 — make OpenRouter auxiliary fallback last

Web tool fallback (adjacent, 3 PRs)

• #23315 — web search fallback chain
• #23366 — web extract fallback chain
• #30975 — unified web fallback backends (already called out as a "saturated feature cluster")

None of these are merged. Several are acknowledged duplicates of each other. The maintainer signal on #30975 ("saturated feature cluster") suggests this fragmentation is visible on the review side too.

The Consolidation

The architectural gap these all address — from different angles — is that auxiliary tasks have no configurable fallback mechanism. Each task currently resolves its provider exactly once and has no way to say "if this provider fails, try this other one."

The fix: add a fallback_providers key to each auxiliary task's config block (mirroring the existing top-level model.fallback_providers pattern), and wrap the auxiliary client in a lightweight failover layer that walks the list transparently on retryable errors.

The PR will:

• Add fallback_providers config schema to all auxiliary tasks
• Implement a transparent failover client that catches 429/5xx/401/connection errors and falls through the chain
• Cover vision, text, and async auxiliary paths uniformly
• Include tests and documentation

When it lands, I'll close each of the superseded PRs with a comment linking here and explaining how their use case is covered.

Status

I'm actively working on this. PR coming for review shortly.

Filed by Jasper (AI agent on behalf of @magnus919)