hermes - 💡(How to fix) Fix bug: fallback routing can select text-only models for image requests and hide the primary failure [1 pull requests]

Problem

When an image request fails on the primary model, Hermes can fall back to a model that does not support image input. The user-visible error then becomes the fallback model's modality error, hiding the original primary failure.

In my case this made the failure look like "Codex cannot do images", even though the primary error happened earlier in the openai-codex path.

Evidence

Primary model:

• provider: openai-codex
• model: gpt-5.5

Fallback model at the time:

• provider: openrouter
• model: deepseek/deepseek-v4-pro

Observed chain in logs:

Provider: openai-codex  Model: gpt-5.5
Error: 'NoneType' object is not iterable
Non-retryable error (HTTP None) — trying fallback...
Primary model failed — switching to fallback: deepseek/deepseek-v4-pro via openrouter
Provider: openrouter  Model: deepseek/deepseek-v4-pro
Error: HTTP 404: No endpoints found that support image input

A fresh direct image request through Hermes to openai-codex / gpt-5.5 succeeded in the same environment, so Codex image support itself is not the blocker.

I reproduced the primary failure with a direct API request to POST /v1/chat/completions using:

• Authorization: Bearer <API_SERVER_KEY>
• X-Hermes-Session-Id: <existing continued session>
• a standard OpenAI image_url user content part

That direct API request still produced the primary Agent-side failure:

Provider: openai-codex  Model: gpt-5.5
Error: 'NoneType' object is not iterable
Primary model failed — switching to fallback: ...

So this is reproducible at the Agent API boundary with a standard OpenAI-style image payload. The Codex NoneType failure is likely related to existing openai-codex issues, but fallback routing made the final error misleading.

Expected fix

For requests containing image/file/audio/video content, fallback routing should:

• skip fallback models that do not support the required input modality
• use provider metadata where available, for example OpenRouter architecture.input_modalities
• preserve the primary failure cause when a fallback also fails
• if no compatible fallback exists, return the primary failure plus a clear message that no compatible fallback is configured

Example error shape:

Primary openai-codex/gpt-5.5 failed with: 'NoneType' object is not iterable.
No configured image-capable fallback is available.

Why this matters

The current behavior points users at the wrong problem. The actionable failure was the primary Codex path; the visible error was from a secondary fallback route that could never handle the request's image input.