hermes - 💡(How to fix) Fix Feature Request: Automatic vision fallback for non-vision primary models

Problem

When using a primary model that does not support vision (e.g. DeepSeek), sending an image causes the entire request to fail with:

unknown variant `image_url`, expected `text`

The failure happens at the API level — DeepSeek rejects image_url before the agent sees a response. The agent cannot recover.

Current workaround

1. /new (lose context)
2. Switch primary model to Gemini/GPT-4o
3. Send image again
4. Switch back to DeepSeek

This is disruptive — especially on Telegram where voice+image messages are common.

Proposed Solution

Pre-processing layer in the agent loop that intercepts image attachments before they reach the primary model:

User sends image + text
       |
       v
+----------------------+
| Vision Pre-processor |
| 1. Detect image_url  |
| 2. Check if primary  |
|    model has vision  |
| 3. If NOT: send to   |
|    auxiliary.vision  |
|    -> get descriptn  |
| 4. Replace image_url |
|    with text block   |
+----------------------+
       |
       v
  DeepSeek receives
  text-only message OK

Config addition

auxiliary:
  vision:
    provider: google
    model: gemini-2.5-flash
    auto_fallback: true   # NEW

Benefits

• Seamless UX — no /new, no model switching, no context loss
• Best-of-both-worlds — cheap text models for reasoning + specialized vision models for images
• Works mid-session — images just work
• No API changes — purely client-side preprocessing

Edge Cases

• Multiple images per message — process sequentially or parallel
• Mixed content — preserve text alongside descriptions
• Vision model failure — graceful degradation with a note
• Cost tracking — auxiliary calls should appear in usage stats
• Streaming — vision must complete before primary model stream begins

Submitted via Hermes Agent on behalf of Vitaliy Li