Overview

When a fallback model is activated in Hermes Agent, the _fallback_sticky flag keeps the fallback active for the entire session. This is currently hardcoded behavior. This issue proposes exposing it as a user-configurable option.

Current Behavior

When the primary model triggers a fallback (e.g., qwen-local gets a TransportError), the system activates the _fallback_sticky mechanism. Once triggered, the fallback model persists for the entire session until /new or /reset is called.

The core logic resides in run_agent.py:

def _restore_primary_runtime(self):
    if getattr(self, "_fallback_sticky", False):
        return False
    # Attempt to restore primary model
    ...

The gateway layer mirrors this with _session_fallback_sticky, ensuring persistence even across agent instance recreations.

Why This Matters

1. Performance optimization — Avoids a "3 retries → fallback" cycle on every turn
2. Cost predictability — Prevents each message from re-triggering the full retry + fallback flow during brief primary model outages
3. Predictability — Users know that once fallback activates, the session uses a single consistent model

Proposed Enhancement

Add a sticky option to the fallback_model configuration in config.yaml:

fallback_model:
  provider: deepseek
  model: deepseek-v4-flash
  base_url: https://api.deepseek.com/v1
  sticky: true  # Default: true, backward compatible

• sticky: true — Current behavior: fallback persists for the entire session
• sticky: false — Each turn attempts to restore the primary model (useful for networks with frequent fluctuations)

Implementation Notes

• Default to true for backward compatibility
• Only requires changing the initial value of _fallback_sticky from a hardcoded False to reading config.fallback_model.sticky
• Gateway layer synchronously updates _session_fallback_sticky initialization logic

References

• run_agent.py — _restore_primary_runtime(), _fallback_sticky
• gateway/run.py — _session_fallback_sticky