hermes - 💡(How to fix) Fix Per-model or per-provider compression threshold overrides [1 participants]

Feature Description

Allow setting different compression.threshold values per model or per provider, instead of a single global value.

Motivation

With 1M+ context models becoming more common (e.g. DeepSeek V4 Flash, MiMo V2.5 Pro, Gemini 2.5 Pro), a single global threshold creates a tension:

• Large context models (1M): A threshold of 0.5 means compression only triggers at 500K tokens — most conversations never reach this, so the feature is effectively unused.
• Small context models (128K): The same 0.5 threshold triggers at 64K, which is reasonable.

Users who switch between models (e.g. MiMo 1M for main chat, Claude 200K for coding) need different compression strategies. A global ratio that works well for 128K models is too conservative for 1M models.

Proposed Solution

Option A — Per-provider override:

compression:
  enabled: true
  threshold: 0.5          # global default
  target_ratio: 0.2
  providers:
    xiaomi:
      threshold: 0.3       # for 1M context models, compress earlier
    anthropic:
      threshold: 0.65      # keep Claude existing behavior

Option B — Per-model override:

compression:
  threshold: 0.5
  model_overrides:
    - model: mimo-v2.5-pro
      threshold: 0.3
    - model: claude-sonnet-4
      threshold: 0.65

Option A is simpler and covers most cases (providers tend to have similar context sizes). Option B is more granular but adds config complexity.

Alternatives Considered

• Dynamic threshold based on context size: Automatically lower the threshold ratio as context size grows (e.g. threshold = min(0.5, 100000 / context_length)). This would be zero-config but less predictable.
• Absolute token threshold: Instead of a ratio, use a fixed token count (e.g. compress at 64K tokens regardless of model). Simpler reasoning but loses the relative scaling.

Context

As a user running MiMo V2.5 Pro (1M context) as main model, I set threshold to 0.5 but this effectively means compression almost never triggers. Lowering to 0.3 would make more sense for 1M models, but that same 0.3 would be too aggressive for 128K models in the fallback chain.