hermes - 💡(How to fix) Fix but the main model's compression threshold was [2 comments, 2 participants]

TL;DR

Edit the config.yaml file to either use a larger compression model or lower the compression threshold to match the compression model's context.

Guidance

• The warning occurs because the compression model's context (256,000 tokens) is lower than the main model's compression threshold (500,000 tokens).
• To fix this, you can either update the config.yaml file to use a larger compression model with a context of 500,000+ tokens.
• Alternatively, you can lower the compression threshold to 0.25 in the config.yaml file to match the current compression model's context.

Example

compression:
  threshold: 0.25

or

auxiliary:
  compression:
    model: <model-with-500,000+-context>

Notes

The exact model name with a 500,000+ context is not specified, so you will need to find a suitable replacement model.

Recommendation

Apply workaround: edit the config.yaml file to lower the compression threshold to 0.25, as this is a simpler change that can be made immediately.