hermes - 💡(How to fix) Fix [Bug]: 128K token， cannot be used [1 participants]

⚠️ API call failed (attempt 1/3): APIError 🔌 Provider: custom Model: Qwen3.6-35B-A3B-UD-Q4_K_M.gguf 🌐 Endpoint: http://192.168.1.13:8080/v1 📝 Error: Context size has been exceeded. ⚠️ Context length exceeded — stepping down: 131,072 → 128,000 tokens 🗜️ Context too large (~42,664 tokens) — compressing (1/3)...

TL;DR

Reduce the context length to below 68K tokens to prevent the API call from failing due to exceeded context size.

Guidance

• Verify the current context length and adjust it to a value below 68K tokens to prevent errors.
• Check the Hermes Agent and LLM Inference Platform configurations to ensure they are set to handle context lengths below 68K tokens.
• Consider implementing context compression or splitting to handle larger context sizes.
• Review the error logs to confirm that the issue is indeed related to context size exceeding the limit.

Example

No code snippet is provided as the issue does not imply a specific code change.

Notes

The root cause of the issue appears to be the context size exceeding the limit, but the exact solution may depend on the specific requirements of the Hermes Agent and LLM Inference Platform.

Recommendation

Apply workaround: Reduce the context length to prevent errors, as the exact fix may require further investigation into the Hermes Agent and LLM Inference Platform configurations.