hermes - 💡(How to fix) Fix Proposal: progressive tool-result compression to reduce token waste in long conversations [1 participants]

Problem

In long conversations (40+ turns), old tool results — file contents, command outputs, search results — consume thousands of tokens that the model no longer needs verbatim. The model only needs to remember: WHAT tool was called, and the OUTCOME (success/failure + key result).

Current behavior

The existing ContextCompressor (threshold-triggered LLM summarization) handles this, but only after a 413/overflow event — it's a reactive, heavyweight mechanism that permanently mutates self.messages.

Before that trigger point, every API call sends the full verbatim content of all historical tool results. In a 50-turn session with 30 tool calls, this can easily be 50K+ tokens of stale tool output that the model has already acted upon.

Why it matters

1. Token waste — Each API call pays for tokens the model doesn't need. In a coding session with many read_file and terminal calls, old outputs are pure waste after the model has moved on.
2. Earlier 413 triggers — Stale tool results push the context toward the threshold faster, causing more frequent full compression events (which are expensive, irreversible, and disruptive).
3. Degraded reasoning — More tokens in context = more noise for the model to sift through. Compact summaries of old results can actually improve focus.

Proposed solution: Progressive tool-result compression

An ephemeral, per-API-call optimization that compresses old tool results to one-line summaries before each LLM call — complementing (not replacing) the existing ContextCompressor.

How it works

Before each API call:
  1. Identify all role="tool" messages in api_messages
  2. Keep the last N tool results intact (recent context)
  3. For older tool results, replace content with a compact summary:
     [read_file] OK (12847 chars) | import os → from pathlib import Path

Key design decisions

Relationship to existing compression

The two are orthogonal: progressive compression reduces token waste on every call, which delays the need for a full ContextCompressor trigger.

Configuration

agent:
  compression:
    progressive:
      enabled: true           # defaults to compression.enabled
      recent_tool_keep: 20    # defaults to compression.protect_last_n
      min_messages: 16        # only activate in long conversations
      max_compressed_len: 300 # skip results shorter than this

Evidence

Token savings (benchmark)

Simulated conversations with read_file tool calls (the most common token-heavy pattern):

Per-result compression ratio in the Large scenario: 64.7x (5,144 chars → 80 chars per compressed result).

For typical coding sessions (20-30 tool calls), this means 40-70K fewer tokens per API call, directly translating to lower cost and later 413 triggers.

Implementation status

I have a working implementation with 32 unit tests covering all branches (no-op paths, boundary conditions, immutability, edge cases). Happy to submit a PR if there's interest.

Questions for maintainers

1. Is this direction something you'd want in the core?
2. Any preference on the summary format or the default thresholds?
3. Should this be opt-in (default false) or opt-out (default true, respecting compression.enabled)?