PR #17337: fix(tools): isolate get_tool_definitions quiet_mode cache + dedup LCM injection (#17335)

• Repository: NousResearch/hermes-agent
• Author: Sanjays2402
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/17337

Description (problem / solution / changelog)

Closes #17335.

Problem

Long-lived Gateway processes (Feishu, etc.) were sending duplicate tool names to providers that enforce uniqueness:

• DeepSeek: Tool names must be unique.
• Xiaomi MiMo: tools contains duplicate names: lcm_expand
• Moonshot/Kimi: function name lcm_grep is duplicated

TUI was unaffected because TUI uses quiet_mode=False and skips the cache.

Root Cause (two layered bugs)

1. model_tools.get_tool_definitions(quiet_mode=True) aliased the cached object on the first call. The cache-hit path (line 278) already returned list(cached) — safe. But the first uncached call stored and returned the same object. run_agent.py then mutates self.tools in-place (appending memory + LCM context-engine schemas), so the very first agent init in a Gateway process poisoned the cache, and every subsequent init appended LCM schemas again on top of the already-polluted list.

2. run_agent.py's context-engine injection had no dedup. Memory-tools injection (lines 1728–1748) already skips already-present names. The LCM injection right below it (lines 1986–1993) didn't. So even after fixing the cache, plugin paths that register schemas via ctx.register_tool() could still produce duplicates.

Fix (defense in depth, exactly as the issue suggested)

model_tools.py — on the uncached branch, cache the result but return list(result) to the caller, mirroring the cache-hit path:

result = _compute_tool_definitions(...)
if quiet_mode:
    _tool_defs_cache[cache_key] = result
    return list(result)
return result

run_agent.py — build _existing_tool_names from self.tools and skip already-present schemas, mirroring the memory-tools block above:

_existing_tool_names = {t.get("function", {}).get("name") for t in self.tools if isinstance(t, dict)}
for _schema in self.context_compressor.get_tool_schemas():
    _tname = _schema.get("name", "")
    if _tname and _tname in _existing_tool_names:
        continue
    ...
    _existing_tool_names.add(_tname)

Tests

New file tests/test_get_tool_definitions_cache_isolation.py:

• test_first_uncached_call_returns_fresh_list — pins the fix; without it, the first-call alias is the entire bug.
• test_cache_hit_returns_fresh_list — pre-existing #17098 behavior stays.
• test_caller_mutation_does_not_poison_cache — simulates run_agent appending lcm_grep / lcm_expand to the returned list and asserts the next call doesn't see them.
• test_repeated_caller_mutation_does_not_accumulate — reproduces the long-lived Gateway accumulation across 5 agent inits.
• test_non_quiet_mode_does_not_use_cache — sanity, explains why TUI was unaffected.

$ python -m pytest tests/test_get_tool_definitions_cache_isolation.py tests/test_model_tools.py -q
............................                                            [100%]
28 passed in 0.78s

5/5 new tests pass; 23/23 existing tests/test_model_tools.py still pass.

Changed files

• model_tools.py (modified, +8/-0)
• run_agent.py (modified, +17/-2)
• tests/test_get_tool_definitions_cache_isolation.py (added, +94/-0)

PR #5713: feat(caching): multi-block system prompt with tiered TTLs (v2)

• Repository: NousResearch/hermes-agent
• Author: Deland78
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/5713

Description (problem / solution / changelog)

Summary

Refactor Anthropic prompt caching to use a structured multi-block system prompt with per-block cache_control markers instead of a single monolithic system message. This maximizes cache hits by isolating volatile content (timestamps, platform hints) from stable content (identity, skills, memory).

Architecture

The system prompt is now assembled as three SystemPromptBlock instances with different cache TTLs:

At API call time, blocks are converted to Anthropic content block format (`[{type: text, text: ..., cache_control: ...}, ...]`) and sent as the system message. Non-caching models fall through to the flat-string path unchanged.

New public API in `agent/prompt_caching.py`

• `SystemPromptBlock`, `CacheMetrics`, `AggregatedCacheMetrics` dataclasses
• `build_system_content_blocks(blocks)` — convert blocks to Anthropic format
• `apply_anthropic_cache_control_v2(messages, tools, cache_ttl, native_anthropic)` — multi-block + tool caching with budget management (max 4 breakpoints across tools + system + messages)
• `extract_cache_metrics(usage, api_mode)` — per-call cache extraction supporting both native Anthropic (`cache_read_input_tokens`, `cache_creation_input_tokens`) and OpenRouter (`prompt_tokens_details.cached_tokens`) response formats
• `aggregate_cache_metrics(metrics_list)` — cross-turn aggregation

The v1 `apply_anthropic_cache_control` function and `_apply_cache_marker` helper are preserved unchanged for backward compatibility.

Integration in `run_agent.py`

• New `_build_system_prompt_blocks()` method assembles the three tiered blocks and caches them on `self._cached_system_blocks`
• The existing `_build_system_prompt()` method still returns a flat string (for backward compatibility with code paths that expect one) but now delegates to the block builder
• Cached blocks are invalidated on context compression (`_cached_system_blocks = None` alongside `_cached_system_prompt = None`)
• At API call time, when `_use_prompt_caching` is enabled and `_cached_system_blocks` is populated, a multi-block path builds `{role: system, content: [...]}` with cache_control markers already set per block
• Plugin turn context (`_plugin_turn_context`) remains reserved for future system-level plugin instructions; plugin context from pre_llm_call hooks still goes into user messages (unchanged)
• Fallback flat-string path handles non-caching models and pre-structured content correctly

Test coverage

• `tests/agent/test_prompt_caching.py` — 46 unit tests covering v1 (preserved) and v2 functions: data structures, cache markers, content block conversion, pre-structured detection, breakpoint budgeting, metrics extraction and aggregation
• `tests/agent/test_prompt_caching_v2.py` — 38 additional integration tests for v2 behavior (tool caching interaction with system blocks, budget with pre-structured content, backward compatibility with v1 code paths)
• `tests/test_prompt_caching_integration.py` — 10 integration tests against `run_agent.py` block assembly (three-block structure, tier TTLs, timestamp in ephemeral block only, cache invalidation, backward-compat string return, non-caching models unaffected)

Verified: 317 tests passing (all of the above plus `tests/test_run_agent.py` regression suite).

Test plan

• All new v2 unit tests pass (`pytest tests/agent/test_prompt_caching.py tests/agent/test_prompt_caching_v2.py`)
• Integration tests against `run_agent.py` block assembly pass (`pytest tests/test_prompt_caching_integration.py`)
• Full run_agent.py regression suite passes (`pytest tests/test_run_agent.py`)
• `run_agent` imports cleanly
• Manual: verify cache hit rate improves on a multi-turn conversation with stable context files (reviewer action)
• Manual: verify non-caching models (e.g. local Ollama) still work via flat-string fallback (reviewer action)

Platforms tested

Linux (WSL2, Ubuntu 22.04), Python 3.11

🤖 Generated with Claude Code

Changed files

• agent/prompt_caching.py (modified, +252/-6)
• run_agent.py (modified, +169/-83)
• tests/agent/test_prompt_caching.py (modified, +316/-0)
• tests/agent/test_prompt_caching_v2.py (added, +469/-0)
• tests/test_prompt_caching_integration.py (added, +209/-0)

PR #10006: fix: codex_responses prompt caching — session routing headers + cache_write_tokens field

• Repository: NousResearch/hermes-agent
• Author: zicochaos
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/10006

Description (problem / solution / changelog)

Problem

Prompt caching does not work when using codex_responses API mode with OpenAI-compatible providers (e.g. theclawbay). Every request is a cache miss despite prompt_cache_key being set in the request body.

Root cause

Two issues:

1. Missing session routing headers (run_agent.py)

The OpenAI client is initialized without session_id or x-client-request-id headers for codex providers. These headers are required for server-side cache routing — they tell the backend to route requests to the same server that holds the cached prompt prefix.

The official Codex CLI sends these unconditionally. Hermes sets default_headers for OpenRouter, GitHub Copilot, Kimi, and Qwen — but never for Codex/theclawbay.

2. Wrong field name for cache_write_tokens (agent/usage_pricing.py)

The codex_responses branch reads cache_creation_tokens (Anthropic naming convention) instead of cache_write_tokens (OpenAI Responses API naming). This means cache write tokens are always reported as 0.

Fix

Patch 1: Session routing headers

After session_id is assigned during __init__, inject session_id and x-client-request-id into default_headers for codex_responses mode. Also applied in _apply_client_headers_for_base_url() so headers survive /model switches.

Patch 2: cache_write_tokens field

Read cache_write_tokens first (OpenAI naming), fall back to cache_creation_tokens for backward compatibility.

Tests

• test_codex_responses_reads_cache_write_tokens_field — verifies correct field is read
• test_codex_responses_falls_back_to_cache_creation_tokens — backward compat
• test_codex_responses_injects_session_routing_headers — verifies headers are set

Affected files

Changed files

• agent/usage_pricing.py (modified, +3/-1)
• run_agent.py (modified, +22/-0)
• tests/agent/test_usage_pricing.py (modified, +36/-0)
• tests/run_agent/test_run_agent.py (modified, +19/-0)

PR #17736: fix: dedup context engine tools + fix kimi aux deadcode model name

• Repository: NousResearch/hermes-agent
• Author: wjameswen888
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/17736

Description (problem / solution / changelog)

What does this PR do?

Two small bugfixes from local-mods:

1. Dedup context engine tools — lcm_grep/lcm_describe/lcm_expand were injected without checking valid_tool_names. On session restore or context-engine re-init, get_tool_schemas() returns the same tools → duplicate function names → 400 from DeepSeek/Moonshot.

2. Fix Kimi aux deadcode — _API_KEY_PROVIDER_AUX_MODELS["kimi-coding-cn"] pointed to nonexistent model kimi-k2-turbo-preview. Changed to kimi-k2.6.

Related Issue

Fixes #

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

How to Test

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: <!-- e.g. Ubuntu 24.04, macOS 15.2, Windows 11 -->

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

• This skill is broadly useful to most users (if bundled) — see Contributing Guide
• SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
• No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
• I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

Changed files

• agent/auxiliary_client.py (modified, +1/-1)
• run_agent.py (modified, +5/-1)