hermes - 💡(How to fix) Fix Feature Request: Support Native Gemini Context Caching

Feature Description

Support Google Gemini context caching (prompt caching) in the native Gemini adapter (agent/gemini_native_adapter.py) to reduce input token latency and cost for sessions with large system prompts and long multi-turn history.

Current Behavior

Currently, prompt caching is only implemented for Anthropic models. The agent uses the system_and_3 strategy in agent/prompt_caching.py to inject cache_control blocks into the messages payload.

For Gemini models running natively via GeminiNativeClient (configured via gemini or google-gemini-cli profiles), the request builder build_gemini_request constructs a payload that sends the entire message history and system instruction fresh on every turn. Because Gemini context caching requires calling the dedicated cachedContents API endpoint to create a cache resource, the current client implementation results in linear, uncached token billing on every turn.

For large-context agents (e.g., 40k+ tokens of system prompt + history), this leads to excessive API usage costs.

Proposed Implementation

To support Gemini context caching, the Gemini native adapter should implement the following workflow:

1. Threshold Check: Context caching on Gemini requires a minimum of 32,768 tokens (both for Google AI Studio and Vertex AI). The adapter should check if the stable prefix (system instructions + stable message history) exceeds this limit.
2. Cache Resource Creation: If the threshold is met, the client should send a POST request to the v1beta.cachedContents endpoint:

   POST https://generativelanguage.googleapis.com/v1beta/cachedContents

   With a body specifying the model, the system instruction, and contents to cache, along with a TTL (e.g., mapped from prompt_caching.cache_ttl in config.yaml, default 300s):

   {
     "model": "models/gemini-2.5-flash",
     "contents": [...],
     "systemInstruction": {...},
     "ttl": "300s"
   }

3. Cache Resource Reuse: The API returns a resource name (e.g., cachedContents/abc123xyz). The client must store this cache reference and attach it to subsequent chat completion requests in the request payload:

   {
     "contents": [... new turns ...],
     "cachedContent": "cachedContents/abc123xyz"
   }

4. Integration with prompt_caching.py: Align the Gemini cache invalidation logic with the existing prompt_caching.cache_ttl and session management configuration.

Additional Context

With the release of Gemini 1.5/2.5/3.5 models and their massive context windows, many users run Hermes Agent with highly detailed system prompts and deep memories. Adding native caching will significantly lower the cost barrier for running Hermes on Google's model suite.

If there is interest in this feature, our team is happy to implement this and submit a PR.