Issue Station
Sign In
Tools hereToolbox

openclaw - ✅(Solved) Fix Bug: OpenRouter path does not apply cache_control to conversation messages (only system prompt) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#63034•Fetched 2026-04-09 07:59:15
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
liu51115liu51115
Participants
liu51115liu51115
Timeline (top)
cross-referenced ×1referenced ×1

The OpenRouter code path (applyAnthropicEphemeralCacheControlMarkers) only applies cache_control markers to system/developer role messages. It does NOT apply markers to conversation messages (user/assistant roles). This means on OpenRouter, only the system prompt is cached — the entire conversation history is re-processed on every turn.

In contrast, the direct Anthropic path (applyAnthropicPayloadPolicyToParams) applies cache_control to both the system prompt stable prefix AND the last user message, enabling the full conversation prefix to be cached.

Root Cause

Two distinct functions in anthropic-payload-policy:

  1. applyAnthropicPayloadPolicyToParams (direct Anthropic path):

    • applyAnthropicCacheControlToSystem: splits at <!-- OPENCLAW_CACHE_BOUNDARY -->, marks stable prefix
    • applyAnthropicCacheControlToMessages: marks last user message → full conversation prefix cached

  2. applyAnthropicEphemeralCacheControlMarkers (OpenRouter path):

    • Marks all system/developer blocks with cache_control: { type: "ephemeral" }
    • Does NOT mark any conversation messages
    • Does NOT split at cache boundary marker

Additionally, the OpenRouter path does not split the system prompt at <!-- OPENCLAW_CACHE_BOUNDARY -->, so dynamic content below the boundary (LCM systemPromptAddition) is in the same cached block — any change invalidates the entire system cache too.

Fix Action

Fixed

PR fix notes

PR #63062: fix: apply cache_control to conversation messages on OpenRouter path

Description (problem / solution / changelog)

Summary

The OpenRouter code path (applyAnthropicEphemeralCacheControlMarkers) only applies cache_control markers to system/developer messages. Conversation messages (user/assistant) are never marked. This means on OpenRouter, only the system prompt is cached — the entire conversation history is re-sent uncached on every turn.

The direct Anthropic path (applyAnthropicPayloadPolicyToParams) correctly marks both system prompt AND the last user message via applyAnthropicCacheControlToMessages.

Fix

One line added at the end of applyAnthropicEphemeralCacheControlMarkers:

applyAnthropicCacheControlToMessages(messages, { type: "ephemeral" });

This reuses the existing function from the direct Anthropic path to mark the last user message with cache_control: { type: "ephemeral" }, enabling the full conversation prefix to be cached.

Test Results

Before patch:

  • Agent A (80K context): 14% cache hit rate, $0.30/turn
  • Agent B (170K context): 12% cache hit rate, $0.90/turn

After patch:

  • Agent A (80K context): 86% cache hit rate, $0.03/turn
  • 10× cost reduction on OpenRouter fallback

Impact

Session ContextPre-patch CachePost-patch CacheCost Reduction
80K (Agent A)14%86%10×
170K (Agent B)12%~85% (estimated)~10×
15K (subagent)70-89%70-89% (no change)—

Subagents were unaffected because system prompt dominates their total context. Main sessions with large conversation history benefit dramatically.

Fixes #63034

Changed files

  • src/agents/anthropic-payload-policy.ts (modified, +1/-0)
RAW_BUFFERClick to expand / collapse

Summary

The OpenRouter code path (applyAnthropicEphemeralCacheControlMarkers) only applies cache_control markers to system/developer role messages. It does NOT apply markers to conversation messages (user/assistant roles). This means on OpenRouter, only the system prompt is cached — the entire conversation history is re-processed on every turn.

In contrast, the direct Anthropic path (applyAnthropicPayloadPolicyToParams) applies cache_control to both the system prompt stable prefix AND the last user message, enabling the full conversation prefix to be cached.

Impact

For a main session at 170K context on OpenRouter:

  • System prompt: ~20K tokens (cached)
  • Conversation messages: ~150K tokens (never cached)
  • Effective cache hit rate: 12%
  • Cost per turn at Opus pricing: ~$0.90 instead of ~$0.09

For the same session on Anthropic direct:

  • Full prefix cached (system + conversation)
  • Effective cache hit rate: 85-100%
  • Cost per turn: ~$0.05-0.09

Subagents appear to cache well (70-89%) only because their system prompt dominates total context — conversation messages are small.

Root Cause

Two distinct functions in anthropic-payload-policy:

  1. applyAnthropicPayloadPolicyToParams (direct Anthropic path):

    • applyAnthropicCacheControlToSystem: splits at <!-- OPENCLAW_CACHE_BOUNDARY -->, marks stable prefix
    • applyAnthropicCacheControlToMessages: marks last user message → full conversation prefix cached

  2. applyAnthropicEphemeralCacheControlMarkers (OpenRouter path):

    • Marks all system/developer blocks with cache_control: { type: "ephemeral" }
    • Does NOT mark any conversation messages
    • Does NOT split at cache boundary marker

Additionally, the OpenRouter path does not split the system prompt at <!-- OPENCLAW_CACHE_BOUNDARY -->, so dynamic content below the boundary (LCM systemPromptAddition) is in the same cached block — any change invalidates the entire system cache too.

Expected Behavior

OpenRouter path should apply cache_control to conversation messages the same way the direct Anthropic path does, at minimum marking the last user message to establish a conversation-prefix cache breakpoint.

Environment

  • OpenClaw 4.5 (3e72c03)
  • OR models: openrouter/anthropic/claude-opus-4-6, openrouter/anthropic/claude-sonnet-4-6
  • cacheRetention: "short" configured
  • Main session context: 170K tokens
  • Observed cache hit: 12-14% on OR vs 85-100% on direct Anthropic

extent analysis

TL;DR

Modify the applyAnthropicEphemeralCacheControlMarkers function in the OpenRouter path to apply cache_control markers to conversation messages, similar to the direct Anthropic path.

Guidance

  • Identify the applyAnthropicEphemeralCacheControlMarkers function and update it to mark conversation messages with cache_control markers, specifically the last user message.
  • Split the system prompt at the <!-- OPENCLAW_CACHE_BOUNDARY --> marker to separate dynamic content and prevent invalidating the entire system cache.
  • Verify the updated function by checking the cache hit rate for conversation messages and ensuring it approaches the rates seen in the direct Anthropic path.
  • Test the changes with different session contexts and cache retention policies to ensure the fix is robust.

Example

// Pseudo-code example, actual implementation may vary
function applyAnthropicEphemeralCacheControlMarkers(messages) {
  // Mark system/developer blocks with cache_control
  messages.forEach((message) => {
    if (message.role === 'system' || message.role === 'developer') {
      message.cache_control = { type: 'ephemeral' };
    }
  });
  
  // Mark last user message with cache_control
  const lastUserMessage = messages.find((message) => message.role === 'user');
  if (lastUserMessage) {
    lastUserMessage.cache_control = { type: 'ephemeral' };
  }
  
  // Split system prompt at cache boundary marker
  const systemPrompt = messages.find((message) => message.role === 'system');
  if (systemPrompt && systemPrompt.text.includes('<!-- OPENCLAW_CACHE_BOUNDARY -->')) {
    const [stablePrefix, dynamicContent] = systemPrompt.text.split('<!-- OPENCLAW_CACHE_BOUNDARY -->');
    systemPrompt.text = stablePrefix;
    // Handle dynamic content separately
  }
}

Notes

The provided example is a simplified illustration and may require adjustments to fit the actual implementation. The applyAnthropicEphemeralCacheControlMarkers function should be updated to handle conversation messages and system prompts according to the direct Anthropic path's logic.

Recommendation

Apply the workaround by modifying the applyAnthropicEphemeralCacheControlMarkers function to apply cache_control markers to conversation messages and split the system prompt at the cache boundary marker. This should improve the cache hit rate for conversation messages in the OpenRouter path.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#installation#tensor shape#autograd error#model save/load#conversation history

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING