openclaw - ✅(Solved) Fix feat(agents): forward prompt_cache_key on openai-completions transport when compat.supportsPromptCacheKey is set [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

buildOpenAIResponsesParams forwards prompt_cache_key: options?.sessionId on every Responses API turn (src/agents/openai-transport-stream.ts:794), and parseTransportChunkUsage already reads prompt_tokens_details.cached_tokens on Completions responses (src/agents/openai-transport-stream.ts:1546).

buildOpenAICompletionsParams (src/agents/openai-transport-stream.ts:1486-1540) does not set prompt_cache_key or any other cache-routing field. Any model routed through openai-completions depends purely on the provider-side content hash for cache lookup, with no opt-in way for operators to pin a stable cache key across turns.

OpenAI's prompt caching guide documents prompt_cache_key as valid on chat.completions.create as well as responses.create, and several OpenAI-compatible providers (vLLM, SGLang, Chutes, and similar) honor it on the completions path when routed through a stable session.

Root Cause

buildOpenAIResponsesParams forwards prompt_cache_key: options?.sessionId on every Responses API turn (src/agents/openai-transport-stream.ts:794), and parseTransportChunkUsage already reads prompt_tokens_details.cached_tokens on Completions responses (src/agents/openai-transport-stream.ts:1546).

buildOpenAICompletionsParams (src/agents/openai-transport-stream.ts:1486-1540) does not set prompt_cache_key or any other cache-routing field. Any model routed through openai-completions depends purely on the provider-side content hash for cache lookup, with no opt-in way for operators to pin a stable cache key across turns.

OpenAI's prompt caching guide documents prompt_cache_key as valid on chat.completions.create as well as responses.create, and several OpenAI-compatible providers (vLLM, SGLang, Chutes, and similar) honor it on the completions path when routed through a stable session.

Fix Action

Fixed

PR fix notes

PR #69411: agents: forward prompt_cache_key on openai-completions when compat.supportsPromptCacheKey is set

Description (problem / solution / changelog)

Summary

Fixes #69272.

buildOpenAICompletionsParams never emitted prompt_cache_key on the Completions path, so operators running OpenAI-compatible proxies (vLLM, SGLang, Chutes, etc.) could not opt into stable prompt caching per session the way buildOpenAIResponsesParams already does for the Responses API.

This change forwards options.sessionId as prompt_cache_key on the Completions path under all three of the following conditions:

  1. model.compat.supportsPromptCacheKey === true (explicit opt-in, defaults to the existing behavior — nothing emitted — when unset or false).
  2. resolveCacheRetention(options?.cacheRetention) !== "none" (mirrors the Responses API gate at src/agents/openai-transport-stream.ts:794).
  3. options?.sessionId is present.

Conservative default preserves #49877 / #48155 (Volcano Engine DeepSeek rejects unknown fields) and the scope maintainers set in #67427.

Why not covered by #67427

#67427 added compat.supportsPromptCacheKey to gate the Responses API strip. That is the right mechanism, but it only affected Responses API payloads. The Completions builder had no knob at all. This PR wires the same flag through the second code path.

Changes

  • src/agents/openai-transport-stream.ts — added the conditional emit in buildOpenAICompletionsParams with an explanatory comment matching the Responses API emission.
  • src/agents/openai-transport-stream.test.ts — 5 new tests covering:
    1. Default off (no compat.supportsPromptCacheKey, sessionId present) → not emitted
    2. compat.supportsPromptCacheKey=true + sessionId → emitted
    3. compat.supportsPromptCacheKey=true + cacheRetention: "none" → suppressed
    4. compat.supportsPromptCacheKey=true + no sessionId → suppressed
    5. compat.supportsPromptCacheKey=false → suppressed

Scope

Nothing else touched:

  • No new provider surface, no new compat field, no schema changes.
  • No change to Responses API, Anthropic cacheRetention, or Gemini cachedContents.
  • No change to tool assembly, system prompt shaping, or usage parsing.

Test plan

  • pnpm test src/agents/openai-transport-stream.test.ts — 70/70 passed
  • pnpm tsgo:core — clean
  • pnpm tsgo:core:test — clean
  • pnpm lint — clean (via committer check:changed)
  • pnpm format — clean
  • Import cycle checks — 0 cycles

AI-assisted: yes (Claude, under sk7n4k3d supervision).

Changed files

  • src/agents/openai-transport-stream.test.ts (modified, +104/-0)
  • src/agents/openai-transport-stream.ts (modified, +12/-0)

Code Example

if (
  compat.supportsPromptCacheKey === true &&
  resolveCacheRetention(options?.cacheRetention) !== "none" &&
  options?.sessionId
) {
  params.prompt_cache_key = options.sessionId;
}
RAW_BUFFERClick to expand / collapse

Summary

buildOpenAIResponsesParams forwards prompt_cache_key: options?.sessionId on every Responses API turn (src/agents/openai-transport-stream.ts:794), and parseTransportChunkUsage already reads prompt_tokens_details.cached_tokens on Completions responses (src/agents/openai-transport-stream.ts:1546).

buildOpenAICompletionsParams (src/agents/openai-transport-stream.ts:1486-1540) does not set prompt_cache_key or any other cache-routing field. Any model routed through openai-completions depends purely on the provider-side content hash for cache lookup, with no opt-in way for operators to pin a stable cache key across turns.

OpenAI's prompt caching guide documents prompt_cache_key as valid on chat.completions.create as well as responses.create, and several OpenAI-compatible providers (vLLM, SGLang, Chutes, and similar) honor it on the completions path when routed through a stable session.

Why this is not covered by #67427

#67427 added compat.supportsPromptCacheKey to gate the Responses API strip. That is the right mechanism, and it is already wired into resolveProviderRequestCapabilities, ModelCompatConfig, and the generated schema. But it only affects Responses API payloads. Operators who run OpenAI-compatible proxies on openai-completions get no knob today: the Completions builder simply never emits prompt_cache_key.

Proposal (conservative)

Extend buildOpenAICompletionsParams so that when compat.supportsPromptCacheKey === true AND cacheRetention !== "none" AND options?.sessionId is present, the field is forwarded:

if (
  compat.supportsPromptCacheKey === true &&
  resolveCacheRetention(options?.cacheRetention) !== "none" &&
  options?.sessionId
) {
  params.prompt_cache_key = options.sessionId;
}

Default behavior (undefined) stays as it is today: nothing emitted. This preserves the conservative default from #49877 / #48155 (Volcano Engine DeepSeek rejects unknown fields) and the scope boundary maintainers set in #67427.

Out of scope

  • No new provider surface
  • No change to default behavior
  • No change to Responses API, Anthropic cacheRetention, or Gemini cachedContents
  • No change to tool assembly or system prompt shaping

Motivation

A sibling effort on a separate OpenClaw deployment measured, on turn 2+ of multi-turn sessions routed through an OpenAI-compatible provider that honors prompt_cache_key on completions:

  • TTFT: ~22-31% lower
  • cached_tokens / prompt_tokens: ~0.98
  • Pricing: cached input tokens at half rate vs fresh input

The current behavior on openai-completions leaves that on the table with no operator opt-in.

Before I open a PR

CONTRIBUTING.md says to start here first for anything that looks like a feature. Would maintainers prefer:

  1. The conservative variant above (default off, gated on the existing compat.supportsPromptCacheKey), or
  2. A tri-state compat.supportsPromptCacheKey that also covers completions when true, mirroring the existing Responses semantics, or
  3. Keep completions off that path entirely and point users at Responses API?

Happy to send the PR in whichever shape you prefer. Code-path pointers above are against main at 042c117342.

AI-assisted disclosure

Designed by a human, delivered with Claude Opus 4.7.

extent analysis

TL;DR

The proposed fix involves extending buildOpenAICompletionsParams to forward prompt_cache_key when compat.supportsPromptCacheKey is true and a session ID is present.

Guidance

  • Review the buildOpenAICompletionsParams function in src/agents/openai-transport-stream.ts to understand how parameters are currently being built for OpenAI completions.
  • Consider the proposed conservative extension that conditionally adds prompt_cache_key to the parameters based on compat.supportsPromptCacheKey, cacheRetention, and the presence of a session ID.
  • Evaluate the potential impact of this change on existing behavior, particularly for operators using OpenAI-compatible proxies on the openai-completions path.
  • Assess whether the proposed fix aligns with the project's contributing guidelines and maintainers' preferences regarding feature additions.

Example

if (
  compat.supportsPromptCacheKey === true &&
  resolveCacheRetention(options?.cacheRetention) !== "none" &&
  options?.sessionId
) {
  params.prompt_cache_key = options.sessionId;
}

Notes

The proposed fix aims to provide a way for operators to opt-in to using a stable cache key across turns for OpenAI-compatible providers that honor prompt_cache_key on completions. However, the maintainers' preference regarding the implementation approach (conservative variant, tri-state compat.supportsPromptCacheKey, or keeping completions off the path) needs to be considered before proceeding with a PR.

Recommendation

Apply the conservative workaround by extending buildOpenAICompletionsParams as proposed, pending maintainer feedback, as it preserves the default behavior and aligns with the existing compat.supportsPromptCacheKey mechanism.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING