PR #15768: feat(runner): expose prompt cache hit count in completion response

• Repository: ollama/ollama
• Author: anishesg
• State: open | merged: False
• Link: https://github.com/ollama/ollama/pull/15768

Description (problem / solution / changelog)

When the runner processes a completion request it consults the KV cache to skip re-evaluating prompt tokens that were already processed in a previous turn. The number of tokens actually served from that cache was tracked internally but never returned to callers, making it impossible to observe cache efficiency.

A new numCachedInputs field is added to the Sequence struct in runner/ollamarunner/runner.go. Immediately after loadCache populates seq.cache.Inputs, the length of that slice is captured as seq.numCachedInputs. That value is then included in the outgoing CompletionResponse as PromptCachedCount (omitted when zero so existing clients see no change).

On the server side, llm/server.go gains the corresponding PromptCachedCount int field with the prompt_cached_count,omitempty JSON tag so the information surfaces through the HTTP API without breaking backward compatibility.

Fixes #15758

Changed files

• llm/server.go (modified, +1/-0)
• runner/ollamarunner/runner.go (modified, +3/-0)