Summary

Redis Semantic Cache currently appears to skip true semantic/vector caching for the Responses API (/v1/responses). Exact caching can work once responses / aresponses are in supported_call_types, but the Redis semantic backend extracts prompt text only from messages. The Responses API uses input, so semantic store/lookup returns before calling RedisVL store / check, and the RediSearch index remains empty.

This was observed with RediSearch-enabled Redis and Azure OpenAI embeddings. The local code-level repro below does not require a live Redis instance because it shows the early return before RedisVL is called.

Evidence from current upstream checkout

Checked on BerriAI/litellm default branch litellm_internal_staging at commit c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3 on 2026-05-19.

• Cache.__init__ default supported_call_types already includes responses and aresponses: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/caching.py#L70-L83
• responses and aresponses take input, not messages: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/responses/main.py#L416-L418 https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/responses/main.py#L901-L903
• RedisSemanticCache.set_cache() only reads kwargs["messages"] and returns when it is absent, before self.llmcache.store(...): https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/redis_semantic_cache.py#L281-L298
• RedisSemanticCache.get_cache() also only reads kwargs["messages"] and returns None when it is absent, before self.llmcache.check(...): https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/redis_semantic_cache.py#L317-L332
• The async Redis semantic paths have the same messages-only extraction: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/redis_semantic_cache.py#L430-L456 https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/redis_semantic_cache.py#L473-L493
• The sync wrapper lookup path also drops most original kwargs and passes only messages=messages to the backend cache, so even if RedisSemanticCache.get_cache() learned input, sync lookup would still need passthrough: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/caching.py#L526-L531
• Async wrapper lookup already passes **kwargs, but RedisSemanticCache still ignores input: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/caching/caching.py#L562-L569

Minimal repro

Run from the repo root:

from unittest.mock import Mock
from litellm.caching.redis_semantic_cache import RedisSemanticCache

cache = RedisSemanticCache.__new__(RedisSemanticCache)
cache.llmcache = Mock()
cache._get_cache_filters = Mock(return_value={})
cache._get_ttl = Mock(return_value=None)

cache.set_cache("responses-key", {"ok": True}, input="hello responses")
print("store_calls", cache.llmcache.store.call_count)

cache.llmcache.check = Mock(return_value=[])
metadata = {}
result = cache.get_cache("responses-key", input="hello responses", metadata=metadata)
print("lookup_result", result)
print("check_calls", cache.llmcache.check.call_count)
print("metadata", metadata)

Observed output:

store_calls 0
lookup_result None
check_calls 0
metadata {'semantic-similarity': 0.0}

This shows the Responses-style input payload never reaches RedisVL store/check. In a live RediSearch-enabled Redis deployment, that matches the observed empty semantic index.

Current test coverage

The existing Redis semantic cache unit tests pass, but they do not cover Responses input extraction:

uv run pytest tests/test_litellm/caching/test_redis_semantic_cache.py -q
..............                                                           [100%]
14 passed in 0.20s

The existing tests mock RedisVL and exercise message-based semantic caching, cache-key filtering, and semantic similarity metadata. They do not assert that /responses input is converted into a semantic prompt.

Expected behavior

For responses / aresponses calls using Redis Semantic Cache:

1. The semantic cache should extract prompt text from Responses input.
2. Store path should call RedisVL store/astore and populate the RediSearch index.
3. Lookup path should call RedisVL check/acheck and return semantic hits when similarity threshold is met.
4. Exact cache and semantic cache behavior should remain distinct and observable.

Suggested fix scope

A focused patch would likely include:

• Add a shared prompt extraction helper for RedisSemanticCache that supports both:
  • chat-style messages
  • Responses-style input, including string input and structured ResponseInputParam list items
• Update sync and async Redis semantic set/get paths to use that helper.
• Update sync Cache.get_cache() to pass the original kwargs through to backend caches, or otherwise pass input / metadata along with messages.
• Add unit tests in tests/test_litellm/caching/test_redis_semantic_cache.py for:
  • set_cache(..., input="...") calls RedisVL store
  • get_cache(..., input="...") calls RedisVL check
  • async async_set_cache / async_get_cache behavior for Responses input
  • structured Responses input list extraction

Observability and preflight gaps seen while investigating

Two related improvements would make this easier to operate:

• Observability: today the proxy exposes generic cache hit headers and x-litellm-semantic-similarity only when metadata contains semantic-similarity: https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/proxy/common_utils/callback_utils.py#L419-L420

  It would be useful to distinguish exact cache hits from semantic cache hits explicitly in headers/log metadata, especially when both exact and semantic cache paths are configured.

• Redis Semantic Cache preflight: readiness can report Redis semantic index info through _index_info(): https://github.com/BerriAI/litellm/blob/c2efe9e422b6ce62f0001d847d578d1e7d7ea6e3/litellm/proxy/health_endpoints/_health_endpoints.py#L1497-L1505

  But the Helm chart path does not appear to provide a clear preflight guard that the configured Redis actually has RediSearch/Redis Stack module support. A safer preflight / Helm value check would prevent deploying redis-semantic against a plain Redis instance where vector indexing cannot work.

Related issue search

I searched open issues for RedisSemanticCache responses semantic cache RediSearch input and redis semantic cache responses input and did not find an exact duplicate. The closest related open issue I found is #23441, but that is focused on Qdrant semantic cache failures rather than Redis Semantic Cache and Responses input extraction.