litellm - 💡(How to fix) Fix [Feature]: Support Valkey for semantic prompt caching (AWS ElastiCache)

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Add support for Valkey as a backend for LiteLLM's semantic prompt cache, so users can run it against AWS ElastiCache for Valkey (or any Valkey cluster with the valkey-search module loaded).

Root Cause

Add support for Valkey as a backend for LiteLLM's semantic prompt cache, so users can run it against AWS ElastiCache for Valkey (or any Valkey cluster with the valkey-search module loaded).

RAW_BUFFERClick to expand / collapse

Summary

Add support for Valkey as a backend for LiteLLM's semantic prompt cache, so users can run it against AWS ElastiCache for Valkey (or any Valkey cluster with the valkey-search module loaded).

Problem

LiteLLM's semantic cache (type: redis-semantic) is built on RedisVL, which targets the RediSearch FT.* API. RediSearch is not available on Redis OSS or ElastiCache for Redis OSS, which has historically blocked using ElastiCache as a semantic cache backend for LiteLLM.

Valkey is otherwise already de-facto usable as the standard cache (see merged #16207, fixing the ElastiCache-for-Valkey float redis_version), but Valkey is never named in the caching docs and the semantic-cache path is undocumented and untested against it.

Proposed Solution

Treat Valkey as a recognized backend for the semantic cache. The exact shape is intentionally left open — reasonable options include:

  • Allowing type: redis-semantic to work against a Valkey cluster running valkey-search (which exposes the same FT.* surface), with whatever client / probe changes are needed.
  • A dedicated type: valkey-semantic cache type.
  • Or any other approach the implementer prefers.

Plus a documented end-to-end example for AWS ElastiCache for Valkey in the caching docs.

Expected Behavior

  • Pointing LiteLLM's semantic cache at an ElastiCache for Valkey cluster (with vector search enabled) just works.
  • The supported configuration is documented alongside the existing Redis / Qdrant semantic-cache examples.
  • Existing Redis-based setups continue to work unchanged.

Benefits

  • Unblocks semantic prompt caching on AWS ElastiCache, which today requires standing up Redis Stack or Qdrant separately.
  • Makes Valkey a first-class, named option in the docs rather than a de-facto compatibility accident.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING