litellm - 💡(How to fix) Fix [Bug]: with Redis Cluster, TTL preservation failed, falling back to regular pipeline: EVALSHA - all keys must map to the same key slot warning occurs [1 participants]

TL;DR

The warning message "TTL preservation failed, falling back to regular pipeline" in LiteLLM Proxy with Redis Cluster setup can be addressed by reviewing the Redis Cluster configuration and ensuring that all keys used by LiteLLM Proxy map to the same key slot.

Guidance

• Review the Redis Cluster configuration to ensure that the keys used by LiteLLM Proxy are properly hashed and mapped to the same key slot, as required by the EVALSHA command.
• Verify that the team budget limit increase is properly propagated to all LiteLLM pods and the Redis Cluster nodes to prevent inconsistencies.
• Check the LiteLLM Proxy documentation for any specific recommendations or guidelines for using Redis Cluster with multiple master nodes.
• Consider testing the scenario with a smaller team budget limit to see if the issue is related to the budget limit increase or the Redis Cluster setup.

Example

No specific code snippet is provided as the issue seems to be related to the configuration and setup of Redis Cluster and LiteLLM Proxy.

Notes

The issue may be specific to the combination of LiteLLM Proxy version v1.81.1 and the Redis Cluster setup with multiple master nodes. Further investigation and testing may be required to determine the root cause and find a permanent solution.

Recommendation

Apply workaround: Review and adjust the Redis Cluster configuration to ensure that all keys used by LiteLLM Proxy map to the same key slot, as this is the most likely cause of the warning message.