litellm - 💡(How to fix) Fix [Bug]: When an agent with tools turns on either redis-semanticor qdrant-semantic cache mode and carries tool return JSON requests to the gateway, the tool return content will be lost.

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Observed Behavior​ After tool invocation, the agent began repeating requests. Logs confirmed that tool responses were correctly returned, so the issue was not an infinite loop. Root Cause Analysis​ The most probable cause is improper injection of tool results into the model context: the results lacked the required role=tooland corresponding tool_call_id. Langfuse Tracing​ Multiple ToolNodes were observed, each showing normal tool input and output. However, once execution returned to the parent Chat Model node, the tool outputs disappeared. Only the tool request and assistant tool_call_idremained. Hypothesis​ When tool invocation is combined with semantic similarity caching, tool response payloads are lost. The agent, seeing only the tool request, assumes the tool has not been executed and repeatedly issues new calls, resulting in persistent errors.

Steps to Reproduce

1. create a simple agent with a tool
2. set up litellm on your machine with the following config: model_list:

• model_name: tokenhub-deepseek-v4-pro litellm_params: model: openai/tokenhub-deepseek-v4-pro api_base: os.environ/UPSTREAM_OPENAI_BASE_URL api_key: os.environ/UPSTREAM_OPENAI_API_KEY timeout: 600 temperature: 0.6 top_p: 0.95

• model_name: venus-embedding litellm_params: model: openai/bge-large-zh api_base: os.environ/VENUS_PROXY_EMBEDDING_BASE_URL api_key: os.environ/VENUS_PROXY_API_TOKEN timeout: 60

litellm_settings: set_verbose: true cache: true cache_params: type: redis-semantic # mode: default_off ttl: 600 similarity_threshold: 0.5 # similarity threshold for semantic cache redis_semantic_cache_embedding_model: openai/bge-large-zh

general_settings: master_key: os.environ/LITELLM_MASTER_KEY database_url: os.environ/DATABASE_URL disable_master_key_return: true store_model_in_db: true store_prompts_in_spend_logs: true 3. ask the agent to call the tool and observe.

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.6

Twitter / LinkedIn details

No response