litellm - 💡(How to fix) Fix [Bug]: LiteLLM cache init ignores AWS role for Bedrock [1 participants]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When configuring a Bedrock embedding model with aws_role_arn for cross-account access, LiteLLM's Redis semantic cache initialization bypasses that role configuration and attempts to invoke Bedrock using the pod's default IAM role instead. This causes authorization failures because the default role lacks Bedrock permissions, even though the model's configured role has them. The cache init should respect and assume the model's specified role before attempting embeddings, but currently it doesn't, making cross-account Bedrock semantic caching impossible without also granting the default pod role direct Bedrock access.

Steps to Reproduce

1. Create a Bedrock embedding model in a separate AWS account (Account B) with cross-account access
2. Configure k8s pod with IAM role in Account A that only has sts:AssumeRole permission to the cross-account role in Account B
3. Configure LiteLLM model with aws_role_arn pointing to the cross-account role in Account B
4. Enable Redis semantic cache in config.yaml, specifying the Bedrock model as redis_semantic_cache_embedding_model
5. Start the LiteLLM proxy
6. Observe: Cache initialization fails with bedrock:InvokeModel is not authorized error, even though the model's configured role has Bedrock permissions
7. Verify: The error shows the pod's default role (assumed-role/k8s-app-litellm..../botocore-session-xxxx) is being used instead of the cross-account role specified in aws_role_arn.

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.83.14

Twitter / LinkedIn details

No response