litellm - 💡(How to fix) Fix [Bug]: Credential/deployment tags leak into fallback routing, causing spurious 401 errors [1 participants]

1. The fallback fails with the tag routing error

TL;DR

Disable enable_tag_filtering on the router or modify the tags configuration to exclude the 'Credential: openai' tag from influencing fallback routing decisions.

Guidance

• Verify that the enable_tag_filtering feature is the cause of the issue by temporarily disabling it and checking if the fallback routing works as expected.
• Check the configuration of the model groups and their associated tags to ensure that the 'Credential: openai' tag is not being incorrectly applied to the fallback model.
• Review the documentation for the enable_tag_filtering feature to understand how it interacts with the fallback routing mechanism and how to configure it to achieve the desired behavior.
• Consider modifying the code that handles the fallback routing to explicitly exclude the 'Credential: openai' tag from the tags passed to the fallback model.

Notes

The exact solution may depend on the specific requirements and constraints of the LiteLLM deployment, and may require additional configuration changes or code modifications.

Recommendation

Apply workaround: Modify the tags configuration to exclude the 'Credential: openai' tag from influencing fallback routing decisions, as this is a more targeted and less disruptive change than disabling the enable_tag_filtering feature entirely.