litellm - 💡(How to fix) Fix [Bug]: langfuse_otel callback logs raw upstream model name instead of proxy alias for streaming requests [1 comments, 1 participants]

TL;DR

The issue can be resolved by modifying the langfuse_otel callback to handle streaming requests correctly, ensuring it logs the proxy alias instead of the raw upstream model identifier.

Guidance

• Review the langfuse_otel callback implementation to identify why it's not handling streaming requests as expected, focusing on how model identifiers are processed.
• Verify that the model_name from the config.yaml is being correctly passed to the langfuse_otel callback for streaming requests.
• Check if there's an existing option or configuration in langfuse_otel that allows specifying the model alias for logging purposes, especially for streaming requests.
• Consider adding custom logging logic within the langfuse_otel callback to override the model identifier with the proxy alias for streaming requests.

Example

No specific code example can be provided without modifying the existing langfuse_otel callback, but a potential approach could involve overriding the model identifier in the callback:

# Hypothetical example, actual implementation may vary
def langfuse_otel_callback(request):
    # ...
    if request.stream:
        # Override model identifier with proxy alias
        model_identifier = request.model_name
    else:
        model_identifier = get_upstream_model_identifier()
    # Log using model_identifier

Notes

The exact solution depends on the internal workings of the langfuse_otel callback and how it interacts with LiteLLM's proxy functionality. The provided steps are guidance toward identifying and potentially resolving the issue.

Recommendation

Apply a workaround by modifying the langfuse_otel callback to correctly handle streaming requests, as upgrading to a fixed version is not explicitly mentioned as an option in the provided issue context.