level : "ERROR", message : "litellm.proxy.proxy_server.async_data_generator(): Exception occured - internalServerException {"message":"The system encountered an unexpected error during processing. Try your request again."}", stacktrace : "Traceback (most recent call last): File "/usr/local/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py", line 1375, in async_streaming_data_generator async for chunk in proxy_logging_obj.async_post_call_streaming_iterator_hook( ...<33 lines>... yield serialize_chunk(chunk) File "/usr/local/lib/python3.13/site-packages/litellm/proxy/utils.py", line 2196, in async_post_call_streaming_iterator_hook async for chunk in current_response: yield chunk File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item [Previous line repeated 3 more times] File "/usr/local/lib/python3.13/site-packages/litellm/proxy/hooks/responses_id_security.py", line 286, in async_post_call_streaming_iterator_hook async for chunk in response: ...<9 lines>... yield chunk File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item File "/usr/local/lib/python3.13/site-packages/litellm/integrations/custom_logger.py", line 465, in async_post_call_streaming_iterator_hook async for item in response: yield item [Previous line repeated 5 more times] File "/usr/local/lib/python3.13/site-packages/litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py", line 511, in bedrock_sse_wrapper async for chunk in handler.async_sse_wrapper(completion_stream): yield chunk File "/usr/local/lib/python3.13/site-packages/litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py", line 102, in async_sse_wrapper async for chunk in completion_stream: ...<2 lines>... yield encoded_chunk File "/usr/local/lib/python3.13/site-packages/litellm/llms/bedrock/chat/invoke_handler.py", line 1712, in aiter_bytes message = self._parse_message_from_event(event) File "/usr/local/lib/python3.13/site-packages/litellm/llms/bedrock/chat/invoke_handler.py", line 1731, in _parse_message_from_event raise BedrockError( ...<6 lines>... ) litellm.llms.bedrock.common_utils.BedrockError: internalServerException {"message":"The system encountered an unexpected error during processing. Try your request again."}",

Fix Plan

To fix the issue, we need to add a try/except block to the async_sse_wrapper function to catch the BedrockError exception and map it to a proper LiteLLM exception.

Step-by-Step Solution

1. Open the streaming_iterator.py file and navigate to the async_sse_wrapper function.
2. Add a try/except block around the async for loop to catch the BedrockError exception.
3. In the except block, log the error and raise a proper LiteLLM exception.

Example Code

async def async_sse_wrapper(self, completion_stream):
    try:
        async for chunk in completion_stream:
            encoded_chunk = self._encode_chunk(chunk)
            yield encoded_chunk
    except BedrockError as e:
        # Log the error
        logger.error(f"BedrockError: {e}")
        # Raise a proper LiteLLM exception
        raise LiteLLMException(f"Error processing request: {e}")

Verification

To verify that the fix worked, you can test the endpoint again and check the logs to see if the BedrockError exception is being caught and mapped to a proper LiteLLM exception.

Extra Tips

• Make sure to test the fix thoroughly to ensure that it does not introduce any new issues.
• Consider adding additional logging or monitoring to detect similar issues in the future.
• If you are using a version control system, make sure to commit the changes and update the version number accordingly.