litellm - 💡(How to fix) Fix [Bug]: SGLang rerank bug [1 pull requests]

'list' object has no attribute 'get'

---

C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Scripts\python.exe C:/Users/xxx/PycharmProjects/phoenix-gc/script.py
16:26:59 - LiteLLM:WARNING: get_model_cost_map.py:271 - LiteLLM: Failed to fetch remote model cost map from https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json: [Errno 11001] getaddrinfo failed. Falling back to local backup.
16:27:11 - LiteLLM:DEBUG: utils.py:482 - 
16:27:11 - LiteLLM:DEBUG: utils.py:482 - Request to litellm:
16:27:11 - LiteLLM:DEBUG: utils.py:482 - litellm.rerank(model='hosted_vllm/QWEN3 RERANKER 8B', query='What is the capital of the United States?', documents=['Carson City is the capital city of the American state of Nevada.', 'The Commonwealth of the Northern Mariana Islands is its capital is Saipan.', 'Washington, D.C. is the capital of the United States.', 'Capital punishment has existed in the United States since before it was a country.'], return_documents=False, top_n=2)
16:27:11 - LiteLLM:DEBUG: utils.py:482 - 
16:27:11 - LiteLLM:DEBUG: litellm_logging.py:563 - self.optional_params: {}
16:27:11 - LiteLLM:DEBUG: utils.py:482 - SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
16:27:11 - LiteLLM:INFO: main.py:160 - optional_rerank_params: {'query': 'What is the capital of the United States?', 'documents': ['Carson City is the capital city of the American state of Nevada.', 'The Commonwealth of the Northern Mariana Islands is its capital is Saipan.', 'Washington, D.C. is the capital of the United States.', 'Capital punishment has existed in the United States since before it was a country.'], 'top_n': 2, 'rank_fields': None, 'return_documents': False}
16:27:11 - LiteLLM:DEBUG: litellm_logging.py:563 - self.optional_params: {'query': 'What is the capital of the United States?', 'documents': ['Carson City is the capital city of the American state of Nevada.', 'The Commonwealth of the Northern Mariana Islands is its capital is Saipan.', 'Washington, D.C. is the capital of the United States.', 'Capital punishment has existed in the United States since before it was a country.'], 'top_n': 2, 'rank_fields': None, 'return_documents': False}
16:27:11 - LiteLLM:DEBUG: litellm_logging.py:1166 - 
POST Request Sent from LiteLLM:
curl -X POST \
https://litellm.scm-test.int.gazprombank.ru/rerank \
-H 'Authorization: Be****Ow' -H 'accept: application/json' -H 'content-type: application/json' \
-d '{'model': 'QWEN3 RERANKER 8B', 'query': 'What is the capital of the United States?', 'top_n': 2, 'documents': ['Carson City is the capital city of the American state of Nevada.', 'The Commonwealth of the Northern Mariana Islands is its capital is Saipan.', 'Washington, D.C. is the capital of the United States.', 'Capital punishment has existed in the United States since before it was a country.'], 'return_documents': False}'
16:27:17 - LiteLLM:ERROR: main.py:567 - Error in rerank: {"error":{"message":"'list' object has no attribute 'get'","type":"None","param":"None","code":"500"}}
16:27:17 - LiteLLM:DEBUG: get_api_base.py:62 - Error occurred in getting api base - litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=QWEN3 RERANKER 8B
 Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers
16:27:17 - LiteLLM:DEBUG: exception_mapping_utils.py:2487 - Logging Details: logger_fn - None | callable(logger_fn) - False
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
Provider List: https://docs.litellm.ai/docs/providers
16:27:17 - LiteLLM:DEBUG: litellm_logging.py:2934 - Logging Details LiteLLM-Failure Call: []
Traceback (most recent call last):
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\llms\custom_httpx\llm_http_handler.py", line 1039, in rerank
    response = sync_httpx_client.post(
^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\llms\custom_httpx\http_handler.py", line 1085, in post
    _raise_masked_sync_error(e, stream)
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\llms\custom_httpx\http_handler.py", line 354, in _raise_masked_sync_error
    raise MaskedHTTPStatusError(e, message=_text, text=_text) from None
litellm.llms.custom_httpx.http_handler.MaskedHTTPStatusError: Server error '500 Internal Server Error' for url 'https://litellm.scm-test.int.gazprombank.ru/rerank'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\rerank_api\main.py", line 399, in rerank
    response = base_llm_http_handler.rerank(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\llms\custom_httpx\llm_http_handler.py", line 1046, in rerank
    raise self._handle_error(
          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\llms\custom_httpx\llm_http_handler.py", line 5091, in _handle_error
    raise provider_config.get_error_class(
litellm.llms.hosted_vllm.rerank.transformation.HostedVLLMRerankError: {"error":{"message":"'list' object has no attribute 'get'","type":"None","param":"None","code":"500"}}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\script.py", line 25, in <module>
    response = litellm.rerank(
               ^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\utils.py", line 1772, in wrapper
    raise e
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\utils.py", line 1593, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\rerank_api\main.py", line 568, in rerank
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "C:\Users\xxx\PycharmProjects\phoenix-gc\venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2425, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: None - {"error":{"message":"'list' object has no attribute 'get'","type":"None","param":"None","code":"500"}}
Process finished with exit code 1
(log from SDK, in proxy: 
12:50:25 - LiteLLM Proxy:ERROR: endpoints.py:110 - litellm.proxy.proxy_server.rerank(): Exception occured - 'list' object has no attribute 'get'
INFO:     193.160.11.56:39040 - "POST /v1/rerank HTTP/1.1" 500 Internal Server Error
)