litellm - 💡(How to fix) Fix [Bug]: auto_router OpenAI embedding sometimes returns "'$.input' is invalid.” [1 comments, 1 participants]

litellm2026-03-28 06:36:22

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

BerriAI/litellm#24724•Fetched 2026-04-08 01:41:56

View on GitHub

Comments

Participants

Timeline

Reactions

Author

shahrin014

Participants

shahrin014

Timeline (top)

labeled ×3commented ×1mentioned ×1subscribed ×1

Error Message

2026-03-28 15:28:24 - LiteLLM Router - DEBUG - async_function_with_fallbacks_common_utils:5271 - TracebackTraceback (most recent call last): File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1363, in embedding headers, sync_embedding_response = self.make_sync_openai_embedding_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 344, in sync_wrapper result = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1236, in make_sync_openai_embedding_request raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1228, in make_sync_openai_embedding_request raw_response = openai_client.embeddings.with_raw_response.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_legacy_response.py", line 367, in wrapped return cast(LegacyAPIResponse[R], func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 132, in create return self._post( ^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1297, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1070, in request raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 4923, in embedding response = openai_chat_completions.embedding( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1394, in embedding raise OpenAIError( litellm.llms.openai.common_utils.OpenAIError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 93, in encode_queries embeds = self.litellm_router_instance.embedding( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3984, in embedding raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3981, in embedding response = self.function_with_fallbacks(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5875, in function_with_fallbacks return run_async_function(self.async_function_with_fallbacks, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 112, in run_async_function return future.result() ^^^^^^^^^^^^^^^ File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 100, in run_in_new_loop return new_loop.run_until_complete(async_function(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5489, in async_function_with_fallbacks return await self.async_function_with_fallbacks_common_utils( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5446, in async_function_with_fallbacks_common_utils raise original_exception File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks response = await self.async_function_with_retries(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5635, in async_function_with_retries self.should_retry_this_error( File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5839, in should_retry_this_error raise error File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries response = await self.make_call(original_function, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5750, in make_call response = original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4040, in _embedding raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4020, in _embedding response = litellm.embedding( ^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1775, in wrapper raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1596, in wrapper result = original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 5777, in embedding raise exception_type( ^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type raise BadRequestError( litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding Available Model Group Fallbacks=None

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries response = await self.make_call(original_function, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call response = await response ^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion deployment = await self.async_get_available_deployment( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment pre_routing_hook_response = await self.async_pre_routing_hook( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook return await self.auto_routers[model].async_pre_routing_hook( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer( ^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in call vector = self._encode(text=[text], input_type="queries") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode else self.encoder.encode_queries(text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries raise ValueError( ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding Available Model Group Fallbacks=None

During handling of the above exception, another exception occurred:

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks response = await self.async_function_with_retries(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5743, in async_function_with_retries raise original_exception File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5672, in async_function_with_retries response = await self.make_call(original_function, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call response = await response ^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion deployment = await self.async_get_available_deployment( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment raise e File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment pre_routing_hook_response = await self.async_pre_routing_hook( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook return await self.auto_routers[model].async_pre_routing_hook( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer( ^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in call vector = self._encode(text=[text], input_type="queries") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode else self.encoder.encode_queries(text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries raise ValueError( ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding Available Model Group Fallbacks=None

Code Example

- model_name: semantic-router
    litellm_params:
      model: auto_router/auto_router_1
      auto_router_config_path: router.yaml
      auto_router_default_model: claude-model # This model works when selected directly
      auto_router_embedding_model: custom-embedding

---

2026-03-28 15:28:24 - LiteLLM Router - DEBUG - async_function_with_fallbacks_common_utils:5271 - TracebackTraceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1363, in embedding
    headers, sync_embedding_response = self.make_sync_openai_embedding_request(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 344, in sync_wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1236, in make_sync_openai_embedding_request
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1228, in make_sync_openai_embedding_request
    raw_response = openai_client.embeddings.with_raw_response.create(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_legacy_response.py", line 367, in wrapped
    return cast(LegacyAPIResponse[R], func(*args, **kwargs))
                                      ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 132, in create
    return self._post(
           ^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1297, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1070, in request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 4923, in embedding
    response = openai_chat_completions.embedding(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1394, in embedding
    raise OpenAIError(
litellm.llms.openai.common_utils.OpenAIError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 93, in encode_queries
    embeds = self.litellm_router_instance.embedding(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3984, in embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3981, in embedding
    response = self.function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5875, in function_with_fallbacks
    return run_async_function(self.async_function_with_fallbacks, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 112, in run_async_function
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 100, in run_in_new_loop
    return new_loop.run_until_complete(async_function(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5489, in async_function_with_fallbacks
    return await self.async_function_with_fallbacks_common_utils(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5446, in async_function_with_fallbacks_common_utils
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5635, in async_function_with_retries
    self.should_retry_this_error(
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5839, in should_retry_this_error
    raise error
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5750, in make_call
    response = original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4040, in _embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4020, in _embedding
    response = litellm.embedding(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1775, in wrapper
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1596, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 5777, in embedding
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion
    deployment = await self.async_get_available_deployment(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment
    pre_routing_hook_response = await self.async_pre_routing_hook(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook
    return await self.auto_routers[model].async_pre_routing_hook(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook
    route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer(
                                                                    ^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in __call__
    vector = self._encode(text=[text], input_type="queries")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode
    else self.encoder.encode_queries(text)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries
    raise ValueError(
ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 93, in encode_queries
    embeds = self.litellm_router_instance.embedding(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3984, in embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3981, in embedding
    response = self.function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5875, in function_with_fallbacks
    return run_async_function(self.async_function_with_fallbacks, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 112, in run_async_function
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 100, in run_in_new_loop
    return new_loop.run_until_complete(async_function(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5489, in async_function_with_fallbacks
    return await self.async_function_with_fallbacks_common_utils(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5446, in async_function_with_fallbacks_common_utils
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5635, in async_function_with_retries
    self.should_retry_this_error(
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5839, in should_retry_this_error
    raise error
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5750, in make_call
    response = original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4040, in _embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4020, in _embedding
    response = litellm.embedding(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1775, in wrapper
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1596, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 5777, in embedding
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5743, in async_function_with_retries
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5672, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion
    deployment = await self.async_get_available_deployment(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment
    pre_routing_hook_response = await self.async_pre_routing_hook(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook
    return await self.auto_routers[model].async_pre_routing_hook(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook
    route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer(
                                                                    ^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in __call__
    vector = self._encode(text=[text], input_type="queries")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode
    else self.encoder.encode_queries(text)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries
    raise ValueError(
ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

RAW_BUFFERClick to expand / collapse

Check for existing issues

I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

semantic router could not pick the correct model. I expect it to not have this error, and at the very least, fall back to the default model

  - model_name: semantic-router
    litellm_params:
      model: auto_router/auto_router_1
      auto_router_config_path: router.yaml
      auto_router_default_model: claude-model # This model works when selected directly
      auto_router_embedding_model: custom-embedding

Steps to Reproduce

Use semantic router
Use OpenAI embedding
Send a message through OpenCode, but attach a file for example @filename It also happens intermittently without attaching a file

Relevant log output

2026-03-28 15:28:24 - LiteLLM Router - DEBUG - async_function_with_fallbacks_common_utils:5271 - TracebackTraceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1363, in embedding
    headers, sync_embedding_response = self.make_sync_openai_embedding_request(
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 344, in sync_wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1236, in make_sync_openai_embedding_request
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1228, in make_sync_openai_embedding_request
    raw_response = openai_client.embeddings.with_raw_response.create(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_legacy_response.py", line 367, in wrapped
    return cast(LegacyAPIResponse[R], func(*args, **kwargs))
                                      ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 132, in create
    return self._post(
           ^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1297, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1070, in request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 4923, in embedding
    response = openai_chat_completions.embedding(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1394, in embedding
    raise OpenAIError(
litellm.llms.openai.common_utils.OpenAIError: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 93, in encode_queries
    embeds = self.litellm_router_instance.embedding(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3984, in embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3981, in embedding
    response = self.function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5875, in function_with_fallbacks
    return run_async_function(self.async_function_with_fallbacks, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 112, in run_async_function
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 100, in run_in_new_loop
    return new_loop.run_until_complete(async_function(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5489, in async_function_with_fallbacks
    return await self.async_function_with_fallbacks_common_utils(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5446, in async_function_with_fallbacks_common_utils
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5635, in async_function_with_retries
    self.should_retry_this_error(
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5839, in should_retry_this_error
    raise error
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5750, in make_call
    response = original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4040, in _embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4020, in _embedding
    response = litellm.embedding(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1775, in wrapper
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1596, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 5777, in embedding
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion
    deployment = await self.async_get_available_deployment(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment
    pre_routing_hook_response = await self.async_pre_routing_hook(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook
    return await self.auto_routers[model].async_pre_routing_hook(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook
    route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer(
                                                                    ^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in __call__
    vector = self._encode(text=[text], input_type="queries")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode
    else self.encoder.encode_queries(text)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries
    raise ValueError(
ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 93, in encode_queries
    embeds = self.litellm_router_instance.embedding(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3984, in embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 3981, in embedding
    response = self.function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5875, in function_with_fallbacks
    return run_async_function(self.async_function_with_fallbacks, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 112, in run_async_function
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/asyncify.py", line 100, in run_in_new_loop
    return new_loop.run_until_complete(async_function(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/.asdf/installs/python/3.12.11/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5489, in async_function_with_fallbacks
    return await self.async_function_with_fallbacks_common_utils(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5446, in async_function_with_fallbacks_common_utils
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5635, in async_function_with_retries
    self.should_retry_this_error(
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5839, in should_retry_this_error
    raise error
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5586, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5750, in make_call
    response = original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4040, in _embedding
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 4020, in _embedding
    response = litellm.embedding(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1775, in wrapper
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1596, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/main.py", line 5777, in embedding
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2456, in exception_type
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 478, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5480, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5743, in async_function_with_retries
    raise original_exception
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5672, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 5754, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2220, in _acompletion
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 2071, in _acompletion
    deployment = await self.async_get_available_deployment(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9209, in async_get_available_deployment
    raise e
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9082, in async_get_available_deployment
    pre_routing_hook_response = await self.async_pre_routing_hook(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router.py", line 9376, in async_pre_routing_hook
    return await self.auto_routers[model].async_pre_routing_hook(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py", line 124, in async_pre_routing_hook
    route_choice: Optional[Union[RouteChoice, List[RouteChoice]]] = self.routelayer(
                                                                    ^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/base.py", line 601, in __call__
    vector = self._encode(text=[text], input_type="queries")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/semantic_router/routers/semantic.py", line 58, in _encode
    else self.encoder.encode_queries(text)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shahrin/workspace/ai/litellm/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/litellm_encoder.py", line 98, in encode_queries
    raise ValueError(
ValueError: Internal_litellm_router API call failed. Error: litellm.BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}. Received Model Group=custom-embedding
Available Model Group Fallbacks=None

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.6

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To fix the issue with the semantic router not picking the correct model and failing with a 400 error from OpenAI, follow these steps:

Validate Input Data: Ensure that the input data being sent to the OpenAI API is valid and correctly formatted. The error message indicates that the $.input is invalid, so review the API documentation and verify that the input data conforms to the expected format.
Check Model Configuration: Verify that the custom-embedding model is correctly configured and available. Make sure that the model is properly set up in the OpenAI dashboard and that the API credentials are correct.
Implement Fallback Logic: Modify the code to implement fallback logic when the primary model fails. This can be done by catching the BadRequestError exception and attempting to use a different model or configuration.
Update LiteLLM Configuration: Update the LiteLLM configuration to include fallback models or configurations. This can be done by modifying the litellm_params section of the configuration file to include additional models or configurations.

Example code snippet to implement fallback logic:

try:
    # Attempt to use the primary model
    response = litellm.embedding(input_data, model="custom-embedding")
except BadRequestError as e:
    # Fallback to a different model or configuration
    fallback_model = "claude-model"
    response = litellm.embedding(input_data, model=fallback_model)

Verification

To verify that the fix worked, test the semantic router with different input data and models. Ensure that the router correctly picks the primary model and falls back to the secondary model when the primary model fails.

Extra Tips

Regularly review and update the LiteLLM configuration to ensure that it is correctly set up and configured.
Implement logging and monitoring to detect and respond to errors and exceptions.
Consider implementing additional error handling and fallback logic to improve the robustness and reliability of the semantic router.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #runtime error #dependency conflict #environment setup #docker error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

litellm - 💡(How to fix) Fix [Bug]: auto_router OpenAI embedding sometimes returns "'$.input' is invalid.” [1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Code Example

Check for existing issues

What happened?

Steps to Reproduce

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

TRENDING

litellm - 💡(How to fix) Fix [Bug]: auto_router OpenAI embedding sometimes returns "'$.input' is invalid.” [1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Code Example

Check for existing issues

What happened?

Steps to Reproduce

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING