litellm - ✅(Solved) Fix [Bug]: '>' not supported between instances of 'NoneType' and 'int' [2 pull requests, 1 participants]

litellm2026-03-11 02:16:33

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

BerriAI/litellm#23316•Fetched 2026-04-08 00:37:35

View on GitHub

Comments

Participants

Timeline

Reactions

Author

ohmer1

Participants

ohmer1

Timeline (top)

labeled ×3cross-referenced ×2referenced ×2

Error Message

"error_information": { "traceback": " File "/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 6264, in chat_completion\n result = await base_llm_response_processor.base_process_llm_request(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<16 lines>...\n )\n ^\n File "/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py", line 856, in base_process_llm_request\n responses = await llm_responses\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 1533, in acompletion\n raise e\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 1509, in acompletion\n response = await self.async_function_with_fallbacks(**kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5104, in async_function_with_fallbacks\n return await self.async_function_with_fallbacks_common_utils(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n ...<8 lines>...\n )\n ^\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5061, in async_function_with_fallbacks_common_utils\n raise original_exception\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5095, in async_function_with_fallbacks\n response = await self.async_function_with_retries(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib/python3.13/site-packages/litellm/router.py", line 5262, in async_function_with_retries\n if num_retries > 0:\n ^^^^^^^^^^^^^^^\n", "error_code": "", "error_class": "TypeError", "llm_provider": "", "error_message": "'>' not supported between instances of 'NoneType' and 'int'" },

Fix Action

Fixed

Fixed by PR: fix(router): handle num_retries=None to prevent TypeError in retry logic (https://github.com/BerriAI/litellm/pull/23317)
Fixed by PR: fix(router): guard num_retries against None in async_function_with_retries (https://github.com/BerriAI/litellm/pull/23356)

PR fix notes

PR #23317: fix(router): handle num_retries=None to prevent TypeError in retry logic

Repository: BerriAI/litellm
Author: alvinttang
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/23317

Description (problem / solution / changelog)

Summary

Fixes #23316

When num_retries is None (either passed explicitly by the caller or through certain proxy/SDK configurations), the comparison if num_retries > 0 in async_function_with_retries raises:

TypeError: '>' not supported between instances of 'NoneType' and 'int'

Root cause

kwargs.get("num_retries", self.num_retries) returns None when the key exists in kwargs with an explicit None value — Python's dict.get() only uses the default when the key is missing, not when the value is None. This bypasses the fallback to self.num_retries (which is always set to a valid int in Router.__init__).

Fix

All 8 call sites that set kwargs["num_retries"] now use an explicit is not None check instead of relying on dict.get()'s default parameter.
Added a None guard in async_function_with_retries where num_retries is popped from kwargs, falling back to self.num_retries.

Affected methods

_update_kwargs_before_fallbacks, image_generation, aimage_generation, atext_completion, aadapter_completion, acreate_file, acreate_batch, acancel_batch

Test plan

Verify that requests with num_retries=None no longer raise TypeError
Verify that explicit num_retries=0 still disables retries (not overridden by fallback)
Verify that requests without num_retries in kwargs still use self.num_retries as default

Changed files

litellm/router.py (modified, +11/-9)

PR #23356: fix(router): guard num_retries against None in async_function_with_retries

Repository: BerriAI/litellm
Author: NIK-TIGER-BILL
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/23356

Description (problem / solution / changelog)

Root cause

When a Router is created without an explicit num_retries= value and the request itself doesn't carry num_retries in kwargs, the following assignment (present in several code paths):

kwargs['num_retries'] = kwargs.get('num_retries', self.num_retries)

produces kwargs['num_retries'] = None because self.num_retries defaults to None.

async_function_with_retries then pops that None and later evaluates:

if num_retries > 0:   # TypeError: '>' not supported between NoneType and int

This manifests as an unhandled TypeError crash on every request that hits the retry path after an upstream error (e.g. Mistral models receiving image payloads — fixes #23316):

"error_class": "TypeError",
"error_message": "'>' not supported between instances of 'NoneType' and 'int'"

Fix

Coerce num_retries to int at pop-time with or 0, so a missing/None value is treated as no retries instead of crashing:

num_retries: int = kwargs.pop("num_retries") or 0

Fixes #23316

Changed files

litellm/model_prices_and_context_window_backup.json (modified, +30/-0)
litellm/router.py (modified, +1/-1)
model_prices_and_context_window.json (modified, +30/-0)

Code Example

"error_information": {
    "traceback": "  File \"/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py\", line 6264, in chat_completion\n    result = await base_llm_response_processor.base_process_llm_request(\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<16 lines>...\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py\", line 856, in base_process_llm_request\n    responses = await llm_responses\n                ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 1533, in acompletion\n    raise e\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 1509, in acompletion\n    response = await self.async_function_with_fallbacks(**kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5104, in async_function_with_fallbacks\n    return await self.async_function_with_fallbacks_common_utils(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<8 lines>...\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5061, in async_function_with_fallbacks_common_utils\n    raise original_exception\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5095, in async_function_with_fallbacks\n    response = await self.async_function_with_retries(*args, **kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5262, in async_function_with_retries\n    if num_retries > 0:\n       ^^^^^^^^^^^^^^^\n",
    "error_code": "",
    "error_class": "TypeError",
    "llm_provider": "",
    "error_message": "'>' not supported between instances of 'NoneType' and 'int'"
  },

RAW_BUFFERClick to expand / collapse

Check for existing issues

I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using Litellm as a proxy between Openwebui (frontend) and Mistral.ai models (backend), including an image in a conversation leads to a processing failure within LiteLLM. This issue appears to be specific to Mistral.ai models.

Scope & Observations:

This issue only occurs when LiteLLM is proxying requests for Mistral.ai models. It doesn't occur when LiteLLM is proxying requests for Gemini models, suggesting the issue is specific to the Mistral.ai integration or LiteLLM's handling of image data for it.
Not all images trigger the issue; it may be related to specific image characteristics such as maximum resolution or file size.
Reproducible Test Image: https://images.unsplash.com/photo-1768006273852-07a34dede095?ixlib=rb-4.1.0&q=85&fm=jpg&crop=entropy&cs=srgb&dl=zero-take-sTpSJDvZyEM-unsplash.jpg&w=1920

Screenshot:

Steps to Reproduce

Add a mistral.ai model to litellm.
In openwebui, select a mistral model and create a new chat with an image file with or without text.
Boom!

Relevant log output

"error_information": {
    "traceback": "  File \"/usr/lib/python3.13/site-packages/litellm/proxy/proxy_server.py\", line 6264, in chat_completion\n    result = await base_llm_response_processor.base_process_llm_request(\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<16 lines>...\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/proxy/common_request_processing.py\", line 856, in base_process_llm_request\n    responses = await llm_responses\n                ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 1533, in acompletion\n    raise e\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 1509, in acompletion\n    response = await self.async_function_with_fallbacks(**kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5104, in async_function_with_fallbacks\n    return await self.async_function_with_fallbacks_common_utils(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n    ...<8 lines>...\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5061, in async_function_with_fallbacks_common_utils\n    raise original_exception\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5095, in async_function_with_fallbacks\n    response = await self.async_function_with_retries(*args, **kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/lib/python3.13/site-packages/litellm/router.py\", line 5262, in async_function_with_retries\n    if num_retries > 0:\n       ^^^^^^^^^^^^^^^\n",
    "error_code": "",
    "error_class": "TypeError",
    "llm_provider": "",
    "error_message": "'>' not supported between instances of 'NoneType' and 'int'"
  },

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.14

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

The issue seems to be related to the handling of image data in the LiteLLM proxy. To fix this, we need to modify the proxy_server.py file to properly handle image data.

Here are the steps to fix the issue:

Update the base_process_llm_request function in common_request_processing.py to check for None values before comparing them with integers.
Modify the async_function_with_fallbacks function in router.py to handle the TypeError exception.

Example code changes:

# common_request_processing.py
async def base_process_llm_request(self, request):
    # ...
    if responses is not None and len(responses) > 0:
        # ...
    # ...

# router.py
async def async_function_with_fallbacks(self, *args, **kwargs):
    try:
        # ...
    except TypeError as e:
        # Handle the TypeError exception
        logging.error(f"TypeError: {e}")
        return None

Verification

To verify that the fix worked, you can try the following steps:

Add a Mistral.ai model to LiteLLM.
In OpenWebUI, select a Mistral model and create a new chat with an image file.
Check the LiteLLM logs to see if the error is still occurring.

Extra Tips

Make sure to update the LiteLLM version to the latest one to ensure that you have the latest bug fixes and features.
If you are still experiencing issues, try to debug the proxy_server.py file to see where the error is occurring.
You can also try to increase the logging level to get more detailed error messages.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

litellm - ✅(Solved) Fix [Bug]: '>' not supported between instances of 'NoneType' and 'int' [2 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Fix Action

Fixed

PR fix notes

PR #23317: fix(router): handle num_retries=None to prevent TypeError in retry logic

Description (problem / solution / changelog)

Summary

Root cause

Fix

Affected methods

Test plan

Changed files

PR #23356: fix(router): guard num_retries against None in async_function_with_retries

Description (problem / solution / changelog)

Root cause

Fix

Changed files

Code Example

Check for existing issues

What happened?

Steps to Reproduce

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING