Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: Connection to OpenAI failed due to Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer' [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24357•Fetched 2026-04-08 01:18:14
View on GitHub
Comments
0
Participants
1
Timeline
6
Reactions
0
Author
tibor-reisstibor-reiss
Participants
tibor-reisstibor-reiss
Timeline (top)
labeled ×3cross-referenced ×2referenced ×1

Error Message

{ "event": "litellm.proxy.proxy_server.async_data_generator(): Exception occured - Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer'", "exc_info": [ "<class 'pydantic_core._pydantic_core.PydanticSerializationError'>", "PydanticSerializationError(Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer')", "<traceback object at 0xffff5026ef80>" ], "logger": "LiteLLM Proxy", "level": "error", ... }

Fix Action

Fix / Workaround

v1.81.14-stable.gpt-5.4-patch6

PR fix notes

PR #24358: fix: MockValSer json serialization error

Description (problem / solution / changelog)

Relevant issues

Fixes #24357

Pre-Submission checklist

Couple of things to note:

  • black was not run before, so I had to edit audit_logs.py
  • make test-unit errors out because of "responses" directory present e.g. in tests/test_litellm/llms/databricks - had to patch it locally - please update the docs about how to run tests
  • e2e tests are flaky

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Type

🐛 Bug Fix

Changes

Fix json serialization error in streaming

Changed files

  • litellm/proxy/management_helpers/audit_logs.py (modified, +9/-5)
  • litellm/proxy/proxy_server.py (modified, +8/-0)
  • tests/test_litellm/proxy/test_response_model_sanitization.py (modified, +62/-9)

PR #24362: fix: handle pydantic serialization errors in safe_dumps (fixes #24357)

Description (problem / solution / changelog)

Problem

When streaming chunks contain empty ChoiceLogprobs objects (e.g. content=None, refusal=None), model_dump() raises PydanticSerializationError: 'MockValSer' object cannot be converted to 'SchemaSerializer'. This causes the proxy to return a 500 error on the last streaming chunk.

Related issue: Fixes #24357

Root Cause

In safe_json_dumps.py, safe_dumps() calls obj.model_dump() on BaseModel objects during serialization. Pydantic v2 has a known issue where empty ChoiceLogprobs objects cannot be serialized with the standard model_dump() method.

Fix

Wrap the model_dump() call with a try/except that:

  1. First tries model_dump() (standard serialization)
  2. Falls back to model_dump(mode='json') (json-compatible serialization)
  3. Falls back to str(obj) as a last resort

Changes

  • litellm/litellm_core_utils/safe_json_dumps.py — add error handling around model_dump()
  • tests/test_litellm/litellm_core_utils/test_safe_json_dumps.py — add test for serialization failure fallback

Testing

Added test_pydantic_model_with_serialization_error that verifies graceful degradation when model_dump() raises an error.

Changed files

  • litellm/litellm_core_utils/safe_json_dumps.py (modified, +9/-1)
  • tests/test_litellm/litellm_core_utils/test_safe_json_dumps.py (modified, +23/-0)

Code Example

stream = await client.chat.completions.create(
    model=...,
    messages=...,
    stream=True,
)
async for chunk in stream:
    choice = chunk.choices[0]
    content = geattr(choice.delta, "content", "")
    print(content)

---

{ "event": "litellm.proxy.proxy_server.async_data_generator(): Exception occured - Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer'", "exc_info": [ "<class 'pydantic_core._pydantic_core.PydanticSerializationError'>", "PydanticSerializationError(Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer')", "<traceback object at 0xffff5026ef80>" ], "logger": "LiteLLM Proxy", "level": "error", ... }
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

TLDR: the last chunk can’t be serialized due to logprobs attribute

What you do: You call the AI (e.g. OpenAI's GPT‑5) in streaming mode.

What happens in the background: The model sends the answer in many small pieces (chunks) instead of one big block at the end. Where it breaks: It works until the very last chunk. That last chunk is special: it doesn’t contain new text (content=None), it just says “I’m done.” (finish_reason='stop'). It can still carry a logprobs object that describes how “confident” the model was. In that final chunk that object is always (at least very often) empty (no real content, no refusal info—just placeholders: ChoiceLogprobs(content=None, refusal=None)).

Why it crashes: When the proxy tries to turn that last chunk into JSON to send to the client, the library (pydantic) that does the conversion runs into a bug: it doesn’t know how to serialize this particular empty logprobs structure. So you get: 'MockValSer' object cannot be converted to 'SchemaSerializer'. In other words: “I have this thing that looks like logprobs but isn’t in a form I can turn into JSON.” So the actual issue is: the last streaming chunk has an empty logprobs object that cannot be serialized to JSON, and that triggers the error.

Related LiteLLM issues - none of them solved the core problem:

Steps to Reproduce

stream = await client.chat.completions.create(
    model=...,
    messages=...,
    stream=True,
)
async for chunk in stream:
    choice = chunk.choices[0]
    content = geattr(choice.delta, "content", "")
    print(content)

Relevant log output

{ "event": "litellm.proxy.proxy_server.async_data_generator(): Exception occured - Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer'", "exc_info": [ "<class 'pydantic_core._pydantic_core.PydanticSerializationError'>", "PydanticSerializationError(Error serializing to JSON: TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer')", "<traceback object at 0xffff5026ef80>" ], "logger": "LiteLLM Proxy", "level": "error", ... }

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.14-stable.gpt-5.4-patch6

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To solve the serialization issue with the empty logprobs object in the last chunk, we need to modify the serialization process to handle this special case.

Here are the steps:

  • Check if the logprobs object is empty before attempting to serialize it.
  • If it's empty, either remove it from the object being serialized or replace it with a default value that can be serialized.

Example code:

import json
from pydantic import BaseModel

class Chunk(BaseModel):
    content: str = None
    logprobs: object = None
    finish_reason: str = None

def serialize_chunk(chunk):
    # Check if logprobs is empty
    if chunk.logprobs and not hasattr(chunk.logprobs, 'content') and not hasattr(chunk.logprobs, 'refusal'):
        # Remove or replace logprobs if it's empty
        chunk.logprobs = None
    
    # Attempt to serialize the chunk
    try:
        return chunk.json()
    except Exception as e:
        # Handle serialization errors
        print(f"Error serializing chunk: {e}")
        return None

# Example usage:
chunk = Chunk(content=None, logprobs=object(), finish_reason='stop')
serialized_chunk = serialize_chunk(chunk)
print(serialized_chunk)

Verification

To verify that the fix worked, you can test the serialization of the last chunk with an empty logprobs object. If the serialization is successful, it should no longer throw a PydanticSerializationError.

Extra Tips

  • Make sure to handle potential errors that may occur during serialization, such as logging the error and returning a default value or an error message.
  • Consider adding a check for other potential edge cases that may cause serialization issues.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#output truncation#response parsing#generation error#database connection#vector store

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING