litellm - 💡(How to fix) Fix [Bug]: JSON repair utility breaks on tool call arguments ending inside a string or containing raw newlines [1 pull requests]

Description

The JSON repair utility in litellm/litellm_core_utils/prompt_templates/common_utils.py has two bugs that cause JSONDecodeError when processing tool call arguments:

Bug 1: Premature bracket closure when truncated inside a string

When a tool call argument is truncated inside a string value (e.g. a long shell heredoc cut off before the closing "), _repair_json() closes all open brackets and returns prematurely. This produces invalid JSON because the unterminated string value is never properly closed.

Example: A tool call with a long Write command like:

Write(code="very long file content that gets truncated...

The repair utility closes the outer {} without closing the string, resulting in a JSONDecodeError.

Bug 2: Raw newlines inside JSON string values are not escaped

LLM tool call arguments containing shell heredocs or multi-line strings often include literal \n and \r characters inside JSON string values, which is not valid JSON. The repair utility does not escape these before attempting to parse, causing parse failures.

Example: A Bash tool call emitting:

Write(code="line1\nline2\nline3")

The raw newlines inside the quoted string cause the JSON parser to fail.

Proposed fix

For Bug 1: Check in_string state before returning from the bracket-repair loop, and close any open string value with a ".

For Bug 2: Add _escape_raw_newlines_in_json() that walks the string character-by-character and escapes \n/\r to \\n/\\r only when inside a quoted string.

Related to #25985 "JSONDecodeError Unterminated string when parsing tool call arguments" (partially overlapping).