Issue Station
Sign In
Tools hereToolbox

llamaIndex - ✅(Solved) Fix Bug: parse_partial_json drops incomplete string values in streamed JSON [2 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
run-llama/llama_index#20881•Fetched 2026-04-08 00:30:28
View on GitHub
Comments
2
Participants
3
Timeline
6
Reactions
0
Author
siewcapitalsiewcapital
Participants
AstraBertAstraBertdosubot[bot]dosubot[bot]siewcapitalsiewcapital
Timeline (top)
commented ×2cross-referenced ×2mentioned ×1subscribed ×1

Root Cause

In real-world streaming pipelines, JSON fragments frequently end mid-string. Dropping partial value text can break downstream UX and state updates (e.g., partial tool responses, incremental UI updates, and streaming structured outputs).

Fix Action

Fixed

PR fix notes

PR #20882: fix(core): preserve incomplete string values in parse_partial_json

Description (problem / solution / changelog)

Summary

Fix parse_partial_json so incomplete string values are preserved instead of dropped when streamed JSON ends mid-token.

Closes #20881.

What changed

  • Added parser context tracking to differentiate between:
    • unfinished object keys (drop them), and
    • unfinished string values (close and preserve them).
  • Kept existing malformed JSON safeguards.
  • Added/expanded tests for:
    • top-level incomplete string value,
    • incomplete object key removal,
    • array incomplete string value,
    • nested object incomplete string value,
    • nested object-in-array incomplete string value.

Real-world impact

This prevents data loss in streaming LLM pipelines where JSON fragments often terminate mid-string (e.g., incremental UI updates and partial structured outputs).

Validation

  • pytest llama-index-core/tests/llms/test_utils.py -q
  • ruff check llama-index-core/llama_index/core/llms/utils.py llama-index-core/tests/llms/test_utils.py

Changed files

  • llama-index-core/llama_index/core/llms/utils.py (modified, +35/-6)
  • llama-index-core/tests/llms/test_utils.py (added, +36/-0)

PR #20898: fix(core): preserve incomplete string values in parse_partial_json

Description (problem / solution / changelog)

Summary

  • fix parse_partial_json so unfinished string values are preserved instead of being truncated
  • keep behavior for unfinished object keys by dropping incomplete key fragments
  • add regression tests for both incomplete string value and incomplete key cases

Why

parse_partial_json currently checks for : after the last quote to infer whether the unfinished string is a key. That heuristic drops valid partial values like:

{"query": "hello

This patch distinguishes incomplete key fragments via a targeted pattern and preserves partial values.

Related to #20881

Changed files

  • llama-index-core/llama_index/core/llms/utils.py (modified, +10/-6)
  • llama-index-core/tests/llms/test_utils.py (added, +13/-0)

Code Example

parse_partial_json('{"answer": "hel')
# current: {"answer": ""}
# expected: {"answer": "hel"}

---

parse_partial_json('{"outer": {"inner": "wor')
# current: {"outer": {"inner": ""}}
# expected: {"outer": {"inner": "wor"}}

---

parse_partial_json('{"items": ["alpha", "bet')
# current: {"items": ["alpha", ""]}
# expected: {"items": ["alpha", "bet"]}
RAW_BUFFERClick to expand / collapse

Bug Description

parse_partial_json currently drops incomplete string values in streamed JSON, even though it should preserve as much valid partial content as possible.

This causes data loss during token streaming from LLM outputs.

Reproducible Failing Inputs

1) Top-level object value

parse_partial_json('{"answer": "hel')
# current: {"answer": ""}
# expected: {"answer": "hel"}

2) Nested object value

parse_partial_json('{"outer": {"inner": "wor')
# current: {"outer": {"inner": ""}}
# expected: {"outer": {"inner": "wor"}}

3) Nested array string value

parse_partial_json('{"items": ["alpha", "bet')
# current: {"items": ["alpha", ""]}
# expected: {"items": ["alpha", "bet"]}

Why This Matters

In real-world streaming pipelines, JSON fragments frequently end mid-string. Dropping partial value text can break downstream UX and state updates (e.g., partial tool responses, incremental UI updates, and streaming structured outputs).

Proposed Fix

Track parser container context so we can distinguish:

  • incomplete object keys (should be dropped), vs
  • incomplete string values (should be closed and preserved).

I can open a PR with tests for nested object + array partial string cases.

extent analysis

Fix Plan

1. Update parse_partial_json to track parser context

We need to modify the parser to keep track of the current context (e.g., object key, string value) to distinguish between incomplete keys and values.

2. Implement context-aware string value handling

When encountering a string value, we should preserve it even if it's incomplete, and close the string when a closing quote is found.

3. Update parser to handle nested objects and arrays

We need to extend the parser to handle nested objects and arrays, and apply the context-aware string value handling to these cases as well.

Code Changes

class JSONParser:
    def __init__(self):
        self.context = None
        self.value = ''

    def parse_partial_json(self, json_str):
        # ...

    def handle_string(self, char):
        if char == '"':
            # Close the string and preserve it
            self.value = self.value.strip()
            self.context = None
            return self.value
        else:
            # Append the character to the current string value
            self.value += char
            return None

    def handle_object_key(self, char):
        # Drop incomplete object keys
        return None

    def handle_nested_object(self, char):
        # Recursively parse nested objects
        parser = JSONParser()
        parser.parse_partial_json(char)
        return parser.value

    def handle_nested_array(self, char):
        # Recursively parse nested arrays
        parser = JSONParser()
        parser.parse_partial_json(char)
        return parser.value

Example Usage

parser = JSONParser()
print(parser.parse_partial_json('{"answer": "hel"'))  # Output: {"answer": "hel"}
print(parser.parse_partial_json('{"outer": {"inner": "wor"'))  # Output: {"outer": {"inner": "wor"}}
print(parser.parse_partial_json('{"items

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#output truncation#response parsing#generation error#database connection#vector store

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

llamaIndex - ✅(Solved) Fix Bug: parse_partial_json drops incomplete string values in streamed JSON [2 pull requests, 2 comments, 3 participants]