Issue Station
Sign In
Tools hereToolbox

llamaIndex - ✅(Solved) Fix [Bug]: LLMChatEndEvent.model_dump() mutates ChatResponse.raw in-place, corrupting caller's response object [1 pull requests]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Error Message

from llama_index.observability.otel import LlamaIndexOpenTelemetry from llama_index.core.llms import OpenAI from pydantic import BaseModel

class MyModel(BaseModel): answer: str

register telemetry (activates dispatcher subscriber)

LlamaIndexOpenTelemetry(...).start_registering()

llm = OpenAI(model="gpt-4o-mini") structured_llm = llm.as_structured_llm(output_cls=MyModel)

from llama_index.core.llms import ChatMessage output = await structured_llm.achat([ChatMessage.from_str("Say hello")]) print(type(output.raw)) # <class 'dict'> ← expected MyModel instance output.raw.answer # AttributeError: 'dict' object has no attribute 'answer'

Root Cause

Because self.response holds a reference to the same ChatResponse object that was returned to the caller, this converts ChatResponse.raw from a Pydantic model instance to a plain dict — permanently, for the caller.

Fix Action

Fix / Workaround

Any code using StructuredLLM.achat() that runs with an active OpenTelemetry/dispatcher subscriber (e.g. LlamaIndexOpenTelemetry.start_registering()) will receive a dict instead of the expected Pydantic model from ChatResponse.raw. This causes AttributeError: 'dict' object has no attribute '...' in all callers that access fields on the structured response.

register telemetry (activates dispatcher subscriber)

LlamaIndexOpenTelemetry(...).start_registering()

PR fix notes

PR #21424: fix: avoid mutating response.raw in LLM event model_dump methods

Description (problem / solution / changelog)

What this fixes

Closes #21422.

The model_dump() overrides in LLMCompletionInProgressEvent, LLMCompletionEndEvent, LLMChatInProgressEvent, and LLMChatEndEvent were mutating self.response.raw in-place:

# Before — corrupts the original object
self.response.raw = self.response.raw.model_dump()
return super().model_dump(**kwargs)

Because self.response is the same object held by the caller, this permanently replaced the Pydantic model with a plain dict. Any code that accessed response.raw after the span handler fired would get a dict instead of the expected model — silent data corruption that only surfaces when the raw response is later inspected or passed to another layer (e.g. retries, streaming handlers, structured output parsers).

Fix

Use model_copy() to create a throwaway copy of the event with raw pre-serialised, dump that copy, and leave the original response object untouched:

# After — original response is never modified
def model_dump(self, **kwargs):
    if isinstance(self.response.raw, BaseModel):
        return self.model_copy(
            update={
                "response": self.response.model_copy(
                    update={"raw": self.response.raw.model_dump()}
                )
            }
        ).model_dump(**kwargs)
    return super().model_dump(**kwargs)

The same pattern is applied to all four affected event classes.

Testing

The bug is reproducible by calling model_dump() on any of the four event classes and then checking that event.response.raw is still a BaseModel instance afterwards. Without this fix, it would be a dict.

Happy to add a unit test if that would help — let me know if there's a preferred location for instrumentation event tests.

Changed files

  • llama-index-core/llama_index/core/instrumentation/events/llm.py (modified, +28/-8)

Code Example

# llama_index/core/instrumentation/events/llm.py
def model_dump(self, **kwargs: Any) -> Dict[str, Any]:
    if self.response is not None and isinstance(self.response.raw, BaseModel):
        self.response.raw = self.response.raw.model_dump()  # ← mutates original object!
    return super().model_dump(**kwargs)

---

from llama_index.observability.otel import LlamaIndexOpenTelemetry
from llama_index.core.llms import OpenAI
from pydantic import BaseModel

class MyModel(BaseModel):
     answer: str

# register telemetry (activates dispatcher subscriber)
LlamaIndexOpenTelemetry(...).start_registering()

llm = OpenAI(model="gpt-4o-mini")
structured_llm = llm.as_structured_llm(output_cls=MyModel)

from llama_index.core.llms import ChatMessage
output = await structured_llm.achat([ChatMessage.from_str("Say hello")])
print(type(output.raw))  # <class 'dict'>  ← expected MyModel instance
output.raw.answer        # AttributeError: 'dict' object has no attribute 'answer'

---

def model_dump(self, **kwargs: Any) -> Dict[str, Any]:
    if self.response is not None and isinstance(self.response.raw, BaseModel):
        # Do NOT mutate self.response.raw — create a copy for serialization only
        data = super().model_dump(**kwargs)
        data["response"]["raw"] = self.response.raw.model_dump()
        return data
    return super().model_dump(**kwargs)

---

[2026-04-20 12:58:39,879: WARNING/ForkPoolWorker-1] [LLM_DBG] ruleset_selection attempt=1 llm_type=StructuredLLM output_cls=<class 'XXX'> output_type=ChatResponse has_raw=True raw_type=dict output_repr=ChatResponse(message=ChatMessage(role=<MessageRole.ASSISTANT: 'assistant'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='{"reasoning":"The [Response] value \'XXX\' corresponds to the XXX condition, so per the rule no ruleset is selected.","ruleset_id":null}')]), raw={'reasoning': "The [Response] value 'XXX' corresponds to the XXX condition, so per the rule no ruleset is selected.", 'ruleset_id': None}
[2026-04-20 12:58:39,879: WARNING/ForkPoolWorker-1] [LLM_DBG] ruleset_selection attempt=1 response_type=dict is_dict=True response_repr={'reasoning': "The [Response] value 'XXX' corresponds to the XXX condition, so per the rule no ruleset is selected.",'ruleset_id': None}
[2026-04-20 12:58:39,879: ERROR/ForkPoolWorker-1] _decide_which_ruleset_to_use: LLM FAILED. Error type: AttributeError, Message: 'dict' object has no attribute 'ruleset_id'
RAW_BUFFERClick to expand / collapse

Bug Description

LLMChatEndEvent.model_dump() (and LLMCompletionEndEvent.model_dump()) mutates ChatResponse.raw in-place when serializing the event for telemetry spans:

# llama_index/core/instrumentation/events/llm.py
def model_dump(self, **kwargs: Any) -> Dict[str, Any]:
    if self.response is not None and isinstance(self.response.raw, BaseModel):
        self.response.raw = self.response.raw.model_dump()  # ← mutates original object!
    return super().model_dump(**kwargs)

Because self.response holds a reference to the same ChatResponse object that was returned to the caller, this converts ChatResponse.raw from a Pydantic model instance to a plain dict — permanently, for the caller.

Impact

Any code using StructuredLLM.achat() that runs with an active OpenTelemetry/dispatcher subscriber (e.g. LlamaIndexOpenTelemetry.start_registering()) will receive a dict instead of the expected Pydantic model from ChatResponse.raw. This causes AttributeError: 'dict' object has no attribute '...' in all callers that access fields on the structured response.

The bug is silent in local development (no telemetry subscriber → model_dump() never called) and only manifests in environments with telemetry enabled.

Version

0.14.20

Steps to Reproduce

from llama_index.observability.otel import LlamaIndexOpenTelemetry
from llama_index.core.llms import OpenAI
from pydantic import BaseModel

class MyModel(BaseModel):
     answer: str

# register telemetry (activates dispatcher subscriber)
LlamaIndexOpenTelemetry(...).start_registering()

llm = OpenAI(model="gpt-4o-mini")
structured_llm = llm.as_structured_llm(output_cls=MyModel)

from llama_index.core.llms import ChatMessage
output = await structured_llm.achat([ChatMessage.from_str("Say hello")])
print(type(output.raw))  # <class 'dict'>  ← expected MyModel instance
output.raw.answer        # AttributeError: 'dict' object has no attribute 'answer'

Expected Behavior model_dump() should serialize a copy of the data, not mutate the original object. ChatResponse.raw must remain a Pydantic model instance after model_dump() is called on the event.

Proposed Fix - same in LLMCompletionEndEvent

def model_dump(self, **kwargs: Any) -> Dict[str, Any]:
    if self.response is not None and isinstance(self.response.raw, BaseModel):
        # Do NOT mutate self.response.raw — create a copy for serialization only
        data = super().model_dump(**kwargs)
        data["response"]["raw"] = self.response.raw.model_dump()
        return data
    return super().model_dump(**kwargs)

Or alternatively, override model_dump to deep-copy the response before mutation.

Relevant Logs/Tracebacks

[2026-04-20 12:58:39,879: WARNING/ForkPoolWorker-1] [LLM_DBG] ruleset_selection attempt=1 llm_type=StructuredLLM output_cls=<class 'XXX'> output_type=ChatResponse has_raw=True raw_type=dict output_repr=ChatResponse(message=ChatMessage(role=<MessageRole.ASSISTANT: 'assistant'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='{"reasoning":"The [Response] value \'XXX\' corresponds to the XXX condition, so per the rule no ruleset is selected.","ruleset_id":null}')]), raw={'reasoning': "The [Response] value 'XXX' corresponds to the XXX condition, so per the rule no ruleset is selected.", 'ruleset_id': None}
[2026-04-20 12:58:39,879: WARNING/ForkPoolWorker-1] [LLM_DBG] ruleset_selection attempt=1 response_type=dict is_dict=True response_repr={'reasoning': "The [Response] value 'XXX' corresponds to the XXX condition, so per the rule no ruleset is selected.",'ruleset_id': None}
[2026-04-20 12:58:39,879: ERROR/ForkPoolWorker-1] _decide_which_ruleset_to_use: LLM FAILED. Error type: AttributeError, Message: 'dict' object has no attribute 'ruleset_id'

extent analysis

TL;DR

The model_dump() method should be modified to create a copy of the ChatResponse.raw data for serialization instead of mutating the original object.

Guidance

  • Identify the model_dump() method in LLMChatEndEvent and LLMCompletionEndEvent classes and modify it to create a copy of self.response.raw before serializing it.
  • Verify that the ChatResponse.raw remains a Pydantic model instance after model_dump() is called on the event by checking its type and attributes.
  • Consider overriding model_dump() to deep-copy the response before mutation to avoid any potential issues.
  • Test the modified code with the provided steps to reproduce the issue to ensure the fix works as expected.

Example

def model_dump(self, **kwargs: Any) -> Dict[str, Any]:
    if self.response is not None and isinstance(self.response.raw, BaseModel):
        # Create a copy of self.response.raw for serialization
        data = super().model_dump(**kwargs)
        data["response"]["raw"] = self.response.raw.model_dump()
        return data
    return super().model_dump(**kwargs)

Notes

The proposed fix assumes that creating a copy of self.response.raw is sufficient to avoid mutating the original object. However, if the model_dump() method is called recursively or if there are other references to the same object, additional modifications may be necessary.

Recommendation

Apply the proposed workaround by modifying the model_dump() method to create a copy of self.response.raw for serialization. This should fix the issue and prevent the AttributeError from occurring.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#environment setup#docker error#permission error#memory optimization#batch processing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING