llamaIndex - 💡(How to fix) Fix [Bug]: obj serialization mismatch [2 pull requests]

Bug Description

In pr #18447, serialization using node.dict() was replaced with the newer node.model_dump(...) in multiple places, e.g. in node_to_metadata_dict(). The previous .dict() was subclassed for IndexNodes to wrap the contents of the obj field with an additional __type__ field (using doc_to_json()). model_dump() is not subclassed.

It does not seem like the deserialization was updated to reflect this. metadata_dict_to_node() calls BaseComponent.from_json(node_json), which calls IndexNode.from_dict(...), which calls json_to_doc(), which expectes the __type__ wrapper. It thus throws an exception and replaces the obj with the backup variant of a new TextNode with the serialized obj as text content.

Version

0.14.21

Steps to Reproduce

This occured for me when using a BM25 retriever, which serializes the nodes for its corpus. When inputting IndexNodes, the underlying TextNodes are not deserialized correctly on retrieval.