llamaIndex - 💡(How to fix) Fix [Question]: How large data set can be reingested to milvus and query it using llamaindex when I need to change the embedding model to a new one? [1 comments, 2 participants]

Fix: Rebuild Indices for New Embedding Model

Rebuilding indices for the new embedding model will allow Milvus to retrieve information related to the old indices.

Step-by-Step Solution Plan:

1. Drop Old Indices

DROP INDEX <old_index_name> ON <collection_name>;

Replace <old_index_name> and <collection_name> with the actual names of your index and collection.

2. Rebuild Indices for New Embedding Model

CREATE INDEX <new_index_name> ON <collection_name> USING IVF_FLAT (embedding);

Replace <new_index_name> and <collection_name> with the actual names of your new index and collection.

3. Rebuild the Vector Table

ALTER TABLE <collection_name> REBUILD;

Replace <collection_name> with the actual name of your collection.

4. Rebuild the Index for the Vector Table

REBUILD INDEX <new_index_name> ON <collection_name>;

Replace <new_index_name> and <collection_name> with the actual names of your new index and collection.

Verification

• Check if the new embedding model is being used correctly by querying the collection.
• Verify that the information related to the old indices is being retrieved correctly.

Extra Tips

• Make sure to back up your data before rebuilding indices.
• Rebuilding indices can be a time-consuming process, so plan accordingly.
• Consider using a more efficient indexing strategy for your use case.