litellm - 💡(How to fix) Fix [Bug]: Cannot remove `extra_param` from model's `litellm_info` - PUT requests succeeds but value persists after refresh [1 participants]

Fix Plan

To resolve the issue of the custom field not being removed from litellm_info, we need to ensure that the update operation correctly persists the changes. Here are the steps:

• Update the model update endpoint:
  • Check if the litellm_info object is being correctly updated in the database.
  • Verify that the update operation is not ignoring the removal of the extra_param field.
• Implement a cache invalidation mechanism:
  • If the UI is caching the model data, implement a cache invalidation mechanism to ensure that the updated model data is fetched from the database after an update operation.
• Example code snippet:

# Assuming a Python backend with a database model
from models import Model

def update_model(model_id, updated_data):
    model = Model.objects.get(id=model_id)
    model.litellm_info = updated_data.get('litellm_info', {})
    model.save()
    # Invalidate cache
    cache.delete(f'model_{model_id}')

• Verify database updates:
  • Check the database to ensure that the litellm_info object is being correctly updated.

Verification

To verify that the fix worked:

• Update a model to add a custom field to litellm_info.
• Remove the custom field and update the model again.
• Refresh the page and navigate to the model details to ensure that the custom field is no longer present.

Extra Tips

• Ensure that the update endpoint is correctly handling the removal of fields from the litellm_info object.
• Implement logging to track any issues with the update operation or cache invalidation.
• Consider implementing a retry mechanism to handle any transient errors during the update operation.