litellm - 💡(How to fix) Fix [Bug]: Prompts in Experimental not fetched when server shutdown [1 participants]

TL;DR

The issue can likely be resolved by implementing a mechanism to load prompts from the Postgres SQL database into the in-memory registry upon server startup.

Guidance

• The root cause appears to be the loss of in-memory registry data upon server restart, while the data persists in the Postgres SQL tables.
• To verify this, check the server logs for any errors or warnings related to data loading from the database upon startup.
• Consider implementing a data loading mechanism that populates the in-memory registry from the Postgres SQL database when the server starts.
• Review the existing code for any data synchronization or caching mechanisms that may be relevant to this issue.

Example

No code snippet is provided due to lack of specific implementation details.

Notes

The solution may require modifications to the server startup process or the data access layer to ensure that the in-memory registry is populated from the database.

Recommendation

Apply a workaround by implementing a data loading mechanism to populate the in-memory registry from the Postgres SQL database upon server startup, as upgrading to a fixed version is not explicitly implied in the issue.