openclaw - 💡(How to fix) Fix models.list is slow because it does provider startup and dependency installs on the hot path [2 comments, 2 participants]

TL;DR

The slowdown in models.list can be mitigated by optimizing the model catalog load process, potentially by caching or pre-loading dependencies and avoiding slow network fetches.

Guidance

• Investigate the possibility of caching or pre-loading provider plugins and bundled runtime dependencies to avoid repeated npm install work during model listing.
• Consider optimizing or disabling remote pricing fetches that are timing out at 30s, or increasing the timeout value if necessary.
• Review the gateway/catalog path for potential bottlenecks or areas for improvement, as the local patch to the gateway hot path resulted in faster models.list responses.
• Verify that the QMD Node ABI mismatch is resolved, as it may be contributing to the issue or causing other problems.

Example

No code snippet is provided as the issue does not imply a specific code change.

Notes

The root cause of the slowdown appears to be related to the model catalog load process, but further investigation is needed to determine the best solution. The QMD Node ABI mismatch may be a separate issue that needs to be addressed.

Recommendation

Apply a workaround to optimize the model catalog load process, such as caching or pre-loading dependencies, to improve the performance of models.list. This is recommended as it directly addresses the observed slowdown and can potentially provide a significant improvement in response time.