ollama - 💡(How to fix) Fix MLX runner fails to load dynamic library on Ollama 0.20.5 (Apple Silicon, Homebrew install) [1 participants]

After upgrading from 0.20.3 to 0.20.5, MLX-based models fail to run due to a dynamic library loading error. Non-MLX models continue to work normally. Restarting the Ollama service does not resolve the issue. Error: 500 Internal Server Error: mlx runner failed: Error: MLX not available: failed to load MLX dynamic library (searched: [/opt/homebrew/Cellar/ollama/0.20.5/bin/lib/ollama /opt/homebrew/Cellar/ollama/0.20.5/bin /opt/homebrew/var/build/lib/ollama]) (exit: exit status 1) Error suggests missing or incorrectly packaged MLX dynamic library rather than a model issue

TL;DR

Downgrade to Ollama version 0.20.3 to restore functionality of MLX-based models until the dynamic library loading issue in version 0.20.5 is resolved.

Guidance

• Verify that the issue is specific to MLX-backed models by attempting to run a non-MLX model, as the problem seems to be isolated to MLX models.
• Check the installation paths and library dependencies for the Ollama 0.20.5 version to ensure that the MLX dynamic library is correctly packaged and accessible.
• Consider reinstalling Ollama 0.20.5 via Homebrew to ensure a clean installation, although the issue suggests a packaging problem rather than an installation error.
• If downgrading to 0.20.3 is not feasible due to other dependencies, monitor the Ollama project for updates that address the MLX dynamic library loading issue in version 0.20.5.

Example

No specific code snippet is applicable in this scenario as the issue appears to be related to packaging and library loading rather than code implementation.

Notes

The solution provided is based on the information given and the observed behavior. The actual fix may depend on further investigation into the packaging and distribution of the MLX dynamic library with the Ollama 0.20.5 Homebrew build.

Recommendation

Apply workaround: Downgrade to version 0.20.3, as it is confirmed to work with MLX-based models, allowing for continued use of these models until the issue in version 0.20.5 is resolved.