ollama - 💡(How to fix) Fix mlx runner failed [3 comments, 2 participants]

I was able to download this model on intel, yet when running it get this error: Error: 500 Internal Server Error: mlx runner failed: Error: MLX not available: failed to load MLX dynamic library (searched: [/usr/lib/ollama /build/lib/ollama]) (exit: exit status 1)

TL;DR

The issue may be related to a compatibility problem between the new ollama version and the system's GPU or Metal driver, causing a command buffer execution failure.

Guidance

• Verify that the system's Metal driver is up-to-date, as outdated drivers might cause compatibility issues with the new ollama version.
• Check if other users have reported similar issues with ollama 0.19 on macOS with Apple GPU, to determine if it's a known problem.
• Consider testing the same prompt on a different system or with a different GPU to isolate the issue.
• Look into ollama's documentation or community forums for any known workarounds or fixes for Metal-related errors.

Example

No specific code snippet can be provided without more context, but checking the Metal driver version can be done using the system_profiler SPDisplaysDataType command in the Terminal on macOS.

Notes

The issue might be specific to the combination of ollama 0.19, macOS, and Apple GPU, and more information or debugging would be required to determine the root cause.

Recommendation

Apply workaround: Try to update the Metal driver or test on a different system to see if the issue persists, as this might help identify if the problem is specific to the current setup.