ollama - 💡(How to fix) Fix Regression in running flux2-klein on iMac since v0.15.5 [1 comments, 1 participants]

$ ollama run --verbose x/flux2-klein:9b-bf16 Error: 500 Internal Server Error: mlx runner failed: Error: failed to create server: failed to load image model: insufficient memory for image generation: need 32 GB, have 30 GB (exit: exit status 1)

Fix Plan

To resolve the "insufficient memory" issue with ollama version 0.15.5 and later, we need to increase the memory allocation for the model.

Step-by-Step Solution

• Check available memory: Verify that your system has sufficient memory to allocate to the model. In this case, the model requires 32 GB of memory.
• Increase memory allocation: Update the ollama configuration to allocate more memory to the model. You can do this by setting the --memory flag when running the ollama run command.
• Example command: Run the following command to allocate 32 GB of memory to the model:

ollama run --verbose --memory 32G x/flux2-klein:9b-bf16

• Alternative solution: If increasing the memory allocation is not feasible, consider using a model that requires less memory or optimizing the model to reduce its memory footprint.

Verification

To verify that the fix worked, run the ollama run command with the increased memory allocation and check the output for any error messages related to insufficient memory. If the model runs successfully, it should indicate that the memory issue has been resolved.

Extra Tips

• Make sure to monitor your system's memory usage when running the model to avoid over-allocating memory and causing other system issues.
• Consider using a more efficient model or optimizing your existing model to reduce its memory requirements.