ollama - 💡(How to fix) Fix M1 Mac: z-image-turbo:fp8 VRAM regression 0.1.8.1→0.1.8.2 & 0.1.8.3 ... (11.9>11.3 GiB) [2 comments, 2 participants]

1. Error trotz kleiner Auflösung Error: 500 Internal Server Error: model requires 11.9 GiB but only 11.3 GiB are available (after 512.0 MiB overhead)

Fix Plan

The fix involves increasing the available memory for the model by adjusting the memory allocation settings.

Steps to Fix

• Increase the memory limit by setting the --memory-limit flag when running the ollama command. For example:

ollama run x/z-image-turbo:fp8 "test" --width 512 --height 512 --memory-limit 12GiB

• Alternatively, you can also try reducing the model's memory requirements by using a smaller model or optimizing the model's architecture.
• If using a Docker container, ensure that the container has sufficient memory allocated to it. You can do this by setting the --memory flag when running the Docker container. For example:

docker run -m 12GiB x/z-image-turbo:fp8 "test" --width 512 --height 512

Verification

To verify that the fix worked, run the ollama command with the increased memory limit and check that the model loads successfully without any memory-related errors.

Example Code

You can also modify the ollama command to automatically increase the memory limit when running the model. For example:

import subprocess

# Set the memory limit to 12GiB
memory_limit = "12GiB"

# Run the ollama command with the increased memory limit
subprocess.run([
    "ollama", "run", "x/z-image-turbo:fp8", "test", 
    "--width", "512", "--height", "512", 
    "--memory-limit", memory_limit
])