ollama - 💡(How to fix) Fix qwen3.5:9b model runner crashes on Apple M4 16GB - "model runner has unexpectedly stopped" [1 comments, 2 participants]

crashes immediately when loaded on Apple M4 16GB unified memory with error:

or an internal error, check ollama server logs for details" {"error":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details"}

Fix Plan

The fix involves increasing the memory allocation for the model runner to prevent it from crashing due to resource limitations.

• Increase the VRAM_LIMIT environment variable to allocate more memory for the model runner.
• Modify the ollama configuration to increase the default_num_ctx value to allow for more context switches.
• Update the model loading code to handle large models by increasing the batch_size and reducing the sequence_length.

Example code snippet to increase VRAM_LIMIT:

import os
os.environ['VRAM_LIMIT'] = '12GiB'

Example configuration update:

ollama config set default_num_ctx 8192

Example model loading code update:

model_loader = ModelLoader(batch_size=16, sequence_length=128)

Verification

To verify that the fix worked, load the qwen3.5:9b model and check for any errors.

• Run the curl command to load the model and generate text.
• Check the server logs for any error messages.
• Verify that the model responds normally and generates text correctly.

Example verification command:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "qwen-vialetto-fix",
  "prompt": "ciao",
  "keep_alive": "24h"
}'

Extra Tips

• Monitor the server logs and adjust the VRAM_LIMIT and default_num_ctx values as needed to prevent resource limitations.
• Consider updating the ollama version to the latest release to ensure that any known issues are fixed.
• Test the model with different batch_size and sequence_length values to optimize performance.