ollama - 💡(How to fix) Fix Metal backend crash on Apple Silicon M5: MTLLibrary bfloat/half mismatch causes llama runner termination (500) [3 comments, 2 participants]

Summary

Ollama consistently fails to run any model due to a Metal backend initialization failure. The runner process terminates and the API returns 500.

Environment

• Ollama version: 0.20.6
• Platform: macOS 26.3.1 (build 25D771280a)
• Hardware: Apple Silicon M5
• Install type: Ollama desktop app + CLI
• Reproducibility: 100% (all tested models)

Steps to Reproduce

1. Start Ollama.
2. Run any model prompt, for example:
   • ollama run gemma2:2b "Hello"
3. Observe server-side runner crash and client-side 500 error.

Expected Behavior

Model loads successfully and returns a response.

Actual Behavior

Model load fails during Metal initialization; llama runner exits; client receives:

• 500 Internal Server Error: llama runner process has terminated: %!w(<nil>)

Error Signature (sanitized)

• static_assert failed: Input types must match cooperative tensor types
• bfloat/half mismatch in MetalPerformancePrimitives matmul path
• ggml_metal_init: error: failed to initialize the Metal library
• llama_init_from_model: failed to initialize the context: failed to initialize Metal backend
• panic: unable to create llama context
• llama runner terminated: exit status 2

Representative framework references:

• /System/Library/Frameworks/MetalPerformancePrimitives.framework/Headers/__impl/MPPTensorOpsMatMul2dImpl.h:3266
• /System/Library/Frameworks/MetalPerformancePrimitives.framework/Headers/__impl/MPPTensorOpsMatMul2dImpl.h:3267

Additional Observations

• Failure occurs across multiple models, not model-specific.
• Model blobs download and verification succeed.
• Runtime fails at backend initialization stage.

Workarounds Tried

• Reinstall Ollama
• Install/update Xcode and Metal toolchain components
• Restart app/processes
• Alternate host/port setup
• Disable flash attention
• KV cache type adjustments

Result: same failure path in Metal backend.

Impact

Ollama is unusable for inference in this environment because all model runs terminate before serving.

Privacy Note

All user-identifying information and local absolute paths have been redacted.