ollama - 💡(How to fix) Fix Apple M5 backend/runtime compatibility issue in Ollama [1 participants]

Summary

On Apple M5, Ollama fails to run qwen2.5-coder:14b due to a Metal backend initialization crash. The server starts and /api/tags works, but /api/generate fails when loading the model.

Environment

• Ollama: 0.20.7 (also reproduced with 0.21.0 binary from cached updater bundle)
• OS: macOS 15.2 (Darwin 25.2.0)
• Hardware: Apple M5
• RAM: 32 GB
• Model: qwen2.5-coder:14b

Repro Steps

1. Ensure no existing instance: pkill -f "Ollama|ollama" || true
2. Start server: OLLAMA_DEBUG=1 ollama serve > /tmp/ollama-m5.log 2>&1 &
3. Confirm server readiness: curl -s http://127.0.0.1:11434/api/tags
4. Trigger generation: curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5-coder:14b","prompt":"hello","stream":false}'

Expected

Model should load and return a normal response.

Actual

Returns server error:

• {"error":"llama runner process has terminated: %!w(<nil>)"}

Key Log Evidence

• ggml_metal_init: picking default device: Apple M5
• ggml_metal_init: the device does not have a precompiled Metal library - this is unexpected
• ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3
• static_assert failed due to requirement '__tensor_ops_detail::__is_same_v<bfloat, half>' "Input types must match cooperative tensor types"
• ggml_metal_init: error: failed to initialize the Metal library
• llama_init_from_model: failed to initialize the context: failed to initialize Metal backend
• panic: unable to create llama context
• llama runner terminated (exit status 2)

Additional Notes

• OLLAMA_LLM_LIBRARY=cpu was also tested, but the run path still attempts Metal initialization and fails with the same signature.
• This appears to be a backend compatibility/regression issue on Apple M5 Metal path.

Control Tests

• Same model works outside Ollama:
  • Using llama.cpp llama-server with the exact same GGUF blob from ~/.ollama/models/blobs/... serves requests successfully on /v1/chat/completions.
  • This indicates the model artifact itself is valid.
• Different model also fails in Ollama:
  • tinyllama:latest was tested in Ollama and failed with the same llama runner process has terminated behavior.
  • This suggests the issue is runtime/backend-wide in Ollama on this machine, not specific to qwen2.5-coder:14b.