ollama - 💡(How to fix) Fix Metal shader compile fails on macOS 26.3.1 — MPPTensorOpsMatMul2dImpl static_assert (bfloat/half) on Apple M5 [1 participants]

ggml_metal_init: picking default device: Apple M5 ggml_metal_init: the device does not have a precompiled Metal library - this is unexpected ggml_metal_init: will try to compile it on the fly ggml_metal_library_init: error: Error Domain=MTLLibraryErrorDomain Code=3 MPPTensorOpsMatMul2dImpl.h:3266: static_assert failed due to requirement '__tensor_ops_detail::__is_same_v<bfloat, half>' "Input types must match cooperative tensor types" MPPTensorOpsMatMul2dImpl.h:3267: static_assert failed due to requirement '__tensor_ops_detail::__is_same_v<half, bfloat>' ggml_metal_init: error: failed to initialize the Metal library ggml_backend_metal_device_init: error: failed to allocate context llama_init_from_model: failed to initialize the context: failed to initialize Metal backend panic: unable to create llama context

TL;DR

Update the GGML shader source to use matching types for mpp::tensor_ops::matmul2d operands to resolve the Metal library compilation failure.

Guidance

• Verify that the issue is indeed caused by the type mismatch in the mpp::tensor_ops::matmul2d instantiation by checking the Metal library compilation logs for the specific error message.
• Identify the embedded GGML shader source code that instantiates mpp::tensor_ops::matmul2d with mismatched types and update it to use matching types (e.g., both half or both bfloat).
• Consider temporarily using a previous version of macOS or disabling the Metal library compilation to work around the issue, if possible.
• Test the updated GGML shader source with the 0.22.0-rc1 version to ensure the fix works as expected.

Example

No code snippet is provided as the issue is specific to the GGML shader source code, which is not included in the issue body.

Notes

The fix may require updates to the GGML library or the llama runner process to handle the new strict type-matching requirements in the MetalPerformancePrimitives headers.

Recommendation

Apply a workaround by updating the GGML shader source code to use matching types for mpp::tensor_ops::matmul2d operands, as this is the most direct way to resolve the Metal library compilation failure.