ollama - 💡(How to fix) Fix MLX model support for gemma 4 [1 comments, 2 participants]

hf download mlx-community/gemma-4-26b-a4b-it-mxfp4

cd ~/.cache/huggingface/hub/models--mlx-community--gemma-4-26b-a4b-it-mxfp4 cp -rL snapshots/4d7ba9cbe85182e0b6882dfe72fc2fc6ea2b4afb final echo "FROM final" > Modelfile

ollama create --experimental gemma-4-26b-a4b-it-mxfp4 importing safetensors model  importing safetensors model  importing model-00001-of-00003.safetensors (444 tensors)  importing model-00002-of-00003.safetensors (452 tensors)  importing model-00003-of-00003.safetensors (594 tensors)  importing config config.json  importing config generation_config.json  importing config processor_config.json  importing config tokenizer.json  importing config tokenizer_config.json  writing manifest for gemma-4-26b-a4b-it-mxfp4  successfully imported gemma-4-26b-a4b-it-mxfp4 with 1048 layers

rm -Rf final

ollama list

NAME                                                     ID              SIZE     MODIFIED           

gemma-4-26b-a4b-it-mxfp4:latest                          2d1b607d3ef8    14 GB    About a minute ago

ollama --verbose run gemma-4-26b-a4b-it-mxfp4:latest 

Error: 500 Internal Server Error: mlx runner failed: Error: unsupported architecture: Gemma4ForConditionalGeneration (exit: exit status 1)

TL;DR

The error message indicates that the Gemma 26B model is not supported yet in the mlx runner, suggesting a potential issue with the model's architecture or compatibility.

Guidance

• The error message specifically mentions "unsupported architecture: Gemma4ForConditionalGeneration", implying that the model's architecture is not compatible with the mlx runner.
• Verify the model's compatibility with the mlx runner by checking the documentation or release notes for any updates on supported models.
• Consider using a different model that is known to be compatible with the mlx runner, or wait for an update that adds support for the Gemma 26B model.
• Check the model's configuration files (e.g. config.json, generation_config.json, etc.) to ensure that they are correctly formatted and compatible with the mlx runner.

Notes

The issue seems to be related to the model's architecture or compatibility, rather than a problem with the import process or the ollama tool itself. The error message is quite specific, which suggests that the solution may involve updating the model or using a different one.

Recommendation

Apply workaround: Use a different model that is known to be compatible with the mlx runner, as the Gemma 26B model is not supported yet.