ollama - 💡(How to fix) Fix Any way to run large model locally instead of cloud? GLM, Deepseek v4, minimax, etc. [1 participants]

It always results in an error. such as llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe' llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'glm-dsa'

TL;DR

Check if Ollama supports the specific quantized model architectures ('qwen35moe' and 'glm-dsa') and consider updating Ollama or using a different loader to support local inference of these models.

Guidance

• Verify that Ollama is configured to use local mode instead of cloud mode by checking the documentation or configuration options.
• Check the Hugging Face model cards for 'Qwen3.6-35B-A3B-GGUF:UD-Q8_K_XL' and 'GLM-5-REAP-50pct-UD-IQ2_M' to ensure they are compatible with Ollama and local inference.
• Investigate if there are any known issues or limitations with loading quantized models in Ollama, specifically with the 'qwen35moe' and 'glm-dsa' architectures.
• Consider reaching out to the Ollama community or developers to ask about support for these specific models and architectures.

Notes

The error messages suggest that Ollama does not recognize the model architectures, which may indicate a limitation or incompatibility with the quantized models from Hugging Face.

Recommendation

Apply workaround: Try to update Ollama to the latest version or use a different model loader that supports the specific architectures, as it seems that the current version of Ollama may not support the 'qwen35moe' and 'glm-dsa' models.