ollama - 💡(How to fix) Fix 500 Internal Server Error when loading hf.co/HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive:Q4_K_M via ollama run [1 participants]

I'm encountering a 500 Internal Server Error when attempting to run a quantized GGUF model from Hugging Face using Ollama. arch ollama[1158318]: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35'

Fix Plan

The fix involves updating the Ollama version to support the latest GGUF model architecture.

Steps to Fix

• Update Ollama to the latest version using the following command:

pip install --upgrade ollama

• If the issue persists, try specifying the model architecture explicitly when loading the model:

from ollama import LLaMA

model = LLaMA.from_pretrained('path/to/model', architecture='qwen35')

Alternatively, you can try updating the ollama configuration file to include the missing architecture:

# ollama_config.json
{
    "architectures": {
        "qwen35": {
            "type": "q4_K",
            "num_tensors": 203
        }
    }
}

Verification

After applying the fix, verify that the model loads successfully by running the following command:

from ollama import LLaMA

model = LLaMA.from_pretrained('path/to/model')
print(model)

If the model loads successfully, you should see the model's architecture and configuration printed to the console.

Extra Tips

• Make sure to check the Ollama documentation for the latest supported architectures and models.
• If you're using a custom model, ensure that it's compatible with the latest Ollama version.