ollama - 💡(How to fix) Fix The new version encounters an error when running gpt-oss！！！ [1 comments, 2 participants]

When I upgraded ollama to the latest version, I found that my original gpt-oss:120b stopped working, with the following error: Error: 500 Internal Server Error: model failed to load, this may be due to resource limitations or an internal error, check ollama server logs for details Later, I tried gpt-oss:20b, but it also failed to run. Due to the significant upgrade, it was functioning properly on version 0.13.0 for me. I am posting the relevant error logs.

Fix Plan

The fix involves adjusting the GPU memory allocation to accommodate the larger model sizes.

• Step 1: Update Ollama Configuration Update the ollama configuration file to increase the GPU memory allocation. Add the following lines to the configuration file:

  gpu:
    memory_limit: 32G

• Step 2: Set Environment Variables Set the following environment variables before running ollama:

  export CUDA_VISIBLE_DEVICES=0
  export NVIDIA_DRIVER_CAPABILITIES=compute,utility

• Step 3: Run Ollama with Increased Memory Run ollama with the updated configuration and environment variables:

  ollama --config /path/to/config.yml --model gpt-oss:120b

• Step 4: Monitor GPU Memory Usage Monitor the GPU memory usage using nvidia-smi to ensure that the increased memory allocation is sufficient.

Verification

Verify that the fix worked by checking the ollama server logs for successful model loading and by monitoring the GPU memory usage.

Extra Tips

• Ensure that the NVIDIA driver version is compatible with the CUDA version.
• Consider upgrading the GPU drivers to the latest version for improved performance and stability.
• Refer to the ollama documentation for more information on configuring GPU memory allocation and environment variables.