ollama - 💡(How to fix) Fix deepseek-r1:70b model now takes 69 GB of RAM to run a 43 GB model? [3 comments, 2 participants]

Error: 500 Internal Server Error: model requires more system memory (69.0 GiB) than is available (65.6 GiB)

Fix Plan

The fix involves increasing the available system memory or optimizing the model to require less memory.

Steps to Fix

• Increase Swap Space: Increase the swap space to act as an extension of the RAM.
• Optimize Model: Optimize the model to use less memory.
• Update Ollama: Update Ollama to the latest version.

Example Code

To increase swap space, you can use the following commands:

# Create a 4GB swap file
sudo fallocate -l 4G /swapfile

# Make the swap file readable and writable only by root
sudo chmod 600 /swapfile

# Initialize the swap space
sudo mkswap /swapfile

# Activate the swap space
sudo swapon /swapfile

To optimize the model, you can try reducing the batch size or using a more efficient model architecture.

Verification

After applying the fix, try running the model again with ollama run deepseek-r1:70b. If the model runs successfully, the fix was successful. You can also check the available memory using free -h to verify that the swap space is being used.

Extra Tips

• Monitor your system's memory usage to avoid running out of memory in the future.
• Consider upgrading your hardware to increase the available RAM.
• Refer to the Ollama documentation for more information on optimizing models and managing memory usage.