ollama - 💡(How to fix) Fix Models load into ram first and then into gpu on latest update [1 comments, 2 participants]

Fix Plan

The fix is to optimize model loading by downgrading to a stable version of Ollama, as the issue is version-specific.

Steps to Fix

• Downgrade Ollama to version 1.17.4 using the following command:

pip install ollama==1.17.4

• Verify that the downgrade was successful by checking the Ollama version:

ollama --version

• Update your code to ensure compatibility with the downgraded version, if necessary.

Verification

• Load the models and measure the loading time to ensure it has returned to the expected 15 seconds.
• Monitor the application for any other issues that may have arisen from the downgrade.

Example Use Case

import ollama

# Load the model
model = ollama.load_model('model_name')

# Measure the loading time
import time
start_time = time.time()
model.load()
end_time = time.time()
print(f"Loading time: {end_time - start_time} seconds")