What is the issue?

When Ollama runs in a container with a memory limit (this is very important), it incorrectly detects available memory.

When running on bare metal, Ollama uses MemAvailable from /proc/meminfo. This corresponds to an estimate of how much memory applications have to operate with. Available memory doesn't include the used page cache because page cache can be instantly freed.

When running in a container with a memory limit, Ollama reads the available memory from cgroups. But cgroups don't expose a metric comparable to MemAvailable from /proc/meminfo. The closest there is is /sys/fs/cgroup/memory.current but that also includes the used page cache. What this means in practice is that after Ollama loads a model once, Linux caches it into page cache and even after the model gets unloaded, the page cache stays filled and since the page cache is included in memory.current, Ollama thinks that there isn't enough available memory.

TLDR; mem.FreeMemory = /sys/fs/cgroup/memory.max - /sys/fs/cgroup/memory.current is too naive and causes Ollama to think it has less memory available than what it has in reality.

Relevant lines: https://github.com/ollama/ollama/blob/main/discover/cpu_linux.go#L69-L79

I'll have an LLM-generated PR ready to fix this soon. It's a fairly minimal change and it looks good to me but I'm not a Go programmer.

Relevant log output

OS

Docker, Linux

GPU

No response

CPU

Intel, Other

Ollama version

0.20.5