ollama - 💡(How to fix) Fix ollama ps reports ~2x SIZE and a false CPU/GPU split for memory-mapped MoE models (RAM mmap double-counted)

What is the issue?

For a large MoE GGUF, ollama ps reports a SIZE roughly equal to (VRAM copy + the mmap'd weights in RAM) and derives a misleading "CPU/GPU XX%/YY%" split, even though the model is fully resident and running on the GPU. Disabling mmap (use_mmap=false) collapses SIZE to ≈ VRAM and the split jumps to ~100% GPU — with identical actual GPU placement.

Model: hf.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL — Gemma 4 26B-A4B, 128-expert MoE, Q4_0, weights file size = 13.26 GiB.

Pinned to a single 16 GB GPU (CUDA_VISIBLE_DEVICES = one card), num_ctx 8192:

Key arithmetic: 13.4 GiB (vram) + 13.26 GiB (weights mmap'd into RAM) ≈ 26.7 GiB (runner.size). The mmap'd file pages in RAM are counted into the runner "size", and the CPU% appears to be derived as (size − vram) / size ≈ 50%.

Why this is a display/accounting bug, not real offload:

• nvidia-smi shows ~15.5 GiB actually resident on the single GPU in both cases.
• Generation runs at ~100 tok/s in both cases. A genuine 50% CPU compute offload of a 26B model would be ~an order of magnitude slower.
• KV cache is tiny (~460 MiB total via gemma4 iSWA) and does not scale the SIZE (num_ctx 2048→32768 barely changes VRAM).
• With both GPUs visible (28 GB total), the same model loads at 14–15 GB / 100% GPU and never reports the inflated 28 GB.

Expected behavior

ollama ps SIZE and the CPU/GPU split should reflect actual compute placement (this model is 100% on the GPU). The OS page cache from mmap should not be counted as "CPU offload".

Impact

On a single ~16 GB host this inflated estimate can make the scheduler unnecessarily refuse to co-load other models or partial-offload, and it misleads users into believing a model that fits does not fit.

Steps to reproduce

ollama pull hf.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL

# Constrain Ollama to a single ~16 GB GPU (env on the server process), e.g.:
#   CUDA_VISIBLE_DEVICES=<uuid-of-one-16GB-card>

# default (mmap on):
curl -s localhost:11434/api/generate -d '{"model":"hf.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL","prompt":"hi","stream":false,"options":{"num_predict":1,"num_ctx":8192}}' >/dev/null
ollama ps          # SIZE ~28 GB, "50%/50% CPU/GPU"

# mmap off:
curl -s localhost:11434/api/generate -d '{"model":"hf.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL","prompt":"hi","stream":false,"options":{"num_predict":1,"num_ctx":8192,"use_mmap":false}}' >/dev/null
ollama ps          # SIZE ~15 GB, "94% GPU"

With OLLAMA_DEBUG=1 the server logs runner.size / runner.vram for each load.

Relevant log output

# use_mmap=true
runner.size="26.7 GiB"  runner.vram="13.4 GiB"
ollama ps: ... 28 GB  50%/50% CPU/GPU  ...

# use_mmap=false
runner.size="14.2 GiB"  runner.vram="13.4 GiB"
ollama ps: ... 15 GB   6%/94% CPU/GPU  ...

# model metadata
n_expert = 128, n_expert_used = 8
print_info: file size = 13.26 GiB (4.51 BPW)

OS

Linux (WSL2, Ubuntu)

GPU

NVIDIA RTX 5070 Ti 16 GB (also present: RTX 4070 Ti 12 GB; the bug is shown when Ollama is pinned to the single 16 GB card)

CPU

x86_64

Ollama version

Reproduced on 0.30.4 and 0.30.7 (identical behavior — not version-specific)