print_info: ssm_dt_b_c_rms = 0 print_info: model type = 1B print_info: model params = 6.94 B print_info: general.name = Granite 4.0 H Tiny print_info: f_embedding_scale = 12.000000 print_info: f_residual_scale = 0.220000 print_info: f_attention_scale = 0.007813 print_info: n_ff_shexp = 1024 print_info: vocab type = BPE print_info: n_vocab = 100352 print_info: n_merges = 100000 print_info: BOS token = 100257 '<|end_of_text|>' print_info: EOS token = 100257 '<|end_of_text|>' print_info: EOT token = 100257 '<|end_of_text|>' print_info: UNK token = 100269 '<|unk|>' print_info: PAD token = 100256 '<|pad|>' print_info: LF token = 198 'Ċ' print_info: FIM PRE token = 100258 '<|fim_prefix|>' print_info: FIM SUF token = 100260 '<|fim_suffix|>' print_info: FIM MID token = 100259 '<|fim_middle|>' print_info: FIM PAD token = 100261 '<|fim_pad|>' print_info: EOG token = 100257 '<|end_of_text|>' print_info: EOG token = 100261 '<|fim_pad|>' print_info: max token length = 256 load_tensors: loading model tensors, this can take a while... (mmap = false) load_tensors: offloading 10 repeating layers to GPU load_tensors: offloaded 10/41 layers to GPU load_tensors: CPU model buffer size = 120.59 MiB load_tensors: CUDA0 model buffer size = 1003.36 MiB load_tensors: CUDA_Host model buffer size = 3028.19 MiB time=2026-03-04T09:37:10.728+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server not responding" ggml_cuda_host_malloc: failed to allocate 1.00 MiB of pinned memory: out of memory CUDA error: out of memory current device: 0, in function ggml_backend_cuda_device_event_new at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:4957 cudaEventCreateWithFlags(&event, 0x02) C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:94: CUDA error time=2026-03-04T09:37:23.287+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-03-04T09:37:24.385+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server not responding" time=2026-03-04T09:37:26.735+01:00 level=ERROR source=server.go:303 msg="llama runner terminated" error="exit status 1" time=2026-03-04T09:37:26.902+01:00 level=INFO source=sched.go:518 msg="Load failed" model=C:\Users\user.ollama\models\blobs\sha256-491ba81786c46a345a5da9a60cdb9f9a3056960c8411dd857153c194b1f91313 error="llama runner process has terminated: CUDA error" [GIN] 2026/03/04 - 09:37:27 | 500 | 29.7846277s | 127.0.0.1 | POST "/api/chat"

Fix Plan

The issue is caused by a CUDA out-of-memory error. To fix this, we need to reduce the memory usage of the model or increase the available GPU memory.

Step 1: Reduce Model Memory Usage

We can try to reduce the model size or use a more efficient model architecture. However, since the model is already loaded, we can try to reduce the batch size or sequence length to reduce memory usage.

Step 2: Increase Available GPU Memory

We can try to close other GPU-intensive applications or increase the GPU memory allocation for the Ollama process.

Step 3: Modify Ollama Configuration

We can modify the Ollama configuration to use a smaller model or reduce the GPU memory allocation. We can add the following configuration options to the ollama.yaml file:

model:
  size: small
gpu:
  memory_allocation: 2048

Alternatively, we can use the --gpu-memory flag when running the Ollama command:

ollama run --gpu-memory 2048 command-r7b:latest "Summarize this text in 3 bullet points: ..."

Step 4: Update Ollama Version

If the issue persists, we can try updating Ollama to the latest version:

ollama update

Verification

To verify that the fix worked, we can run the Ollama command again and check for any errors:

ollama run command-r7b:latest "Summarize this text in 3 bullet points: ..."

If the command runs successfully without any errors, the fix has worked.

Extra Tips

• Make sure to close other GPU-intensive applications before running Ollama.
• Consider using a more efficient model architecture or reducing the model size to reduce memory usage.
• If the issue persists, try increasing the GPU memory allocation or using a different GPU device.