ollama - 💡(How to fix) Fix Not able to run ollama on top of RTX PRO 4500 Server Edition + GPU mig enabled + driver 590.48.01 + Cuda13.1 [7 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
ollama/ollama#15178Fetched 2026-04-08 01:58:16
View on GitHub
Comments
7
Participants
2
Timeline
8
Reactions
0
Author
Participants
Timeline (top)
commented ×7labeled ×1

Error Message

the error logs: error logs.

Code Example

[root@nvidia-driver-daemonset-9 drivers]# nvidia-smi
Tue Mar 31 16:50:24 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01 Driver Version: 590.48.01 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX PRO 4500 Blac... On | 00000000:0B:00.0 Off | On |
| N/A 29C P8 17W / 165W | 116MiB / 32623MiB | N/A Default |
| | | Enabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+----------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Shared Memory-Usage | Vol| Shared |
| ID ID Dev | Shared BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+==================================+===========+=======================|
| 0 1 0 0 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+
| 0 2 0 1 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+

---

root@ollama-0:/# root@ollama-0:/# nvidia-smi
Tue Mar 31 16:52:04 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01 Driver Version: 590.48.01 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX PRO 4500 Blac... On | 00000000:0B:00.0 Off | On |
| N/A 29C P8 17W / 165W | N/A | N/A Default |
| | | Enabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+----------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Shared Memory-Usage | Vol| Shared |
| ID ID Dev | Shared BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+==================================+===========+=======================|
| 0 1 0 0 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+

---

[core@quanta-tmo-4500-cran mig]$ cat ollama-deployment.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: ollama
namespace: open-webui
spec:
serviceName: "ollama"
selector:
matchLabels:
app: ollama
template:
metadata:
labels:
app: ollama
spec:
containers:

name: ollama
image: docker.io/ollama/ollama:latest
imagePullPolicy: IfNotPresent
env:
- name: OLLAMA_DEBUG
value: "1"
ports:
containerPort: 11434
resources:
requests:
nvidia.com/gpu: "1"
limits:
nvidia.com/gpu: "1"

---

[core@XXXX]$ oc describe node |grep -i "nvidia.com/gpu:" -B 8
Capacity:
cpu: 128
ephemeral-storage: 1873933640Ki
hugepages-1Gi: 50Gi
hugepages-2Mi: 0
memory: 263630500Ki
nvidia.com/gpu: 2

---

[core@xxxx]$ oc get pods -n open-webui
NAME READY STATUS RESTARTS AGE
ollama-0 1/1 Running 0 61s

---

[core@xxxxx]$ oc logs ollama-0 -n open-webui
Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
Your new public key is:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDfkDTv/WVAHAEBTB3xxg80OJRJCi/IsOMNJUfVSyn+f

time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1742 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1744 msg="Ollama cloud disabled: false"
time=2026-03-31T16:54:49.030Z level=INFO source=images.go:477 msg="total blobs: 0"
time=2026-03-31T16:54:49.030Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1800 msg="Listening on [::]:11434 (version 0.19.0)"
time=2026-03-31T16:54:49.030Z level=DEBUG source=sched.go:145 msg="starting llm scheduler"
time=2026-03-31T16:54:49.031Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-31T16:54:49.033Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 42887"
time=2026-03-31T16:54:49.033Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12
time=2026-03-31T16:54:49.413Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=382.320152ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v12]" extra_envs=map[]
time=2026-03-31T16:54:49.414Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 38815"
time=2026-03-31T16:54:49.414Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=245.184387ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v13]" extra_envs=map[]
time=2026-03-31T16:54:49.659Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled. To enable, set OLLAMA_VULKAN=1"
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:124 msg="evaluating which, if any, devices to filter out" initial_count=2
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:146 msg="verifying if device is supported" library=/usr/lib/ollama/cuda_v12 description="NVIDIA RTX PRO 4500 Blackwell Server Edition MIG 1g.16gb" compute=12.0 id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 pci_id=0000:0b:00.0
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:146 msg="verifying if device is supported" library=/usr/lib/ollama/cuda_v13 description="NVIDIA RTX PRO 4500 Blackwell Server Edition MIG 1g.16gb" compute=12.0 id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 pci_id=0000:0b:00.0
time=2026-03-31T16:54:49.659Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34101"
time=2026-03-31T16:54:49.659Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12 GGML_CUDA_INIT=1 CUDA_VISIBLE_DEVICES=GPU-01991e90-02de-d9b8-d852-cd5265aefb02
time=2026-03-31T16:54:49.659Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34747"
time=2026-03-31T16:54:49.659Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13 GGML_CUDA_INIT=1 CUDA_VISIBLE_DEVICES=GPU-01991e90-02de-d9b8-d852-cd5265aefb02
time=2026-03-31T16:54:49.758Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=99.18846ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v12]" extra_envs="map[CUDA_VISIBLE_DEVICES:GPU-01991e90-02de-d9b8-d852-cd5265aefb02 GGML_CUDA_INIT:1]"
time=2026-03-31T16:54:49.758Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 libdir=/usr/lib/ollama/cuda_v12 pci_id=0000:0b:00.0 library=CUDA
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=105.845603ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v13]" extra_envs="map[CUDA_VISIBLE_DEVICES:GPU-01991e90-02de-d9b8-d852-cd5265aefb02 GGML_CUDA_INIT:1]"
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 libdir=/usr/lib/ollama/cuda_v13 pci_id=0000:0b:00.0 library=CUDA
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=733.993471ms
time=2026-03-31T16:54:49.765Z level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="251.4 GiB" available="251.3 GiB"
time=2026-03-31T16:54:49.765Z level=INFO source=routes.go:1850 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

---
RAW_BUFFERClick to expand / collapse

What is the issue?

Not able to run ollama on top of RTX PRO 4500 Server Edition + GPU mig enabled + driver 590.48.01 + Cuda13.1

the error logs:

In host server side

[root@nvidia-driver-daemonset-9 drivers]# nvidia-smi
Tue Mar 31 16:50:24 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01 Driver Version: 590.48.01 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX PRO 4500 Blac... On | 00000000:0B:00.0 Off | On |
| N/A 29C P8 17W / 165W | 116MiB / 32623MiB | N/A Default |
| | | Enabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+----------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Shared Memory-Usage | Vol| Shared |
| ID ID Dev | Shared BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+==================================+===========+=======================|
| 0 1 0 0 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+
| 0 2 0 1 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+

in the ollama pod

root@ollama-0:/# root@ollama-0:/# nvidia-smi
Tue Mar 31 16:52:04 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01 Driver Version: 590.48.01 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX PRO 4500 Blac... On | 00000000:0B:00.0 Off | On |
| N/A 29C P8 17W / 165W | N/A | N/A Default |
| | | Enabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| MIG devices: |
+------------------+----------------------------------+-----------+-----------------------+
| GPU GI CI MIG | Shared Memory-Usage | Vol| Shared |
| ID ID Dev | Shared BAR1-Usage | SM Unc| CE ENC DEC OFA JPG |
| | | ECC| |
|==================+==================================+===========+=======================|
| 0 1 0 0 | 58MiB / 16032MiB | 42 0 | 1 1 1 0 1 |
| | 0MiB / 16383MiB | | |
+------------------+----------------------------------+-----------+-----------------------+

pod spec to deploy ollama

[core@quanta-tmo-4500-cran mig]$ cat ollama-deployment.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: ollama
namespace: open-webui
spec:
serviceName: "ollama"
selector:
matchLabels:
app: ollama
template:
metadata:
labels:
app: ollama
spec:
containers:

name: ollama
image: docker.io/ollama/ollama:latest
imagePullPolicy: IfNotPresent
env:
- name: OLLAMA_DEBUG
value: "1"
ports:
containerPort: 11434
resources:
requests:
nvidia.com/gpu: "1"
limits:
nvidia.com/gpu: "1"

the GPU resource pools in host side (MIG policy is "single")

[core@XXXX]$ oc describe node |grep -i "nvidia.com/gpu:" -B 8
Capacity:
cpu: 128
ephemeral-storage: 1873933640Ki
hugepages-1Gi: 50Gi
hugepages-2Mi: 0
memory: 263630500Ki
nvidia.com/gpu: 2

pod is running

[core@xxxx]$ oc get pods -n open-webui
NAME READY STATUS RESTARTS AGE
ollama-0 1/1 Running 0 61s

error logs.

[core@xxxxx]$ oc logs ollama-0 -n open-webui
Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
Your new public key is:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDfkDTv/WVAHAEBTB3xxg80OJRJCi/IsOMNJUfVSyn+f

time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1742 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1744 msg="Ollama cloud disabled: false"
time=2026-03-31T16:54:49.030Z level=INFO source=images.go:477 msg="total blobs: 0"
time=2026-03-31T16:54:49.030Z level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-03-31T16:54:49.030Z level=INFO source=routes.go:1800 msg="Listening on [::]:11434 (version 0.19.0)"
time=2026-03-31T16:54:49.030Z level=DEBUG source=sched.go:145 msg="starting llm scheduler"
time=2026-03-31T16:54:49.031Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-31T16:54:49.033Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 42887"
time=2026-03-31T16:54:49.033Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12
time=2026-03-31T16:54:49.413Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=382.320152ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v12]" extra_envs=map[]
time=2026-03-31T16:54:49.414Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 38815"
time=2026-03-31T16:54:49.414Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=245.184387ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v13]" extra_envs=map[]
time=2026-03-31T16:54:49.659Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled. To enable, set OLLAMA_VULKAN=1"
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:124 msg="evaluating which, if any, devices to filter out" initial_count=2
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:146 msg="verifying if device is supported" library=/usr/lib/ollama/cuda_v12 description="NVIDIA RTX PRO 4500 Blackwell Server Edition MIG 1g.16gb" compute=12.0 id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 pci_id=0000:0b:00.0
time=2026-03-31T16:54:49.659Z level=DEBUG source=runner.go:146 msg="verifying if device is supported" library=/usr/lib/ollama/cuda_v13 description="NVIDIA RTX PRO 4500 Blackwell Server Edition MIG 1g.16gb" compute=12.0 id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 pci_id=0000:0b:00.0
time=2026-03-31T16:54:49.659Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34101"
time=2026-03-31T16:54:49.659Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v12 GGML_CUDA_INIT=1 CUDA_VISIBLE_DEVICES=GPU-01991e90-02de-d9b8-d852-cd5265aefb02
time=2026-03-31T16:54:49.659Z level=INFO source=server.go:432 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34747"
time=2026-03-31T16:54:49.659Z level=DEBUG source=server.go:433 msg=subprocess PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin OLLAMA_DEBUG=1 LD_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_LIBRARY_PATH=/usr/lib/ollama:/usr/lib/ollama/cuda_v13 GGML_CUDA_INIT=1 CUDA_VISIBLE_DEVICES=GPU-01991e90-02de-d9b8-d852-cd5265aefb02
time=2026-03-31T16:54:49.758Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=99.18846ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v12]" extra_envs="map[CUDA_VISIBLE_DEVICES:GPU-01991e90-02de-d9b8-d852-cd5265aefb02 GGML_CUDA_INIT:1]"
time=2026-03-31T16:54:49.758Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 libdir=/usr/lib/ollama/cuda_v12 pci_id=0000:0b:00.0 library=CUDA
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=105.845603ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/cuda_v13]" extra_envs="map[CUDA_VISIBLE_DEVICES:GPU-01991e90-02de-d9b8-d852-cd5265aefb02 GGML_CUDA_INIT:1]"
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=GPU-01991e90-02de-d9b8-d852-cd5265aefb02 libdir=/usr/lib/ollama/cuda_v13 pci_id=0000:0b:00.0 library=CUDA
time=2026-03-31T16:54:49.765Z level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=733.993471ms
time=2026-03-31T16:54:49.765Z level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="251.4 GiB" available="251.3 GiB"
time=2026-03-31T16:54:49.765Z level=INFO source=routes.go:1850 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

Relevant log output

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

extent analysis

TL;DR

The issue is likely due to the ollama container not being able to utilize the GPU resources properly, and a potential fix involves setting the CUDA_VISIBLE_DEVICES environment variable to the correct GPU device ID.

Guidance

  • Verify that the nvidia.com/gpu resource is properly allocated to the ollama pod by checking the pod's configuration and the node's capacity.
  • Check the ollama container's logs to see if there are any error messages related to GPU initialization or usage.
  • Try setting the CUDA_VISIBLE_DEVICES environment variable to the correct GPU device ID (e.g., GPU-01991e90-02de-d9b8-d852-cd5265aefb02) in the ollama container.
  • Ensure that the ollama image is compatible with the CUDA version (13.1) and the NVIDIA driver version (590.48.01) installed on the host machine.

Example

No code example is provided as the issue seems to be related to configuration and environment variables rather than code.

Notes

The provided logs suggest that the ollama container is trying to use the GPU, but there might be an issue with the GPU initialization or the environment variables. The CUDA_VISIBLE_DEVICES environment variable is set to GPU-01991e90-02de-d9b8-d852-cd5265aefb02, but it's not clear if this is the correct device ID.

Recommendation

Apply a workaround by setting the CUDA_VISIBLE_DEVICES environment variable to the correct GPU device ID in the ollama container, and verify that the ollama image is compatible with the CUDA version and NVIDIA driver version installed on the host machine.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

ollama - 💡(How to fix) Fix Not able to run ollama on top of RTX PRO 4500 Server Edition + GPU mig enabled + driver 590.48.01 + Cuda13.1 [7 comments, 2 participants]