ollama - 💡(How to fix) Fix ollama docker loses access to nvidia gpu [3 comments, 2 participants]

ollama2026-03-06 04:52:52

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

ollama/ollama#14658•Fetched 2026-04-08 00:33:14

View on GitHub

Comments

Participants

Timeline

Reactions

Author

ACodingfreak

Participants

ACodingfreak

rick-github

Timeline (top)

commented ×3closed ×1labeled ×1mentioned ×1

Error Message

Failed to initialize NVML: Unknown Error

Code Example

$ sudo docker exec -it ollama nvidia-smi
Failed to initialize NVML: Unknown Error

$ nvidia-smi
Thu Mar  5 20:45:27 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.288.01             Driver Version: 535.288.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A30                     Off | 00000000:9B:00.0 Off |                    0 |
| N/A   33C    P0              28W / 165W |      0MiB / 24576MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

$ sudo docker exec -it ollama ollama --version
ollama version is 0.15.4

$ sudo docker stop ollama
ollama
$ sudo docker rm ollama
ollama
$sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
0815be239db8b142db97f6d67571db8f8af99156162777a141c76c226b5d04c7
$ sudo docker exec -it ollama nvidia-smi
Fri Mar  6 04:54:05 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.288.01             Driver Version: 535.288.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A30                     Off | 00000000:9B:00.0 Off |                    0 |
| N/A   33C    P0              28W / 165W |      0MiB / 24576MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

RAW_BUFFERClick to expand / collapse

What is the issue?

As shown in below logs, after sometime ollama running as a docker container is not able to access the underlying NVIDIA GPU as shown below. Eventhough I am able to access the same GPUs by running nvidia-smi from the host machine. If I stop and restart the same ollama container everything work fine as expected.

Relevant log output

$ sudo docker exec -it ollama nvidia-smi
Failed to initialize NVML: Unknown Error

$ nvidia-smi
Thu Mar  5 20:45:27 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.288.01             Driver Version: 535.288.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A30                     Off | 00000000:9B:00.0 Off |                    0 |
| N/A   33C    P0              28W / 165W |      0MiB / 24576MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

$ sudo docker exec -it ollama ollama --version
ollama version is 0.15.4

$ sudo docker stop ollama
ollama
$ sudo docker rm ollama
ollama
$sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
0815be239db8b142db97f6d67571db8f8af99156162777a141c76c226b5d04c7
$ sudo docker exec -it ollama nvidia-smi
Fri Mar  6 04:54:05 2026
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.288.01             Driver Version: 535.288.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A30                     Off | 00000000:9B:00.0 Off |                    0 |
| N/A   33C    P0              28W / 165W |      0MiB / 24576MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

OS

Docker

GPU

Nvidia

CPU

No response

Ollama version

0.15.4

extent analysis

Fix Plan

The issue seems to be related to the NVIDIA driver or CUDA version compatibility with the ollama container. To fix this, we can try the following steps:

Update the NVIDIA driver to the latest version.
Ensure that the CUDA version is compatible with the ollama version.
Use the --gpus=all flag when running the ollama container to enable GPU access.
Use the --privileged flag to give the container privileged access to the host system.

Here are the concrete steps:

Update NVIDIA driver:

sudo apt update sudo apt install nvidia-driver-535

2. **Verify CUDA version**:
   ```bash
nvidia-smi

Ensure that the CUDA version is compatible with ollama version 0.15.4. 3. Run ollama container with GPU access:

sudo docker run -d --gpus=all --privileged -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Verify ollama container:

sudo docker exec -it ollama nvidia-smi

   This should show the NVIDIA GPU information and verify that the ollama container has access to the GPU.

### Verification
To verify that the fix worked, run the following command:
```bash
sudo docker exec -it ollama ollama --version

This should show the ollama version and verify that the container is running correctly.

Extra Tips

Ensure that the ollama version is compatible with the NVIDIA driver and CUDA version.
Use the --privileged flag with caution, as it gives the container privileged access to the host system.
Consider using a Docker Compose file to manage the ollama container and its dependencies.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #API routing #API middleware #SSR setup #ISR setup #authentication setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

ollama - 💡(How to fix) Fix ollama docker loses access to nvidia gpu [3 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Code Example

What is the issue?

Relevant log output

OS

GPU

CPU

Ollama version

extent analysis

Fix Plan

Extra Tips

Still need to ship something?

TRENDING

ollama - 💡(How to fix) Fix ollama docker loses access to nvidia gpu [3 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Code Example

What is the issue?

Relevant log output

OS

GPU

CPU

Ollama version

extent analysis

Fix Plan

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING