Issue Station
Sign In
Tools hereToolbox

ollama - 💡(How to fix) Fix 'ollama serve' should report what GPUs if any it found [6 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
ollama/ollama#15148•Fetched 2026-04-08 01:53:01
View on GitHub
Comments
6
Participants
2
Timeline
8
Reactions
0
Author
yurivictyurivict
Participants
rick-githubrick-githubyurivictyurivict
Timeline (top)
commented ×6closed ×1labeled ×1

Code Example

$ grep GPU ollama-serve.log
time=2026-03-30T12:32:42.143-07:00 level=INFO source=routes.go:1742 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:65536 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/yuri/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:true ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-30T12:32:42.186-07:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-30T12:32:42.201-07:00 level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=14.817587ms
time=2026-03-30T12:32:44.599-07:00 level=INFO source=runner.go:895 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Auto KvSize:16384 KvCacheType: NumThreads:16 GPULayers:[] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
RAW_BUFFERClick to expand / collapse

'ollama serve' doesn't report the list of GPUs itis going to use:

$ grep GPU ollama-serve.log
time=2026-03-30T12:32:42.143-07:00 level=INFO source=routes.go:1742 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:65536 OLLAMA_DEBUG:DEBUG OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/yuri/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:true ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-03-30T12:32:42.186-07:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-30T12:32:42.201-07:00 level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=14.817587ms
time=2026-03-30T12:32:44.599-07:00 level=INFO source=runner.go:895 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Auto KvSize:16384 KvCacheType: NumThreads:16 GPULayers:[] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"

It says "discovering available GPUs..." but it never says: "discovered no GPUs", or "discovered these GPUs: Vulkan/GPU0 (NVIDIA ....)"

It should definitively say in human readable English what GPUs if any it is going to use.

extent analysis

Fix Plan

To fix the issue of 'ollama serve' not reporting the list of GPUs it is going to use, we need to modify the logging in the runner.go file.

Here are the steps:

  • Modify the msg in the logger.Info call in runner.go:67 to include the list of discovered GPUs.
  • Add a new log statement after the GPU discovery to print the list of GPUs.

Example code:

// runner.go
func discoverGPUs() {
    // ... existing code ...
    log.Info("discovering available GPUs...")
    gpus, err := getAvailableGPUs()
    if err != nil {
        log.Error("error discovering GPUs: %v", err)
        return
    }
    log.Info("discovered %d GPUs: %v", len(gpus), gpus)
    // ... existing code ...
}

func getAvailableGPUs() ([]string, error) {
    // ... existing code to get available GPUs ...
    // Return a list of GPU names, e.g. "Vulkan/GPU0 (NVIDIA ...)"
}

In the getAvailableGPUs function, you should return a list of GPU names in a human-readable format.

Verification

To verify that the fix worked, run ollama serve and check the logs for the message "discovered X GPUs: ..." where X is the number of GPUs discovered.

Extra Tips

Make sure to handle the case where no GPUs are discovered and log a corresponding message. Additionally, consider adding more detailed logging for GPU discovery errors to aid in debugging.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#prompt template#agent execution#callback error#memory management#API rate limit

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING