ollama - 💡(How to fix) Fix CUDA out of memory error when using Qwen3.5:27B after external application (e.g., Mattermost plugin) invokes the model [7 comments, 2 participants]

ollama2026-03-10 01:40:55

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

ollama/ollama#14751•Fetched 2026-04-08 00:32:06

View on GitHub

Comments

Participants

Timeline

Reactions

Author

cedarice

Participants

cedarice

rick-github

Timeline (top)

commented ×7closed ×1labeled ×1

When invoking the Qwen3.5 series models (specifically qwen3.5:27b) via Ollama, an intermittent CUDA out-of-memory error occurs. The issue appears to be triggered after an external application (such as the Mattermost Agent Plugin) calls the model. Subsequent manual inference attempts fail with a CUDA OOM error, even though ollama ps shows the model as actively loaded.

Error Message

Thinking... OkayError: an error was encountered while running the model: CUDA error: out of memory current device: 0, in function evaluate_and_capture_cuda_graph at //ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:3831 cudaGraphInstantiate(&cuda_ctx->cuda_graph->instance, cuda_ctx->cuda_graph->graph, __null, __null, 0) //ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:94: CUDA error

Root Cause

Code Example

ollama run qwen3.5:27b

---

Thinking...
OkayError: an error was encountered while running the model: CUDA error: out of memory
  current device: 0, in function evaluate_and_capture_cuda_graph at //ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:3831
  cudaGraphInstantiate(&cuda_ctx->cuda_graph->instance, cuda_ctx->cuda_graph->graph, __null, __null, 0)
//ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:94: CUDA error

---

3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 111 gp=0xc00053a8c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e984a?, 0x3?, 0x63?, 0x4?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000534f38 sp=0xc000534f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000534fc8 sp=0xc000534f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000534fe0 sp=0xc000534fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000534fe8 sp=0xc000534fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 112 gp=0xc00053aa80 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xd4?, 0x8?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000535738 sp=0xc000535718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005357c8 sp=0xc000535738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005357e0 sp=0xc0005357c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005357e8 sp=0xc0005357e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 113 gp=0xc00053ac40 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a5d2a0f4?, 0x3?, 0xc3?, 0x4b?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000535f38 sp=0xc000535f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000535fc8 sp=0xc000535f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000535fe0 sp=0xc000535fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000535fe8 sp=0xc000535fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 114 gp=0xc00053ae00 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e93e8?, 0x1?, 0x5e?, 0xcf?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000540738 sp=0xc000540718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005407c8 sp=0xc000540738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005407e0 sp=0xc0005407c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005407e8 sp=0xc0005407e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 115 gp=0xc00053afc0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e5f9b?, 0x3?, 0x4d?, 0xa2?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000540f38 sp=0xc000540f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000540fc8 sp=0xc000540f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000540fe0 sp=0xc000540fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000540fe8 sp=0xc000540fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 116 gp=0xc00053b180 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e3b7e?, 0x1?, 0x11?, 0x92?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000541738 sp=0xc000541718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005417c8 sp=0xc000541738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005417e0 sp=0xc0005417c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005417e8 sp=0xc0005417e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 117 gp=0xc00053b340 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57d0855?, 0x1?, 0x6c?, 0x8d?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000541f38 sp=0xc000541f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000541fc8 sp=0xc000541f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000541fe0 sp=0xc000541fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000541fe8 sp=0xc000541fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 118 gp=0xc00053b500 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0x26?, 0x79?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000542738 sp=0xc000542718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005427c8 sp=0xc000542738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005427e0 sp=0xc0005427c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005427e8 sp=0xc0005427e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 119 gp=0xc00053b6c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57f16b8?, 0x1?, 0xc0?, 0x72?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000542f38 sp=0xc000542f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000542fc8 sp=0xc000542f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000542fe0 sp=0xc000542fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000542fe8 sp=0xc000542fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 120 gp=0xc00053b880 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a5ddaecf?, 0x1?, 0xed?, 0xb4?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000543738 sp=0xc000543718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005437c8 sp=0xc000543738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005437e0 sp=0xc0005437c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005437e8 sp=0xc0005437e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 121 gp=0xc00053ba40 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x3?, 0x19?, 0x5?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000543f38 sp=0xc000543f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000543fc8 sp=0xc000543f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000543fe0 sp=0xc000543fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000543fe8 sp=0xc000543fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 122 gp=0xc00053bc00 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xa1?, 0x20?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053c738 sp=0xc00053c718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053c7c8 sp=0xc00053c738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053c7e0 sp=0xc00053c7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053c7e8 sp=0xc00053c7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 123 gp=0xc00053bdc0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57dad07?, 0x1?, 0xf?, 0x4b?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053cf38 sp=0xc00053cf18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053cfc8 sp=0xc00053cf38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053cfe0 sp=0xc00053cfc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053cfe8 sp=0xc00053cfe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 124 gp=0xc000544000 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xf6?, 0x8c?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053d738 sp=0xc00053d718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053d7c8 sp=0xc00053d738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053d7e0 sp=0xc00053d7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053d7e8 sp=0xc00053d7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 125 gp=0xc0005441c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57effab?, 0x1?, 0xe0?, 0xdd?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053df38 sp=0xc00053df18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053dfc8 sp=0xc00053df38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053dfe0 sp=0xc00053dfc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053dfe8 sp=0xc00053dfe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 126 gp=0xc000544380 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e5ff6?, 0x1?, 0x26?, 0x27?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053e738 sp=0xc00053e718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053e7c8 sp=0xc00053e738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053e7e0 sp=0xc00053e7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053e7e8 sp=0xc00053e7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 20 gp=0xc00027b880 m=nil [chan receive]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x30?, 0x564acc813260?, 0x1?, 0x0?, 0xc0008bf798?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc0008bf750 sp=0xc0008bf730 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv(0xc000744310, 0x0, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:664 +0x445 fp=0xc0008bf7c8 sp=0xc0008bf750 pc=0x564acafd6085
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv1(0x564acc333bbb?, 0x29?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:506 +0x12 fp=0xc0008bf7f0 sp=0xc0008bf7c8 pc=0x564acafd5c12
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).forwardBatch(_, {0x2, {0x564acc8e4130, 0xc000730300}, {0x564acc8f1570, 0xc0006d5db8}, {0xc000b1c148, 0x1, 0x1}, {{0x564acc8f1570, ...}, ...}, ...})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:476 +0xfa fp=0xc0008bfb58 sp=0xc0008bf7f0 pc=0x564acb65a7ba
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0002f90e0, {0x564acc8d7bd0, 0xc0001c3d60})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:453 +0x18c fp=0xc0008bffb8 sp=0xc0008bfb58 pc=0x564acb65a46c
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:1442 +0x28 fp=0xc0008bffe0 sp=0xc0008bffb8 pc=0x564acb663f68
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0008bffe8 sp=0xc0008bffe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:1442 +0x4c9
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 21 gp=0xc00027ba40 m=nil [select]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0xc0000459e8?, 0x2?, 0x4?, 0x0?, 0xc000045810?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000045620 sp=0xc000045600 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.selectgo(0xc0000459e8, 0xc00004580c, 0xc00088c000?, 0x0, 0x1?, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/select.go:351 +0x837 fp=0xc000045758 sp=0xc000045620 pc=0x564acb018c17
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).completion(0xc0002f90e0, {0x564acc8d5440, 0xc00079a0e0}, 0xc000476280)
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:974 +0xc86 fp=0xc000045ac0 sp=0xc000045758 pc=0x564acb65ee26
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).completion-fm({0x564acc8d5440?, 0xc00079a0e0?}, 0xc0008bfb40?)
3月 10 09:33:22 jzswai ollama[3872788]:         <autogenerated>:1 +0x36 fp=0xc000045af0 sp=0xc000045ac0 pc=0x564acb664456
3月 10 09:33:22 jzswai ollama[3872788]: net/http.HandlerFunc.ServeHTTP(0xc0005ba780?, {0x564acc8d5440?, 0xc00079a0e0?}, 0xc0008bfb60?)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2294 +0x29 fp=0xc000045b18 sp=0xc000045af0 pc=0x564acb33a8c9
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*ServeMux).ServeHTTP(0x564acafdeac5?, {0x564acc8d5440, 0xc00079a0e0}, 0xc000476280)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2822 +0x1c4 fp=0xc000045b68 sp=0xc000045b18 pc=0x564acb33c7c4
3月 10 09:33:22 jzswai ollama[3872788]: net/http.serverHandler.ServeHTTP({0x564acc8d1730?}, {0x564acc8d5440?, 0xc00079a0e0?}, 0x1?)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3301 +0x8e fp=0xc000045b98 sp=0xc000045b68 pc=0x564acb35a24e
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*conn).serve(0xc0001ae750, {0x564acc8d7b98, 0xc0005df800})
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2102 +0x625 fp=0xc000045fb8 sp=0xc000045b98 pc=0x564acb338dc5
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*Server).Serve.gowrap3()
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3454 +0x28 fp=0xc000045fe0 sp=0xc000045fb8 pc=0x564acb33e688
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000045fe8 sp=0xc000045fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by net/http.(*Server).Serve in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3454 +0x485
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 1758 gp=0xc000583dc0 m=nil [IO wait]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0xc009445e00?, 0xc009445e50?, 0x0?, 0x80?, 0xb?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc0006f45d8 sp=0xc0006f45b8 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.netpollblock(0x564acb05d798?, 0xcafd34a6?, 0x4a?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/netpoll.go:575 +0xf7 fp=0xc0006f4610 sp=0xc0006f45d8 pc=0x564acafff097
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.runtime_pollWait(0x7f9cb20525b8, 0x72)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/netpoll.go:351 +0x85 fp=0xc0006f4630 sp=0xc0006f4610 pc=0x564acb039105
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*pollDesc).wait(0xc0005c1500?, 0xc0005df901?, 0x0)
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006f4658 sp=0xc0006f4630 pc=0x564acb0c1487
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*pollDesc).waitRead(...)
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_poll_runtime.go:89
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*FD).Read(0xc0005c1500, {0xc0005df901, 0x1, 0x1})
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_unix.go:165 +0x27a fp=0xc0006f46f0 sp=0xc0006f4658 pc=0x564acb0c277a
3月 10 09:33:22 jzswai ollama[3872788]: net.(*netFD).Read(0xc0005c1500, {0xc0005df901?, 0xc0007141d8?, 0xc0006f4770?})
3月 10 09:33:22 jzswai ollama[3872788]:         net/fd_posix.go:55 +0x25 fp=0xc0006f4738 sp=0xc0006f46f0 pc=0x564acb137da5
3月 10 09:33:22 jzswai ollama[3872788]: net.(*conn).Read(0xc00013ab00, {0xc0005df901?, 0xc009449c70?, 0xc009449cc0?})
3月 10 09:33:22 jzswai ollama[3872788]:         net/net.go:194 +0x45 fp=0xc0006f4780 sp=0xc0006f4738 pc=0x564acb146165
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*connReader).backgroundRead(0xc0005df8f0)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:690 +0x37 fp=0xc0006f47c8 sp=0xc0006f4780 pc=0x564acb332c97
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*connReader).startBackgroundRead.gowrap2()
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:686 +0x25 fp=0xc0006f47e0 sp=0xc0006f47c8 pc=0x564acb332bc5
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0006f47e8 sp=0xc0006f47e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by net/http.(*connReader).startBackgroundRead in goroutine 21
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:686 +0xb6
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 1439 gp=0xc0007b6e00 m=nil [chan receive]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x30?, 0x564acc813260?, 0x1?, 0xba?, 0xc000b26b18?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000b26ad0 sp=0xc000b26ab0 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv(0xc0007442a0, 0x0, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:664 +0x445 fp=0xc000b26b48 sp=0xc000b26ad0 pc=0x564acafd6085
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv1(0x564acc3378ec?, 0x2c?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:506 +0x12 fp=0xc000b26b70 sp=0xc000b26b48 pc=0x564acafd5c12
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).computeBatch(0xc0002f90e0, {0x2, {0x564acc8e4130, 0xc000730300}, {0x564acc8f1570, 0xc0006d5db8}, {0xc000b1c148, 0x1, 0x1}, {{0x564acc8f1570, ...}, ...}, ...})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:646 +0x185 fp=0xc000b26ef0 sp=0xc000b26b70 pc=0x564acb65c2c5
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:459 +0x58 fp=0xc000b26fe0 sp=0xc000b26ef0 pc=0x564acb65a698
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000b26fe8 sp=0xc000b26fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by github.com/ollama/ollama/runner/ollamarunner.(*Server).run in goroutine 20
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:459 +0x2cd
3月 10 09:33:22 jzswai ollama[3872788]: rax    0x0
3月 10 09:33:22 jzswai ollama[3872788]: rbx    0x7f9ad1fff640
3月 10 09:33:22 jzswai ollama[3872788]: rcx    0x7f9cb1e969fc
3月 10 09:33:22 jzswai ollama[3872788]: rdx    0x6
3月 10 09:33:22 jzswai ollama[3872788]: rdi    0x3cf144
3月 10 09:33:22 jzswai ollama[3872788]: rsi    0x3cf1a2
3月 10 09:33:22 jzswai ollama[3872788]: rbp    0x3cf1a2
3月 10 09:33:22 jzswai ollama[3872788]: rsp    0x7f9ad1ffdd20
3月 10 09:33:22 jzswai ollama[3872788]: r8     0x7f9ad1ffddf0
3月 10 09:33:22 jzswai ollama[3872788]: r9     0x7f94e9ba9e90
3月 10 09:33:22 jzswai ollama[3872788]: r10    0x8
3月 10 09:33:22 jzswai ollama[3872788]: r11    0x246
3月 10 09:33:22 jzswai ollama[3872788]: r12    0x6
3月 10 09:33:22 jzswai ollama[3872788]: r13    0x16
3月 10 09:33:22 jzswai ollama[3872788]: r14    0x7f9c5029242e
3月 10 09:33:22 jzswai ollama[3872788]: r15    0x7f9ac90d5508
3月 10 09:33:22 jzswai ollama[3872788]: rip    0x7f9cb1e969fc
3月 10 09:33:22 jzswai ollama[3872788]: rflags 0x246
3月 10 09:33:22 jzswai ollama[3872788]: cs     0x33
3月 10 09:33:22 jzswai ollama[3872788]: fs     0x0
3月 10 09:33:22 jzswai ollama[3872788]: gs     0x0
3月 10 09:33:22 jzswai ollama[3872788]: time=2026-03-10T09:33:22.799+08:00 level=ERROR source=server.go:303 msg="llama runner terminated" error="exit status 2"

RAW_BUFFERClick to expand / collapse

What is the issue?

Description

Steps to Reproduce

Start Ollama service with GPU support (device 0).
Use an external application (e.g., Mattermost Agent Plugin) to send a request to qwen3.5:27b.
Observe that the application receives no response or an empty result from the model.
In terminal, run:
```
ollama run qwen3.5:27b
```
The model starts normally and accepts input.
After submitting a prompt (e.g., who are you), the inference fails with a CUDA OOM error (see logs below).
Run ollama ps — the model still appears as "running".

Expected Behavior

The model should handle inference requests consistently, whether invoked via API/plugin or CLI.
If a prior request exhausts VRAM, subsequent requests should either:
- Gracefully release and reload the model, or
- Return a clear error indicating resource unavailability (rather than a low-level CUDA crash).

Actual Behavior

After an external plugin invokes the model, manual CLI inference fails with:

Thinking...
OkayError: an error was encountered while running the model: CUDA error: out of memory
  current device: 0, in function evaluate_and_capture_cuda_graph at //ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:3831
  cudaGraphInstantiate(&cuda_ctx->cuda_graph->instance, cuda_ctx->cuda_graph->graph, __null, __null, 0)
//ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu:94: CUDA error

Despite the error, ollama ps continues to list qwen3.5:27b as running, suggesting the model context was not properly cleaned up.

Qwen3 series (e.g., qwen3:32b, qwen3:14b) works as expected under the same conditions

Relevant log output

3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 111 gp=0xc00053a8c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e984a?, 0x3?, 0x63?, 0x4?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000534f38 sp=0xc000534f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000534fc8 sp=0xc000534f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000534fe0 sp=0xc000534fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000534fe8 sp=0xc000534fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 112 gp=0xc00053aa80 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xd4?, 0x8?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000535738 sp=0xc000535718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005357c8 sp=0xc000535738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005357e0 sp=0xc0005357c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005357e8 sp=0xc0005357e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 113 gp=0xc00053ac40 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a5d2a0f4?, 0x3?, 0xc3?, 0x4b?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000535f38 sp=0xc000535f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000535fc8 sp=0xc000535f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000535fe0 sp=0xc000535fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000535fe8 sp=0xc000535fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 114 gp=0xc00053ae00 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e93e8?, 0x1?, 0x5e?, 0xcf?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000540738 sp=0xc000540718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005407c8 sp=0xc000540738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005407e0 sp=0xc0005407c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005407e8 sp=0xc0005407e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 115 gp=0xc00053afc0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e5f9b?, 0x3?, 0x4d?, 0xa2?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000540f38 sp=0xc000540f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000540fc8 sp=0xc000540f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000540fe0 sp=0xc000540fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000540fe8 sp=0xc000540fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 116 gp=0xc00053b180 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e3b7e?, 0x1?, 0x11?, 0x92?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000541738 sp=0xc000541718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005417c8 sp=0xc000541738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005417e0 sp=0xc0005417c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005417e8 sp=0xc0005417e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 117 gp=0xc00053b340 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57d0855?, 0x1?, 0x6c?, 0x8d?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000541f38 sp=0xc000541f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000541fc8 sp=0xc000541f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000541fe0 sp=0xc000541fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000541fe8 sp=0xc000541fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 118 gp=0xc00053b500 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0x26?, 0x79?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000542738 sp=0xc000542718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005427c8 sp=0xc000542738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005427e0 sp=0xc0005427c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005427e8 sp=0xc0005427e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 119 gp=0xc00053b6c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57f16b8?, 0x1?, 0xc0?, 0x72?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000542f38 sp=0xc000542f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000542fc8 sp=0xc000542f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000542fe0 sp=0xc000542fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000542fe8 sp=0xc000542fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 120 gp=0xc00053b880 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a5ddaecf?, 0x1?, 0xed?, 0xb4?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000543738 sp=0xc000543718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005437c8 sp=0xc000543738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc0005437e0 sp=0xc0005437c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005437e8 sp=0xc0005437e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 121 gp=0xc00053ba40 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x3?, 0x19?, 0x5?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000543f38 sp=0xc000543f18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc000543fc8 sp=0xc000543f38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc000543fe0 sp=0xc000543fc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000543fe8 sp=0xc000543fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 122 gp=0xc00053bc00 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xa1?, 0x20?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053c738 sp=0xc00053c718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053c7c8 sp=0xc00053c738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053c7e0 sp=0xc00053c7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053c7e8 sp=0xc00053c7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 123 gp=0xc00053bdc0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57dad07?, 0x1?, 0xf?, 0x4b?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053cf38 sp=0xc00053cf18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053cfc8 sp=0xc00053cf38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053cfe0 sp=0xc00053cfc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053cfe8 sp=0xc00053cfe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 124 gp=0xc000544000 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x564acd3e44c0?, 0x1?, 0xf6?, 0x8c?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053d738 sp=0xc00053d718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053d7c8 sp=0xc00053d738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053d7e0 sp=0xc00053d7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053d7e8 sp=0xc00053d7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 125 gp=0xc0005441c0 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57effab?, 0x1?, 0xe0?, 0xdd?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053df38 sp=0xc00053df18 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053dfc8 sp=0xc00053df38 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053dfe0 sp=0xc00053dfc8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053dfe8 sp=0xc00053dfe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 126 gp=0xc000544380 m=nil [GC worker (idle)]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x103b297a57e5ff6?, 0x1?, 0x26?, 0x27?, 0x0?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc00053e738 sp=0xc00053e718 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkWorker(0xc00016f570)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1423 +0xe9 fp=0xc00053e7c8 sp=0xc00053e738 pc=0x564acafe7aa9
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gcBgMarkStartWorkers.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x25 fp=0xc00053e7e0 sp=0xc00053e7c8 pc=0x564acafe7985
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00053e7e8 sp=0xc00053e7e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by runtime.gcBgMarkStartWorkers in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/mgc.go:1339 +0x105
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 20 gp=0xc00027b880 m=nil [chan receive]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x30?, 0x564acc813260?, 0x1?, 0x0?, 0xc0008bf798?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc0008bf750 sp=0xc0008bf730 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv(0xc000744310, 0x0, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:664 +0x445 fp=0xc0008bf7c8 sp=0xc0008bf750 pc=0x564acafd6085
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv1(0x564acc333bbb?, 0x29?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:506 +0x12 fp=0xc0008bf7f0 sp=0xc0008bf7c8 pc=0x564acafd5c12
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).forwardBatch(_, {0x2, {0x564acc8e4130, 0xc000730300}, {0x564acc8f1570, 0xc0006d5db8}, {0xc000b1c148, 0x1, 0x1}, {{0x564acc8f1570, ...}, ...}, ...})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:476 +0xfa fp=0xc0008bfb58 sp=0xc0008bf7f0 pc=0x564acb65a7ba
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0002f90e0, {0x564acc8d7bd0, 0xc0001c3d60})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:453 +0x18c fp=0xc0008bffb8 sp=0xc0008bfb58 pc=0x564acb65a46c
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:1442 +0x28 fp=0xc0008bffe0 sp=0xc0008bffb8 pc=0x564acb663f68
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0008bffe8 sp=0xc0008bffe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:1442 +0x4c9
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 21 gp=0xc00027ba40 m=nil [select]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0xc0000459e8?, 0x2?, 0x4?, 0x0?, 0xc000045810?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000045620 sp=0xc000045600 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.selectgo(0xc0000459e8, 0xc00004580c, 0xc00088c000?, 0x0, 0x1?, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/select.go:351 +0x837 fp=0xc000045758 sp=0xc000045620 pc=0x564acb018c17
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).completion(0xc0002f90e0, {0x564acc8d5440, 0xc00079a0e0}, 0xc000476280)
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:974 +0xc86 fp=0xc000045ac0 sp=0xc000045758 pc=0x564acb65ee26
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).completion-fm({0x564acc8d5440?, 0xc00079a0e0?}, 0xc0008bfb40?)
3月 10 09:33:22 jzswai ollama[3872788]:         <autogenerated>:1 +0x36 fp=0xc000045af0 sp=0xc000045ac0 pc=0x564acb664456
3月 10 09:33:22 jzswai ollama[3872788]: net/http.HandlerFunc.ServeHTTP(0xc0005ba780?, {0x564acc8d5440?, 0xc00079a0e0?}, 0xc0008bfb60?)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2294 +0x29 fp=0xc000045b18 sp=0xc000045af0 pc=0x564acb33a8c9
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*ServeMux).ServeHTTP(0x564acafdeac5?, {0x564acc8d5440, 0xc00079a0e0}, 0xc000476280)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2822 +0x1c4 fp=0xc000045b68 sp=0xc000045b18 pc=0x564acb33c7c4
3月 10 09:33:22 jzswai ollama[3872788]: net/http.serverHandler.ServeHTTP({0x564acc8d1730?}, {0x564acc8d5440?, 0xc00079a0e0?}, 0x1?)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3301 +0x8e fp=0xc000045b98 sp=0xc000045b68 pc=0x564acb35a24e
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*conn).serve(0xc0001ae750, {0x564acc8d7b98, 0xc0005df800})
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:2102 +0x625 fp=0xc000045fb8 sp=0xc000045b98 pc=0x564acb338dc5
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*Server).Serve.gowrap3()
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3454 +0x28 fp=0xc000045fe0 sp=0xc000045fb8 pc=0x564acb33e688
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000045fe8 sp=0xc000045fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by net/http.(*Server).Serve in goroutine 1
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:3454 +0x485
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 1758 gp=0xc000583dc0 m=nil [IO wait]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0xc009445e00?, 0xc009445e50?, 0x0?, 0x80?, 0xb?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc0006f45d8 sp=0xc0006f45b8 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.netpollblock(0x564acb05d798?, 0xcafd34a6?, 0x4a?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/netpoll.go:575 +0xf7 fp=0xc0006f4610 sp=0xc0006f45d8 pc=0x564acafff097
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.runtime_pollWait(0x7f9cb20525b8, 0x72)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/netpoll.go:351 +0x85 fp=0xc0006f4630 sp=0xc0006f4610 pc=0x564acb039105
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*pollDesc).wait(0xc0005c1500?, 0xc0005df901?, 0x0)
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006f4658 sp=0xc0006f4630 pc=0x564acb0c1487
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*pollDesc).waitRead(...)
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_poll_runtime.go:89
3月 10 09:33:22 jzswai ollama[3872788]: internal/poll.(*FD).Read(0xc0005c1500, {0xc0005df901, 0x1, 0x1})
3月 10 09:33:22 jzswai ollama[3872788]:         internal/poll/fd_unix.go:165 +0x27a fp=0xc0006f46f0 sp=0xc0006f4658 pc=0x564acb0c277a
3月 10 09:33:22 jzswai ollama[3872788]: net.(*netFD).Read(0xc0005c1500, {0xc0005df901?, 0xc0007141d8?, 0xc0006f4770?})
3月 10 09:33:22 jzswai ollama[3872788]:         net/fd_posix.go:55 +0x25 fp=0xc0006f4738 sp=0xc0006f46f0 pc=0x564acb137da5
3月 10 09:33:22 jzswai ollama[3872788]: net.(*conn).Read(0xc00013ab00, {0xc0005df901?, 0xc009449c70?, 0xc009449cc0?})
3月 10 09:33:22 jzswai ollama[3872788]:         net/net.go:194 +0x45 fp=0xc0006f4780 sp=0xc0006f4738 pc=0x564acb146165
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*connReader).backgroundRead(0xc0005df8f0)
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:690 +0x37 fp=0xc0006f47c8 sp=0xc0006f4780 pc=0x564acb332c97
3月 10 09:33:22 jzswai ollama[3872788]: net/http.(*connReader).startBackgroundRead.gowrap2()
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:686 +0x25 fp=0xc0006f47e0 sp=0xc0006f47c8 pc=0x564acb332bc5
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0006f47e8 sp=0xc0006f47e0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by net/http.(*connReader).startBackgroundRead in goroutine 21
3月 10 09:33:22 jzswai ollama[3872788]:         net/http/server.go:686 +0xb6
3月 10 09:33:22 jzswai ollama[3872788]: goroutine 1439 gp=0xc0007b6e00 m=nil [chan receive]:
3月 10 09:33:22 jzswai ollama[3872788]: runtime.gopark(0x30?, 0x564acc813260?, 0x1?, 0xba?, 0xc000b26b18?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/proc.go:435 +0xce fp=0xc000b26ad0 sp=0xc000b26ab0 pc=0x564acb039eee
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv(0xc0007442a0, 0x0, 0x1)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:664 +0x445 fp=0xc000b26b48 sp=0xc000b26ad0 pc=0x564acafd6085
3月 10 09:33:22 jzswai ollama[3872788]: runtime.chanrecv1(0x564acc3378ec?, 0x2c?)
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/chan.go:506 +0x12 fp=0xc000b26b70 sp=0xc000b26b48 pc=0x564acafd5c12
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).computeBatch(0xc0002f90e0, {0x2, {0x564acc8e4130, 0xc000730300}, {0x564acc8f1570, 0xc0006d5db8}, {0xc000b1c148, 0x1, 0x1}, {{0x564acc8f1570, ...}, ...}, ...})
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:646 +0x185 fp=0xc000b26ef0 sp=0xc000b26b70 pc=0x564acb65c2c5
3月 10 09:33:22 jzswai ollama[3872788]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run.gowrap1()
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:459 +0x58 fp=0xc000b26fe0 sp=0xc000b26ef0 pc=0x564acb65a698
3月 10 09:33:22 jzswai ollama[3872788]: runtime.goexit({})
3月 10 09:33:22 jzswai ollama[3872788]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000b26fe8 sp=0xc000b26fe0 pc=0x564acb041e61
3月 10 09:33:22 jzswai ollama[3872788]: created by github.com/ollama/ollama/runner/ollamarunner.(*Server).run in goroutine 20
3月 10 09:33:22 jzswai ollama[3872788]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:459 +0x2cd
3月 10 09:33:22 jzswai ollama[3872788]: rax    0x0
3月 10 09:33:22 jzswai ollama[3872788]: rbx    0x7f9ad1fff640
3月 10 09:33:22 jzswai ollama[3872788]: rcx    0x7f9cb1e969fc
3月 10 09:33:22 jzswai ollama[3872788]: rdx    0x6
3月 10 09:33:22 jzswai ollama[3872788]: rdi    0x3cf144
3月 10 09:33:22 jzswai ollama[3872788]: rsi    0x3cf1a2
3月 10 09:33:22 jzswai ollama[3872788]: rbp    0x3cf1a2
3月 10 09:33:22 jzswai ollama[3872788]: rsp    0x7f9ad1ffdd20
3月 10 09:33:22 jzswai ollama[3872788]: r8     0x7f9ad1ffddf0
3月 10 09:33:22 jzswai ollama[3872788]: r9     0x7f94e9ba9e90
3月 10 09:33:22 jzswai ollama[3872788]: r10    0x8
3月 10 09:33:22 jzswai ollama[3872788]: r11    0x246
3月 10 09:33:22 jzswai ollama[3872788]: r12    0x6
3月 10 09:33:22 jzswai ollama[3872788]: r13    0x16
3月 10 09:33:22 jzswai ollama[3872788]: r14    0x7f9c5029242e
3月 10 09:33:22 jzswai ollama[3872788]: r15    0x7f9ac90d5508
3月 10 09:33:22 jzswai ollama[3872788]: rip    0x7f9cb1e969fc
3月 10 09:33:22 jzswai ollama[3872788]: rflags 0x246
3月 10 09:33:22 jzswai ollama[3872788]: cs     0x33
3月 10 09:33:22 jzswai ollama[3872788]: fs     0x0
3月 10 09:33:22 jzswai ollama[3872788]: gs     0x0
3月 10 09:33:22 jzswai ollama[3872788]: time=2026-03-10T09:33:22.799+08:00 level=ERROR source=server.go:303 msg="llama runner terminated" error="exit status 2"

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.17.7

extent analysis

Fix Plan

The issue appears to be related to CUDA out-of-memory errors when running the qwen3.5:27b model. To fix this, we can try the following steps:

Increase GPU memory allocation: Try increasing the GPU memory allocation for the Ollama service. This can be done by setting the CUDA_VISIBLE_DEVICES environment variable or by configuring the Ollama service to use a specific GPU with more memory.
Optimize model configuration: Check the model configuration for qwen3.5:27b and optimize it to reduce memory usage. This can include reducing the batch size, sequence length, or other model parameters.
Implement model unloading: Implement a mechanism to unload the model from memory when it is not in use. This can help reduce memory usage and prevent out-of-memory errors.
Update Ollama version: Consider updating the Ollama version to the latest release, as this may include bug fixes or improvements that address the out-of-memory issue.

Example code to implement model unloading:

import torch

# Load the model
model = torch.load('qwen3.5:27b')

# Use the model for inference
output = model(input)

# Unload the model from memory
del model
torch.cuda.empty_cache()

Verification

To verify that the fix worked, try running the ollama run qwen3.5:27b command again and check for any out-of-memory errors. You can also monitor the GPU memory usage using tools like nvidia-smi to ensure that the model is not consuming excessive memory.

Extra Tips

Make sure to monitor the GPU memory usage and adjust the model configuration or GPU allocation as needed to prevent out-of-memory errors.
Consider using a more efficient model or optimizing the existing model to reduce memory usage.
If the issue persists, try debugging the model loading and unloading process to identify any memory leaks or other issues.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #prompt formatting #chain error #conversation history #tool integration

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

ollama - 💡(How to fix) Fix CUDA out of memory error when using Qwen3.5:27B after external application (e.g., Mattermost plugin) invokes the model [7 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Code Example

What is the issue?

Description

Steps to Reproduce

Expected Behavior

Actual Behavior

Relevant log output

OS

GPU

CPU

Ollama version

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

TRENDING

ollama - 💡(How to fix) Fix CUDA out of memory error when using Qwen3.5:27B after external application (e.g., Mattermost plugin) invokes the model [7 comments, 2 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Code Example

What is the issue?

Description

Steps to Reproduce

Expected Behavior

Actual Behavior

Relevant log output

OS

GPU

CPU

Ollama version

extent analysis

Fix Plan

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING