Issue Station
Sign In
Tools hereToolbox

ollama - ✅(Solved) Fix "panic: failed to sample token" with "nemotron-3-nano" [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
ollama/ollama#14882•Fetched 2026-04-08 00:48:09
View on GitHub
Comments
0
Participants
1
Timeline
3
Reactions
0
Author
marianopeckmarianopeck
Participants
marianopeckmarianopeck
Timeline (top)
cross-referenced ×1labeled ×1referenced ×1

Error Message

'{"error":{"message":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details","type":"api_error","param":null,"code":null}} time=2026-03-16T16:50:07.497-03:00 level=ERROR source=server.go:1610 msg="post predict" error="Post "http://127.0.0.1:51411/completion\": EOF"

Fix Action

Fixed

PR fix notes

PR #14896: runner/ollamarunner: handle sampling errors gracefully

Description (problem / solution / changelog)

Fixes #14882

Replace panics with proper error handling when token sampling or decoding fails. Previously, the server would crash with "panic: failed to sample token" when models like nemotron-3-nano returned invalid logits (e.g., NaN values).

Changes:

  • Log the error with context instead of panicking
  • Remove the sequence gracefully using DoneReasonStop
  • Continue processing other sequences normally

This prevents the entire server from crashing when a single sequence encounters a sampling error, which can happen with certain models or under specific conditions.

Changed files

  • runner/ollamarunner/runner.go (modified, +6/-2)

Code Example

'{"error":{"message":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details","type":"api_error","param":null,"code":null}}
'

---

panic: failed to sample token

goroutine 567 [running]:
github.com/ollama/ollama/runner/ollamarunner.(*Server).computeBatch(0x140002330e0, {0x5, {0x105bce850, 0x14000855100}, {0x105bdbdb0, 0x14005ed92d8}, {0x14005e88c08, 0x1de, 0x25f}, {{0x105bdbdb0, ...}, ...}, ...})
	/Users/runner/work/ollama/ollama/runner/ollamarunner/runner.go:762 +0x1668
created by github.com/ollama/ollama/runner/ollamarunner.(*Server).run in goroutine 10
	/Users/runner/work/ollama/ollama/runner/ollamarunner/runner.go:459 +0x22c
time=2026-03-16T16:50:07.497-03:00 level=ERROR source=server.go:1610 msg="post predict" error="Post \"http://127.0.0.1:51411/completion\": EOF"
[GIN] 2026/03/16 - 16:50:07 | 500 | 12.603315458s |     10.211.55.3 | POST     "/v1/responses"
time=2026-03-16T16:50:07.497-03:00 level=DEBUG source=sched.go:585 msg="context for request finished"
time=2026-03-16T16:50:07.498-03:00 level=DEBUG source=sched.go:336 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/nemotron-3-nano:30b runner.inference="[{ID:0 Library:Metal}]" runner.size="26.0 GiB" runner.vram="26.0 GiB" runner.parallel=1 runner.pid=66967 runner.model=/Users/mariano/.ollama/models/blobs/sha256-a70437c41b3b0b768c48737e15f8160c90f13dc963f5226aabb3a160f708d1ce runner.num_ctx=262144 duration=5m0s
time=2026-03-16T16:50:07.498-03:00 level=DEBUG source=sched.go:354 msg="after processing request finished event" runner.name=registry.ollama.ai/library/nemotron-3-nano:30b runner.inference="[{ID:0 Library:Metal}]" runner.size="26.0 GiB" runner.vram="26.0 GiB" runner.parallel=1 runner.pid=66967 runner.model=/Users/mariano/.ollama/models/blobs/sha256-a70437c41b3b0b768c48737e15f8160c90f13dc963f5226aabb3a160f708d1ce runner.num_ctx=262144 refCount=0
RAW_BUFFERClick to expand / collapse

What is the issue?

Hi everyone,

I am trying to call the Responses API to the Ollama server with the model "nemotron-3-nano". But I am getting a crash. I am running Ollama in debug mode. I can attach the whole server.log is useful.

Important to note that same Responses API request (and same code) works great with many other models.

Also interesting is that ollama run nemotron-3-nano does work. It seems to fail only with my Responses API call.

The response I get back in my client is:

'{"error":{"message":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details","type":"api_error","param":null,"code":null}}
'

Relevant log output

panic: failed to sample token

goroutine 567 [running]:
github.com/ollama/ollama/runner/ollamarunner.(*Server).computeBatch(0x140002330e0, {0x5, {0x105bce850, 0x14000855100}, {0x105bdbdb0, 0x14005ed92d8}, {0x14005e88c08, 0x1de, 0x25f}, {{0x105bdbdb0, ...}, ...}, ...})
	/Users/runner/work/ollama/ollama/runner/ollamarunner/runner.go:762 +0x1668
created by github.com/ollama/ollama/runner/ollamarunner.(*Server).run in goroutine 10
	/Users/runner/work/ollama/ollama/runner/ollamarunner/runner.go:459 +0x22c
time=2026-03-16T16:50:07.497-03:00 level=ERROR source=server.go:1610 msg="post predict" error="Post \"http://127.0.0.1:51411/completion\": EOF"
[GIN] 2026/03/16 - 16:50:07 | 500 | 12.603315458s |     10.211.55.3 | POST     "/v1/responses"
time=2026-03-16T16:50:07.497-03:00 level=DEBUG source=sched.go:585 msg="context for request finished"
time=2026-03-16T16:50:07.498-03:00 level=DEBUG source=sched.go:336 msg="runner with non-zero duration has gone idle, adding timer" runner.name=registry.ollama.ai/library/nemotron-3-nano:30b runner.inference="[{ID:0 Library:Metal}]" runner.size="26.0 GiB" runner.vram="26.0 GiB" runner.parallel=1 runner.pid=66967 runner.model=/Users/mariano/.ollama/models/blobs/sha256-a70437c41b3b0b768c48737e15f8160c90f13dc963f5226aabb3a160f708d1ce runner.num_ctx=262144 duration=5m0s
time=2026-03-16T16:50:07.498-03:00 level=DEBUG source=sched.go:354 msg="after processing request finished event" runner.name=registry.ollama.ai/library/nemotron-3-nano:30b runner.inference="[{ID:0 Library:Metal}]" runner.size="26.0 GiB" runner.vram="26.0 GiB" runner.parallel=1 runner.pid=66967 runner.model=/Users/mariano/.ollama/models/blobs/sha256-a70437c41b3b0b768c48737e15f8160c90f13dc963f5226aabb3a160f708d1ce runner.num_ctx=262144 refCount=0

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.18.0

extent analysis

Fix Plan

The fix involves increasing the resource limits for the Ollama server to prevent it from crashing due to resource limitations.

  • Increase the memory limit for the Ollama server by setting the OLLAMA_SERVER_MEMORY_LIMIT environment variable.
  • Increase the VRAM limit for the Ollama server by setting the OLLAMA_SERVER_VRAM_LIMIT environment variable.

Example:

export OLLAMA_SERVER_MEMORY_LIMIT=32G
export OLLAMA_SERVER_VRAM_LIMIT=32G
  • Update the Ollama server configuration to increase the parallelism limit for the model.
// In ollamarunner/runner.go, update the computeBatch function to increase the parallelism limit
func (s *Server) computeBatch(ctx context.Context, batch *Batch) {
    // ...
    parallelismLimit := 2 // Increase the parallelism limit
    // ...
}

Alternatively, you can also try updating the Ollama version to the latest one, as the issue might be fixed in a newer version.

Verification

To verify that the fix worked, try making the same Responses API request again and check if the Ollama server crashes. You can also check the server logs to see if there are any error messages related to resource limitations.

Extra Tips

  • Make sure to monitor the Ollama server's resource usage and adjust the resource limits accordingly to prevent crashes.
  • Consider implementing a retry mechanism for the Responses API request to handle temporary resource limitations.
  • If the issue persists, try attaching the full server log to the issue report for further debugging.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#tool integration#LLM response#prompt template#agent execution

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING