ollama - 💡(How to fix) Fix [0.17.5][macOS Apple Silicon] model runner unexpectedly stopped (EOF/exit status 2) on /api/generate [8 comments, 3 participants]

• {"error":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error"} {"error":"model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details"} time=2026-03-04T17:10:17.873+09:00 level=ERROR source=server.go:1611 msg="post predict" error="Post "http://127.0.0.1:53947/completion\": EOF" time=2026-03-04T17:10:17.873+09:00 level=ERROR source=server.go:303 msg="llama runner terminated" error="exit status 2" /api/generate should return a normal completion or a stable, actionable error without runner termination.

Fix Plan

To address the intermittent failure of inference calls, we will focus on adjusting the resource allocation and handling of the Ollama API. The steps below aim to stabilize the /api/generate endpoint.

1. Increase Resource Limits:

   • Edit the ollama configuration to allocate more resources to the model runner.
   • Specifically, increase the max_memory and max_cpu settings in your ollama.yaml or equivalent configuration file.

2. Implement Retry Mechanism:

   • Modify the client-side code to implement a retry mechanism with exponential backoff for handling temporary failures.
   • Example Python code using requests and tenacity libraries:

     import requests
     from tenacity import retry, stop_after_attempt, wait_exponential
     
     @retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=4, max=10))
     def generate_text(prompt, model="gpt-oss:20b"):
         url = "http://127.0.0.1:11434/api/generate"
         data = {
             "model": model,
             "prompt": prompt,
             "stream": False,
             "options": {"num_ctx": 512, "num_predict": 8}
         }
         response = requests.post(url, json=data)
         response.raise_for_status()
         return response.json()
     
     # Example usage
     try:
         result = generate_text("Hello, world!")
         print(result)
     except requests.RequestException as e:
         print(f"Request failed: {e}")

3. Monitor Server Logs:

   • Regularly inspect server logs for patterns or specific error messages that could indicate underlying issues.
   • Adjust logging levels or add custom logging to capture more detailed information around failures.

4. Update Ollama Version:

   • If possible, update Ollama to the latest version to ensure any known issues related to resource management or stability have been addressed.

Verification

• After implementing these changes, repeat the reproduction steps to verify that the /api/generate endpoint no longer intermittently fails.
• Monitor server logs for any error messages related to resource limitations or internal errors.

Extra Tips

• Regularly review and adjust resource allocations based on workload demands to prevent similar issues.
• Consider implementing health checks for the model runner to proactively detect and restart it if necessary.
• For long-term stability, explore options for horizontal scaling or load balancing to distribute the workload across multiple instances.