ollama - 💡(How to fix) Fix Incorrectly calculates available system memory for qwen3.6:35b-a3b [1 comments, 2 participants]

Error: 500 Internal Server Error: model requires more system memory (9.7 GiB) than is available (8.1 GiB)

TL;DR

The most likely fix is to investigate and adjust Ollama's memory calculation to accurately reflect available system memory, considering the discrepancy between reported and actual available RAM.

Guidance

• Verify that Ollama's memory calculation is correctly accounting for the buff/cache memory, as Linux reports this as available but Ollama may not.
• Check if there are any environment variables or configuration options in Ollama that can be adjusted to influence the memory calculation or to override the pre-flight memory check.
• Consider testing with a different model or version to see if the issue is specific to qwen3.6:35b-a3b or a more general problem with Ollama's memory management.
• Review related issues (#14501, #14557, #14719) for any insights or fixes that might apply to this scenario.

Example

No specific code snippet can be provided without more details on Ollama's internal memory calculation logic. However, if Ollama provides an API or configuration for adjusting memory thresholds, an example might look like setting an environment variable: OLLAMA_MEMORY_THRESHOLD_OVERRIDE=true or adjusting a config file to reflect the actual available memory.

Notes

The issue seems to stem from a discrepancy in how Ollama calculates available memory versus the actual available memory as reported by the system. The fact that setting OLLAMA_GPU_LAYERS=999 does not resolve the issue suggests that the problem lies in the pre-flight memory check rather than GPU-specific memory allocation.

Recommendation

Apply a workaround by attempting to adjust Ollama's configuration or environment variables to correctly account for available system memory, as directly upgrading to a fixed version is not mentioned as an option in the provided context. This approach is chosen because it directly addresses the observed discrepancy in memory calculation, which is the root cause of the failure.