ollama - 💡(How to fix) Fix Tool calling is not streaming on macOS with MLX, causing timeout when write tool outputs large code

Summary

When using tool calling with MLX backend on macOS, the tool call responses are not streaming. This causes severe issues when a tool like write needs to output large amounts of code — the client waits a long time without any response and eventually times out.

Environment

• OS: macOS (Apple Silicon)
• Backend: MLX
• Model: qwen3.6:27b-coding-nvfp4
• Ollama version: Latest main

Expected behavior

Tool call outputs should be streamed incrementally to the client, just like regular text completion streaming. This matches the behavior when not using MLX (e.g., CUDA backend), where tool calls stream properly and clients don't timeout.

Actual behavior

When a tool call is triggered (e.g., write tool generating a large file with hundreds or thousands of lines of code), the entire tool output is buffered and only sent at the very end. This means:

1. The client receives no intermediate chunks for an extended period
2. For large code outputs, the wait can be 30 seconds or more
3. Client-side timeouts are triggered (typically around 30-60s depending on the client)
4. The request fails even though Ollama is still generating in the background

Steps to reproduce

1. Pull the model on macOS with MLX support:

ollama pull qwen3.6:27b-coding-nvfp4

1. Use the chat API with tool/function calling enabled, providing a prompt that triggers a tool to generate a large file (e.g., 500+ lines of code)

2. Observe that during tool execution, no streaming chunks are received by the client until the entire tool output is complete

3. The client times out before receiving the final response

Additional context

This is particularly problematic for coding models used in agent workflows where code generation via tools is very common. Non-streaming tool calls significantly degrade the user experience and reliability of AI coding assistants on macOS.

This issue may be related to how the MLX backend handles tool call chunking/streaming compared to other backends.