litellm - 💡(How to fix) Fix [Bug]: minimax-m2.7 via Ollama Cloud fails on 2nd+ request with Internal Server Error [1 comments, 2 participants]

Python version: 3.12 LiteLLM version: 1.82.4 OS: Linux

Description

When using minimax-m2.7:cloud (Ollama's naming — the :cloud suffix denotes the cloud-hosted version) through the Ollama Cloud API, the first request succeeds but every subsequent request fails with Internal Server Error. This includes starting a brand new session — still only one response before it breaks. This rules out conversation history as the cause.

Steps to Reproduce

1. Configure LiteLLM with the Ollama provider pointing to Ollama Cloud:

   import litellm
   litellm.api_base = "https://ollama.com/api"
   litellm.api_key = "your-ollama-cloud-api-key"

2. Send a first chat completion request with model minimax-m2.7:cloud — succeeds
3. Send a second request (same or new conversation) — fails with Internal Server Error

Expected Behavior

Both requests should succeed.

Actual Behavior

First request: ✅ succeeds Second request: ❌ fails with:

litellm.APIConnectionError: Ollama_chatException - {"error":"Internal Server Error (ref: <uuid>)"}

Retries (3/3) also fail. Even starting a brand new session exhibits the same pattern.

Additional Context

Key observation: The same underlying model is available via OpenRouter as minimax/minimax-m2.7 and works correctly there. This suggests the model itself is fine — the bug is in LiteLLM's Ollama adapter handling of this specific cloud model's streaming response.

Working models via Ollama Cloud:

• GLM-5 (also a thinking model) — ✅ works perfectly, multiple requests
• Non-thinking models — ✅ work fine

Not working:

• minimax-m2.7:cloud via Ollama Cloud — ❌ fails after first request
• Same model via OpenRouter (minimax/minimax-m2.7) — ✅ works

Hypothesis: minimax-m2.7:cloud may stream its thinking/reasoning content differently from other thinking models (e.g., GLM-5), causing the Ollama chat transformation or streaming iterator to corrupt subsequent requests. This could be related to how the thinking content field is handled differently between the two models.

Relevant Code

The Ollama chat transformation in litellm/llms/ollama/chat/transformation.py has reasoning content handling:

# thinking is set from reasoning_content in transform_request
if reasoning_content is not None:
    ollama_message["thinking"] = reasoning_content

# response remaps 'thinking' field to 'reasoning_content'
response_json_message["reasoning_content"] = response_json_message.get("thinking")

The streaming iterator OllamaChatCompletionResponseIterator tracks started_reasoning_content and finished_reasoning_content flags to strip <think> XML tags from content. If the model streams thinking differently, these flags could get into a bad state.

Related Issues

• Similar: litellm#15399 — "Ollama Cloud Models Streaming Chunk Parsing Failure" (closed, about deepseek-v3.1:671b-cloud)

Investigation Needed

A raw streaming log of minimax-m2.7:cloud vs GLM-5 chunks from Ollama Cloud would help identify the difference. The issue likely requires comparing the actual SSE chunk format between the two models to see if minimax-m2.7:cloud uses a non-standard thinking field format or streams it differently.