litellm - 💡(How to fix) Fix [Bug]: Issue using litellm with pi.dev

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Code Example

cat models.json
{
  "providers": {
    "ollama": {
      "baseUrl": "http://10.30.0.103:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "gemma4:e4b" },
        { "id": "qwen3.5:9b" }
      ]
    }
  }
}

pi

 pi v0.74.0
 escape interrupt · ctrl+c/ctrl+d clear/exit · / commands · ! bash · ctrl+o more
 Press ctrl+o to show full startup help and loaded resources.

 Pi can explain its own features and look up its docs. Ask it how to use or extend Pi.


 say hello


 {
   "name": "say_hello",
   "arguments": {
     "text": "Hello!"
   }
 }

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Only showing models from configured providers. Use /login to add providers.

>

→ ollama/qwen3.5:9b [litellm]  ollama/gemma4:e4b [litellm]

  Model Name: ollama/qwen3.5:9b

---

cat models.json
{
  "providers": {
    "litellm": {
      "baseUrl": "http://10.30.0.103:4000/v1",
      "api": "openai-completions",
      "apiKey": "sk-1234",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": false
      },
      "models": [
        {
          "id": "ollama/qwen3.5:9b",
          "reasoning": false
        },
        {
          "id": "ollama/gemma4:e4b",
          "reasoning": false
        }
      ]
    }
  }
}

---

All logs are attached in description.
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When I use pi.dev with models in ollama it works as expected. But when i bring in litellm to talk with ollama I see weird tool call output from same ollama model.

On using ollama models directly as shown below I am seeing right output from respective models.

cat models.json
{
  "providers": {
    "ollama": {
      "baseUrl": "http://10.30.0.103:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "gemma4:e4b" },
        { "id": "qwen3.5:9b" }
      ]
    }
  }
}

pi

 pi v0.74.0
 escape interrupt · ctrl+c/ctrl+d clear/exit · / commands · ! bash · ctrl+o more
 Press ctrl+o to show full startup help and loaded resources.

 Pi can explain its own features and look up its docs. Ask it how to use or extend Pi.


 say hello


 {
   "name": "say_hello",
   "arguments": {
     "text": "Hello!"
   }
 }

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Only showing models from configured providers. Use /login to add providers.

>

→ ollama/qwen3.5:9b [litellm] ✓
  ollama/gemma4:e4b [litellm]

  Model Name: ollama/qwen3.5:9b

Now I am using below configuration to use litellm as proxy to talk with same ollama models. I am seeing output in a structured format instead of final output in case of ollama models

cat models.json
{
  "providers": {
    "litellm": {
      "baseUrl": "http://10.30.0.103:4000/v1",
      "api": "openai-completions",
      "apiKey": "sk-1234",
      "compat": {
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": false
      },
      "models": [
        {
          "id": "ollama/qwen3.5:9b",
          "reasoning": false
        },
        {
          "id": "ollama/gemma4:e4b",
          "reasoning": false
        }
      ]
    }
  }
}
<img width="787" height="860" alt="Image" src="https://github.com/user-attachments/assets/7bf255ac-5bd5-427d-93ef-854afafe40c2" />

Steps to Reproduce

  1. Install pi.dev and configure models.json as mentioned in description
  2. run pi command to start the terminal

Relevant log output

All logs are attached in description.

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.83.10

Twitter / LinkedIn details

No response

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - 💡(How to fix) Fix [Bug]: Issue using litellm with pi.dev