litellm - ✅(Solved) Fix [Bug]: ZAI/GLM tool results silently dropped when content is OpenAI list-format [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#25868Fetched 2026-04-17 08:28:28
View on GitHub
Comments
0
Participants
1
Timeline
1
Reactions
0
Author
Participants
Timeline (top)
labeled ×1

Root Cause

The OpenAI API spec allows tool message content as either a plain string or a list of content parts:

// Format A: string (works)
{"role": "tool", "tool_call_id": "call_123", "content": "22.5°C, partly cloudy"}

// Format B: list of content parts (broken — content silently dropped)
{"role": "tool", "tool_call_id": "call_123", "content": [{"type": "text", "text": "22.5°C, partly cloudy"}]}

LiteLLM passes both formats through unchanged to the ZAI API. However, z.ai's GLM backend uses a Jinja chat template that checks m.content is string (the same root cause as vllm-project/vllm#39614). When content is a list, the check fails and the tool result is silently replaced with an empty artifact. The model then responds with things like "I'm sorry, the service didn't return any data".

This was confirmed by inspecting the actual requests in the LiteLLM spend logs database — the Go client's requests had list-format content while Python urllib requests had string content, explaining why some clients worked and others didn't.

Fix Action

Fix

The ZAI provider's _transform_messages() should flatten list-format content in tool and assistant messages to plain strings before forwarding to z.ai. Here is the fix we applied locally:

# litellm/llms/zai/chat/transformation.py

from typing import Any, Coroutine, List, Literal, Optional, Tuple, Union, overload

from litellm.secret_managers.main import get_secret_str
from litellm.types.llms.openai import AllMessageValues, ChatCompletionToolParam

from ...openai.chat.gpt_transformation import OpenAIGPTConfig

ZAI_API_BASE = "https://api.z.ai/api/paas/v4"


def _flatten_content_parts(content):
    """
    Flatten OpenAI multi-part content to a plain string.

    The OpenAI API allows tool message content as either a string or a list
    of content parts like [{"type": "text", "text": "..."}].  GLM's chat
    template checks ``m.content is string`` and silently drops list-format
    content (see vllm-project/vllm#39614).  This helper normalises both
    forms to a plain string so the content always reaches the model.
    """
    if isinstance(content, str) or content is None:
        return content
    if isinstance(content, list):
        parts = []
        for part in content:
            if isinstance(part, dict):
                text = part.get("text")
                if text:
                    parts.append(text)
            elif isinstance(part, str):
                parts.append(part)
        return "\n".join(parts) if parts else ""
    return content


class ZAIChatConfig(OpenAIGPTConfig):
    # ... existing methods unchanged ...

    # fmt: off
    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[True]
    ) -> Coroutine[Any, Any, List[AllMessageValues]]: ...

    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[False] = False,
    ) -> List[AllMessageValues]: ...
    # fmt: on

    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: bool = False
    ) -> Union[List[AllMessageValues], Coroutine[Any, Any, List[AllMessageValues]]]:
        # Flatten list-format content in tool and assistant messages
        for message in messages:
            role = message.get("role")
            content = message.get("content")
            if role == "tool" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)
            elif role == "assistant" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)

        # Delegate to parent for user-message image-URL transforms, etc.
        return super()._transform_messages(messages, model, is_async=is_async)

    # ... rest of class unchanged ...

PR fix notes

PR #25993: fix(zai): flatten list-format content in tool/assistant messages before sending to GLM

Description (problem / solution / changelog)

Fixes #25868

Problem

GLM's Jinja chat template checks m.content is string and silently drops list-format content (same root cause as vllm-project/vllm#39614). When a client sends tool result content as an OpenAI-format list of content parts — which the official Go openai-go client always does — the tool result is lost and the model responds as if no data was returned.

Working (string format):

{"role": "tool", "tool_call_id": "c1", "content": "22.5°C, partly cloudy."}

Broken (list format — silently dropped by GLM):

{"role": "tool", "tool_call_id": "c1", "content": [{"type": "text", "text": "22.5°C, partly cloudy."}]}

Solution

Add ZAIChatConfig._transform_messages() that normalises list-format content in tool and assistant messages to plain strings before delegating to the parent OpenAIGPTConfig transformer. User messages (which may contain images) are intentionally left untouched.

The new _flatten_content_parts() helper joins text parts with \n and is a no-op for plain strings and None values.

Testing

Added TestZAIMessageTransformation with 7 unit tests covering:

  • Tool message with list-format content → flattened to string
  • Assistant message with list-format content → flattened to string
  • String content → passes through unchanged
  • Multi-part list → joined with newline
  • Empty list → empty string
  • NoneNone

All 16 ZAI provider tests pass.

Changed files

  • litellm/llms/zai/chat/transformation.py (modified, +43/-1)
  • tests/test_litellm/llms/zai/test_zai_provider.py (modified, +114/-0)

Code Example

// Format A: string (works)
{"role": "tool", "tool_call_id": "call_123", "content": "22.5°C, partly cloudy"}

// Format B: list of content parts (broken — content silently dropped)
{"role": "tool", "tool_call_id": "call_123", "content": [{"type": "text", "text": "22.5°C, partly cloudy"}]}

---

curl -s http://localhost:8090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -d '{
    "model": "zai/glm-5.1",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "What is the temperature in Tokyo?"}]},
      {"role": "assistant", "content": [{"type": "text", "text": "Let me check."}],
       "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_temp", "arguments": "{\"city\":\"Tokyo\"}"}}]},
      {"role": "tool", "tool_call_id": "call_1",
       "content": [{"type": "text", "text": "The temperature in Tokyo is 22.5°C, partly cloudy."}]}
    ],
    "tools": [{"type": "function", "function": {"name": "get_temp", "description": "Get temperature", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
    "temperature": 0.1
  }'

---

# litellm/llms/zai/chat/transformation.py

from typing import Any, Coroutine, List, Literal, Optional, Tuple, Union, overload

from litellm.secret_managers.main import get_secret_str
from litellm.types.llms.openai import AllMessageValues, ChatCompletionToolParam

from ...openai.chat.gpt_transformation import OpenAIGPTConfig

ZAI_API_BASE = "https://api.z.ai/api/paas/v4"


def _flatten_content_parts(content):
    """
    Flatten OpenAI multi-part content to a plain string.

    The OpenAI API allows tool message content as either a string or a list
    of content parts like [{"type": "text", "text": "..."}].  GLM's chat
    template checks ``m.content is string`` and silently drops list-format
    content (see vllm-project/vllm#39614).  This helper normalises both
    forms to a plain string so the content always reaches the model.
    """
    if isinstance(content, str) or content is None:
        return content
    if isinstance(content, list):
        parts = []
        for part in content:
            if isinstance(part, dict):
                text = part.get("text")
                if text:
                    parts.append(text)
            elif isinstance(part, str):
                parts.append(part)
        return "\n".join(parts) if parts else ""
    return content


class ZAIChatConfig(OpenAIGPTConfig):
    # ... existing methods unchanged ...

    # fmt: off
    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[True]
    ) -> Coroutine[Any, Any, List[AllMessageValues]]: ...

    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[False] = False,
    ) -> List[AllMessageValues]: ...
    # fmt: on

    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: bool = False
    ) -> Union[List[AllMessageValues], Coroutine[Any, Any, List[AllMessageValues]]]:
        # Flatten list-format content in tool and assistant messages
        for message in messages:
            role = message.get("role")
            content = message.get("content")
            if role == "tool" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)
            elif role == "assistant" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)

        # Delegate to parent for user-message image-URL transforms, etc.
        return super()._transform_messages(messages, model, is_async=is_async)

    # ... rest of class unchanged ...
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using LiteLLM to route tool-calling conversations to the ZAI provider (zai/glm-5.1), tool results are silently dropped if the client sends content as an OpenAI-format list of content parts instead of a plain string. The model responds as if the tool returned no data.

This affects any OpenAI-compatible client that sends tool message content in the multi-part format — notably the official Go client (openai-go), which always serializes content this way.

Root cause

The OpenAI API spec allows tool message content as either a plain string or a list of content parts:

// Format A: string (works)
{"role": "tool", "tool_call_id": "call_123", "content": "22.5°C, partly cloudy"}

// Format B: list of content parts (broken — content silently dropped)
{"role": "tool", "tool_call_id": "call_123", "content": [{"type": "text", "text": "22.5°C, partly cloudy"}]}

LiteLLM passes both formats through unchanged to the ZAI API. However, z.ai's GLM backend uses a Jinja chat template that checks m.content is string (the same root cause as vllm-project/vllm#39614). When content is a list, the check fails and the tool result is silently replaced with an empty artifact. The model then responds with things like "I'm sorry, the service didn't return any data".

This was confirmed by inspecting the actual requests in the LiteLLM spend logs database — the Go client's requests had list-format content while Python urllib requests had string content, explaining why some clients worked and others didn't.

Reproduction

curl -s http://localhost:8090/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -d '{
    "model": "zai/glm-5.1",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "What is the temperature in Tokyo?"}]},
      {"role": "assistant", "content": [{"type": "text", "text": "Let me check."}],
       "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_temp", "arguments": "{\"city\":\"Tokyo\"}"}}]},
      {"role": "tool", "tool_call_id": "call_1",
       "content": [{"type": "text", "text": "The temperature in Tokyo is 22.5°C, partly cloudy."}]}
    ],
    "tools": [{"type": "function", "function": {"name": "get_temp", "description": "Get temperature", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}],
    "temperature": 0.1
  }'

Expected: Model responds using the tool result (mentions 22.5°C).

Actual: Model says "I'm sorry, it seems the temperature data for Tokyo couldn't be retrieved at the moment..."

If you change the tool message content to a plain string ("content": "The temperature in Tokyo is 22.5°C, partly cloudy."), the same request works correctly.

Fix

The ZAI provider's _transform_messages() should flatten list-format content in tool and assistant messages to plain strings before forwarding to z.ai. Here is the fix we applied locally:

# litellm/llms/zai/chat/transformation.py

from typing import Any, Coroutine, List, Literal, Optional, Tuple, Union, overload

from litellm.secret_managers.main import get_secret_str
from litellm.types.llms.openai import AllMessageValues, ChatCompletionToolParam

from ...openai.chat.gpt_transformation import OpenAIGPTConfig

ZAI_API_BASE = "https://api.z.ai/api/paas/v4"


def _flatten_content_parts(content):
    """
    Flatten OpenAI multi-part content to a plain string.

    The OpenAI API allows tool message content as either a string or a list
    of content parts like [{"type": "text", "text": "..."}].  GLM's chat
    template checks ``m.content is string`` and silently drops list-format
    content (see vllm-project/vllm#39614).  This helper normalises both
    forms to a plain string so the content always reaches the model.
    """
    if isinstance(content, str) or content is None:
        return content
    if isinstance(content, list):
        parts = []
        for part in content:
            if isinstance(part, dict):
                text = part.get("text")
                if text:
                    parts.append(text)
            elif isinstance(part, str):
                parts.append(part)
        return "\n".join(parts) if parts else ""
    return content


class ZAIChatConfig(OpenAIGPTConfig):
    # ... existing methods unchanged ...

    # fmt: off
    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[True]
    ) -> Coroutine[Any, Any, List[AllMessageValues]]: ...

    @overload
    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: Literal[False] = False,
    ) -> List[AllMessageValues]: ...
    # fmt: on

    def _transform_messages(
        self, messages: List[AllMessageValues], model: str, is_async: bool = False
    ) -> Union[List[AllMessageValues], Coroutine[Any, Any, List[AllMessageValues]]]:
        # Flatten list-format content in tool and assistant messages
        for message in messages:
            role = message.get("role")
            content = message.get("content")
            if role == "tool" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)
            elif role == "assistant" and isinstance(content, list):
                message["content"] = _flatten_content_parts(content)

        # Delegate to parent for user-message image-URL transforms, etc.
        return super()._transform_messages(messages, model, is_async=is_async)

    # ... rest of class unchanged ...

Broader consideration

This same issue likely affects any OpenAI-compatible provider whose backend doesn't handle list-format content in tool messages. It may be worth normalizing tool message content to strings in OpenAIGPTConfig._transform_messages() for all providers, not just ZAI — since tool results are always text and the list format adds no value for them.

Related issues:

  • vllm-project/vllm#39614 — same root cause in vLLM's local inference
  • #25669 — Anthropic content block format issues (different format, same category)
  • #24496 (closed) — hosted_vllm Anthropic tool_result conversion (related pattern)

Environment

  • LiteLLM version: 1.83.8
  • Provider: zai (z.ai API)
  • Model: glm-5.1
  • Client: Go openai-go library (sends list-format content by default)
  • Also reproducible with any client that sends content: [{"type": "text", "text": "..."}]

extent analysis

TL;DR

The issue can be fixed by flattening list-format content in tool and assistant messages to plain strings before forwarding to the ZAI provider.

Guidance

  • Identify if the client is sending content as a list of content parts instead of a plain string, which can cause the tool results to be silently dropped.
  • Verify if the ZAI provider's backend handles list-format content in tool messages correctly.
  • Consider normalizing tool message content to strings in OpenAIGPTConfig._transform_messages() for all providers, not just ZAI.
  • Apply the provided fix in litellm/llms/zai/chat/transformation.py to flatten list-format content.

Example

The provided code snippet in litellm/llms/zai/chat/transformation.py demonstrates how to flatten list-format content:

def _flatten_content_parts(content):
    # ...
    if isinstance(content, list):
        parts = []
        for part in content:
            if isinstance(part, dict):
                text = part.get("text")
                if text:
                    parts.append(text)
            elif isinstance(part, str):
                parts.append(part)
        return "\n".join(parts) if parts else ""
    return content

Notes

This fix may need to be applied to other OpenAI-compatible providers whose backends don't handle list-format content in tool messages. The issue is not specific to the ZAI provider, but rather a general compatibility issue with the OpenAI API.

Recommendation

Apply the workaround by flattening list-format content in tool and assistant messages to plain strings, as shown in the provided code snippet. This fix should be applied to the OpenAIGPTConfig._transform_messages() method to ensure compatibility with all OpenAI-compatible providers.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: ZAI/GLM tool results silently dropped when content is OpenAI list-format [1 pull requests, 1 participants]