litellm - ✅(Solved) Fix [Bug]: Format for sending in post_call for guardrail [1 pull requests, 1 participants]

litellm2026-04-01 19:15:07

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

BerriAI/litellm#24939•Fetched 2026-04-08 02:23:37

View on GitHub

Comments

Participants

Timeline

Reactions

Author

shivamrawat1

Participants

shivamrawat1

Timeline (top)

labeled ×3renamed ×1subscribed ×1

PR fix notes

PR #25891: Guardrail post call structured messages

Repository: BerriAI/litellm
Author: J-Byron
State: open | merged: False
Link: https://github.com/BerriAI/litellm/pull/25891

Description (problem / solution / changelog)

Relevant issues

Fixes: #24939

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Type

🐛 Bug Fix

Changes

Problem

Post-call guardrails only received a flat list of text/tool_call strings — they had no visibility into the full conversation context. This made it impossible to block harmful tool calls or make informed decisions based on the complete message history.

Fix

Populate structured_messages in process_output_response with the original request messages plus the assistant turn appended. Guardrails now receive the full conversation in standard OpenAI chat format on every post_call — including tool calls, tool responses, and assistant content.

Changed files

litellm/llms/openai/chat/guardrail_translation/handler.py (modified, +22/-0)
tests/test_litellm/llms/openai/chat/guardrail_translation/test_openai_guardrail_handler.py (modified, +244/-0)

Code Example

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  }
]

---

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  }
]

---

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  },
  {
    "role": "tool",
    "tool_call_id": "call_123",
    "content": "{\"temperature\": 5, \"condition\": \"snow\"}"
  }
]

---

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  },
  {
    "role": "tool",
    "tool_call_id": "call_123",
    "content": "{\"temperature\": 5, \"condition\": \"snow\"}"
  },
  {
     "role": "assistant",
     "content": "The current temperature is 5F."
  }
]

---

RAW_BUFFERClick to expand / collapse

Check for existing issues

I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

The ideal format being sent over to guardrail in post_call:

In structured_output we should have the latest model response appended to the prior context (which should already be there for the pre_call ) with full info including any tool call and tool responses. This structured_output can stick with the standard openai chat format that you are already using there.

An example input:

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  }
]

Model may respond with a tool call and I'm hoping to see the following on the post_call guardrail so that any dangerous actions taken by the model can be blocked by guardrail at this stage.

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  }
]

after this gets to the agentic system we will see a tool response, getting back to model vendor, so the next input will be

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  },
  {
    "role": "tool",
    "tool_call_id": "call_123",
    "content": "{\"temperature\": 5, \"condition\": \"snow\"}"
  }
]

and now model will respond with a text, and here the post_call should give the following to the guardrail.

[
  {
    "role": "user",
    "content": "What is the weather in Boston?"
  },
  {
    "role": "assistant",
    "tool_call": {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Boston\"}"
      }
    }
  },
  {
    "role": "tool",
    "tool_call_id": "call_123",
    "content": "{\"temperature\": 5, \"condition\": \"snow\"}"
  },
  {
     "role": "assistant",
     "content": "The current temperature is 5F."
  }
]

Output guardrail can only see the text content from the assistant, and there is no way to block any potential harmful tool call in this case or to make a very informed block/pass decision.

Steps to Reproduce

Above

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

main

Twitter / LinkedIn details

No response

extent analysis

TL;DR

Modify the post_call guardrail to include the full conversation context, including tool calls and responses, to enable informed block/pass decisions.

Guidance

Review the current implementation of the post_call guardrail to identify why it only receives the text content from the assistant.
Modify the guardrail to accept and process the full structured_output, including tool calls and responses.
Update the guardrail logic to analyze the full conversation context and make informed decisions about blocking or passing potential harmful tool calls.
Verify that the modified guardrail can correctly handle the expected input formats, such as the examples provided in the issue description.

Notes

The solution requires modifying the post_call guardrail to handle the full conversation context, which may involve updating the API or interface between the guardrail and the agentic system.

Recommendation

Apply workaround: Modify the post_call guardrail to include the full conversation context, as this will enable the guardrail to make informed decisions about blocking or passing potential harmful tool calls.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#output truncation #response parsing #generation error #database connection #vector store

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

litellm - ✅(Solved) Fix [Bug]: Format for sending in post_call for guardrail [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

PR fix notes

PR #25891: Guardrail post call structured messages

Description (problem / solution / changelog)

Relevant issues

Pre-Submission checklist

Screenshots / Proof of Fix

Type

Changes

Problem

Fix

Changed files

Code Example

Check for existing issues

What happened?

Steps to Reproduce

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

TL;DR

Guidance

Notes

Recommendation

Still need to ship something?

TRENDING

litellm - ✅(Solved) Fix [Bug]: Format for sending in post_call for guardrail [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

PR fix notes

PR #25891: Guardrail post call structured messages

Description (problem / solution / changelog)

Relevant issues

Pre-Submission checklist

Screenshots / Proof of Fix

Type

Changes

Problem

Fix

Changed files

Code Example

Check for existing issues

What happened?

Steps to Reproduce

Relevant log output

What part of LiteLLM is this about?

What LiteLLM version are you on ?

Twitter / LinkedIn details

extent analysis

TL;DR

Guidance

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING