Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: `experimental_use_latest_role_message_only` erroneously sending tool and assistant prompts to guardrail [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#23476•Fetched 2026-04-08 00:44:03
View on GitHub
Comments
0
Participants
1
Timeline
6
Reactions
0
Author
gfawcett22gfawcett22
Participants
gfawcett22gfawcett22
Timeline (top)
labeled ×3referenced ×2cross-referenced ×1

Root Cause

Example showing the setting is turned on because the latest system and first user prompt are not sent:

Fix Action

Fixed

PR fix notes

PR #23621: fix: filter tool/assistant messages from guardrail with experimental_use_latest_role_message_only

Description (problem / solution / changelog)

Summary

Fixes #23476

When experimental_use_latest_role_message_only is enabled for Bedrock guardrails, tool and assistant role messages were erroneously being sent to the guardrail. This PR ensures only user-role messages are included.

Changes

litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py

  1. _create_bedrock_input_content_request: Added explicit role check to skip non-user messages when experimental_use_latest_role_message_only is True. This provides defense-in-depth so that even if non-user messages slip through the upstream filter, they are excluded from the Bedrock API request content.

  2. async_post_call_success_hook: Fixed a fallback bug where input_filter.payload_messages or new_messages would bypass filtering when payload_messages was None or empty — falling back to sending ALL messages (including tool/assistant) to the guardrail. Now properly skips INPUT validation when no user messages are found and the flag is enabled.

  3. async_post_call_streaming_iterator_hook: Same fallback fix as above.

tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py

Added three regression tests:

  • Tool message as last message is excluded, only user message is sent
  • Assistant message as last message is excluded, only user message is sent
  • _create_bedrock_input_content_request skips non-user message content when flag is enabled

Test plan

  • Added regression test: conversation ending with tool message only sends latest user message
  • Added regression test: conversation ending with assistant message only sends latest user message
  • Added regression test: _create_bedrock_input_content_request filters non-user content
  • Existing tests continue to pass

Changed files

  • litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py (modified, +71/-36)
  • tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py (modified, +85/-0)

Code Example

guardrails:
  - guardrail_name: litellm-default-guardrail
    litellm_params:
      guardrail: bedrock
      mode: [pre_call]
      guardrailIdentifier: os.environ/GUARDRAIL_ID
      guardrailVersion: os.environ/GUARDRAIL_VERSION
      default_on: false
      aws_region_name: os.environ/AWS_REGION
      experimental_use_latest_role_message_only: true

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Example showing the setting is turned on because the latest system and first user prompt are not sent:

<img width="646" height="1034" alt="Image" src="https://github.com/user-attachments/assets/67804173-b034-4578-a545-b38f7598cf1c" />

Showing the guardrail block when the last user prompt is the offensive:

<img width="781" height="699" alt="Image" src="https://github.com/user-attachments/assets/a29d9b26-a792-4c06-9dee-73561b2c1260" />

Showing role: "tool" prompt getting send to guardrail:

<img width="713" height="689" alt="Image" src="https://github.com/user-attachments/assets/9c296d3e-d156-4edd-a005-a12d107cccf8" />

Showing role: "assistance" prompt getting sent to guardrail even though it's configured for only pre_call:

<img width="737" height="705" alt="Image" src="https://github.com/user-attachments/assets/7f2fe772-2288-4674-9f26-9a68ef311b7c" />

Steps to Reproduce

  1. Deploy a Bedrock guardrail.
  2. Add a section like the following to your config.yml ensuring experimental_use_latest_role_message_only is true:
guardrails:
  - guardrail_name: litellm-default-guardrail
    litellm_params:
      guardrail: bedrock
      mode: [pre_call]
      guardrailIdentifier: os.environ/GUARDRAIL_ID
      guardrailVersion: os.environ/GUARDRAIL_VERSION
      default_on: false
      aws_region_name: os.environ/AWS_REGION
      experimental_use_latest_role_message_only: true
  1. Send requests similar to the images in the description where the last prompts are variable roles (user, tool, assistant).

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.0

Twitter / LinkedIn details

No response

extent analysis

Fix Plan

To fix the issue, we need to modify the config.yml to correctly handle the experimental_use_latest_role_message_only setting.

  • Update the config.yml to include a conditional statement that checks the role of the last prompt and only sends it to the guardrail if it matches the expected role.
  • Add a check to ensure that the role is not user when sending the prompt to the guardrail.

Example code snippet:

guardrails:
  - guardrail_name: litellm-default-guardrail
    litellm_params:
      guardrail: bedrock
      mode: [pre_call]
      guardrailIdentifier: os.environ/GUARDRAIL_ID
      guardrailVersion: os.environ/GUARDRAIL_VERSION
      default_on: false
      aws_region_name: os.environ/AWS_REGION
      experimental_use_latest_role_message_only: true
      role_filter:
        - role: "assistance"
          mode: [pre_call]
        - role: "tool"
          mode: [pre_call]

Alternatively, you can modify the code that sends the prompt to the guardrail to include a conditional statement:

if prompt["role"] in ["assistance", "tool"] and prompt["mode"] in ["pre_call"]:
    # Send the prompt to the guardrail
    send_to_guardrail(prompt)

Verification

To verify that the fix worked, send requests with different roles and modes, and check that the guardrail only receives the expected prompts.

Extra Tips

  • Make sure to test the fix with different scenarios to ensure that it works as expected.
  • Consider adding more logging to help diagnose any future issues.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#model download#tokenizer error#prompt formatting#chain error#conversation history

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING