litellm - 💡(How to fix) Fix [Feature]: Bedrock Guardrails contextual grounding via the guardrail hook (request-side qualifiers) [1 pull requests]

The Feature

Amazon Bedrock Guardrails' contextual grounding check stops hallucinations by scoring a model response against a reference source (grounding_source) and the user question (query). The ApplyGuardrail / Converse API expresses this by tagging each content block with a qualifiers array (grounding_source / query / guard_content).

The Bedrock guardrail hook (litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py) currently sends guardrail content as plain text with no qualifiers, so contextual grounding cannot be driven through the proxy guardrails: config — even though the response side is already wired (_extract_blocked_assessments / _should_raise_guardrail_blocked_exception already parse contextualGroundingPolicy and block on it).

So grounding is "half-wired": responses are handled, but requests can't carry the grounding source/query. This issue tracks completing the request side.

Motivation, pitch

RAG / summarization / QA apps routing Bedrock through LiteLLM want to block hallucinated answers at the proxy. Today they cannot, because there is no way to tell the guardrail which message text is the grounding source vs. the query.

Proposed (mirrors the existing guarded_text selective-guarding marker): let callers tag message content blocks {"type": "grounding_source", "text": ...} and {"type": "query", "text": ...}. At post_call the hook assembles one ApplyGuardrail(source="OUTPUT") carrying the grounding source + query + the model response (as guard_content). Requests without these tags produce a byte-identical payload, so existing behaviour is unchanged.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response