crewai - 💡(How to fix) Fix Native RAG Verification: Adversarial Critic Agent & Groundedness Task Template [2 comments, 2 participants]

Feature Area

Agent capabilities

Is your feature request related to a an existing bug? Please link it here.

When building complex RAG (Retrieval-Augmented Generation) pipelines in CrewAI, it is common to assign an Agent the task of synthesizing a final answer based on retrieved documents. However, there is no built-in, deterministic guardrail to prevent the final Synthesizer Agent from hallucinating information that wasn't actually present in the source text.

Current offline testing frameworks (like RAGAS) or safety guardrails (like Llama Guard) don't solve the problem of live, runtime epistemic verification within a Crew.

Describe the solution you'd like

I would love to see CrewAI introduce native tools or a first-class Process pattern for Adversarial Fact-Checking.

In a custom pipeline I recently built (Pluto Pipeline), I introduced an architecture that I believe would fit perfectly into CrewAI:

The Critic Agent (Fast/Cheap Model): A dedicated agent configured strictly to challenge the generated draft against the original source documents. The Debate Task: A cyclic or conditional task where the Critic Agent identifies ungrounded sentences, and the Synthesizer Agent is forced to either provide a direct quote from the source text or retract the claim before the task is marked Complete. Positional Context Batching: A method within the Task to batch evidence to avoid the "Lost-in-the-Middle" performance degradation. It would be incredible if CrewAI offered a VerificationTask class or a native CriticAgent template that developers could plug into their Crews at the very end of their workflows to guarantee output groundedness.

Describe alternatives you've considered

I have considered external verification APIs (like VeroQ) or writing custom Python loops outside of the Crew string. However, I believe CrewAI's architecture is already perfectly suited for this! Using a smaller, cheaper model (like Llama 3 8B) as the CriticAgent to audit a larger model (like a 70B) natively within the Crew makes the pipeline highly efficient and self-correcting.

Additional context

I am highly motivated to bring this pattern to CrewAI. If the maintainers believe this aligns with the roadmap, I would love to contribute a PR with a Cookbook/Example demonstrating how to build an "Adversarial Fact-Checking Crew", or help design a native VerificationTask component!

Willingness to Contribute

Yes, I'd be happy to submit a pull request