Checked other resources

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

Feature Request

Introduce a RunnableFactChecker to verify LLM outputs against provided context in RAG pipelines.

Motivation

LLMs can generate hallucinated or unsupported information, especially in retrieval-augmented generation (RAG) workflows. A fact-checking step can improve reliability by validating generated outputs.

Proposed Solution

• Sentence-level verification using an LLM
• Confidence score computation based on supported statements
• Optional strict mode to filter unsupported sentences
• Runnable-compatible for LCEL pipelines

Example

fact_checker = RunnableFactChecker(llm=verifier_llm, strict_mode=True)

result = fact_checker.invoke({
    "context": docs,
    "answer": generated_text
})

### Use Case

I'm trying to build a Retrieval-Augmented Generation (RAG) application where LLM responses are generated based on retrieved documents.

Currently, there is no built-in mechanism in LangChain to verify whether the generated response is actually grounded in the provided context. This often leads to hallucinated or unsupported statements, especially in multi-step pipelines.

Right now, I have to manually implement post-processing logic to:
- split responses into sentences
- verify each statement using an additional LLM call
- compute a confidence score
- optionally filter unsupported content

This feature would help users by providing a reusable and composable `Runnable` that:
- verifies LLM outputs against context
- integrates directly into LCEL pipelines
- provides confidence scoring
- optionally enforces strict grounding (filtering unsupported statements)

This would be especially useful for:
- RAG pipelines
- QA systems
- enterprise applications requiring higher reliability

It would reduce repeated boilerplate and provide a standardized approach to post-generation verification.

### Proposed Solution

_No response_

### Alternatives Considered

_No response_

### Additional Context

_No response_