langchain - 💡(How to fix) Fix Enhancement of Refine logic of chain_type refine.py [1 comments, 1 participants]

langchain2026-03-19 07:36:24

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

langchain-ai/langchain#36098•Fetched 2026-04-08 01:01:59

View on GitHub

Comments

Participants

Timeline

Reactions

Author

sonuramashishnpm

Participants

sonuramashishnpm

Timeline (top)

labeled ×3commented ×1issue_type_added ×1

Code Example

# Users can now pass batch_size as an optional parameter
chain.combine_docs(docs, batch_size=3)


### Use Case

## 📖 Use Case: Summarizing a 20-Page Legal Document

### **The Scenario**
A user wants to summarize a long legal contract that has been split into **60 small chunks** (each 500 characters) to maintain high embedding precision.

### **The Problem (Standard LangChain)**
- **API Calls:** 60 sequential calls to the LLM.
- **Time:** If each call takes 2 seconds, the user waits **120 seconds (2 minutes)**.
- **Context:** The AI only sees 500 characters at a time, often losing the connection between consecutive clauses.

### **The Solution (With Batch-Refine)**
By setting `batch_size=3`, the process is optimized:
- **API Calls:** Only **20 calls** (1 initial + 19 refine).
- **Time:** The user waits only **40 seconds**.
- **Context:** The AI sees 1,500 characters in each step, allowing it to understand full legal clauses that span across multiple chunks.

### **Code Example**

RAW_BUFFERClick to expand / collapse

Checked other resources

This is a feature request, not a bug report or usage question.
I added a clear and descriptive title that summarizes the feature request.
I used the GitHub search to find a similar feature request and didn't find it.
I checked the LangChain documentation and API reference to see if this feature already exists.
This is not related to the langchain-community package.

Package (Required)

Feature Description

🚀 Optimization: Batch-Refine Mechanism for `RefineDocumentsChain`

📝 Description

Currently, the RefineDocumentsChain processes documents in a strict 1-by-1 sequential loop. While precise, this approach is highly inefficient for large document sets, leading to high latency and fragmented context.

This PR introduces a Batch-Refine Mechanism to the combine_docs and acombine_docs methods. By grouping documents into batches (defaulting to 3), we significantly reduce the number of API calls and provide the LLM with a broader context window for each refinement step.

✨ Changes

Optimized combine_docs: Added a sliding window logic using range(1, len(docs), batch_size).
Optimized acombine_docs: Implemented the same batching logic for asynchronous execution.
Improved Context Injection: Uses format_document to join multiple documents into a single page_content block before the refine step.
Backward Compatibility: Uses kwargs.get("batch_size", 1) to ensure existing workflows remain unchanged unless explicitly configured by the user.

📊 Performance Impact

For a set of 10 documents:

Original Logic: 1 initial call + 9 refine calls = 10 total calls.
Batch Logic (size=3): 1 initial call + 3 refine calls = 4 total calls.
Result: ~60% reduction in total LLM latency and API costs.

🛠️ How to use

# Users can now pass batch_size as an optional parameter
chain.combine_docs(docs, batch_size=3)


### Use Case

## 📖 Use Case: Summarizing a 20-Page Legal Document

### **The Scenario**
A user wants to summarize a long legal contract that has been split into **60 small chunks** (each 500 characters) to maintain high embedding precision.

### **The Problem (Standard LangChain)**
- **API Calls:** 60 sequential calls to the LLM.
- **Time:** If each call takes 2 seconds, the user waits **120 seconds (2 minutes)**.
- **Context:** The AI only sees 500 characters at a time, often losing the connection between consecutive clauses.

### **The Solution (With Batch-Refine)**
By setting `batch_size=3`, the process is optimized:
- **API Calls:** Only **20 calls** (1 initial + 19 refine).
- **Time:** The user waits only **40 seconds**.
- **Context:** The AI sees 1,500 characters in each step, allowing it to understand full legal clauses that span across multiple chunks.

### **Code Example**
```python
from langchain_classic.chains import RefineDocumentsChain

# Existing setup for initial and refine chains...

chain = RefineDocumentsChain(
    initial_llm_chain=initial_llm_chain,
    refine_llm_chain=refine_llm_chain,
    document_variable_name="context",
    initial_response_name="prev_response",
)

# New: Simply pass batch_size to the call
result = chain.combine_docs(docs, batch_size=3)
print(result[0])


### Proposed Solution

## ✨ Changes
- **Optimized `combine_docs`**: Added a sliding window logic using `range(1, len(docs), batch_size)`.
- **Optimized `acombine_docs`**: Implemented the same batching logic for asynchronous execution.
- **Improved Context Injection**: Uses `format_document` to join multiple documents into a single `page_content` block before the refine step.
- **Backward Compatibility**: Uses `kwargs.get("batch_size", 1)` to ensure existing workflows remain unchanged unless explicitly configured by the user.

## 📊 Performance Impact
For a set of **10 documents**:
- **Original Logic**: 1 initial call + 9 refine calls = **10 total calls**.
- **Batch Logic (size=3)**: 1 initial call + 3 refine calls = **4 total calls**.
- **Result**: **~60% reduction in total LLM latency and API costs.**


### Alternatives Considered

# No 

### Additional Context

# No

extent analysis

Fix Plan

To implement the batch-refine mechanism for RefineDocumentsChain, follow these steps:

Update the combine_docs method to use a sliding window logic with range(1, len(docs), batch_size).
Implement the same batching logic for asynchronous execution in acombine_docs.
Use format_document to join multiple documents into a single page_content block before the refine step.
Ensure backward compatibility by using kwargs.get("batch_size", 1).

Code Example

from langchain_classic.chains import RefineDocumentsChain

class BatchRefineDocumentsChain(RefineDocumentsChain):
    def combine_docs(self, docs, batch_size=1, **kwargs):
        # Sliding window logic
        for i in range(0, len(docs), batch_size):
            batch = docs[i:i+batch_size]
            # Join multiple documents into a single page_content block
            page_content = "\n".join([self.format_document(doc) for doc in batch])
            # Refine step
            refine_input = {"context": page_content}
            refine_output = self.refine_llm_chain.run(refine_input)
            # Process refine output

    def acombine_docs(self, docs, batch_size=1, **kwargs):
        # Asynchronous execution with batching logic
        async def refine_batch(batch):
            # Join multiple documents into a single page_content block
            page_content = "\n".join([self.format_document(doc) for doc in batch])
            # Refine step
            refine_input = {"context": page_content}
            refine_output = await self.refine_llm_chain.arun(refine_input)
            # Process refine output
        # Split docs into batches and execute refine_batch asynchronously
        batches = [docs[i:i+batch_size] for i in range(0, len(docs), batch_size)]
        await asyncio.gather(*[refine_batch(batch) for batch in batches])

# Usage
chain = BatchRefineDocumentsChain(
    initial_llm_chain=initial_llm_chain,
    refine_llm_chain=refine_llm_chain,
    document_variable_name="context",
    initial_response_name="prev_response",
)
result = chain.combine_docs(docs, batch_size=3)
print(result[0])

Verification

To verify the fix, test the BatchRefineDocumentsChain with different batch sizes and document sets. Measure the reduction in total LLM latency and API costs compared to the original logic.

Extra Tips

Monitor the performance impact of the batch-refine mechanism on your specific use case.
Adjust the default batch size based on your performance requirements and LLM constraints.
Consider adding error handling and logging for the refine steps to ensure robustness.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #installation #tensor shape #autograd error #optimization

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Data

Security

Network

Code

UI/UX

Text

System

Multimedia

Protocol

API

Engineering

langchain - 💡(How to fix) Fix Enhancement of Refine logic of chain_type refine.py [1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Code Example

Checked other resources

Package (Required)

Feature Description

🚀 Optimization: Batch-Refine Mechanism for `RefineDocumentsChain`

📝 Description

✨ Changes

📊 Performance Impact

🛠️ How to use

extent analysis

Fix Plan

Code Example

Verification

Extra Tips

Still need to ship something?

TRENDING

langchain - 💡(How to fix) Fix Enhancement of Refine logic of chain_type refine.py [1 comments, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Code Example

Checked other resources

Package (Required)

Feature Description

🚀 Optimization: Batch-Refine Mechanism for RefineDocumentsChain

📝 Description

✨ Changes

📊 Performance Impact

🛠️ How to use

extent analysis

Fix Plan

Code Example

Verification

Extra Tips

Still need to ship something?

RELATED_DISCOVERY

TRENDING

🚀 Optimization: Batch-Refine Mechanism for `RefineDocumentsChain`