langchain - 💡(How to fix) Fix feat(text-splitters): add PySBDTextSplitter [1 participants]

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

Add a PySBDTextSplitter class to langchain-text-splitters that uses the pysbd (Python Sentence Boundary Disambiguation) library for sentence-level text splitting.

pysbd is a rule-based sentence boundary detector that correctly handles edge cases like abbreviations (Dr., U.S.A.), decimal numbers (3.14), and ellipsis. It supports 22 languages.

Use Case

Current splitters like CharacterTextSplitter or RecursiveCharacterTextSplitter split on character count and can break sentences mid-way. SpacyTextSplitter and NLTKTextSplitter exist but require heavy dependencies.

pysbd is lightweight, rule-based, and language-aware — making it a great alternative for users who want accurate sentence splitting without loading a full NLP model.

Proposed Solution

A new class PySBDTextSplitter extending TextSplitter base class, similar to the existing SpacyTextSplitter pattern. Supports language parameter for all 22 pysbd languages. I have already written the implementation and tests, ready to open a PR once approved.

Alternatives Considered

SpacyTextSplitter — heavier dependency, model download required. NLTKTextSplitter — less accurate on abbreviations and edge cases.

Additional Context

No response