PR #35469: fix(text-splitters): restore lazy imports for heavy optional dependencies

• Repository: langchain-ai/langchain
• Author: gitbalaji
• State: open | merged: False
• Link: https://github.com/langchain-ai/langchain/pull/35469

Description (problem / solution / changelog)

Summary

• Moves nltk, spacy, sentence-transformers, and konlpy imports back inside class constructors/functions so they are only loaded when the respective splitter is actually instantiated
• Adds a subprocess-based regression test to verify no heavy packages are imported at langchain_text_splitters load time

Why

PR #32325 moved these optional dependency imports to module-level try/except blocks (to satisfy ruff's PLC0415 rule). Since __init__.py imports all four splitter modules, this caused import langchain_text_splitters to eagerly load all optional heavy packages, resulting in:

• A PyTorch NVML warning (UserWarning: Can't initialize NVML) on non-GPU machines
• A ~650MB memory spike on import (74MB → 736MB), vs ~50MB in 0.3.x

The fix restores the lazy import pattern with # noqa: PLC0415 to suppress the linter rule, which is the correct trade-off when a dependency has high instantiation cost.

Review notes

• The PLC0415 suppressions are intentional — these are optional heavy dependencies that should never be loaded unless the user explicitly instantiates the splitter class
• The regression test uses a subprocess for proper isolation (the test file itself imports langchain_text_splitters at the top, so sys.modules checks within the same process would not reflect a clean import state)

Fixes #35437.

AI disclaimer: This PR was developed with assistance from Claude Code (Anthropic AI).

Changed files

• libs/text-splitters/langchain_text_splitters/konlpy.py (modified, +7/-13)
• libs/text-splitters/langchain_text_splitters/nltk.py (modified, +4/-9)
• libs/text-splitters/langchain_text_splitters/sentence_transformers.py (modified, +7/-12)
• libs/text-splitters/langchain_text_splitters/spacy.py (modified, +12/-15)
• libs/text-splitters/tests/unit_tests/test_text_splitters.py (modified, +20/-0)

PR #35499: fix(text-splitters): lazy-import nltk/spacy/sentence-transformers to avoid ~700MB memory bloat

• Repository: langchain-ai/langchain
• Author: sxu75374
• State: open | merged: False
• Link: https://github.com/langchain-ai/langchain/pull/35499

Description (problem / solution / changelog)

Problem

Importing langchain_text_splitters eagerly pulls in nltk, spacy, and sentence-transformers (plus torch transitively) at module level, adding ~700 MiB RSS even when only lightweight splitters like RecursiveCharacterTextSplitter are used.

This regression was introduced in 1.0.0 when PLC0415 moved lazy imports to top-level (PR #32325).

Root Cause

__init__.py imports from .nltk, .spacy, and .sentence_transformers at module level. Each of these files does try: import heavy_lib at the top, so merely importing the package triggers the full import chain.

Fix

Move the three heavy-dep splitter imports (NLTKTextSplitter, SpacyTextSplitter, SentenceTransformersTokenTextSplitter) behind a __getattr__ lazy loader, following the exact same pattern used throughout langchain-core (e.g., langchain_core.embeddings, langchain_core.callbacks).

• __all__ is unchanged — public API is identical
• TYPE_CHECKING block preserves type-checker visibility
• globals()[attr_name] = result caches the import so subsequent accesses are free

Tests Added

tests/unit_tests/test_lazy_imports.py:

• Bare package import does not pull in heavy deps
• Lazy __getattr__ correctly resolves each class
• Unknown attributes raise AttributeError
• Lightweight splitters remain directly importable
• E2E: using RecursiveCharacterTextSplitter doesn't import heavy deps

Reproduction (from issue)

import os, psutil
def mem(): return psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024

print(f"Start: {mem():.1f} MiB")
from langchain_text_splitters import RecursiveCharacterTextSplitter
print(f"After: {mem():.1f} MiB")
# Before: 74 → 736 MiB (+662 MiB)
# After:  74 → ~80 MiB (no heavy deps loaded)

Fixes #35437

Changed files

• libs/text-splitters/langchain_text_splitters/__init__.py (modified, +33/-5)
• libs/text-splitters/tests/unit_tests/test_lazy_imports.py (added, +99/-0)