Issue Station
Sign In
Tools hereToolbox

langchain - ✅(Solved) Fix Bug: @lru_cache-ed async httpx client causes APIConnectionError across event loops [7 pull requests, 9 comments, 8 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
langchain-ai/langchain#35783•Fetched 2026-04-08 00:24:33
View on GitHub
Comments
9
Participants
8
Timeline
25
Reactions
0
Author
niilooyniilooy
Participants
anushkapunekaranushkapunekarindexedakkiindexedakkiKer102Ker102laniakea001laniakea001maxsnow651-devmaxsnow651-devniilooyniilooysaiprasanth-gitsaiprasanth-gitshivamtiwari3shivamtiwari3
Timeline (top)
commented ×8cross-referenced ×7referenced ×6labeled ×3

Error Message

import asyncio import threading from concurrent.futures import ThreadPoolExecutor, as_completed from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

errors = [] successes = [] lock = threading.Lock()

def worker(thread_id: int): """Each thread runs asyncio.run() which creates a fresh event loop.""" for cycle in range(5): async def call(): return await llm.ainvoke(f"Reply with only: t{thread_id}c{cycle}")

    try:
        result = asyncio.run(call())
        with lock:
            successes.append((thread_id, cycle))
    except Exception as e:
        with lock:
            errors.append((thread_id, cycle, e))
            print(f"Thread {thread_id} cycle {cycle}: {type(e).__name__}: {e}")

with ThreadPoolExecutor(max_workers=8) as pool: futures = [pool.submit(worker, i) for i in range(8)] for f in as_completed(futures): f.result()

print(f"\nTotal: {len(successes) + len(errors)}, OK: {len(successes)}, FAIL: {len(errors)}")

Root Cause

# langchain_openai/chat_models/_client_utils.py

@lru_cache  # ← process-global, not event-loop-aware
def _cached_async_httpx_client(base_url, timeout):
    return _build_async_httpx_client(base_url, timeout)

def _get_default_async_httpx_client(base_url, timeout):
    try:
        hash(timeout)
    except TypeError:
        return _build_async_httpx_client(base_url, timeout)
    else:
        return _cached_async_httpx_client(base_url, timeout)  # ← shared across loops

The cache key is (base_url, timeout) but should also include the event loop identity to prevent cross-loop sharing. The same pattern exists in langchain-anthropic.

Fix Action

Workaround

Pass an explicit http_async_client to bypass the cache entirely:

import httpx
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    http_async_client=httpx.AsyncClient(),
    http_client=httpx.Client(),
)

Or implement a loop-isolated cache:

import asyncio, httpx

_async_client_cache = {}

def get_loop_isolated_async_client(**kwargs):
    loop_id = id(asyncio.get_running_loop())
    if loop_id not in _async_client_cache:
        _async_client_cache[loop_id] = httpx.AsyncClient(**kwargs)
    return _async_client_cache[loop_id]

PR fix notes

PR #1: fix(langchain-openai): use per-event-loop cache for async httpx client to prevent cross-loop errors

Description (problem / solution / changelog)

Summary

Fixes #35783

The _cached_async_httpx_client function in _client_utils.py was decorated with @lru_cache, making it process-global and shared across all event loops. This caused RuntimeError: Event loop is closed errors when ainvoke() is called from multiple threads (each with their own asyncio.run() event loop), sequential asyncio.run() calls, or frameworks like Celery that spin up multiple event loops.

Root Cause

httpx.AsyncClient connections are bound to the event loop they were created on. When a thread's asyncio.run() call completes, it closes the event loop. The next call from a different event loop would get the same cached AsyncClient, which tries to reuse or close connections tied to the now-dead loop, causing the error.

Fix

Replaced the process-global @lru_cache on _cached_async_httpx_client with a WeakValueDictionary that caches clients per event loop using the loop's id() as part of the key. This means:

  • Each event loop gets its own AsyncClient instance — no more cross-loop sharing
  • WeakValueDictionary automatically cleans up entries when a client is garbage collected, preventing memory leaks
  • Sync clients (_cached_sync_httpx_client) are unaffected since they don't have this problem

Testing

The reproduction script from the issue (multi-threaded test with 8 threads × 5 cycles each) should now complete without any APIConnectionError failures.

Changed files

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +60/-9)

PR #35794: fix(langchain-openai): use per-event-loop cache for async httpx client to prevent cross-loop errors

Description (problem / solution / changelog)

Summary

Fixes #35783

The _cached_async_httpx_client function in _client_utils.py was decorated with @lru_cache, making it process-global and shared across all event loops. This caused RuntimeError: Event loop is closed errors when ainvoke() is called from multiple threads (each with their own asyncio.run() event loop), sequential asyncio.run() calls, or frameworks like Celery that spin up multiple event loops.

Root Cause

httpx.AsyncClient connections are bound to the event loop they were created on. When a thread's asyncio.run() call completes, it closes the event loop. The next call from a different event loop would get the same cached AsyncClient, which tries to reuse or close connections tied to the now-dead loop, causing the error.

Fix

Replaced the process-global @lru_cache on _cached_async_httpx_client with a WeakValueDictionary that caches clients per event loop using the loop's id() as part of the key. This means:

  • Each event loop gets its own AsyncClient instance - no more cross-loop sharing
  • WeakValueDictionary automatically cleans up entries when a client is garbage collected, preventing memory leaks
  • Sync clients (_cached_sync_httpx_client) are unaffected since they don't have this problem

Testing

The reproduction script from the issue (multi-threaded test with 8 threads x 5 cycles each) should now complete without any APIConnectionError failures.

Changed files

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +44/-7)

PR #35816: fix: include event loop ID in httpx AsyncClient cache key

Description (problem / solution / changelog)

Summary

  • Fixes #35783: @lru_cache on _cached_async_httpx_client caches by (base_url, timeout), but an httpx.AsyncClient is bound to the event loop on which it was created. When multiple threads each call asyncio.run() (which creates a fresh event loop), the stale cached client is reused on a different loop, causing RuntimeError / connection errors deep in httpcore/anyio.
  • Adds id(asyncio.get_event_loop()) as an additional cache key so each event loop gets its own AsyncClient instance, while still sharing within the same loop.

Changes

libs/partners/openai/langchain_openai/chat_models/_client_utils.py

  1. _cached_async_httpx_client: added _event_loop_id: int = 0 parameter (used only as a cache-key discriminator).
  2. _get_default_async_httpx_client: passes id(asyncio.get_event_loop()) as the _event_loop_id kwarg when calling the cached function.

The sync client is unaffected since httpx.Client does not have event-loop affinity.

Test plan

  • Run the reproduction script from #35783 (multi-threaded asyncio.run() calls) and verify no cross-event-loop errors
  • Verify single-threaded async usage still benefits from client caching (same loop = same client)
  • Existing unit tests pass

🤖 Generated with Claude Code

Changed files

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +3/-2)

PR #35817: fix: make async httpx client cache event-loop-aware

Description (problem / solution / changelog)

Summary

  • Replace @lru_cache on _cached_async_httpx_client with a thread-safe dict keyed on (base_url, timeout, event_loop_id), preventing cross-event-loop reuse of httpx.AsyncClient instances
  • Apply the same fix to both langchain-openai and langchain-anthropic partners
  • Evict stale cache entries when their associated event loop has closed

Problem

_cached_async_httpx_client() uses @lru_cache which is not event-loop-aware. When ainvoke() is called from multiple threads (each running asyncio.run() with its own event loop), the cached AsyncClient tries to reuse connections bound to a closed loop, causing:

RuntimeError: Event loop is closed
openai.APIConnectionError: Connection error.

This affects multi-threaded async evaluation, sequential asyncio.run() calls, and framework scheduling (Celery, etc.).

Fix

Replace the process-global @lru_cache with a threading.Lock-protected dict that includes id(asyncio.get_running_loop()) in the cache key. Each event loop gets its own AsyncClient. Stale entries (where the client is closed) are evicted on each lookup.

The sync client cache (@lru_cache) is unchanged since sync clients are not bound to an event loop.

Files Changed

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py
  • libs/partners/anthropic/langchain_anthropic/_client_utils.py

Test Plan

  • Verify the reproduction script from #35783 no longer raises APIConnectionError
  • Verify single-threaded async usage still benefits from caching (same loop = same client)
  • Verify unhashable timeout values still bypass the cache

Fixes #35783

Changed files

  • .devcontainer/README.md (removed, +0/-49)
  • .devcontainer/devcontainer.json (removed, +0/-58)
  • .devcontainer/docker-compose.yaml (removed, +0/-13)
  • .dockerignore (removed, +0/-34)
  • .editorconfig (removed, +0/-52)
  • .gitattributes (removed, +0/-3)
  • .github/CODEOWNERS (removed, +0/-3)
  • .github/ISSUE_TEMPLATE/bug-report.yml (removed, +0/-151)
  • .github/ISSUE_TEMPLATE/config.yml (removed, +0/-15)
  • .github/ISSUE_TEMPLATE/feature-request.yml (removed, +0/-151)
  • .github/ISSUE_TEMPLATE/privileged.yml (removed, +0/-49)
  • .github/ISSUE_TEMPLATE/task.yml (removed, +0/-120)
  • .github/PULL_REQUEST_TEMPLATE.md (removed, +0/-41)
  • .github/actions/uv_setup/action.yml (removed, +0/-39)
  • .github/dependabot.yml (removed, +0/-95)
  • .github/images/logo-dark.svg (removed, +0/-6)
  • .github/images/logo-light.svg (removed, +0/-6)
  • .github/pr-file-labeler.yml (removed, +0/-128)
  • .github/scripts/check_diff.py (removed, +0/-333)
  • .github/scripts/check_prerelease_dependencies.py (removed, +0/-36)
  • .github/scripts/get_min_versions.py (removed, +0/-199)
  • .github/tools/git-restore-mtime (removed, +0/-756)
  • .github/workflows/_compile_integration_test.yml (removed, +0/-65)
  • .github/workflows/_lint.yml (removed, +0/-81)
  • .github/workflows/_release.yml (removed, +0/-629)
  • .github/workflows/_test.yml (removed, +0/-85)
  • .github/workflows/_test_pydantic.yml (removed, +0/-73)
  • .github/workflows/auto-label-by-package.yml (removed, +0/-109)
  • .github/workflows/check_agents_sync.yml (removed, +0/-42)
  • .github/workflows/check_core_versions.yml (removed, +0/-67)
  • .github/workflows/check_diffs.yml (removed, +0/-267)
  • .github/workflows/integration_tests.yml (removed, +0/-271)
  • .github/workflows/pr_labeler_file.yml (removed, +0/-31)
  • .github/workflows/pr_labeler_title.yml (removed, +0/-47)
  • .github/workflows/pr_lint.yml (removed, +0/-116)
  • .github/workflows/refresh_model_profiles.yml (removed, +0/-93)
  • .github/workflows/tag-external-contributions.yml (removed, +0/-151)
  • .github/workflows/v03_api_doc_build.yml (removed, +0/-167)
  • .gitignore (removed, +0/-168)
  • .markdownlint.json (removed, +0/-14)
  • .mcp.json (removed, +0/-8)
  • .pre-commit-config.yaml (removed, +0/-125)
  • .vscode/extensions.json (removed, +0/-19)
  • .vscode/settings.json (removed, +0/-78)
  • AGENTS.md (removed, +0/-253)
  • CITATION.cff (removed, +0/-8)
  • CLAUDE.md (removed, +0/-253)
  • LICENSE (removed, +0/-21)
  • README.md (removed, +0/-76)
  • libs/Makefile (removed, +0/-20)
  • libs/README.md (removed, +0/-35)
  • libs/core/Makefile (removed, +0/-85)
  • libs/core/README.md (removed, +0/-47)
  • libs/core/extended_testing_deps.txt (removed, +0/-1)
  • libs/core/langchain_core/__init__.py (removed, +0/-20)
  • libs/core/langchain_core/_api/__init__.py (removed, +0/-87)
  • libs/core/langchain_core/_api/beta_decorator.py (removed, +0/-253)
  • libs/core/langchain_core/_api/deprecation.py (removed, +0/-603)
  • libs/core/langchain_core/_api/internal.py (removed, +0/-23)
  • libs/core/langchain_core/_api/path.py (removed, +0/-50)
  • libs/core/langchain_core/_import_utils.py (removed, +0/-41)
  • libs/core/langchain_core/_security/__init__.py (removed, +0/-0)
  • libs/core/langchain_core/_security/_ssrf_protection.py (removed, +0/-361)
  • libs/core/langchain_core/agents.py (removed, +0/-256)
  • libs/core/langchain_core/caches.py (removed, +0/-272)
  • libs/core/langchain_core/callbacks/__init__.py (removed, +0/-132)
  • libs/core/langchain_core/callbacks/base.py (removed, +0/-1157)
  • libs/core/langchain_core/callbacks/file.py (removed, +0/-267)
  • libs/core/langchain_core/callbacks/manager.py (removed, +0/-2697)
  • libs/core/langchain_core/callbacks/stdout.py (removed, +0/-123)
  • libs/core/langchain_core/callbacks/streaming_stdout.py (removed, +0/-152)
  • libs/core/langchain_core/callbacks/usage.py (removed, +0/-149)
  • libs/core/langchain_core/chat_history.py (removed, +0/-246)
  • libs/core/langchain_core/chat_loaders.py (removed, +0/-26)
  • libs/core/langchain_core/chat_sessions.py (removed, +0/-19)
  • libs/core/langchain_core/document_loaders/__init__.py (removed, +0/-39)
  • libs/core/langchain_core/document_loaders/base.py (removed, +0/-155)
  • libs/core/langchain_core/document_loaders/blob_loaders.py (removed, +0/-38)
  • libs/core/langchain_core/document_loaders/langsmith.py (removed, +0/-143)
  • libs/core/langchain_core/documents/__init__.py (removed, +0/-55)
  • libs/core/langchain_core/documents/base.py (removed, +0/-347)
  • libs/core/langchain_core/documents/compressor.py (removed, +0/-74)
  • libs/core/langchain_core/documents/transformers.py (removed, +0/-79)
  • libs/core/langchain_core/embeddings/__init__.py (removed, +0/-31)
  • libs/core/langchain_core/embeddings/embeddings.py (removed, +0/-78)
  • libs/core/langchain_core/embeddings/fake.py (removed, +0/-129)
  • libs/core/langchain_core/env.py (removed, +0/-22)
  • libs/core/langchain_core/example_selectors/__init__.py (removed, +0/-47)
  • libs/core/langchain_core/example_selectors/base.py (removed, +0/-58)
  • libs/core/langchain_core/example_selectors/length_based.py (removed, +0/-128)
  • libs/core/langchain_core/example_selectors/semantic_similarity.py (removed, +0/-358)
  • libs/core/langchain_core/exceptions.py (removed, +0/-111)
  • libs/core/langchain_core/globals.py (removed, +0/-72)
  • libs/core/langchain_core/indexing/__init__.py (removed, +0/-53)
  • libs/core/langchain_core/indexing/api.py (removed, +0/-948)
  • libs/core/langchain_core/indexing/base.py (removed, +0/-661)
  • libs/core/langchain_core/indexing/in_memory.py (removed, +0/-104)
  • libs/core/langchain_core/language_models/__init__.py (removed, +0/-116)
  • libs/core/langchain_core/language_models/_utils.py (removed, +0/-327)
  • libs/core/langchain_core/language_models/base.py (removed, +0/-373)

PR #35830: fix: use loop-aware cache for async httpx clients to prevent cross-event-loop crashes

Description (problem / solution / changelog)

Description

Fixes #35783 — @lru_cache on _cached_async_httpx_client() causes APIConnectionError / RuntimeError("Event loop is closed") when the cached AsyncClient is reused across different event loops.

Root Cause

_cached_async_httpx_client() was decorated with @lru_cache keyed by (base_url, timeout), but ignored event loop identity. When a cached AsyncClient was created in one event loop and later accessed from a different loop (common in multi-threaded applications calling asyncio.run()), the underlying httpx transport was bound to the original (now-closed) loop.

The _AsyncHttpxClientWrapper.__del__ method attempts asyncio.get_running_loop().create_task(self.aclose()) which silently fails, leaving the client in a broken state for the new loop.

Fix

Replaced @lru_cache with a manual dict-based cache that includes id(asyncio.get_running_loop()) as part of the cache key:

key = (base_url, timeout, id(asyncio.get_running_loop()))

The implementation also:

  • Returns a fresh (uncached) client if no event loop is running
  • Checks client.is_closed before returning cached entries
  • Cleans up stale entries for dead loops on each cache miss
  • Applied the same fix to langchain-anthropic which had the identical bug

Files Changed

FileChange
libs/partners/openai/langchain_openai/chat_models/_client_utils.pyLoop-aware async client cache
libs/partners/anthropic/langchain_anthropic/_client_utils.pySame fix for Anthropic partner

Reproduction

import asyncio
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

# First call creates + caches the AsyncClient for loop A
asyncio.run(llm.ainvoke("Hello"))

# Second call reuses the cached client, but loop A is now dead
asyncio.run(llm.ainvoke("Hello"))  # APIConnectionError before fix

Testing

The fix maintains backward compatibility — the sync @lru_cache is unchanged, and the async cache now correctly handles:

  • Same loop reuse (cache hit, same as before)
  • Cross-loop reuse (cache miss, creates new client)
  • No running loop (returns fresh client, no caching)

Changed files

  • libs/partners/anthropic/langchain_anthropic/_client_utils.py (modified, +37/-3)
  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +30/-4)

PR #35847: fix(openai): use loop-aware proxy to prevent cross-loop APIConnectionError (fixes #35783)

Description (problem / solution / changelog)

Summary

Fixes #35783.

_get_default_async_httpx_client returned a process-global @lru_cache'd httpx.AsyncClient. ChatOpenAI stores this client at __init__ time. When a second asyncio.run() call uses the same ChatOpenAI instance (or two instances with the same params), requests go through a client whose connection pool is bound to the now-closed first loop, raising APIConnectionError.

Root Cause

_cached_async_httpx_client used @lru_cache, returning a single _AsyncHttpxClientWrapper for a given (base_url, timeout). httpx.AsyncClient connection pools use asyncio.Lock primitives bound to the event loop at creation time. Sharing one client across loops causes RuntimeError: Event loop is closed inside httpcore.

The bug is latent in ChatOpenAI.__init__ (line ~1085 of base.py): _get_default_async_httpx_client is called once and the result is passed to openai.AsyncOpenAI(http_client=...), which stores it permanently. Fixing the caching in _client_utils.py alone is insufficient — the stale client is embedded at init time.

Solution

Introduce _LoopAwareAsyncHttpxClientWrapper, a proxy that:

  • Subclasses openai.DefaultAsyncHttpxClient (compatible type for the openai SDK's http_client parameter)
  • Is itself @lru_cache'd — one proxy per (base_url, timeout), so ChatOpenAI.__init__ cost is unchanged (no performance regression)
  • Overrides send() to delegate to a per-loop inner _AsyncHttpxClientWrapper stored in a weakref.WeakKeyDictionary[loop → client]
  • When a new event loop first calls send(), it gets a fresh inner client (no dead-loop connections)
  • When a loop is GC'd, WeakKeyDictionary removes the entry automatically

_get_default_async_httpx_client now returns this proxy (for hashable timeouts) instead of the raw cached client.

Testing

  • Added tests/unit_tests/chat_models/test_client_utils.py with 11 tests:
    • Same loop reuses the same inner client
    • Different loops get isolated inner clients
    • Multi-threaded loops get isolated inner clients
    • GC'd loop removes WeakKeyDictionary entry
    • send() delegates to the inner client
    • aclose() closes only the current loop's inner client
    • @lru_cache on the outer proxy (same params → same proxy)
    • Different params → different proxies
    • Hashable timeout returns _LoopAwareAsyncHttpxClientWrapper
    • Unhashable timeout (httpx.Timeout) falls back to plain wrapper
    • Same hashable params share the cached proxy instance
  • All 11 tests pass; no existing tests broken

Checklist

  • Fixes the root cause (stale loop-bound connection pool), not just the symptom
  • New tests cover the exact scenario from the issue
  • No performance regression: outer proxy is @lru_cache'd, __init__ cost unchanged
  • All existing unit tests unaffected
  • No unrelated changes
  • Code style matches project conventions
  • Ruff lint passes

AI disclaimer: This PR was developed with the assistance of an AI coding assistant (Claude Code). All logic was reviewed and verified through unit tests.

Changed files

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +59/-2)
  • libs/partners/openai/tests/unit_tests/chat_models/test_client_utils.py (added, +121/-0)

PR #35888: fix(openai): use thread-local cache for async httpx client to prevent cross-loop reuse

Description (problem / solution / changelog)

Summary

_get_default_async_httpx_client() used @lru_cache (process-global) to share a single AsyncClient across all ChatOpenAI instances with the same (base_url, timeout). This is unsafe in multi-threaded environments where each thread runs its own event loop via asyncio.run().

Failure scenario

Thread-A: asyncio.run(llm.ainvoke(...))  →  loop-1 opens HTTP connections
asyncio.run() exits  →  loop-1 is CLOSED
Thread-B: asyncio.run(llm.ainvoke(...))  →  gets the SAME cached client
                                         →  tries to use connections bound to dead loop-1
                                         →  RuntimeError: Event loop is closed
                                         →  openai.APIConnectionError: Connection error.

This affects:

  • Multi-threaded async evaluation (each thread calls asyncio.run())
  • Sequential asyncio.run() calls (each creates and then closes a fresh loop)
  • Celery async workers, pytest-asyncio test suites, etc.

Fix

Replace @lru_cache with threading.local() for the async client. Each thread gets its own AsyncClient that lives within the event loop created for that thread.

# Before (broken — process-global, not event-loop-aware)
@lru_cache
def _cached_async_httpx_client(base_url, timeout):
    return _build_async_httpx_client(base_url, timeout)

# After (correct — one client per thread = one client per event loop)
_thread_local_async_clients = threading.local()

def _get_default_async_httpx_client(base_url, timeout):
    ...
    if not hasattr(_thread_local_async_clients, "cache"):
        _thread_local_async_clients.cache = {}
    if cache_key not in _thread_local_async_clients.cache:
        _thread_local_async_clients.cache[cache_key] = _build_async_httpx_client(...)
    return _thread_local_async_clients.cache[cache_key]

The sync client (_cached_sync_httpx_client) is not changed — sync httpx.Client is not bound to an event loop so it can safely remain process-global.

Repro

import asyncio, threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

errors = []
lock = threading.Lock()

def worker(thread_id):
    for cycle in range(3):
        try:
            asyncio.run(llm.ainvoke(f"Reply: t{thread_id}c{cycle}"))
        except Exception as e:
            with lock:
                errors.append(e)

with ThreadPoolExecutor(max_workers=4) as pool:
    list(pool.map(worker, range(4)))

# Before fix: errors contains APIConnectionError (Event loop is closed)
# After fix:  errors is empty
assert not errors

Fixes #35783

Changed files

  • libs/partners/openai/langchain_openai/chat_models/_client_utils.py (modified, +35/-10)

Code Example

import asyncio
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

errors = []
successes = []
lock = threading.Lock()

def worker(thread_id: int):
    """Each thread runs asyncio.run() which creates a fresh event loop."""
    for cycle in range(5):
        async def call():
            return await llm.ainvoke(f"Reply with only: t{thread_id}c{cycle}")

        try:
            result = asyncio.run(call())
            with lock:
                successes.append((thread_id, cycle))
        except Exception as e:
            with lock:
                errors.append((thread_id, cycle, e))
                print(f"Thread {thread_id} cycle {cycle}: {type(e).__name__}: {e}")

with ThreadPoolExecutor(max_workers=8) as pool:
    futures = [pool.submit(worker, i) for i in range(8)]
    for f in as_completed(futures):
        f.result()

print(f"\nTotal: {len(successes) + len(errors)}, OK: {len(successes)}, FAIL: {len(errors)}")

---

Traceback (most recent call last):
  File ".../openai/_base_client.py", line 1604, in request
    response = await self._client.send(...)
  File ".../httpx/_client.py", line 1629, in send
    response = await self._send_handling_auth(...)
  ...
  File ".../httpcore/_async/http11.py", line 135, in handle_async_request
    await self._response_closed()
  File ".../httpcore/_async/http11.py", line 250, in _response_closed
    await self.aclose()
  File ".../httpcore/_async/http11.py", line 258, in aclose
    await self._network_stream.aclose()
  File ".../httpcore/_backends/anyio.py", line 53, in aclose
    await self._stream.aclose()
  File ".../anyio/streams/tls.py", line 241, in aclose
    await self.transport_stream.aclose()
  File ".../anyio/_backends/_asyncio.py", line 1352, in aclose
    self._transport.close()
  File ".../asyncio/selector_events.py", line 875, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File ".../asyncio/base_events.py", line 799, in call_soon
    self._check_closed()
  File ".../asyncio/base_events.py", line 545, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

The above exception was the direct cause of the following exception:

openai.APIConnectionError: Connection error.

---

# langchain_openai/chat_models/_client_utils.py

@lru_cache  # ← process-global, not event-loop-aware
def _cached_async_httpx_client(base_url, timeout):
    return _build_async_httpx_client(base_url, timeout)

def _get_default_async_httpx_client(base_url, timeout):
    try:
        hash(timeout)
    except TypeError:
        return _build_async_httpx_client(base_url, timeout)
    else:
        return _cached_async_httpx_client(base_url, timeout)  # ← shared across loops

---

import httpx
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    http_async_client=httpx.AsyncClient(),
    http_client=httpx.Client(),
)

---

import asyncio, httpx

_async_client_cache = {}

def get_loop_isolated_async_client(**kwargs):
    loop_id = id(asyncio.get_running_loop())
    if loop_id not in _async_client_cache:
        _async_client_cache[loop_id] = httpx.AsyncClient(**kwargs)
    return _async_client_cache[loop_id]
RAW_BUFFERClick to expand / collapse

Checked other resources

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

import asyncio
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

errors = []
successes = []
lock = threading.Lock()

def worker(thread_id: int):
    """Each thread runs asyncio.run() which creates a fresh event loop."""
    for cycle in range(5):
        async def call():
            return await llm.ainvoke(f"Reply with only: t{thread_id}c{cycle}")

        try:
            result = asyncio.run(call())
            with lock:
                successes.append((thread_id, cycle))
        except Exception as e:
            with lock:
                errors.append((thread_id, cycle, e))
                print(f"Thread {thread_id} cycle {cycle}: {type(e).__name__}: {e}")

with ThreadPoolExecutor(max_workers=8) as pool:
    futures = [pool.submit(worker, i) for i in range(8)]
    for f in as_completed(futures):
        f.result()

print(f"\nTotal: {len(successes) + len(errors)}, OK: {len(successes)}, FAIL: {len(errors)}")

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File ".../openai/_base_client.py", line 1604, in request
    response = await self._client.send(...)
  File ".../httpx/_client.py", line 1629, in send
    response = await self._send_handling_auth(...)
  ...
  File ".../httpcore/_async/http11.py", line 135, in handle_async_request
    await self._response_closed()
  File ".../httpcore/_async/http11.py", line 250, in _response_closed
    await self.aclose()
  File ".../httpcore/_async/http11.py", line 258, in aclose
    await self._network_stream.aclose()
  File ".../httpcore/_backends/anyio.py", line 53, in aclose
    await self._stream.aclose()
  File ".../anyio/streams/tls.py", line 241, in aclose
    await self.transport_stream.aclose()
  File ".../anyio/_backends/_asyncio.py", line 1352, in aclose
    self._transport.close()
  File ".../asyncio/selector_events.py", line 875, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File ".../asyncio/base_events.py", line 799, in call_soon
    self._check_closed()
  File ".../asyncio/base_events.py", line 545, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

The above exception was the direct cause of the following exception:

openai.APIConnectionError: Connection error.

Description

The problem

_get_default_async_httpx_client() in langchain_openai/chat_models/_client_utils.py uses @lru_cache to share a single httpx.AsyncClient across all ChatOpenAI instances with the same (base_url, timeout). This is unsafe when ainvoke() is called from multiple event loops — which happens in:

  • Multi-threaded async evaluation (each thread runs asyncio.run(), creating a new loop)
  • Sequential asyncio.run() calls (each call creates and then closes a new loop)
  • Framework scheduling (e.g. Celery async workers, multi-process evaluation)

The cached AsyncClient opens HTTP connections that are bound to the event loop where the request was first made. When asyncio.run() finishes, that loop is closed. The next call from a different loop gets the same cached client, which tries to reuse (or close) connections from the dead loop, causing RuntimeError: Event loop is closed.

Root cause

# langchain_openai/chat_models/_client_utils.py

@lru_cache  # ← process-global, not event-loop-aware
def _cached_async_httpx_client(base_url, timeout):
    return _build_async_httpx_client(base_url, timeout)

def _get_default_async_httpx_client(base_url, timeout):
    try:
        hash(timeout)
    except TypeError:
        return _build_async_httpx_client(base_url, timeout)
    else:
        return _cached_async_httpx_client(base_url, timeout)  # ← shared across loops

The cache key is (base_url, timeout) but should also include the event loop identity to prevent cross-loop sharing. The same pattern exists in langchain-anthropic.

Workaround

Pass an explicit http_async_client to bypass the cache entirely:

import httpx
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    http_async_client=httpx.AsyncClient(),
    http_client=httpx.Client(),
)

Or implement a loop-isolated cache:

import asyncio, httpx

_async_client_cache = {}

def get_loop_isolated_async_client(**kwargs):
    loop_id = id(asyncio.get_running_loop())
    if loop_id not in _async_client_cache:
        _async_client_cache[loop_id] = httpx.AsyncClient(**kwargs)
    return _async_client_cache[loop_id]

System Info

OS: Darwin OS Version: Darwin Kernel Version 25.3.0 Python Version: 3.12.11

Package Information

langchain_core: 1.2.18 langchain_openai: 1.1.11 langsmith: 0.4.49 openai: 2.26.0 httpx: 0.28.1 httpcore: 1.0.9 anyio: 4.12.0

extent analysis

Problem Summary

The problem is a RuntimeError: Event loop is closed when using langchain_openai in a multi-threaded async evaluation environment.

Root Cause Analysis

The root cause is a shared cache of httpx.AsyncClient instances across all event loops, which leads to connections being bound to the wrong event loop when asyncio.run() finishes.

Fix Plan

To fix this issue, you can either:

1. Pass an explicit http_async_client to bypass the cache entirely

import httpx
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    http_async_client=httpx.AsyncClient(),
    http_client=httpx.Client(),
)

2. Implement a loop-isolated cache

import asyncio, httpx

_async_client_cache = {}

def get_loop_isolated_async_client(**kwargs):
    loop_id = id(asyncio.get_running_loop())
    if loop_id not in _async_client_cache:
        _async_client_cache[loop_id] = httpx.AsyncClient(**kwargs)
    return _async_client_cache[loop_id]

Verification

To verify that the fix worked, you can run the same reproduction steps as before and check that the RuntimeError: Event loop is closed exception is no longer raised.

Extra Tips

To prevent regressions, make sure to test your code in a multi-threaded async evaluation environment to ensure that the fix does not introduce any new issues. Additionally, consider opening a pull request to update the langchain_openai library to include the loop-isolated cache fix.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error#runtime error#dependency conflict#environment setup

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

langchain - ✅(Solved) Fix Bug: @lru_cache-ed async httpx client causes APIConnectionError across event loops [7 pull requests, 9 comments, 8 participants]