litellm - ✅(Solved) Fix [Bug]: Remote streamable-http MCP servers (e.g. GitHub MCP) fail on startup with cancel scope RuntimeError [1 pull requests, 1 participants]

litellm2026-04-28 13:19:23

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

BerriAI/litellm#26700•Fetched 2026-04-29 06:12:40

View on GitHub

Comments

Participants

Timeline

Reactions

Author

renne

Participants

renne

Timeline (top)

cross-referenced ×1

When adding a remote streamable-http MCP server to config.yaml (e.g. GitHub's hosted MCP at https://api.githubcopilot.com/mcp/), LiteLLM crashes during startup with:

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

This prevents the server from initializing MCP tool discovery and the tools are never registered.

Error Message

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

Root Cause

This is an upstream bug in the MCP Python SDK. AnyIO requires cancel scopes to be entered and exited in the same task. ClientSessionGroup attempts concurrent teardown of session exit stacks from a different task context, violating this constraint.

Upstream tracking:

Issue: modelcontextprotocol/python-sdk #915 — open, P1
Fix: modelcontextprotocol/python-sdk #2443 — open, awaiting review. Changes ClientSessionGroup teardown to sequentially await aclose() instead of concurrent closure.

Fix Action

Fixed

Fixed by PR: fix(client): unmask connection errors in ClientSessionGroup teardown (#915) (https://github.com/modelcontextprotocol/python-sdk/pull/2443)

PR fix notes

PR #2443: fix(client): unmask connection errors in ClientSessionGroup teardown (#915)

Repository: modelcontextprotocol/python-sdk
Author: VrtxOmega
State: open | merged: False
Link: https://github.com/modelcontextprotocol/python-sdk/pull/2443

Description (problem / solution / changelog)

Summary

Fixes #915 — the AnyIO cancel-scope RuntimeError that masks underlying connection failures during ClientSessionGroup teardown when a remote streamable-http MCP server is unavailable.

Downstream: also unblocks BerriAI/litellm#26700, where this masking behavior prevents LiteLLM from reporting real failures when configured against remote streamable-http servers (notably GitHub's hosted MCP at https://api.githubcopilot.com/mcp/). The LiteLLM bug explicitly tracks this PR as the upstream fix.

Motivation and Context

When ClientSessionGroup encounters a connection failure (e.g., when a remote server is unavailable), it triggers __aexit__ due to the exiting context manager. Prior to this fix, __aexit__ attempted to concurrently close all previously established session exit stacks using an anyio task group (async with anyio.create_task_group()).

Because this teardown occurred while unwinding an exception, AnyIO correctly threw RuntimeError: Attempted to exit cancel scope in a different task. This completely masked the original connection error (like ConnectError) and confused consumers — users wrapping connect_to_server in try/except reported their handlers were never reached (see the original repro in #915).

By iterating through _session_exit_stacks and sequentially awaiting aclose(), we preserve AnyIO task contexts and allow the true exception to propagate cleanly. This is consistent with the deeper root-cause analysis in #922: ClientSession.__aenter__ binds its task group's cancel scope to current_task, so concurrent teardown across multiple sessions necessarily violates that contract. Sequential teardown is the minimal fix that respects that invariant.

This PR also bundles a related fix to Accept header parsing (commit 005926c): wildcard patterns are valid per RFC 7231 §5.3.2 but were previously rejected. This is required for the GitHub-hosted MCP server connection path that motivated the LiteLLM report, so both fixes are shipped together to unblock that use case end-to-end. Happy to split into a separate PR if maintainers prefer.

How Has This Been Tested?

Added regression test test_unreachable_streamable_http_error_is_catchable in tests/client/test_session_group.py. It reproduces #915 against an unbound local port and asserts that the resulting exception chain (including BaseExceptionGroup, __cause__, and __context__) is free of the masking 'Attempted to exit cancel scope...' RuntimeError.
Verified manually with a standalone repro script that ConnectError and other arbitrary exceptions raised during connect are now caught by user code instead of being shadowed.
Existing pytest tests/client/test_session_group.py suite passes locally — no regressions in default scope teardown.

Breaking Changes

None. Users will now properly receive ExceptionGroup or ConnectError on connection failure rather than the cryptic AnyIO scope error. Code that was somehow catching the AnyIO RuntimeError directly would need to update, but that pattern is not documented anywhere and was effectively unusable in practice (the original repro in #915 demonstrates this).

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist

I have read the MCP Documentation
My code follows the repository's style guidelines
New and existing tests pass locally
I have added appropriate error handling
I have added or updated documentation as needed

Additional context

Closes #915
Unblocks BerriAI/litellm#26700 (downstream tracking issue)
Related: #922 (multi-session teardown — separate root cause, same family of cancel-scope failures; not addressed here)

Changed files

.claude/commands/review-pr.md (modified, +118/-118)
.git-blame-ignore-revs (modified, +5/-5)
.gitattribute (modified, +2/-2)
.github/actions/conformance/client.py (modified, +367/-367)
.github/actions/conformance/run-server.sh (modified, +30/-30)
.gitignore (modified, +174/-174)
AGENTS.md (modified, +140/-140)
CLAUDE.md (modified, +1/-1)
CODE_OF_CONDUCT.md (modified, +128/-128)
CONTRIBUTING.md (modified, +146/-146)
LICENSE (modified, +21/-21)
README.md (modified, +2570/-2570)
README.v2.md (modified, +2503/-2503)
RELEASE.md (modified, +13/-13)
SECURITY.md (modified, +21/-21)
docs/authorization.md (modified, +5/-5)
docs/concepts.md (modified, +13/-13)
docs/experimental/index.md (modified, +42/-42)
docs/experimental/tasks-client.md (modified, +361/-361)
docs/experimental/tasks-server.md (modified, +577/-577)
docs/experimental/tasks.md (modified, +188/-188)
docs/hooks/gen_ref_pages.py (modified, +35/-35)
docs/index.md (modified, +67/-67)
docs/installation.md (modified, +31/-31)
docs/low-level-server.md (modified, +5/-5)
docs/migration.md (modified, +1138/-1138)
docs/testing.md (modified, +77/-77)
examples/README.md (modified, +5/-5)
examples/clients/simple-auth-client/README.md (modified, +98/-98)
examples/clients/simple-auth-client/mcp_simple_auth_client/__init__.py (modified, +1/-1)
examples/clients/simple-auth-client/mcp_simple_auth_client/main.py (modified, +383/-383)
examples/clients/simple-auth-client/pyproject.toml (modified, +43/-43)
examples/clients/simple-chatbot/.python-version (modified, +1/-1)
examples/clients/simple-chatbot/README.MD (modified, +113/-113)
examples/clients/simple-chatbot/mcp_simple_chatbot/.env.example (modified, +1/-1)
examples/clients/simple-chatbot/mcp_simple_chatbot/main.py (modified, +421/-421)
examples/clients/simple-chatbot/mcp_simple_chatbot/requirements.txt (modified, +4/-4)
examples/clients/simple-chatbot/mcp_simple_chatbot/servers_config.json (modified, +12/-12)
examples/clients/simple-chatbot/pyproject.toml (modified, +47/-47)
examples/clients/simple-task-client/README.md (modified, +43/-43)
examples/clients/simple-task-client/mcp_simple_task_client/__main__.py (modified, +5/-5)
examples/clients/simple-task-client/mcp_simple_task_client/main.py (modified, +56/-56)
examples/clients/simple-task-client/pyproject.toml (modified, +43/-43)
examples/clients/simple-task-interactive-client/README.md (modified, +87/-87)
examples/clients/simple-task-interactive-client/mcp_simple_task_interactive_client/__main__.py (modified, +5/-5)
examples/clients/simple-task-interactive-client/mcp_simple_task_interactive_client/main.py (modified, +137/-137)
examples/clients/simple-task-interactive-client/pyproject.toml (modified, +43/-43)
examples/clients/sse-polling-client/README.md (modified, +30/-30)
examples/clients/sse-polling-client/mcp_sse_polling_client/__init__.py (modified, +1/-1)
examples/clients/sse-polling-client/mcp_sse_polling_client/main.py (modified, +102/-102)
examples/clients/sse-polling-client/pyproject.toml (modified, +36/-36)
examples/mcpserver/complex_inputs.py (modified, +29/-29)
examples/mcpserver/desktop.py (modified, +24/-24)
examples/mcpserver/direct_call_tool_result_return.py (modified, +22/-22)
examples/mcpserver/echo.py (modified, +28/-28)
examples/mcpserver/icons_demo.py (modified, +56/-56)
examples/mcpserver/logging_and_progress.py (modified, +31/-31)
examples/mcpserver/memory.py (modified, +324/-324)
examples/mcpserver/parameter_descriptions.py (modified, +19/-19)
examples/mcpserver/readme-quickstart.py (modified, +18/-18)
examples/mcpserver/screenshot.py (modified, +27/-27)
examples/mcpserver/simple_echo.py (modified, +12/-12)
examples/mcpserver/text_me.py (modified, +67/-67)
examples/mcpserver/unicode_example.py (modified, +59/-59)
examples/mcpserver/weather_structured.py (modified, +224/-224)
examples/servers/everything-server/README.md (modified, +42/-42)
examples/servers/everything-server/mcp_everything_server/__init__.py (modified, +3/-3)
examples/servers/everything-server/mcp_everything_server/__main__.py (modified, +6/-6)
examples/servers/everything-server/mcp_everything_server/server.py (modified, +466/-466)
examples/servers/everything-server/pyproject.toml (modified, +36/-36)
examples/servers/simple-auth/README.md (modified, +135/-135)
examples/servers/simple-auth/mcp_simple_auth/__init__.py (modified, +1/-1)
examples/servers/simple-auth/mcp_simple_auth/__main__.py (modified, +7/-7)
examples/servers/simple-auth/mcp_simple_auth/auth_server.py (modified, +183/-183)
examples/servers/simple-auth/mcp_simple_auth/legacy_as_server.py (modified, +137/-137)
examples/servers/simple-auth/mcp_simple_auth/server.py (modified, +161/-161)
examples/servers/simple-auth/mcp_simple_auth/simple_auth_provider.py (modified, +270/-270)
examples/servers/simple-auth/mcp_simple_auth/token_verifier.py (modified, +106/-106)
examples/servers/simple-auth/pyproject.toml (modified, +33/-33)
examples/servers/simple-pagination/README.md (modified, +77/-77)
examples/servers/simple-pagination/mcp_simple_pagination/__main__.py (modified, +5/-5)
examples/servers/simple-pagination/mcp_simple_pagination/server.py (modified, +176/-176)
examples/servers/simple-pagination/pyproject.toml (modified, +43/-43)
examples/servers/simple-prompt/.python-version (modified, +1/-1)
examples/servers/simple-prompt/README.md (modified, +55/-55)
examples/servers/simple-prompt/mcp_simple_prompt/__main__.py (modified, +5/-5)
examples/servers/simple-prompt/mcp_simple_prompt/server.py (modified, +98/-98)
examples/servers/simple-prompt/pyproject.toml (modified, +43/-43)
examples/servers/simple-resource/.python-version (modified, +1/-1)
examples/servers/simple-resource/README.md (modified, +48/-48)
examples/servers/simple-resource/mcp_simple_resource/__main__.py (modified, +5/-5)
examples/servers/simple-resource/mcp_simple_resource/server.py (modified, +91/-91)
examples/servers/simple-resource/pyproject.toml (modified, +43/-43)
examples/servers/simple-streamablehttp-stateless/README.md (modified, +38/-38)
examples/servers/simple-streamablehttp-stateless/mcp_simple_streamablehttp_stateless/__main__.py (modified, +7/-7)
examples/servers/simple-streamablehttp-stateless/mcp_simple_streamablehttp_stateless/server.py (modified, +116/-116)
examples/servers/simple-streamablehttp-stateless/pyproject.toml (modified, +36/-36)
examples/servers/simple-streamablehttp/README.md (modified, +51/-51)
examples/servers/simple-streamablehttp/mcp_simple_streamablehttp/__main__.py (modified, +4/-4)
examples/servers/simple-streamablehttp/mcp_simple_streamablehttp/event_store.py (modified, +93/-93)

Code Example

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

---

mcp_servers:
  github:
    url: https://api.githubcopilot.com/mcp/
    transport: sse   # also tried: http
    allow_all_keys: true
    headers:
      Authorization: "Bearer os.environ/GITHUB_PAT"

---

LiteLLM:WARNING: client.py:385 - MCP client list_tools was cancelled
LiteLLM:WARNING: mcp_server_manager.py:1610 - Task cancelled while listing tools from github

File "/usr/lib/python3.13/site-packages/mcp/server/streamable_http_manager.py", line 117, in run
    yield  # Let the application run
    ^^^^^
GeneratorExit

...

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

RAW_BUFFERClick to expand / collapse

Description

When adding a remote streamable-http MCP server to config.yaml (e.g. GitHub's hosted MCP at https://api.githubcopilot.com/mcp/), LiteLLM crashes during startup with:

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

This prevents the server from initializing MCP tool discovery and the tools are never registered.

Config

mcp_servers:
  github:
    url: https://api.githubcopilot.com/mcp/
    transport: sse   # also tried: http
    allow_all_keys: true
    headers:
      Authorization: "Bearer os.environ/GITHUB_PAT"

Error (from `docker compose logs`)

LiteLLM:WARNING: client.py:385 - MCP client list_tools was cancelled
LiteLLM:WARNING: mcp_server_manager.py:1610 - Task cancelled while listing tools from github

File "/usr/lib/python3.13/site-packages/mcp/server/streamable_http_manager.py", line 117, in run
    yield  # Let the application run
    ^^^^^
GeneratorExit

...

RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

Root Cause

Upstream tracking:

Issue: modelcontextprotocol/python-sdk #915 — open, P1
Fix: modelcontextprotocol/python-sdk #2443 — open, awaiting review. Changes ClientSessionGroup teardown to sequentially await aclose() instead of concurrent closure.

Environment

LiteLLM image: ghcr.io/berriai/litellm:main-stable
Python: 3.13
Transport tried: sse, http
Remote server: https://api.githubcopilot.com/mcp/

Request

Track upstream PR modelcontextprotocol/python-sdk #2443 and update the mcp SDK dependency once the fix is merged.

extent analysis

TL;DR

Update the mcp SDK dependency to a version that includes the fix from PR modelcontextprotocol/python-sdk #2443 to resolve the RuntimeError caused by concurrent teardown of session exit stacks.

Guidance

Track the upstream PR modelcontextprotocol/python-sdk #2443 for updates on the fix.
Once the PR is merged, update the mcp SDK dependency in your project to the latest version.
Verify that the updated dependency resolves the RuntimeError by restarting the LiteLLM server and checking the logs for any errors.
If the issue persists, check the mcp SDK version to ensure it includes the fix from PR modelcontextprotocol/python-sdk #2443.

Notes

The fix is currently awaiting review in PR modelcontextprotocol/python-sdk #2443, so it's not yet available in a released version of the mcp SDK.

Recommendation

Apply the workaround by waiting for the PR modelcontextprotocol/python-sdk #2443 to be merged and then updating the mcp SDK dependency, as this will provide a permanent fix for the issue.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #index setup #retrieval issue #search optimization #API routing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

litellm - ✅(Solved) Fix [Bug]: Remote streamable-http MCP servers (e.g. GitHub MCP) fail on startup with cancel scope RuntimeError [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Fix Action

Fixed

PR fix notes

PR #2443: fix(client): unmask connection errors in ClientSessionGroup teardown (#915)

Description (problem / solution / changelog)

Summary

Motivation and Context

How Has This Been Tested?

Breaking Changes

Types of changes

Checklist

Additional context

Changed files

Code Example

Description

Config

Error (from docker compose logs)

Root Cause

Environment

Request

extent analysis

TL;DR

Guidance

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING

Error (from `docker compose logs`)