Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: mcp_semantic_tool_filter drops all tools when client sends MCP tool names with its own unique-ID suffix (e.g. LibreChat) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26507•Fetched 2026-04-26 05:06:33
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
dborysenkodborysenko
Participants
dborysenkodborysenko
Timeline (top)
labeled ×3cross-referenced ×1

Error Message

litellm.BadRequestError: Hosted_vllmException - {"error":{"message":"all: Invalid value for 'tool_choice': 'tool_choice' is only allowed when 'tools' are specified."}}

Root Cause

formvalue
MCP registry canonical (proxy side)fc_web_search-firecrawl_scrape
Incoming tools[].name from LibreChatfc_web_search-firecrawl_scrape_a1b2c3d4
Router top-k match returnsfc_web_search-firecrawl_scrape
_name_matches_canonical resultFalse — endswith(canonical) is False because the name ends in _a1b2c3d4
_get_tools_by_names result[]

Fix Action

Fixed

PR fix notes

PR #26533: fix(proxy): handle client-side unique-ID suffixes in MCP semantic tool filter

Description (problem / solution / changelog)

Problem

MCP clients like LibreChat append a unique-ID suffix to tool names (e.g. fc_web_search-firecrawl_scrape_a1b2c3d4) to avoid naming collisions across multiple connected MCP servers. The _name_matches_canonical method in SemanticMCPToolFilter only handled the prefix case (<alias><sep><canonical>) introduced in #26117. The symmetric suffix case (<canonical><sep><uid>) fell through all existing checks, causing the filter to drop every tool and forward tools: [] with tool_choice: auto, which strict upstream providers reject with a 400 error.

Concrete example from the issue:

formvalue
MCP registry canonical (proxy side)fc_web_search-firecrawl_scrape
Incoming tools[].name from LibreChatfc_web_search-firecrawl_scrape_a1b2c3d4
Router top-k match returnsfc_web_search-firecrawl_scrape
_name_matches_canonical resultFalse — endswith(canonical) is False because the name ends in _a1b2c3d4
_get_tools_by_names result[]

Root Cause

_name_matches_canonical only checked if client_name.endswith(canonical) (prefix case). It never checked if client_name.startswith(canonical) followed by a separator (suffix case).

Solution

Add a symmetric suffix-matching branch to _name_matches_canonical:

  • If client_name.startswith(canonical), check that the remainder is a single <sep><unique_id> segment
  • The remainder after the separator must contain no MCP_TOOL_PREFIX_SEPARATOR, preventing svc-search-extra_tool from falsely matching canonical svc-search
  • Both _ and - are accepted as separators (same as the prefix case)
  • The existing MCP_TOOL_PREFIX_SEPARATOR in canonical guard still applies — unprefixed canonicals won't trigger suffix matching

Testing

Added 7 new test cases to TestGetToolsByNames:

  • test_client_suffix_with_underscore_separator — LibreChat pattern
  • test_client_suffix_with_dash_separator — dash variant
  • test_suffix_does_not_match_another_namespaced_tool — prevents false positives
  • test_suffix_without_separator_in_canonical_does_not_match — safety guard
  • test_exact_match_preferred_over_suffixed — exact match wins
  • test_prefix_and_suffix_both_match_same_canonical — both patterns resolve
  • test_name_matches_canonical_suffix_static — direct static-method tests

All existing prefix-match tests continue to pass unchanged.

Does not break

  • The fix is purely additive — a new if branch after the existing prefix check
  • No changes to _get_tools_by_names, filter_tools, or any other method
  • The suffix-match guard (MCP_TOOL_PREFIX_SEPARATOR not in rest) is strictly more conservative than the prefix-match guard — it rejects remainders that look like another namespaced tool

Fixes #26507

Changed files

  • litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py (modified, +64/-18)
  • tests/test_litellm/proxy/_experimental/mcp_server/test_semantic_tool_filter.py (modified, +152/-0)

Code Example

litellm.BadRequestError: Hosted_vllmException -
{"error":{"message":"__all__: Invalid value for 'tool_choice':
'tool_choice' is only allowed when 'tools' are specified."}}

---

mcp_servers:
     fc_web_search:
       url: "os.environ/FIRECRAWL_MCP_URL"
   litellm_settings:
     mcp_semantic_tool_filter:
       enabled: true
       embedding_model: "your-embedding-model"
       top_k: 5
       similarity_threshold: 0.3

---

curl -s https://proxy/v1/chat/completions \
     -H "authorization: Bearer sk-..." \
     -d '{
       "model": "some-model",
       "tool_choice": "auto",
       "messages": [{"role":"user","content":"search the web for shanghai weather"}],
       "tools": [{
         "type": "function",
         "function": {
           "name": "fc_web_search-firecrawl_scrape_a1b2c3d4",
           "description": "Scrape content from a URL.",
           "parameters": {"type":"object","properties":{}}
         }
       }]
     }'

---
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

mcp_semantic_tool_filter consistently filters every tool to zero when the client appends a unique identifier suffix to MCP tool names before sending them in tools[]. This is the normal behavior of MCP clients such as LibreChat, which registers every tool as <canonical>_<unique_id> to avoid naming collisions across multiple connected MCP servers.

The failure is related to but distinct from #26078 (fixed in #26117): that fix introduced anchored suffix matching so that a client-side prefix (litellm_<canonical>) is resolved back to the canonical. The symmetric case — a client-side suffix (<canonical>_<uid>) — is not handled and falls through all existing checks.

Concrete example, one tool:

formvalue
MCP registry canonical (proxy side)fc_web_search-firecrawl_scrape
Incoming tools[].name from LibreChatfc_web_search-firecrawl_scrape_a1b2c3d4
Router top-k match returnsfc_web_search-firecrawl_scrape
_name_matches_canonical resultFalse — endswith(canonical) is False because the name ends in _a1b2c3d4
_get_tools_by_names result[]

Because SemanticToolFilterHook.async_pre_call_hook unconditionally writes data["tools"] = filtered_tools, the downstream request is shipped with tool_choice: "auto" and tools: [], which every strict OpenAI-compatible provider rejects:

litellm.BadRequestError: Hosted_vllmException -
{"error":{"message":"__all__: Invalid value for 'tool_choice':
'tool_choice' is only allowed when 'tools' are specified."}}

Expected: the filter should recognise fc_web_search-firecrawl_scrape_a1b2c3d4 as the client-side representation of canonical fc_web_search-firecrawl_scrape (the unique-ID suffix is additive, the canonical prefix up to the separator is invariant), return the matched tool object unchanged, and preserve the forwarded name so tool-call round-trips continue to work on the client side.

Steps to Reproduce

  1. proxy_config.yaml:

    mcp_servers:
  fc_web_search:
    url: "os.environ/FIRECRAWL_MCP_URL"
litellm_settings:
  mcp_semantic_tool_filter:
    enabled: true
    embedding_model: "your-embedding-model"
    top_k: 5
    similarity_threshold: 0.3

  2. Start the proxy and confirm build_router_from_mcp_registry indexes tools with canonical names (e.g. fc_web_search-firecrawl_scrape).

  3. From LibreChat (or any MCP client that appends a unique-ID suffix to tool names), call /v1/chat/completions with tool_choice: "auto" and a relevant user message. Equivalent curl — note the _<uid> suffix on every tool name:

    curl -s https://proxy/v1/chat/completions \
  -H "authorization: Bearer sk-..." \
  -d '{
    "model": "some-model",
    "tool_choice": "auto",
    "messages": [{"role":"user","content":"search the web for shanghai weather"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "fc_web_search-firecrawl_scrape_a1b2c3d4",
        "description": "Scrape content from a URL.",
        "parameters": {"type":"object","properties":{}}
      }
    }]
  }'

  4. Observe response header x-litellm-semantic-filter: 1->0 and a 400 error from the upstream provider.

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.83.13

Twitter / LinkedIn details

No response

extent analysis

TL;DR

The issue can be fixed by modifying the mcp_semantic_tool_filter to handle client-side suffixes by checking if the tool name starts with the canonical name instead of exact matching.

Guidance

  • The issue arises from the mcp_semantic_tool_filter not handling client-side suffixes correctly, leading to tools being filtered out.
  • To fix this, the _name_matches_canonical function should be modified to check if the tool name starts with the canonical name, rather than relying on an exact match or suffix matching.
  • The SemanticToolFilterHook.async_pre_call_hook function should also be updated to preserve the original tool name when forwarding the request.
  • The proxy_config.yaml file and the LibreChat client configuration do not need to be changed.

Example

def _name_matches_canonical(tool_name, canonical_name):
    # Check if tool_name starts with canonical_name
    return tool_name.startswith(canonical_name)

Notes

  • This fix assumes that the client-side suffix is always appended to the canonical name, and that the canonical name is unique.
  • Further testing may be needed to ensure that this fix does not introduce any regressions.

Recommendation

Apply workaround: Modify the _name_matches_canonical function to check if the tool name starts with the canonical name, and update the SemanticToolFilterHook.async_pre_call_hook function to preserve the original tool name. This should fix the issue without requiring any changes to the proxy_config.yaml file or the LibreChat client configuration.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#task chaining#parallel task#integration issue#index setup#retrieval issue

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: mcp_semantic_tool_filter drops all tools when client sends MCP tool names with its own unique-ID suffix (e.g. LibreChat) [1 pull requests, 1 participants]