Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When litellm_settings.mcp_semantic_tool_filter.enabled: true is set, the proxy silently removes every non-MCP tool from the tools array of a /chat/completions (or /responses) request, even when the caller never referenced MCP at all. Concretely, if a client sends a request with a plain OpenAI-format tool definition (a "native" tool the client implements locally, e.g. weather_lookup), the semantic filter runs, decides that none of its (MCP-only) router canonicals match, and writes data["tools"] = [] before the request is forwarded to the upstream LLM. This has two visible failure modes depending on the caller:

1. Tool call never happens. The model receives no tools, so it answers in prose or apologizes that it cannot call the tool — even though the client did send it.
2. BadRequestError from the upstream provider. If the caller also set tool_choice (any non-"none" value), upstream vLLM / OpenAI-compatible backends reject the mutated request with Invalid value for 'tool_choice': 'tool_choice' is only allowed when 'tools' are specified. This is how I originally noticed the bug in OpenCode.

Expected: native (non-MCP) tools pass through the filter unchanged. The semantic filter should only operate on tools that originate from MCP servers the proxy manages.

Actual: native tools are dropped along with any non-matching MCP tools.

Root cause (tracing)

• SemanticToolFilterHook.async_pre_call_hook (litellm/proxy/hooks/mcp_semantic_filter/hook.py) passes the entire data["tools"] list (MCP references expanded + any native tools mixed in) as available_tools to self.filter.filter_tools(...).
• SemanticMCPToolFilter.filter_tools (litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py) queries self.tool_router, which was built by build_router_from_mcp_registry() and seeded only from global_mcp_server_manager. Its output vocabulary is the set of MCP canonical tool names. Native tools have no route.
• _get_tools_by_names(matched_canonicals, available_tools) then keeps only tools whose name resolves to one of those MCP canonicals (strict equality on main). Native tools cannot resolve → silently dropped. The hook then unconditionally overwrites:

data["tools"] = filtered_tools

so the caller has no recovery path.

Steps to Reproduce

Steps to Reproduce

Run a LiteLLM proxy with a config that enables the semantic filter:

# config.yaml
model_list:
  - model_name: my-model
    litellm_params:
      model: hosted_vllm/my-model
      api_base: https://my-vllm.example.com/v1
litellm_settings:
  mcp_semantic_tool_filter:
    enabled: true
    embedding_model: qwen3-embedding-4b   # any embedding model your router has
    top_k: 5

(MCP servers do not need to be registered for the bug to reproduce, but having at least one configured is the realistic deployment shape.)

Send a request containing ONLY a native (non-MCP) tool:

curl -s -X POST "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -D - -o /tmp/body.json \
  -d '{
    "model": "my-model",
    "messages": [{"role":"user","content":"what is the weather in tokyo?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "weather_lookup",
        "description": "Look up the current weather for a city",
        "parameters": {"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
      }
    }],
    "tool_choice": "auto"
  }'

Observe:

Response header x-litellm-semantic-filter: 1->0 (one tool in, zero out). Response body is a plain-text reply — the model never received weather_lookup. If the upstream backend validates tool_choice (vLLM does), the request fails with 400 Invalid value for 'tool_choice': 'tool_choice' is only allowed when 'tools' are specified. Remove "tools" from the same payload and re-run. The x-litellm-semantic-filter header is absent: confirming it was the filter, not any downstream component, that dropped the tool.

Relevant log output

# Response headers on reproduction
HTTP/2 200 
x-litellm-call-id: 32c29115-3cf3-4302-9bfd-6d2392168db3
x-litellm-model-id: f02277f7...
x-litellm-model-group: DeepSeek-V3
x-litellm-semantic-filter: 1->0    # <-- 1 tool in, 0 out

# Response body: model answers in prose, never emits a tool call
# (truncated)
"content":"To get the current weather in Tokyo, I can check a reliable weather
service for you. Would you like me to fetch the latest weather updates for Tokyo
now? Alternatively, you can check real-time weather on websites like: Weather.com,
AccuWeather, Weather Underground..."

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.82.0

Twitter / LinkedIn details

No response