litellm - 💡(How to fix) Fix [Bug]: Tool registry (LiteLLM_ToolTable / LiteLLM_SpendLogToolIndex) not populated for /v1/messages (anthropic_messages) path [1 participants]

Description

LiteLLM_ToolTable and LiteLLM_SpendLogToolIndex stay empty for all traffic going through the /v1/messages endpoint (the native Anthropic messages API path, route_type="anthropic_messages").

Root Cause

_enqueue_tool_registry_upsert in litellm/proxy/db/db_spend_update_writer.py handles four sources for tool names, but misses the non-passthrough anthropic_messages path:

1. MCP tool calls via standard_logging_object.metadata.mcp_tool_call_metadata ✅
2. Request tools in OpenAI format (kwargs["tools"][].function.name) ✅
3. Request tools in Anthropic pass-through format (kwargs["passthrough_logging_payload"]["request_body"]["tools"][].name) ✅
4. Response tool calls (completion_response.choices[].message.tool_calls) ✅

The gap: when a request is routed through anthropic_messages (i.e. llm_router.anthropic_messages(**data)), the response is an AnthropicMessagesResponse — a plain TypedDict with content[{"type": "tool_use", "name": ..., "input": {...}}]. This shape is never inspected by _enqueue_tool_registry_upsert. The hasattr(completion_response, "choices") guard on path 4 is False for a dict, so response tool calls are silently dropped.

Additionally, the request tools for this path live directly in kwargs["tools"][].name (Anthropic format, no "function" wrapper), which path 2 misses because it looks for tool_def.get("function").get("name").

Reproduction

1. Configure a LiteLLM proxy with a model on the /v1/messages endpoint.
2. Send a request with tools defined and a response that includes tool_use content blocks.
3. Check LiteLLM_ToolTable and LiteLLM_SpendLogToolIndex — both are empty.

Expected Behaviour

Tool names from both the request (tools[].name) and the response (content[].type=="tool_use" blocks) should be tracked in LiteLLM_ToolTable and LiteLLM_SpendLogToolIndex just as they are for /chat/completions traffic.

Suggested Fix

In _enqueue_tool_registry_upsert, add two additional extraction paths:

Request tools (Anthropic format, no function wrapper):

for tool_def in kwargs.get("tools") or []:
    if not isinstance(tool_def, dict):
        continue
    # Anthropic format: name is top-level, no "function" wrapper
    name = tool_def.get("name")
    if name:
        _enqueue(name)

Response tool_use blocks (AnthropicMessagesResponse is a TypedDict/dict):

if isinstance(completion_response, dict):
    for block in completion_response.get("content") or []:
        if isinstance(block, dict) and block.get("type") == "tool_use":
            name = block.get("name")
            if name:
                _enqueue(name)

Version

Reproduced on 1.82.6 and 1.83.10.