Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix MCP tool schemas with $ref definitions fail on Anthropic and Fireworks providers (PointerToNowhere) [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26692•Fetched 2026-04-29 06:12:50
View on GitHub
Comments
0
Participants
1
Timeline
2
Reactions
0
Author
hxrikp1729hxrikp1729
Participants
hxrikp1729hxrikp1729
Timeline (top)
cross-referenced ×1labeled ×1

When using LiteLLM as a gateway with MCP servers that expose tools containing $ref references in their JSON schemas (e.g. DevRev MCP), requests to Anthropic and Fireworks providers fail with schema resolution errors. The same request succeeds on OpenAI/GPT models.

Error Message

litellm.BadRequestError: AnthropicException - {"error":{"type":"invalid_request_error","message":"Error resolving schema reference '#/definitions/_gen:tags': PointerToNowhere(ref='/definitions/_gen:tags', resource=Resource(contents={...tool schema without definitions block...}))"}}

Root Cause

MCP servers can return tool schemas that use $ref with a top-level definitions block, e.g.:

{
  "name": "update_contact",
  "input_schema": {
    "type": "object",
    "properties": {
      "tags": {
        "$ref": "#/definitions/_gen:tags",
        "description": "Tags associated with the contact"
      }
    },
    "definitions": {
      "_gen:tags": {
        "type": "object",
        "properties": { ... }
      }
    }
  }
}

When LiteLLM forwards tool definitions to downstream providers, the definitions block is either not included or not resolved inline. Anthropic and Fireworks validate schemas strictly and reject unresolved $ref pointers. OpenAI is more permissive and resolves them server-side.

Fix Action

Fixed

PR fix notes

PR #26695: fix(tools): resolve legacy definitions $ref in tool schemas for Anthropic and Fireworks

Description (problem / solution / changelog)

Summary

Fixes #26692

MCP servers (e.g. DevRev) emit tool input_schema objects that use the legacy draft-04 definitions keyword with $ref pointers like #/definitions/_gen:tags. When LiteLLM forwards these to Anthropic or Fireworks, the request fails at the provider:

  • Anthropic: "Error resolving schema reference '#/definitions/_gen:tags': PointerToNowhere" — the definitions key is stripped by the _allowed_properties filter in _map_tool_helper while the $ref pointers remain dangling
  • Fireworks: "Error resolving schema reference '#/definitions/_gen:tags': AttributeError('NoneType' object has no attribute 'lookup')" — provider rejects unresolved references

OpenAI/GPT models are unaffected because OpenAI resolves $ref server-side.

Changes

litellm/llms/anthropic/chat/transformation.py — _map_tool_helper

Before applying the _allowed_properties filter, detect definitions in the input schema and inline all $ref pointers using the existing unpack_defs utility (already used for response_format schema handling in the same file). $defs refs — natively supported by Anthropic — are left untouched.

litellm/llms/fireworks_ai/chat/transformation.py — _transform_tools

Same pattern: when definitions is present in tool.function.parameters, inline all refs before forwarding to the provider.

Tests

  • test_anthropic_tool_with_legacy_definitions_ref_resolved_inline — asserts $ref is inlined and definitions is stripped
  • test_anthropic_tool_defs_ref_still_preserved — regression guard: $defs refs must not be touched (Anthropic handles them natively)
  • test_fireworks_transform_tools_resolves_definitions_refs — same for Fireworks path

All 3 new tests pass. Existing test_anthropic_map_openai_params_tools_with_defs and surrounding tests pass unchanged.

Root cause (in one sentence)

definitions (JSON Schema draft-04) is not in AnthropicInputSchema.__annotations__, so it gets silently stripped by the _allowed_properties filter — leaving $ref pointers with nothing to resolve against.

Changed files

  • .circleci/config.yml (modified, +3/-3)
  • .github/pull_request_template.md (modified, +4/-0)
  • docs/my-website/docs/proxy/guardrails/xecguard.md (added, +314/-0)
  • litellm/_redis_credential_provider.py (modified, +57/-7)
  • litellm/constants.py (modified, +10/-0)
  • litellm/exceptions.py (modified, +30/-1)
  • litellm/integrations/custom_guardrail.py (modified, +1/-37)
  • litellm/litellm_core_utils/prompt_templates/factory.py (modified, +22/-13)
  • litellm/llms/anthropic/chat/transformation.py (modified, +22/-0)
  • litellm/llms/bedrock/chat/converse_transformation.py (modified, +2/-2)
  • litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py (modified, +7/-1)
  • litellm/llms/fireworks_ai/chat/transformation.py (modified, +18/-0)
  • litellm/llms/ollama/chat/transformation.py (modified, +7/-2)
  • litellm/llms/predibase/chat/handler.py (modified, +43/-228)
  • litellm/llms/predibase/chat/transformation.py (modified, +212/-7)
  • litellm/model_prices_and_context_window_backup.json (modified, +32/-32)
  • litellm/proxy/common_utils/expired_ui_session_key_cleanup_manager.py (added, +156/-0)
  • litellm/proxy/guardrails/guardrail_hooks/noma/noma_v2.py (modified, +1/-2)
  • litellm/proxy/guardrails/guardrail_hooks/unified_guardrail/unified_guardrail.py (modified, +10/-0)
  • litellm/proxy/guardrails/guardrail_hooks/xecguard/__init__.py (added, +45/-0)
  • litellm/proxy/guardrails/guardrail_hooks/xecguard/xecguard.py (added, +585/-0)
  • litellm/proxy/pass_through_endpoints/pass_through_endpoints.py (modified, +73/-5)
  • litellm/proxy/proxy_cli.py (modified, +33/-0)
  • litellm/proxy/proxy_server.py (modified, +81/-2)
  • litellm/router.py (modified, +4/-2)
  • litellm/types/guardrails.py (modified, +5/-0)
  • litellm/types/llms/ollama.py (modified, +1/-0)
  • litellm/types/proxy/guardrails/guardrail_hooks/xecguard.py (added, +77/-0)
  • model_prices_and_context_window.json (modified, +32/-32)
  • tests/llm_translation/test_anthropic_completion.py (modified, +107/-0)
  • tests/llm_translation/test_fireworks_ai_translation.py (modified, +44/-0)
  • tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py (modified, +10/-10)
  • tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py (modified, +109/-0)
  • tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py (modified, +80/-0)
  • tests/test_litellm/llms/ollama/test_ollama_chat_transformation.py (modified, +95/-0)
  • tests/test_litellm/llms/test_predibase_transformation.py (added, +612/-0)
  • tests/test_litellm/proxy/common_utils/test_expired_ui_session_key_cleanup_manager.py (added, +348/-0)
  • tests/test_litellm/proxy/guardrails/guardrail_hooks/test_noma_v2.py (modified, +38/-0)
  • tests/test_litellm/proxy/guardrails/guardrail_hooks/test_xecguard.py (added, +1904/-0)
  • tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_post_call_guardrails.py (added, +276/-0)
  • tests/test_litellm/proxy/test_proxy_cli.py (modified, +65/-0)
  • tests/test_litellm/test_redis.py (modified, +76/-9)
  • tests/test_litellm/test_router.py (modified, +63/-0)
  • ui/litellm-dashboard/public/assets/logos/xecguard.svg (added, +4/-0)
  • ui/litellm-dashboard/src/app/(dashboard)/hooks/proxyConfig/useProxyConfig.test.ts (modified, +23/-0)
  • ui/litellm-dashboard/src/app/(dashboard)/hooks/proxyConfig/useProxyConfig.ts (modified, +6/-2)
  • ui/litellm-dashboard/src/app/(dashboard)/hooks/storeRequestInSpendLogs/useStoreRequestInSpendLogs.ts (modified, +6/-1)
  • ui/litellm-dashboard/src/components/AdminPanel.tsx (modified, +6/-0)
  • ui/litellm-dashboard/src/components/Settings/AdminSettings/LoggingSettings/LoggingSettings.test.tsx (added, +335/-0)
  • ui/litellm-dashboard/src/components/Settings/AdminSettings/LoggingSettings/LoggingSettings.tsx (added, +145/-0)
  • ui/litellm-dashboard/src/components/guardrails/guardrail_garden_configs.ts (modified, +6/-0)
  • ui/litellm-dashboard/src/components/guardrails/guardrail_garden_data.ts (modified, +10/-0)
  • ui/litellm-dashboard/src/components/guardrails/guardrail_info_helpers.tsx (modified, +2/-0)
  • ui/litellm-dashboard/src/components/mcp_tools/mcp_server_edit.test.tsx (modified, +1/-1)
  • ui/litellm-dashboard/src/components/mcp_tools/mcp_server_edit.tsx (modified, +15/-31)
  • ui/litellm-dashboard/src/components/view_logs/ConfigInfoMessage.test.tsx (modified, +3/-19)
  • ui/litellm-dashboard/src/components/view_logs/ConfigInfoMessage.tsx (modified, +3/-14)
  • ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailContent.test.tsx (modified, +0/-21)
  • ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailContent.tsx (modified, +2/-3)
  • ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx (modified, +0/-3)
  • ui/litellm-dashboard/src/components/view_logs/SpendLogsSettingsModal/SpendLogsSettingsModal.test.tsx (removed, +0/-484)
  • ui/litellm-dashboard/src/components/view_logs/SpendLogsSettingsModal/SpendLogsSettingsModal.tsx (removed, +0/-156)
  • ui/litellm-dashboard/src/components/view_logs/index.tsx (modified, +3/-16)

Code Example

litellm.BadRequestError: AnthropicException - {"error":{"type":"invalid_request_error","message":"Error resolving schema reference '#/definitions/_gen:tags': PointerToNowhere(ref='/definitions/_gen:tags', resource=Resource(contents={...tool schema without definitions block...}))"}}

---

litellm.BadRequestError: Fireworks_aiException - {"error":{"type":"invalid_request_error","message":"Error resolving schema reference '#/definitions/_gen:tags': AttributeError(\"'NoneType' object has no attribute 'lookup'\")"}}

---

{
  "name": "update_contact",
  "input_schema": {
    "type": "object",
    "properties": {
      "tags": {
        "$ref": "#/definitions/_gen:tags",
        "description": "Tags associated with the contact"
      }
    },
    "definitions": {
      "_gen:tags": {
        "type": "object",
        "properties": { ... }
      }
    }
  }
}

---

import jsonschema
from jsonschema import RefResolver

def resolve_refs(schema: dict) -> dict:
    """Flatten $ref references inline so providers don't need to resolve them."""
    resolver = RefResolver.from_schema(schema)
    # walk the schema and inline all $refs
    ...
RAW_BUFFERClick to expand / collapse

Description

When using LiteLLM as a gateway with MCP servers that expose tools containing $ref references in their JSON schemas (e.g. DevRev MCP), requests to Anthropic and Fireworks providers fail with schema resolution errors. The same request succeeds on OpenAI/GPT models.

Error Messages

Anthropic (e.g. claude-haiku, custom models routed to Anthropic):

litellm.BadRequestError: AnthropicException - {"error":{"type":"invalid_request_error","message":"Error resolving schema reference '#/definitions/_gen:tags': PointerToNowhere(ref='/definitions/_gen:tags', resource=Resource(contents={...tool schema without definitions block...}))"}}

Fireworks (e.g. kimi-k2p5, kimi-k2p6, minimax-m2p7):

litellm.BadRequestError: Fireworks_aiException - {"error":{"type":"invalid_request_error","message":"Error resolving schema reference '#/definitions/_gen:tags': AttributeError(\"'NoneType' object has no attribute 'lookup'\")"}}

Root Cause

MCP servers can return tool schemas that use $ref with a top-level definitions block, e.g.:

{
  "name": "update_contact",
  "input_schema": {
    "type": "object",
    "properties": {
      "tags": {
        "$ref": "#/definitions/_gen:tags",
        "description": "Tags associated with the contact"
      }
    },
    "definitions": {
      "_gen:tags": {
        "type": "object",
        "properties": { ... }
      }
    }
  }
}

When LiteLLM forwards tool definitions to downstream providers, the definitions block is either not included or not resolved inline. Anthropic and Fireworks validate schemas strictly and reject unresolved $ref pointers. OpenAI is more permissive and resolves them server-side.

Reproduction

  1. Set up a LiteLLM gateway with an MCP server that exposes tools with $ref in their schemas (e.g. DevRev MCP)
  2. Make a request routed to an Anthropic-backed or Fireworks-backed model group
  3. LiteLLM forwards tool schemas with unresolved $ref → provider rejects with schema resolution error
  4. Same request to an OpenAI-backed model group → succeeds

Expected Behavior

LiteLLM should resolve/flatten all $ref references in tool input_schema before forwarding to providers that don't support them. This can be done using Python's jsonschema library:

import jsonschema
from jsonschema import RefResolver

def resolve_refs(schema: dict) -> dict:
    """Flatten $ref references inline so providers don't need to resolve them."""
    resolver = RefResolver.from_schema(schema)
    # walk the schema and inline all $refs
    ...

Alternatively, apply this only for providers known to be strict (Anthropic, Fireworks) and skip for OpenAI which handles refs natively.

Environment

  • LiteLLM version: latest (via Razorpay internal gateway)
  • Failing providers: anthropic, fireworks_ai
  • Working providers: openai
  • MCP source: DevRev MCP (tools with $ref to #/definitions/_gen:* types)
  • Client: opencode (sst/opencode) using @ai-sdk/openai provider pointed at LiteLLM gateway

Related

  • PR #22952 (fix(mcp): resolve $ref params in OpenAPI spec) — handles $ref in OpenAPI spec parsing but not in tool input_schema forwarding
  • PR #24128 (fix: strip $schema from tool input_schema) — adjacent schema handling work
  • Issue #11372 — $ref resolution failure in structured outputs (closed)

This issue affects any MCP server that uses $ref with definitions in its tool schemas, not just DevRev. It's a systematic gap in LiteLLM's provider compatibility layer.

extent analysis

TL;DR

Resolve $ref references in tool input_schema before forwarding to providers that don't support them, such as Anthropic and Fireworks.

Guidance

  • Identify providers that strictly validate schemas and reject unresolved $ref pointers, such as Anthropic and Fireworks.
  • Use a library like jsonschema to resolve/flatten all $ref references in tool input_schema before forwarding to these providers.
  • Consider applying this resolution only for providers known to be strict and skipping it for providers like OpenAI that handle refs natively.
  • Review related PRs (#22952, #24128) and issue (#11372) for adjacent schema handling work and potential solutions.

Example

import jsonschema
from jsonschema import RefResolver

def resolve_refs(schema: dict) -> dict:
    """Flatten $ref references inline so providers don't need to resolve them."""
    resolver = RefResolver.from_schema(schema)
    # walk the schema and inline all $refs
    # implementation details omitted for brevity

Notes

This solution assumes that the jsonschema library can effectively resolve the $ref references in the tool input_schema. Additional error handling and logging may be necessary to ensure robustness.

Recommendation

Apply a workaround to resolve $ref references for strict providers, as upgrading to a fixed version is not mentioned in the issue. This approach allows for a targeted solution without waiting for a potential version update.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ISR setup#authentication setup#request error#file not found

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix MCP tool schemas with $ref definitions fail on Anthropic and Fireworks providers (PointerToNowhere) [1 pull requests, 1 participants]