Issue Station
Sign In
Tools hereToolbox

langchain - ✅(Solved) Fix `wrap_model_call` middleware ToolStrategy narrowing has no effect — model still sees all structured output tools [2 pull requests, 2 comments, 3 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
langchain-ai/langchain#36568•Fetched 2026-04-08 03:01:02
View on GitHub
Comments
2
Participants
3
Timeline
11
Reactions
0
Author
leforebs95leforebs95
Participants
leforebs95leforebs95seuthootDevseuthootDevVishwaspatel2401Vishwaspatel2401
Timeline (top)
cross-referenced ×3labeled ×3commented ×2issue_type_added ×1

When a @wrap_model_call middleware narrows response_format from a union ToolStrategy to a single-type subset via request.override(response_format=ToolStrategy(SubsetType)), the narrowing is validated (lines 1241-1248 of factory.py confirm the subset is valid) but never enforced. The model is still bound with all structured output tools from the original union, and can freely choose any of them.

This contradicts:

  • The code comment at factory.py:1234-1239: "Middleware is allowed to change the response format to a subset of the original structured tools when using ToolStrategy"
  • The discussion on #34239 where this narrowing pattern is described as the intended approach for state-dependent structured output with ToolStrategy

Error Message

Error Message and Stack Trace (if applicable)

Root Cause

structured_output_tools is a closure variable built once from the initial response_format at create_agent() time. Five locations in _get_bound_model() and the graph edge functions use this full set instead of the runtime-narrowed effective_response_format:

LocationCodeImpact
factory.py:1220structured_tools = [info.tool for info in structured_output_tools.values()]All original structured tools are added to final_tools and bound to the model, regardless of narrowed format
factory.py:1071tc["name"] in structured_output_toolsOutput handler accepts tool calls for types outside the narrowed format
factory.py:1100structured_output_tools[tool_call["name"]]Successfully parses responses of the wrong type
factory.py:1728c["name"] not in structured_output_toolsEdge routing treats wrong-type calls as "handled" structured output
factory.py:1808t.name in structured_output_toolsAgent loop exits on wrong-type structured response

The validation at lines 1241-1248 correctly confirms the narrowed format is a valid subset, but none of the downstream code filters to that subset.

Fix Action

Fix / Workaround

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Other Dependencies ------------------ > anthropic: 0.85.0 > blockbuster: 1.5.25 > click: 8.3.1 > cloudpickle: 3.1.2 > croniter: 6.0.0 > cryptography: 46.0.6 > grpcio: 1.78.0 > grpcio-health-checking: 1.78.0 > grpcio-tools: 1.78.0 > httpx: 0.28.1 > jsonpatch: 1.33 > jsonschema-rs: 0.29.1 > langgraph: 1.1.6 > langgraph-checkpoint: 3.0.1 > openai: 2.26.0 > opentelemetry-api: 1.38.0 > opentelemetry-exporter-otlp-proto-http: 1.38.0 > opentelemetry-sdk: 1.38.0 > orjson: 3.11.7 > packaging: 25.0 > protobuf: 6.33.5 > pydantic: 2.12.4 > pyjwt: 2.12.0 > pytest: 9.0.2 > python-dotenv: 1.2.2 > pyyaml: 6.0.3 > requests: 2.33.0 > requests-toolbelt: 1.0.0 > rich: 14.2.0 > sse-starlette: 3.3.2 > starlette: 0.49.3 > structlog: 25.5.0 > tenacity: 9.1.2 > tiktoken: 0.12.0 > truststore: 0.10.4 > typing-extensions: 4.15.0 > uuid-utils: 0.12.0 > uvicorn: 0.38.0 > vcrpy: 7.0.0 > watchfiles: 1.1.1 > wrapt: 2.0.1 > xxhash: 3.6.0 > zstandard: 0.25.0

PR fix notes

PR #36577: fix(langchain): enforce middleware-narrowed ToolStrategy in tool binding

Description (problem / solution / changelog)

Summary

Fixes #36568 — when wrap_model_call middleware narrows response_format from a union ToolStrategy to a single-type subset via request.override(response_format=ToolStrategy(SubsetType)), the narrowing was validated against the original spec but never enforced. The model was still bound with all structured output tools from the original union and could freely choose any of them, contradicting the intended behavior documented at factory.py:1234-1239.

Root cause

In _get_bound_model(), structured_tools was built from the closure-captured structured_output_tools (the full set from initial_response_format), not from the runtime effective_response_format that middleware may have narrowed:

structured_tools = [info.tool for info in structured_output_tools.values()]

This meant every variant of the original union was bound to the model regardless of any middleware narrowing.

Fix

Filter structured_output_tools by the names actually present in effective_response_format.schema_specs before building the final tools list. Because schema_specs is what middleware can narrow at runtime, the model now only sees the variants the effective format permits.

narrowed_tool_names = {tc.name for tc in effective_response_format.schema_specs}
structured_tools = [
    info.tool
    for name, info in structured_output_tools.items()
    if name in narrowed_tool_names
]

The existing validation at factory.py:1241-1248 already guarantees every name in effective_response_format.schema_specs exists in structured_output_tools, so this filter is always a non-empty subset (or equal to the original set when nothing was narrowed).

Tests

Two new regression tests in TestDynamicModelWithResponseFormat:

  1. test_middleware_narrows_tool_strategy_filters_bound_tools — middleware narrows ToolStrategy(WeatherBaseModel | LocationResponse) to ToolStrategy(WeatherBaseModel) via wrap_model_call → asserts the model is bound with only WeatherBaseModel and not LocationResponse, and that the end-to-end structured response is parsed correctly.

  2. test_no_middleware_narrowing_keeps_full_tool_strategy_binding — regression guard ensuring that when no middleware override is in play, both variants of the union are still bound (the fix must not accidentally drop variants in the default case).

Both tests use a RecordingModel subclass of FakeToolCallingModel that captures the names of tools passed to bind_tools, mirroring the existing CustomModel pattern in test_middleware_model_swap_provider_to_tool_strategy.

Pre-Submission checklist

  • Added regression tests in libs/langchain_v1/tests/unit_tests/agents/test_response_format.py
  • PR scope is isolated to a single problem
  • Fix preserves existing behavior when no middleware narrowing is involved
  • No changes to _handle_model_output or edge functions: with the model bound to only the narrowed tools, well-behaved providers physically cannot emit dropped variants, so the existing closure-captured structured_output_tools references in those locations remain correct.

AI assistance disclosure

This contribution was prepared with assistance from an AI coding agent. I reviewed, validated, and finalized the proposed changes and test coverage before submission.

Changed files

  • libs/langchain_v1/langchain/agents/factory.py (modified, +11/-2)
  • libs/langchain_v1/tests/unit_tests/agents/test_response_format.py (modified, +133/-1)

PR #36578: fix(langchain): enforce toolstrategy narrowing in middleware overrides

Description (problem / solution / changelog)

Fixes #36568

This PR fixes a bug where wrap_model_call middleware narrowing of ToolStrategy was validated but not enforced at runtime. The model now binds and accepts only the structured-output tools in the effective narrowed response format, and regression tests were added for middleware narrowing behavior.

No breaking changes.

AI assistance disclaimer: This contribution was prepared with help from an AI coding assistant (GitHub Copilot), and the final code and tests were reviewed and validated by me.

How I verified:

  • Added targeted unit tests for ToolStrategy narrowing behavior.
  • Ran:
    • uv run --group test pytest tests/unit_tests/agents/test_response_format.py -v
    • uv run --group test pytest tests/unit_tests/agents/middleware/core/test_wrap_model_call.py -q

Changed files

  • libs/langchain_v1/langchain/agents/factory.py (modified, +16/-2)
  • libs/langchain_v1/tests/unit_tests/agents/test_response_format.py (modified, +179/-0)

Code Example

from typing import Literal, Union
from collections.abc import Callable

from pydantic import BaseModel, Field
from langchain.agents import create_agent
from langchain.agents.middleware import (
    ModelRequest,
    ModelResponse,
    wrap_model_call,
    before_agent,
)
from langchain.agents.structured_output import ToolStrategy


# Two possible response types
class DetailedAnswer(BaseModel):
    type: Literal["DetailedAnswer"] = "DetailedAnswer"
    content: str = Field(description="A detailed, thorough answer")

class BriefAnswer(BaseModel):
    type: Literal["BriefAnswer"] = "BriefAnswer"
    content: str = Field(description="A brief, one-sentence answer")

FullResponse = Union[DetailedAnswer, BriefAnswer]


# Middleware that should narrow to only DetailedAnswer
@wrap_model_call
async def force_detailed(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
    """Narrow response format to only allow DetailedAnswer."""
    narrowed = request.override(
        response_format=ToolStrategy(DetailedAnswer)
    )
    return await handler(narrowed)


agent = create_agent(
    model="anthropic:claude-haiku-4-5-20251001",
    tools=[],
    middleware=[force_detailed],
    response_format=ToolStrategy(FullResponse),
)

# Ask the model something — it CAN still choose BriefAnswer
# even though middleware narrowed to DetailedAnswer only
result = agent.invoke({
    "messages": [{"role": "user", "content": "What is 2+2? Be brief."}]
})

# BUG: model may return BriefAnswer despite middleware narrowing
print(type(result["structured_response"]))
# Expected: always DetailedAnswer
# Actual: may be BriefAnswer because both tools are bound to the model

---



---

# Current (line 1220): adds ALL structured tools
structured_tools = [info.tool for info in structured_output_tools.values()]

# Fix: filter to only tools in the narrowed format
if isinstance(effective_response_format, ToolStrategy):
    narrowed_names = {tc.name for tc in effective_response_format.schema_specs}
    structured_tools = [
        info.tool for name, info in structured_output_tools.items()
        if name in narrowed_names
    ]
RAW_BUFFERClick to expand / collapse

Checked other resources

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Related Issues / PRs

Issue #36420

Reproduction Steps / Example Code (Python)

from typing import Literal, Union
from collections.abc import Callable

from pydantic import BaseModel, Field
from langchain.agents import create_agent
from langchain.agents.middleware import (
    ModelRequest,
    ModelResponse,
    wrap_model_call,
    before_agent,
)
from langchain.agents.structured_output import ToolStrategy


# Two possible response types
class DetailedAnswer(BaseModel):
    type: Literal["DetailedAnswer"] = "DetailedAnswer"
    content: str = Field(description="A detailed, thorough answer")

class BriefAnswer(BaseModel):
    type: Literal["BriefAnswer"] = "BriefAnswer"
    content: str = Field(description="A brief, one-sentence answer")

FullResponse = Union[DetailedAnswer, BriefAnswer]


# Middleware that should narrow to only DetailedAnswer
@wrap_model_call
async def force_detailed(
    request: ModelRequest,
    handler: Callable[[ModelRequest], ModelResponse],
) -> ModelResponse:
    """Narrow response format to only allow DetailedAnswer."""
    narrowed = request.override(
        response_format=ToolStrategy(DetailedAnswer)
    )
    return await handler(narrowed)


agent = create_agent(
    model="anthropic:claude-haiku-4-5-20251001",
    tools=[],
    middleware=[force_detailed],
    response_format=ToolStrategy(FullResponse),
)

# Ask the model something — it CAN still choose BriefAnswer
# even though middleware narrowed to DetailedAnswer only
result = agent.invoke({
    "messages": [{"role": "user", "content": "What is 2+2? Be brief."}]
})

# BUG: model may return BriefAnswer despite middleware narrowing
print(type(result["structured_response"]))
# Expected: always DetailedAnswer
# Actual: may be BriefAnswer because both tools are bound to the model

Error Message and Stack Trace (if applicable)

Description

Description

When a @wrap_model_call middleware narrows response_format from a union ToolStrategy to a single-type subset via request.override(response_format=ToolStrategy(SubsetType)), the narrowing is validated (lines 1241-1248 of factory.py confirm the subset is valid) but never enforced. The model is still bound with all structured output tools from the original union, and can freely choose any of them.

This contradicts:

  • The code comment at factory.py:1234-1239: "Middleware is allowed to change the response format to a subset of the original structured tools when using ToolStrategy"
  • The discussion on #34239 where this narrowing pattern is described as the intended approach for state-dependent structured output with ToolStrategy

Root Cause

structured_output_tools is a closure variable built once from the initial response_format at create_agent() time. Five locations in _get_bound_model() and the graph edge functions use this full set instead of the runtime-narrowed effective_response_format:

LocationCodeImpact
factory.py:1220structured_tools = [info.tool for info in structured_output_tools.values()]All original structured tools are added to final_tools and bound to the model, regardless of narrowed format
factory.py:1071tc["name"] in structured_output_toolsOutput handler accepts tool calls for types outside the narrowed format
factory.py:1100structured_output_tools[tool_call["name"]]Successfully parses responses of the wrong type
factory.py:1728c["name"] not in structured_output_toolsEdge routing treats wrong-type calls as "handled" structured output
factory.py:1808t.name in structured_output_toolsAgent loop exits on wrong-type structured response

The validation at lines 1241-1248 correctly confirms the narrowed format is a valid subset, but none of the downstream code filters to that subset.

Expected behavior

When middleware narrows response_format to ToolStrategy(SubsetType):

  1. Only the structured output tools in the narrowed format should be bound to the model
  2. The output handler should only accept tool calls matching the narrowed format
  3. Edge routing should only treat narrowed-format tool calls as structured output

Actual behavior

All structured output tools from the original create_agent(response_format=...) are always bound to the model. The model can choose any of them regardless of middleware narrowing. The narrowed effective_response_format is validated but never used for filtering.

Suggested fix

In _get_bound_model(), filter structured output tools to only those present in effective_response_format.schema_specs:

# Current (line 1220): adds ALL structured tools
structured_tools = [info.tool for info in structured_output_tools.values()]

# Fix: filter to only tools in the narrowed format
if isinstance(effective_response_format, ToolStrategy):
    narrowed_names = {tc.name for tc in effective_response_format.schema_specs}
    structured_tools = [
        info.tool for name, info in structured_output_tools.items()
        if name in narrowed_names
    ]

The same filtering pattern would need to be applied to _handle_model_output (line 1071) and the edge functions (lines 1728, 1808) — though these are trickier since they're closures that don't have access to the runtime-narrowed format. One approach: store the effective_response_format in agent state so edge functions can read it, or restructure the edge functions to check both the full set and the state-stored narrowed set.

A simpler alternative for the edge functions: since _handle_model_output already receives effective_response_format, it could reject tool calls not in the narrowed set (returning them as "unhandled" so the agent loop retries), which would make the edge function changes unnecessary.

System Info

⎿  System Information ------------------ > OS: Darwin > OS Version: Darwin Kernel Version 25.3.0: Wed Jan 28 20:54:46 PST 2026; root:xnu-12377.91.3~2/RELEASE_ARM64_T6000 > Python Version: 3.13.1 (main, Jan 14 2025, 23:31:50) [Clang 19.1.6 ]

 Package Information
 -------------------
 > langchain_core: 1.2.25
 > langchain: 1.2.15
 > langsmith: 0.7.24
 > langchain_anthropic: 1.4.0
 > langchain_openai: 1.1.12
 > langgraph_api: 0.7.96
 > langgraph_cli: 0.4.19
 > langgraph_runtime_inmem: 0.27.0
 > langgraph_sdk: 0.3.12

 Optional packages not installed
 -------------------------------
 > deepagents
 > deepagents-cli

 Other Dependencies
 ------------------
 > anthropic: 0.85.0
 > blockbuster: 1.5.25
 > click: 8.3.1
 > cloudpickle: 3.1.2
 > croniter: 6.0.0
 > cryptography: 46.0.6
 > grpcio: 1.78.0
 > grpcio-health-checking: 1.78.0
 > grpcio-tools: 1.78.0
 > httpx: 0.28.1
 > jsonpatch: 1.33
 > jsonschema-rs: 0.29.1
 > langgraph: 1.1.6
 > langgraph-checkpoint: 3.0.1
 > openai: 2.26.0
 > opentelemetry-api: 1.38.0
 > opentelemetry-exporter-otlp-proto-http: 1.38.0
 > opentelemetry-sdk: 1.38.0
 > orjson: 3.11.7
 > packaging: 25.0
 > protobuf: 6.33.5
 > pydantic: 2.12.4
 > pyjwt: 2.12.0
 > pytest: 9.0.2
 > python-dotenv: 1.2.2
 > pyyaml: 6.0.3
 > requests: 2.33.0
 > requests-toolbelt: 1.0.0
 > rich: 14.2.0
 > sse-starlette: 3.3.2
 > starlette: 0.49.3
 > structlog: 25.5.0
 > tenacity: 9.1.2
 > tiktoken: 0.12.0
 > truststore: 0.10.4
 > typing-extensions: 4.15.0
 > uuid-utils: 0.12.0
 > uvicorn: 0.38.0
 > vcrpy: 7.0.0
 > watchfiles: 1.1.1
 > wrapt: 2.0.1
 > xxhash: 3.6.0
 > zstandard: 0.25.0

extent analysis

TL;DR

The issue can be fixed by filtering structured output tools to only those present in the narrowed effective_response_format.schema_specs in the _get_bound_model() function.

Guidance

  • Identify the locations in the code where the full set of structured output tools is used instead of the narrowed format, specifically in _get_bound_model(), _handle_model_output(), and edge functions.
  • Apply filtering to only include tools in the narrowed format, as suggested in the provided fix for _get_bound_model().
  • Consider storing the effective_response_format in agent state to make it accessible to edge functions, or restructure edge functions to check both the full set and the narrowed set.
  • Alternatively, modify _handle_model_output() to reject tool calls not in the narrowed set, which could eliminate the need for changes to edge functions.

Example

The suggested fix for _get_bound_model() is:

if isinstance(effective_response_format, ToolStrategy):
    narrowed_names = {tc.name for tc in effective_response_format.schema_specs}
    structured_tools = [
        info.tool for name, info in structured_output_tools.items()
        if name in narrowed_names
    ]

This code filters the structured output tools to only those present in the narrowed format.

Notes

The provided fix only addresses the issue in _get_bound_model() and may need to be adapted for other locations in the code. Additionally, the suggested alternative approach for edge functions requires further implementation details.

Recommendation

Apply the suggested fix for _get_bound_model() and investigate the necessary changes for _handle_model_output() and edge functions to ensure the narrowed format is enforced throughout the code.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

When middleware narrows response_format to ToolStrategy(SubsetType):

  1. Only the structured output tools in the narrowed format should be bound to the model
  2. The output handler should only accept tool calls matching the narrowed format
  3. Edge routing should only treat narrowed-format tool calls as structured output
#api#ssr#callback error#memory management#API rate limit

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

langchain - ✅(Solved) Fix `wrap_model_call` middleware ToolStrategy narrowing has no effect — model still sees all structured output tools [2 pull requests, 2 comments, 3 participants]