Issue Station
Sign In
Tools hereToolbox

langchain - ✅(Solved) Fix Fake chat models can't be bound to structured output [1 pull requests, 4 comments, 4 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
langchain-ai/langchain#36277•Fetched 2026-04-08 01:36:17
View on GitHub
Comments
4
Participants
4
Timeline
13
Reactions
0
Author
joaopaulo7joaopaulo7
Participants
joaopaulo7joaopaulo7majorelalexis-stackmajorelalexis-stacknihalnihalaninihalnihalanividitkbhatnagarviditkbhatnagar
Timeline (top)
commented ×4labeled ×4cross-referenced ×3issue_type_added ×1

I tried testing the structured output with the fake chat models to test a possible bug, but I ran into another. It seems mock structure outputs aren't implemented for the fake models; this specific example is for GenericFakeChatModel.

The fix doesn't seem to be very complex. I'd be happy to try implementing it and make a pull request.

Error Message

NotImplementedError Traceback (most recent call last) Cell In[1], line 32 9 response: str = Field(description="Parrot's response.") 11 model = GenericFakeChatModel( 12 messages=iter( 13 [ (...) 29 ) 30 ) ---> 32 structured_output_model = model.with_structured_output(Parrot) 33 response = structured_output_model.invoke("Hello, Polly!!") 34 assert isinstance(response, Parrot)

File ~/Documents/Projetos/test-langchain/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py:1773, in BaseChatModel.with_structured_output(self, schema, include_raw, **kwargs) 1771 if type(self).bind_tools is BaseChatModel.bind_tools: 1772 msg = "with_structured_output is not implemented for this model." -> 1773 raise NotImplementedError(msg) 1775 llm = self.bind_tools( 1776 [schema], 1777 tool_choice="any", (...) 1781 }, 1782 ) 1783 if isinstance(schema, type) and is_basemodel_subclass(schema):

NotImplementedError: with_structured_output is not implemented for this model.

Root Cause

I tried testing the structured output with the fake chat models to test a possible bug, but I ran into another. It seems mock structure outputs aren't implemented for the fake models; this specific example is for GenericFakeChatModel.

The fix doesn't seem to be very complex. I'd be happy to try implementing it and make a pull request.

Fix Action

Fix / Workaround

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Other Dependencies

httpx: 0.28.1 jsonpatch: 1.33 langgraph: 1.1.3 orjson: 3.11.7 packaging: 26.0 pydantic: 2.12.5 pyyaml: 6.0.3 requests: 2.33.0 requests-toolbelt: 1.0.0 tenacity: 9.1.4 typing-extensions: 4.15.0 uuid-utils: 0.14.1 xxhash: 3.6.0 zstandard: 0.25.0

PR fix notes

PR #36287: fix(core): enable structured output binding for fake chat models

Description (problem / solution / changelog)

BaseChatModel.with_structured_output() requires bind_tools() to be implemented. None of the 5 fake chat model classes overrode it, causing NotImplementedError.

Added bind_tools() to FakeMessagesListChatModel, FakeListChatModel, FakeChatModel, GenericFakeChatModel, and ParrotFakeChatModel. Each delegates to self.bind(tools=tools, **kwargs).

Added 13 tests. All 1753 core unit tests pass.

Fixes #36277

Fixes #

<!-- Replace everything above this line with a 1-2 sentence description of your change. Keep the "Fixes #xx" keyword and update the issue number. -->

Read the full contributing guidelines: https://docs.langchain.com/oss/python/contributing/overview

All contributions must be in English. See the language policy.

If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!

Thank you for contributing to LangChain! Follow these steps to have your pull request considered as ready for review.

  1. PR title: Should follow the format: TYPE(SCOPE): DESCRIPTION
  1. PR description:
  • Write 1-2 sentences summarizing the change.
  • The Fixes #xx line at the top is required for external contributions — update the issue number and keep the keyword. This links your PR to the approved issue and auto-closes it on merge.
  • If there are any breaking changes, please clearly describe them.
  • If this PR depends on another PR being merged first, please include "Depends on #PR_NUMBER" in the description.
  1. Run make format, make lint and make test from the root of the package(s) you've modified.
  • We will not consider a PR unless these three are passing in CI.
  1. How did you verify your code works?

Additional guidelines:

  • All external PRs must link to an issue or discussion where a solution has been approved by a maintainer, and you must be assigned to that issue. PRs without prior approval will be closed.
  • PRs should not touch more than one package unless absolutely necessary.
  • Do not update the uv.lock files or add dependencies to pyproject.toml files (even optional ones) unless you have explicit permission to do so by a maintainer.

Social handles (optional)

<!-- If you'd like a shoutout on release, add your socials below -->

Twitter: @ LinkedIn: https://linkedin.com/in/

Changed files

  • libs/core/langchain_core/language_models/fake_chat_models.py (modified, +64/-2)
  • libs/core/tests/unit_tests/fake/test_fake_chat_model.py (modified, +135/-0)

Code Example

from pydantic import BaseModel, Field

from langchain.messages import AIMessage, ToolCall
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel

class Parrot(BaseModel):
    name: str = Field(description="Parrot name.")
    species: str = Field(description="Parrot species.")
    response: str = Field(description="Parrot's response.")

model = GenericFakeChatModel(
    messages=iter(
        [
            AIMessage(
                content="",
                tool_calls=[
                    ToolCall(
                        name="Parrot",
                        args={
                            "name": "Polly",
                            "species": "Norwegian Blue",
                            "response": "...",
                        },
                        id="call_1",
                    )
                ],
            ),
        ]
    )
)

structured_output_model = model.with_structured_output(Parrot)
response = structured_output_model.invoke("Hello, Polly!!")
assert isinstance(response, Parrot)

---

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[1], line 32
      9     response: str = Field(description="Parrot's response.")
     11 model = GenericFakeChatModel(
     12     messages=iter(
     13         [
   (...)     29     )
     30 )
---> 32 structured_output_model = model.with_structured_output(Parrot)
     33 response = structured_output_model.invoke("Hello, Polly!!")
     34 assert isinstance(response, Parrot)

File ~/Documents/Projetos/test-langchain/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py:1773, in BaseChatModel.with_structured_output(self, schema, include_raw, **kwargs)
   1771 if type(self).bind_tools is BaseChatModel.bind_tools:
   1772     msg = "with_structured_output is not implemented for this model."
-> 1773     raise NotImplementedError(msg)
   1775 llm = self.bind_tools(
   1776     [schema],
   1777     tool_choice="any",
   (...)   1781     },
   1782 )
   1783 if isinstance(schema, type) and is_basemodel_subclass(schema):

NotImplementedError: with_structured_output is not implemented for this model.
RAW_BUFFERClick to expand / collapse

Checked other resources

  • This is a bug, not a usage question.
  • I added a clear and descriptive title that summarizes this issue.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
  • This is not related to the langchain-community package.
  • I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-openrouter
  • langchain-perplexity
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

from pydantic import BaseModel, Field

from langchain.messages import AIMessage, ToolCall
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel

class Parrot(BaseModel):
    name: str = Field(description="Parrot name.")
    species: str = Field(description="Parrot species.")
    response: str = Field(description="Parrot's response.")

model = GenericFakeChatModel(
    messages=iter(
        [
            AIMessage(
                content="",
                tool_calls=[
                    ToolCall(
                        name="Parrot",
                        args={
                            "name": "Polly",
                            "species": "Norwegian Blue",
                            "response": "...",
                        },
                        id="call_1",
                    )
                ],
            ),
        ]
    )
)

structured_output_model = model.with_structured_output(Parrot)
response = structured_output_model.invoke("Hello, Polly!!")
assert isinstance(response, Parrot)

Error Message and Stack Trace (if applicable)

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[1], line 32
      9     response: str = Field(description="Parrot's response.")
     11 model = GenericFakeChatModel(
     12     messages=iter(
     13         [
   (...)     29     )
     30 )
---> 32 structured_output_model = model.with_structured_output(Parrot)
     33 response = structured_output_model.invoke("Hello, Polly!!")
     34 assert isinstance(response, Parrot)

File ~/Documents/Projetos/test-langchain/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py:1773, in BaseChatModel.with_structured_output(self, schema, include_raw, **kwargs)
   1771 if type(self).bind_tools is BaseChatModel.bind_tools:
   1772     msg = "with_structured_output is not implemented for this model."
-> 1773     raise NotImplementedError(msg)
   1775 llm = self.bind_tools(
   1776     [schema],
   1777     tool_choice="any",
   (...)   1781     },
   1782 )
   1783 if isinstance(schema, type) and is_basemodel_subclass(schema):

NotImplementedError: with_structured_output is not implemented for this model.

Description

I tried testing the structured output with the fake chat models to test a possible bug, but I ran into another. It seems mock structure outputs aren't implemented for the fake models; this specific example is for GenericFakeChatModel.

The fix doesn't seem to be very complex. I'd be happy to try implementing it and make a pull request.

System Info

System Information

OS: Linux OS Version: #106-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 6 07:58:08 UTC 2026 Python Version: 3.12.3 (main, Mar 3 2026, 12:15:18) [GCC 13.3.0]

Package Information

langchain_core: 1.2.22 langchain: 1.2.13 langsmith: 0.7.22 langgraph_sdk: 0.3.12

Optional packages not installed

deepagents deepagents-cli

Other Dependencies

httpx: 0.28.1 jsonpatch: 1.33 langgraph: 1.1.3 orjson: 3.11.7 packaging: 26.0 pydantic: 2.12.5 pyyaml: 6.0.3 requests: 2.33.0 requests-toolbelt: 1.0.0 tenacity: 9.1.4 typing-extensions: 4.15.0 uuid-utils: 0.14.1 xxhash: 3.6.0 zstandard: 0.25.0

extent analysis

Fix Plan

To fix the NotImplementedError when using with_structured_output with GenericFakeChatModel, we need to implement this method for the fake chat models.

Here are the steps to implement the fix:

  • Implement the with_structured_output method in the GenericFakeChatModel class.
  • This method should return a new instance of GenericFakeChatModel with the structured output schema.

Example code:

from langchain_core.language_models.chat_models import BaseChatModel
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel

class GenericFakeChatModel(GenericFakeChatModel):
    def with_structured_output(self, schema, include_raw=False, **kwargs):
        # Implement the logic to create a new instance with structured output
        class StructuredOutputModel(GenericFakeChatModel):
            def __init__(self, *args, **kwargs):
                super().__init__(*args, **kwargs)
                self.schema = schema

            def invoke(self, *args, **kwargs):
                response = super().invoke(*args, **kwargs)
                # Parse the response using the schema
                parsed_response = self.schema.parse_obj(response)
                return parsed_response

        return StructuredOutputModel(
            messages=self.messages,
            # Other arguments...
        )

Verification

To verify that the fix worked, you can use the following example code:

from pydantic import BaseModel, Field

class Parrot(BaseModel):
    name: str = Field(description="Parrot name.")
    species: str = Field(description="Parrot species.")
    response: str = Field(description="Parrot's response.")

model = GenericFakeChatModel(
    messages=iter(
        [
            AIMessage(
                content="",
                tool_calls=[
                    ToolCall(
                        name="Parrot",
                        args={
                            "name": "Polly",
                            "species": "Norwegian Blue",
                            "response": "...",
                        },
                        id="call_1",
                    )
                ],
            ),
        ]
    )
)

structured_output_model = model.with_structured_output(Parrot)
response = structured_output_model.invoke("Hello, Polly!!")
assert isinstance(response, Parrot)

This code should now run without raising a NotImplementedError.

Extra Tips

  • Make sure to test the with_structured_output method thoroughly to ensure it works as expected.
  • Consider adding documentation to the GenericFakeChatModel class to explain how to use the with_structured_output method.
  • If you're planning to submit a pull request, make sure to follow the project's contribution guidelines.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#GPU compatibility#latency issue#model loading

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING