langchain - 💡(How to fix) Fix Structured Output Fails When Appending Assistant Response to Context [1 participants]

ErfanMojibi · 2026-04-21T15:48:03Z

[langchain] When using with structured output in a multi-turn conversation, appending the previous assistant response in my case, serialized via model dump jso… When using `with_structured_output()` in a multi-turn conversation, appending the previous assistant response (in my case, serialized via `model_dump_json()`) to the message history causes structured output to fail or behave unexpectedly on subsequent invocations. ## Fix / Workaround - [x] This is a bug, not a usage question. - [ ] I added a clear and descriptive title that summarizes this issue. - [x] I used the GitHub search to find a similar question and didn't find it. - [ ] I am sure that this is a bug in LangChain rather than my code. - [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package). - [x] This is not related to the langchain-community package. - [x] I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS. Other Dependencies ------------------ > aiohttp: 3.13.5 > dataclasses-json: 0.6.7 > filetype: 1.2.0 > httpx: 0.28.1 > httpx-sse: 0.4.3 > jsonpatch: 1.33 > numpy: 2.4.4 > openai: 2.30.0 > orjson: 3.11.8 > packaging: 26.1 > pydantic: 2.12.5 > pydantic-settings: 2.13.1 > PyYAML: 6.0.3 > pyyaml: 6.0.3 > requests: 2.33.1 > requests-toolbelt: 1.0.0 > SQLAlchemy: 2.0.49 > sqlalchemy: 2.0.49 > tenacity: 9.1.4 > tiktoken: 0.12.0 > typing-extensions: 4.15.0 > uuid-utils: 0.14.1 > xxhash: 3.6.0 > zstandard: 0.25.0 ### Submission checklist - [x] This is a bug, not a usage question. - [ ] I added a clear and descriptive title that summarizes this issue. - [x] I used the GitHub search to find a similar question and didn't find it. - [ ] I am sure that this is a bug in LangChain rather than my code. - [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package). - [x] This is not related to the langchain-community package. - [x] I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS. ### Package (Required) - [x] langchain - [x] langchain-openai - [ ] langchain-anthropic - [ ] langchain-classic - [ ] langchain-core - [ ] langchain-model-profiles - [ ] langchain-tests - [ ] langchain-text-splitters - [ ] langchain-chroma - [ ] langchain-deepseek - [ ] langchain-exa - [ ] langchain-fireworks - [ ] langchain-groq - [ ] langchain-huggingface - [ ] langchain-mistralai - [ ] langchain-nomic - [ ] langchain-ollama - [ ] langchain-openrouter - [ ] langchain-perplexity - [ ] langchain-qdrant - [ ] langchain-xai - [ ] Other / not sure / general ### Related Issues / PRs _No response_ ### Reproduction Steps / Example Code (Python) ```python from pydantic import BaseModel from langchain_openai import ChatOpenAI class Step(BaseModel): explanation: str output: str class MathReasoning(BaseModel): steps: list[Step] final_answer: str llm = ChatOpenAI(model="openai/gpt-oss-20b", base_url=BASE_URL, api_key=API_KEY) structured_llm = llm.with_structured_output(MathReasoning) in_list = [ {"role": "system", "content": "You are a helpful math tutor."}, {"role": "user", "content": "how can I solve 8x + 7 = -23"}, ] # First call — works response = structured_llm.invoke(in_list) # Append assistant response as context in_list.append({"role": "assistant", "content": MathReasoning.model_dump_json(response)}) in_list.append({"role": "user", "content": "what is the final answer?"}) # Second call — structured output fails response = structured_llm.invoke(in_list) ``` ### Error Message and Stack Trace (if applicable) ```shell ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field. Received message: content='' additional_kwargs={'parsed': None, 'refusal': None} response_metadata={'token_usage': {'completion_tokens': 98, 'prompt_tokens': 392, 'total_tokens': 490, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'openai/gpt-oss-120b', 'system_fingerprint': None, 'id': 'chatcmpl-8ca96bb38540b711', 'finish_reason': 'stop', 'logprobs': None} id='lc_run--019db0b6-a8ae-7833-9392-87341a50c1e2-0' tool_calls=[] invalid_tool_calls=[] usage_metadata={'input_tokens': 392, 'output_tokens': 98, 'total_tokens': 490, 'input_token_details': {}, 'output_token_details': {}} ``` ### Description When using `with_structured_output()` in a multi-turn conversation, appending the previous assistant response (in my case, serialized via `model_dump_json()`) to the message history causes structured output to fail or behave unexpectedly on subsequent invocations. ### Steps to Reproduce 1. Define a Pydantic model and create a structured LLM via `llm.with_structured_output(MyModel)` 2. Invoke it with an initial system + user message — works fine 3. Serialize the response with `MyModel.model_dump_json(response)` and append it as an `assistant` message 4. Add a new `user` message and invoke again 5. Structured out

langchain2026-04-21 15:48:03

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

langchain-ai/langchain#36916•Fetched 2026-04-22 07:43:33

View on GitHub

Comments

Participants

Timeline

Reactions

Author

ErfanMojibi

Participants

ErfanMojibi

Timeline (top)

labeled ×4issue_type_added ×1

When using with_structured_output() in a multi-turn conversation, appending the previous assistant response (in my case, serialized via model_dump_json()) to the message history causes structured output to fail or behave unexpectedly on subsequent invocations.

Error Message

ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field. Received message:

content='' additional_kwargs={'parsed': None, 'refusal': None} response_metadata={'token_usage': {'completion_tokens': 98, 'prompt_tokens': 392, 'total_tokens': 490, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'openai/gpt-oss-120b', 'system_fingerprint': None, 'id': 'chatcmpl-8ca96bb38540b711', 'finish_reason': 'stop', 'logprobs': None} id='lc_run--019db0b6-a8ae-7833-9392-87341a50c1e2-0' tool_calls=[] invalid_tool_calls=[] usage_metadata={'input_tokens': 392, 'output_tokens': 98, 'total_tokens': 490, 'input_token_details': {}, 'output_token_details': {}}

Root Cause

Fix Action

Fix / Workaround

This is a bug, not a usage question.
I added a clear and descriptive title that summarizes this issue.
I used the GitHub search to find a similar question and didn't find it.
I am sure that this is a bug in LangChain rather than my code.
The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
This is not related to the langchain-community package.
I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Other Dependencies

aiohttp: 3.13.5 dataclasses-json: 0.6.7 filetype: 1.2.0 httpx: 0.28.1 httpx-sse: 0.4.3 jsonpatch: 1.33 numpy: 2.4.4 openai: 2.30.0 orjson: 3.11.8 packaging: 26.1 pydantic: 2.12.5 pydantic-settings: 2.13.1 PyYAML: 6.0.3 pyyaml: 6.0.3 requests: 2.33.1 requests-toolbelt: 1.0.0 SQLAlchemy: 2.0.49 sqlalchemy: 2.0.49 tenacity: 9.1.4 tiktoken: 0.12.0 typing-extensions: 4.15.0 uuid-utils: 0.14.1 xxhash: 3.6.0 zstandard: 0.25.0

Code Example

from pydantic import BaseModel
from langchain_openai import ChatOpenAI

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

llm = ChatOpenAI(model="openai/gpt-oss-20b", base_url=BASE_URL, api_key=API_KEY)
structured_llm = llm.with_structured_output(MathReasoning)

in_list = [
    {"role": "system", "content": "You are a helpful math tutor."},
    {"role": "user", "content": "how can I solve 8x + 7 = -23"},
]

# First call — works
response = structured_llm.invoke(in_list)

# Append assistant response as context
in_list.append({"role": "assistant", "content": MathReasoning.model_dump_json(response)})
in_list.append({"role": "user", "content": "what is the final answer?"})

# Second call — structured output fails
response = structured_llm.invoke(in_list)

---

ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field. Received message:

content='' additional_kwargs={'parsed': None, 'refusal': None} response_metadata={'token_usage': {'completion_tokens': 98, 'prompt_tokens': 392, 'total_tokens': 490, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'openai/gpt-oss-120b', 'system_fingerprint': None, 'id': 'chatcmpl-8ca96bb38540b711', 'finish_reason': 'stop', 'logprobs': None} id='lc_run--019db0b6-a8ae-7833-9392-87341a50c1e2-0' tool_calls=[] invalid_tool_calls=[] usage_metadata={'input_tokens': 392, 'output_tokens': 98, 'total_tokens': 490, 'input_token_details': {}, 'output_token_details': {}}

RAW_BUFFERClick to expand / collapse

Submission checklist

This is a bug, not a usage question.
I added a clear and descriptive title that summarizes this issue.
I used the GitHub search to find a similar question and didn't find it.
I am sure that this is a bug in LangChain rather than my code.
The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
This is not related to the langchain-community package.
I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

Related Issues / PRs

No response

Reproduction Steps / Example Code (Python)

from pydantic import BaseModel
from langchain_openai import ChatOpenAI

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

llm = ChatOpenAI(model="openai/gpt-oss-20b", base_url=BASE_URL, api_key=API_KEY)
structured_llm = llm.with_structured_output(MathReasoning)

in_list = [
    {"role": "system", "content": "You are a helpful math tutor."},
    {"role": "user", "content": "how can I solve 8x + 7 = -23"},
]

# First call — works
response = structured_llm.invoke(in_list)

# Append assistant response as context
in_list.append({"role": "assistant", "content": MathReasoning.model_dump_json(response)})
in_list.append({"role": "user", "content": "what is the final answer?"})

# Second call — structured output fails
response = structured_llm.invoke(in_list)

Error Message and Stack Trace (if applicable)

ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field. Received message:

content='' additional_kwargs={'parsed': None, 'refusal': None} response_metadata={'token_usage': {'completion_tokens': 98, 'prompt_tokens': 392, 'total_tokens': 490, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_provider': 'openai', 'model_name': 'openai/gpt-oss-120b', 'system_fingerprint': None, 'id': 'chatcmpl-8ca96bb38540b711', 'finish_reason': 'stop', 'logprobs': None} id='lc_run--019db0b6-a8ae-7833-9392-87341a50c1e2-0' tool_calls=[] invalid_tool_calls=[] usage_metadata={'input_tokens': 392, 'output_tokens': 98, 'total_tokens': 490, 'input_token_details': {}, 'output_token_details': {}}

Description

Steps to Reproduce

Define a Pydantic model and create a structured LLM via llm.with_structured_output(MyModel)
Invoke it with an initial system + user message — works fine
Serialize the response with MyModel.model_dump_json(response) and append it as an assistant message
Add a new user message and invoke again
Structured output fails on the second call

If you change the message in line in_list.append({"role": "assistant", "content": MathReasoning.model_dump_json(response)}) to a user message, it works fine!

Expected Behavior

Multi-turn conversations with structured output should support appending prior responses to context gracefully.

Environment

langchain-openai: 1.1.14
pydantic: 2.12.5
Model: openai/gpt-oss-20b
Provider: NVIDIA_NIM

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 25.3.0: Wed Jan 28 20:49:24 PST 2026; root:xnu-12377.81.4~5/RELEASE_ARM64_T8132 Python Version: 3.14.4 (main, Apr 7 2026, 13:13:20) [Clang 21.0.0 (clang-2100.0.123.102)]

Package Information

langchain_core: 1.3.0 langchain_community: 0.4.1 langsmith: 0.7.32 langchain_classic: 1.0.4 langchain_nvidia_ai_endpoints: 1.2.1 langchain_openai: 1.1.14 langchain_text_splitters: 1.1.2 langgraph_sdk: 0.3.13

Optional packages not installed

deepagents deepagents-cli

Other Dependencies

aiohttp: 3.13.5 dataclasses-json: 0.6.7 filetype: 1.2.0 httpx: 0.28.1 httpx-sse: 0.4.3 jsonpatch: 1.33 numpy: 2.4.4 openai: 2.30.0 orjson: 3.11.8 packaging: 26.1 pydantic: 2.12.5 pydantic-settings: 2.13.1 PyYAML: 6.0.3 pyyaml: 6.0.3 requests: 2.33.1 requests-toolbelt: 1.0.0 SQLAlchemy: 2.0.49 sqlalchemy: 2.0.49 tenacity: 9.1.4 tiktoken: 0.12.0 typing-extensions: 4.15.0 uuid-utils: 0.14.1 xxhash: 3.6.0 zstandard: 0.25.0

extent analysis

TL;DR

The issue can be resolved by modifying how the previous assistant response is appended to the message history, potentially by changing the serialization or handling of the structured output.

Guidance

Verify that the MathReasoning.model_dump_json(response) is correctly serializing the response and that the resulting JSON string is properly formatted and can be parsed by the structured_llm.
Check if the issue is related to the specific model or provider being used, and try switching to a different model or provider to see if the issue persists.
Consider modifying the code to handle the structured output differently, such as by using a different serialization method or by parsing the response in a different way.
Review the documentation for langchain_openai and pydantic to ensure that the code is being used correctly and that there are no known issues or limitations that could be contributing to the problem.

Example

import json
from pydantic import BaseModel

class Step(BaseModel):
    explanation: str
    output: str

class MathReasoning(BaseModel):
    steps: list[Step]
    final_answer: str

# ...

response = structured_llm.invoke(in_list)
response_json = json.dumps(response.dict())  # Try using json.dumps instead of model_dump_json
in_list.append({"role": "assistant", "content": response_json})

Notes

The issue may be related to the specific version of langchain_openai or pydantic being used, so trying to reproduce the issue with different versions may be helpful. Additionally, the problem may be specific to the openai/gpt-oss-20b model, so trying a different model could also be useful.

Recommendation

Apply a workaround by modifying the code to handle the structured output differently, such as by using a different serialization method or by parsing the response in a different way. This is because the issue seems to be related to how the previous assistant response is being appended to the message history, and modifying the code to handle this differently may resolve the issue.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #ssr #device allocation #model download #tokenizer error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

langchain - 💡(How to fix) Fix Structured Output Fails When Appending Assistant Response to Context [1 participants]

Recommended Tools

GitHub issue graph ai analysis

Error Message

Root Cause

Fix Action

Fix / Workaround

Other Dependencies

Code Example

Submission checklist

Package (Required)

Related Issues / PRs

Reproduction Steps / Example Code (Python)

Error Message and Stack Trace (if applicable)

Description

Steps to Reproduce

Expected Behavior

Environment

System Info

System Information

Package Information

Optional packages not installed

Other Dependencies

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING