Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Bug]: ChatGPT responses allowlist drops text, so response_format/json_schema never reaches ChatGPT Codex backend [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#24356•Fetched 2026-04-08 01:18:16
View on GitHub
Comments
0
Participants
1
Timeline
4
Reactions
0
Author
HeMulingHeMuling
Participants
HeMulingHeMuling
Timeline (top)
cross-referenced ×2labeled ×1referenced ×1

Error Message

2026-03-22 21:15:11,147 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0) 2026-03-22 21:15:53,303 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0) 2026-03-22 21:16:14,516 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)

Root Cause

Because of that, the structured-output constraint never reaches the backend. The backend then returns normal text instead of schema-conforming JSON, and downstream parsing fails.

Fix Action

Fixed

PR fix notes

PR #24359: [#24356] Preserve text and parallel_tool_calls in ChatGPT responses allowlist

Description (problem / solution / changelog)

Relevant issues

Fixes #24356

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

  • Branch creation CI run
    Link: pending

  • CI run for the last commit
    Link: pending

  • Merge / cherry-pick CI run
    Links: pending

Type

🐛 Bug Fix ✅ Test

Changes

Summary

  • Add text and parallel_tool_calls back to the ChatGPT responses allowlist.
  • Add regression coverage to preserve text.format (json_schema, json_object, text) and parallel_tool_calls while still dropping unsupported params.

Background

  • response_format was already transformed into text.format by the chat/completions -> responses bridge, but ChatGPTResponsesAPIConfig.transform_responses_api_request() dropped text because it was missing from allowed_keys.
  • Direct backend validation against https://chatgpt.com/backend-api/codex/responses shows the backend accepts both text and text.format.type=json_schema.
  • I also verified parallel_tool_calls=false is accepted and echoed by the backend, so this was another allowlist omission.

Relevant logs

Direct backend proof:

=== plain ===
status 200
event: response.created
... "text":{"format":{"type":"text"},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"ok"

=== json_schema ===
status 200
event: response.created
... "text":{"format":{"type":"json_schema","name":"answer_schema","schema":{"type":"object","properties":{"answer":{"type":"string"}},"required":["answer"],"additionalProperties":false},"strict":true},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"{\"answer\":\"ok\"}"

Downstream failure before fix:

2026-03-22 21:15:11,147 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:15:11,147 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current?page=2 未提取到有效选择器
2026-03-22 21:15:53,303 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)

LiteLLM transformation repro before fix:

bridge keys: ['client', 'input', 'litellm_logging_obj', 'model', 'stream', 'text']
provider keys: ['include', 'input', 'instructions', 'model', 'store', 'stream']
text missing from final provider request

Validation after fix:

completion(..., response_format=json_schema, parallel_tool_calls=False)
CONTENT {"answer":"ok"}
PARSED {"answer": "ok"}

responses(..., text_format=Answer, parallel_tool_calls=False)
OUTPUT_TEXT {"answer":"ok"}
PARSED {"answer": "ok"}

Verification

  • pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -vv -n 4 -> 18 passed
  • pytest tests/test_litellm/proxy/client/test_chat.py tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -vv -n 4 -> 27 passed
  • ruff check litellm/llms/chatgpt/responses/transformation.py tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -> passed
  • black --check litellm/llms/chatgpt/responses/transformation.py tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -> passed

Current repo-wide blockers

  • pytest tests/test_litellm -x -vv -n 4 currently fails at collection with AttributeError: module 'responses' has no attribute 'activate' in tests/test_litellm/proxy/client/test_chat.py, but that file passes when run alone and when run alongside the new ChatGPT tests.
  • repo-wide black --check . also reports unrelated pre-existing formatting diffs under litellm/proxy/enterprise/litellm_enterprise/....

Changed files

PR #24363: [#24356] Preserve text and parallel_tool_calls in ChatGPT responses allowlist

Description (problem / solution / changelog)

Relevant issues

Fixes #24356

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.

Type

🐛 Bug Fix ✅ Test

Changes

Summary

  • Add text and parallel_tool_calls back to the ChatGPT responses allowlist.
  • Add regression coverage to preserve text.format (json_schema, json_object, text) and parallel_tool_calls while still dropping unsupported params.

Background

  • The chat/completions -> responses bridge already transforms response_format into text.format, but ChatGPTResponsesAPIConfig.transform_responses_api_request() dropped text because it was missing from allowed_keys.
  • Direct backend validation against https://chatgpt.com/backend-api/codex/responses shows the ChatGPT Codex backend accepts both text and text.format.type=json_schema.
  • I also verified parallel_tool_calls=false is accepted and echoed by the backend, so this was another allowlist omission.

Relevant logs

Direct backend proof:

=== plain ===
status 200
event: response.created
... "text":{"format":{"type":"text"},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"ok"

=== json_schema ===
status 200
event: response.created
... "text":{"format":{"type":"json_schema","name":"answer_schema","schema":{"type":"object","properties":{"answer":{"type":"string"}},"required":["answer"],"additionalProperties":false},"strict":true},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"{\"answer\":\"ok\"}"

Downstream failure before fix:

2026-03-22 21:15:11,147 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:15:11,147 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current?page=2 未提取到有效选择器
2026-03-22 21:15:53,303 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)

LiteLLM transformation repro before fix:

bridge keys: ['client', 'input', 'litellm_logging_obj', 'model', 'stream', 'text']
provider keys: ['include', 'input', 'instructions', 'model', 'store', 'stream']
text missing from final provider request

Verification

  • ruff check litellm/llms/chatgpt/responses/transformation.py tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -> passed
  • black --check litellm/llms/chatgpt/responses/transformation.py tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -> passed
  • PYTHONPATH=/Users/hemuling/Projects/litellm-pr24356-clean /Users/hemuling/Projects/journal-database/.venv/bin/python -m pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -vv -n 4 -> 18 passed

Changed files

  • litellm/llms/chatgpt/responses/transformation.py (modified, +2/-0)
  • tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py (modified, +61/-4)

Code Example

from litellm.completion_extras.litellm_responses_transformation.transformation import LiteLLMResponsesTransformationHandler
from litellm.llms.chatgpt.responses.transformation import ChatGPTResponsesAPIConfig

handler = LiteLLMResponsesTransformationHandler()
request = handler.transform_request(
    model="chatgpt/gpt-5.4",
    messages=[{"role": "user", "content": "Reply with ok as JSON."}],
    optional_params={
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "answer_schema",
                "schema": {
                    "type": "object",
                    "properties": {"answer": {"type": "string"}},
                    "required": ["answer"],
                    "additionalProperties": False,
                },
                "strict": True,
            },
        }
    },
    litellm_params={},
    headers={},
)

assert "text" in request
provider_request = ChatGPTResponsesAPIConfig().transform_responses_api_request(request)
assert "text" not in provider_request

---

2026-03-22 21:15:11,147 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:15:11,147 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current?page=2 未提取到有效选择器
2026-03-22 21:15:11,578 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current 未提取到有效选择器
2026-03-22 21:15:53,303 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:16:14,516 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)

---

=== plain ===
status 200
event: response.created
... "text":{"format":{"type":"text"},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"ok"

=== json_schema ===
status 200
event: response.created
... "text":{"format":{"type":"json_schema","description":null,"name":"answer_schema","schema":{"type":"object","properties":{"answer":{"type":"string"}},"required":["answer"],"additionalProperties":false},"strict":true},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"{\"answer\":\"ok\"}"
RAW_BUFFERClick to expand / collapse

I searched the existing issues and did not find an exact duplicate. The closest related items are #21193, #21209, #13789, #14206, and #19030.

What happened?

When using LiteLLM Chat Completions with chatgpt/gpt-5.4 and response_format=json_schema, the chat/completions -> responses bridge correctly converts response_format into text.format, but the ChatGPT responses provider drops text before sending the final request to https://chatgpt.com/backend-api/codex/responses.

Because of that, the structured-output constraint never reaches the backend. The backend then returns normal text instead of schema-conforming JSON, and downstream parsing fails.

Expected behavior:

  • LiteLLM should preserve text for the ChatGPT responses request once response_format has already been transformed into text.format.
  • This appears to be fallout from the conservative allowlist added in #21209 for #21193, where text was omitted.
  • Direct backend validation shows the ChatGPT Codex backend currently accepts both text.format and text.format.type=json_schema.

Relevant code paths:

  • Bridge creates text from response_format in litellm/completion_extras/litellm_responses_transformation/transformation.py.
  • response_format is mapped to responses_api_request["text"] at line 248.
  • _transform_response_format_to_text_format() builds the text.format payload at lines 875-920.
  • ChatGPT provider filters it out in litellm/llms/chatgpt/responses/transformation.py because text is missing from allowed_keys at lines 92-104, and the filtered dict is returned at line 106.

Steps to Reproduce

  1. Use LiteLLM Chat Completions with model="chatgpt/gpt-5.4" and a response_format that requests structured output.
  2. The bridge layer transforms that into a Responses API request with text.format.
  3. The ChatGPT responses provider applies an allowlist and drops text.
  4. The backend receives no schema constraint, returns plain text, and downstream JSON parsing fails.

Minimal transform-level repro:

from litellm.completion_extras.litellm_responses_transformation.transformation import LiteLLMResponsesTransformationHandler
from litellm.llms.chatgpt.responses.transformation import ChatGPTResponsesAPIConfig

handler = LiteLLMResponsesTransformationHandler()
request = handler.transform_request(
    model="chatgpt/gpt-5.4",
    messages=[{"role": "user", "content": "Reply with ok as JSON."}],
    optional_params={
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "answer_schema",
                "schema": {
                    "type": "object",
                    "properties": {"answer": {"type": "string"}},
                    "required": ["answer"],
                    "additionalProperties": False,
                },
                "strict": True,
            },
        }
    },
    litellm_params={},
    headers={},
)

assert "text" in request
provider_request = ChatGPTResponsesAPIConfig().transform_responses_api_request(request)
assert "text" not in provider_request

Relevant log output

Downstream failure symptom:

2026-03-22 21:15:11,147 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:15:11,147 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current?page=2 未提取到有效选择器
2026-03-22 21:15:11,578 - selector_updater - WARNING - 从URL https://www.cell.com/cell/current 未提取到有效选择器
2026-03-22 21:15:53,303 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)
2026-03-22 21:16:14,516 - html_analyzer - ERROR - JSON解析失败: Expecting value: line 1 column 1 (char 0)

Direct backend support proof from https://chatgpt.com/backend-api/codex/responses:

=== plain ===
status 200
event: response.created
... "text":{"format":{"type":"text"},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"ok"

=== json_schema ===
status 200
event: response.created
... "text":{"format":{"type":"json_schema","description":null,"name":"answer_schema","schema":{"type":"object","properties":{"answer":{"type":"string"}},"required":["answer"],"additionalProperties":false},"strict":true},"verbosity":"medium"} ...
event: response.output_text.done
... "text":"{\"answer\":\"ok\"}"

This suggests the backend itself supports text / json_schema; the request is being stripped inside LiteLLM before it gets there.

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on?

1.82.6

extent analysis

Fix Plan

To resolve the issue, we need to modify the ChatGPTResponsesAPIConfig class to include text in the allowed_keys list. Here are the steps:

  • Open the file litellm/llms/chatgpt/responses/transformation.py
  • Locate the allowed_keys list (lines 92-104)
  • Add 'text' to the allowed_keys list

Example code:

allowed_keys = [
    # ... existing keys ...
    'text',  # Add this line
    # ... existing keys ...
]

Alternatively, you can modify the transform_responses_api_request method to explicitly include the text key in the filtered dictionary:

def transform_responses_api_request(self, request):
    # ... existing code ...
    filtered_request = {k: v for k, v in request.items() if k in allowed_keys or k == 'text'}
    return filtered_request

Verification

To verify that the fix worked, run the minimal transform-level repro code again:

provider_request = ChatGPTResponsesAPIConfig().transform_responses_api_request(request)
assert "text" in provider_request

This should pass without raising an AssertionError.

Extra Tips

  • Make sure to test the fix thoroughly to ensure that it does not introduce any regressions.
  • Consider adding a test case to the LiteLLM test suite to prevent similar issues in the future.
  • If you are using a version control system, create a new branch for the fix and merge it into the main branch after testing and verification.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#response parsing#generation error#database connection#vector store

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING