PR #25371: fix(chatgpt): normalize Responses input and health checks

• Repository: BerriAI/litellm
• Author: afoninsky
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25371

Description (problem / solution / changelog)

Summary

This PR fixes two ChatGPT Responses regressions that showed up when routing ChatGPT models through LiteLLM.

The first commit makes the ChatGPT Responses provider normalize plain string input values into the message-list shape that ChatGPT expects. The second commit updates the health-check path to send the canonical structured Responses payload instead of relying on provider-specific recovery.

Root Cause

The generic Responses API flow allows string inputs, but ChatGPT's /responses endpoint rejects them with 400 and {"detail":"Input must be a list"}. That surfaced in two places:

1. Provider calls that reached ChatGPT with a plain string input
2. UI health checks for mode: responses, which were sending input=prompt or "test"

Changes

• normalize string ChatGPT Responses input into a single user message with input_text
• add a ChatGPT provider regression test for string input normalization
• add a small helper for canonical Responses health-check input
• update the Responses health handler to use structured input
• add health-check tests covering the structured Responses path

Impact

• chatgpt/* models using the Responses API no longer fail when a caller passes a plain string input through LiteLLM
• ChatGPT model health checks stop returning 400 due to invalid Responses input shape
• the health-check path now uses the canonical Responses message format instead of relying on provider-specific normalization

Validation

Ran in Docker with Python 3.11 and LiteLLM proxy extras installed:

docker run --rm -v "$PWD:/work" -w /work python:3.11-slim sh -lc \
  'pip install -q --upgrade pip setuptools wheel && \
   pip install -q -e ".[proxy]" pytest pytest-asyncio httpx && \
   pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py \
          tests/test_litellm/litellm_core_utils/test_health_check_helpers.py -q'

Result: 21 passed

Commit Structure

• fix(chatgpt): normalize string responses input
• fix(health): use structured responses input

Changed files

• litellm/litellm_core_utils/health_check_helpers.py (modified, +19/-1)
• litellm/llms/chatgpt/responses/transformation.py (modified, +13/-0)
• tests/test_litellm/litellm_core_utils/test_health_check_helpers.py (modified, +35/-1)
• tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py (modified, +18/-0)

PR #25403: fix(chatgpt): preserve responses routing and recover empty output

• Repository: BerriAI/litellm
• Author: afoninsky
• State: open | merged: False
• Link: https://github.com/BerriAI/litellm/pull/25403

Description (problem / solution / changelog)

Summary

Fix ChatGPT /v1/chat/completions proxy failures when multiple aliases share the same gpt-5.4 backend model.

Root cause

There were two separate breakpoints in the ChatGPT proxy path:

1. Router startup re-registered shared backend keys like chatgpt/gpt-5.4 using deployment-level model_info. If one alias set mode: chat, that last write downgraded the shared backend key from responses to chat, which sent /v1/chat/completions to the legacy ChatGPT chat-completions endpoint and surfaced as 403 Forbidden.
2. Even after routing was corrected, ChatGPT non-stream Responses payloads could still end with response.completed.output=[] while the real assistant message had already been emitted in earlier SSE events. The bridge then raised Unknown items in responses API response: [].

What changed

• preserve the existing shared backend mode when router deployment registration reuses a provider/model key that already exists in litellm.model_cost
• teach the ChatGPT Responses parser to recover response.output_item.done entries when response.completed.output is empty
• add a defensive /responses -> /chat/completions bridge fallback that reconstructs output items from raw SSE when raw_response.output is empty
• add regression coverage for:
  • shared ChatGPT aliases reusing the same backend model without downgrading responses routing
  • ChatGPT SSE parsing when response.completed.output == []
  • bridge recovery from raw SSE response.output_text.done events

Validation

Ran targeted regression tests in the LiteLLM container runtime:

• tests/test_litellm/test_router_model_cost_isolation.py -k downgrade
• tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -k recovers_output_items
• tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py -k recovers_empty_output_from_raw_sse

Changed files

• litellm/completion_extras/litellm_responses_transformation/transformation.py (modified, +157/-20)
• litellm/llms/chatgpt/responses/transformation.py (modified, +19/-4)
• litellm/router.py (modified, +12/-0)
• tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py (modified, +234/-0)
• tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py (modified, +83/-0)
• tests/test_litellm/proxy/prompts/test_prompt_endpoints.py (modified, +6/-3)
• tests/test_litellm/test_router_model_cost_isolation.py (modified, +76/-0)