PR #23577: feat(openai): route gpt-5.4+ tools+reasoning to Responses API

• Repository: BerriAI/litellm
• Author: Sameerlite
• State: closed | merged: True
• Link: https://github.com/BerriAI/litellm/pull/23577

Description (problem / solution / changelog)

Summary

This PR changes how gpt-5.4+ requests with both tools and reasoning_effort are handled:

1. Route instead of drop: Instead of silently dropping reasoning_effort when tools are present, we route these requests to the Responses API, which supports both tools and reasoning.
2. Dict normalization: We normalize reasoning_effort dicts (e.g. {"effort": "high", "summary": "detailed"}) to strings for the Chat Completions API, and restore the full dict when routing to the Responses API.
3. Summary preservation: When routing to Responses, we restore the original reasoning_effort dict (including summary) before calling the Responses API bridge.

---

Response to Review Comments

1. Hardcoded model detection in main.py

The routing decision is driven by an API constraint, not a model capability. OpenAI's Chat Completions API rejects reasoning_effort when tools are present for gpt-5.4+. The Responses API supports both. This is an API boundary, not a model-specific flag.

is_model_gpt_5_4_plus_model() uses version parsing (gpt-5.4, gpt-5.5, gpt-5.6, etc.), so future models like gpt-5.5 and gpt-5.6 are handled without code changes. The only "hardcoded" part is the version threshold (4), which reflects when this API behavior was introduced.

model_prices_and_context_window.json is used for pricing and context windows. This routing rule is about which API endpoint to use, not pricing or capabilities. Adding a JSON flag would require updates for every new model; the version-based check keeps behavior consistent and self-documenting.

2. Provider-specific import in main.py

main.py already imports provider-specific modules for routing (e.g. OpenAIWebSearchOptions, VertexAIModelRoute). The routing logic needs to know whether the model is an OpenAI gpt-5.4+ model. That check is centralized in OpenAIGPT5Config, which is the same pattern used elsewhere for OpenAI-specific behavior.

3. Fragile summary-field restoration

Restoration uses reasoning_effort from the completion call's kwargs, which is the common path when users pass it directly. Deployment configs that inject reasoning_effort into optional_params before the call are a separate, less common path. The current change fixes the main case (direct argument) and aligns with how other params are handled. Improving the deployment-config path can be done in a follow-up if needed.

4. Silent summary drop for non-gpt-5.4+ models

The Chat Completions API expects reasoning_effort as a string ("none", "low", "medium", "high", "xhigh"), not a dict. Sending {"effort": "high", "summary": "detailed"} would be rejected by OpenAI. Normalizing to a string is required for chat completions. The summary field is only used by the Responses API. For chat-path models, we must send a string; the normalization is correct and not a "silent drop."

5. Pre-submission checklist / GitHub issue

Checklist items will be completed before merge. A related issue can be linked if one exists.

Changed files

• tests/test_litellm/llms/openai/chat/test_openai_gpt_transformation.py (modified, +27/-31)

---

PR #23926: fix(azure): auto-route gpt-5.4+ tools+reasoning to Responses API

• Repository: BerriAI/litellm
• Author: Chesars
• State: closed | merged: True
• Link: https://github.com/BerriAI/litellm/pull/23926

Description (problem / solution / changelog)

Relevant issues

Fixes #23914

Pre-Submission checklist

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

Azure GPT-5.4+ models silently drop reasoning_effort when tools are also present in litellm.completion(). OpenAI already auto-routes these requests to the Responses API (which supports both params), but Azure was excluded from this routing.

Fix

1. litellm/main.py — Moved the gpt-5.4+ auto-routing check out of the try block (Azure models aren't in model_cost map) and extended it to include custom_llm_provider == "azure" alongside "openai".

2. litellm/llms/azure/chat/gpt_5_transformation.py — Removed the code that silently dropped reasoning_effort when tools were present for gpt-5.4+ models. This is no longer needed since requests are now routed to the Responses API bridge.

3. docs/my-website/docs/reasoning_content.md — Updated the docs tip to reflect that auto-routing now works for both OpenAI and Azure.

Testing

• 2 new unit tests for Azure routing in test_main.py
• Updated existing test from asserting reasoning_effort is dropped to asserting it's preserved
• Verified e2e with Azure gpt-5-nano deployment (same deployment serves both /chat/completions and /responses endpoints)

Changed files

• docs/my-website/docs/reasoning_content.md (modified, +19/-2)
• litellm/llms/azure/chat/gpt_5_transformation.py (modified, +3/-8)
• litellm/main.py (modified, +13/-10)
• tests/test_litellm/llms/azure/chat/test_azure_gpt5_transformation.py (modified, +5/-4)
• tests/test_litellm/test_main.py (modified, +34/-0)