Issue Station
Sign In
Tools hereToolbox

litellm - ✅(Solved) Fix [Feature]: azure_ai: use real /responses upstream for litellm.responses() when api_base includes /projects/ [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#25407•Fetched 2026-04-10 03:41:18
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
PenpikaPenpika
Participants
PenpikaPenpika
Timeline (top)
labeled ×3cross-referenced ×1referenced ×1

Please support direct upstream /responses calls for azure_ai in litellm.responses(...) when api_base includes /projects/.

Root Cause

Please support direct upstream /responses calls for azure_ai in litellm.responses(...) when api_base includes /projects/.

PR fix notes

PR #25462: feat(azure_ai): native /responses routing for project-based endpoints

Description (problem / solution / changelog)

Summary

  • Adds AzureAIResponsesAPIConfig so that litellm.responses() routes to the real upstream /responses endpoint when api_base contains /projects/ (Azure AI Foundry project-based endpoints), instead of falling back to the completions-style bridge.
  • Handles Azure-specific auth (api-key header for *.services.ai.azure.com / *.openai.azure.com, Bearer token otherwise).
  • URL construction: /projects/ base appends /openai/v1/responses; plain services.ai.azure.com base appends /models/responses.

Fixes #25407

Test plan

  • 10 new unit tests in tests/test_litellm/llms/azure_ai/responses/test_azure_ai_responses.py covering:
    • Provider config registration returns AzureAIResponsesAPIConfig for azure_ai
    • URL construction for project-based, services.ai.azure.com, and generic api_base patterns
    • api-version query param propagation
    • api-key vs Bearer auth header selection based on host
    • ValueError when api_base is missing
    • E2E mock test verifying litellm.responses() routes through native endpoint

Changed files

  • litellm/__init__.py (modified, +69/-54)
  • litellm/_lazy_imports_registry.py (modified, +5/-0)
  • litellm/llms/azure_ai/responses/__init__.py (added, +0/-0)
  • litellm/llms/azure_ai/responses/transformation.py (added, +136/-0)
  • litellm/responses/main.py (modified, +25/-25)
  • litellm/utils.py (modified, +25/-25)
  • tests/test_litellm/llms/azure_ai/responses/test_azure_ai_responses.py (added, +221/-0)
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

The Feature

Summary

Please support direct upstream /responses calls for azure_ai in litellm.responses(...) when api_base includes /projects/.

Current behavior

With azure_ai, litellm.responses(...) currently falls back to completions-style bridge behavior in my environment.

Requested behavior

If api_base includes /projects/, please allow litellm.responses(...) to call the real upstream /responses endpoint.

Suggested behavior:

  • If the configured URL includes /projects/, use direct /responses routing.
  • Otherwise, existing behavior can remain as-is.

URL example

  • https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/responses

Why this is needed

I want litellm.responses(...) to follow the Azure AI Foundry Responses API path directly when using a project-based endpoint.

References

Motivation, pitch

I'm using azure_ai with Azure AI Foundry.

I found that, to call models such as Claude and DeepSeek in Responses API format on Azure AI Foundry, a project-based endpoint with /projects/ in api_base is needed. However, when I use that endpoint through LiteLLM, I found that litellm.responses(...) does not route to the actual upstream /responses endpoint and instead falls back to completions-style bridge behavior.

I'd like LiteLLM to support direct /responses upstream routing for this Azure AI Foundry project-based endpoint pattern so that models such as Claude and DeepSeek can be called consistently through the Responses API interface.

What part of LiteLLM is this about?

SDK (litellm Python package)

LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

Twitter / LinkedIn details

No response

extent analysis

TL;DR

Modify the litellm.responses(...) function to directly call the upstream /responses endpoint when the api_base includes /projects/.

Guidance

  • Check the api_base URL for the presence of /projects/ to determine the routing behavior.
  • Update the litellm.responses(...) function to use direct /responses routing when the condition is met.
  • Verify the new behavior by testing litellm.responses(...) with a project-based endpoint, such as https://<resource>.services.ai.azure.com/api/projects/<project>/openai/v1/responses.
  • Review the Azure AI Foundry Responses API documentation to ensure compatibility with the updated routing behavior.

Example

No code snippet is provided due to the lack of specific implementation details in the issue.

Notes

The solution assumes that the litellm.responses(...) function can be modified to support direct /responses routing. Additional testing and validation may be necessary to ensure the updated behavior works correctly with different endpoint configurations.

Recommendation

Apply workaround: Modify the litellm.responses(...) function to support direct /responses routing for project-based endpoints, as this will allow for consistent calling of models like Claude and DeepSeek through the Responses API interface.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#mixed precision#training loop#device allocation#model download

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING