litellm - ✅(Solved) Fix [Bug]: GPT-5.4+ Responses bridge fails with "api_base=None" when calling via langchain create_agent on Azure Cognitive Services endpoint [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
BerriAI/litellm#26897Fetched 2026-05-01 05:34:28
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
1
Participants
Timeline (top)
labeled ×3cross-referenced ×1renamed ×1

Error Message

litellm.APIConnectionError: AzureException APIConnectionError - api_base is required for Azure AI Studio. Please set the api_base parameter. Passed api_base=None Traceback (most recent call last): File ".../litellm/responses/main.py", line 558, in aresponses response = await init_response File ".../litellm/llms/custom_httpx/llm_http_handler.py", line 2261, in async_response_api_handler api_base = responses_api_provider_config.get_complete_url( File ".../litellm/llms/azure/responses/transformation.py", line 180, in get_complete_url return BaseAzureLLM._get_base_azure_url( File ".../litellm/llms/azure/common_utils.py", line 756, in _get_base_azure_url raise ValueError( ValueError: api_base is required for Azure AI Studio. Please set the api_base parameter. Passed api_base=None

Fix Action

Fix / Workaround

This was not fixed by upgrading to litellm==1.83.14 (PR #13526 from issue #13523 addressed default api-version handling for Responses but did not resolve this configuration). Switching AZURE_OPENAI_API_VERSION to 2025-04-01-preview did not change the behavior.

Workaround: monkey-patch OpenAIGPT5Config.is_model_gpt_5_4_plus_model to return False, which keeps the call on Chat Completions and the
request goes through.

PR fix notes

PR #26907: fix: preserve router deployment credentials on none overrides

Description (problem / solution / changelog)

Summary

  • treat api_base=None / api_key=None request values as absent for Router client-side credential detection
  • prevent wrapper defaults from erasing non-null deployment credentials before the LiteLLM call is dispatched
  • keep non-null request credentials working as explicit client-side overrides

Fixes #26897.

Tests

  • /tmp/uv-0.10.9-env/bin/uv run --extra proxy pytest tests/test_litellm/test_router.py -q -k 'preserves_deployment_credentials or non_none_request_credentials'\n- /tmp/uv-0.10.9-env/bin/uv run --extra proxy ruff check --ignore PLR0915 litellm/router.py litellm/router_utils/clientside_credential_handler.py\n- /tmp/uv-0.10.9-env/bin/uv run --extra proxy ruff check --ignore PLR0915,F841,T201,E712,F401 tests/test_litellm/test_router.py\n- git diff --check\n\nNote: tests/test_litellm/test_router.py has existing lint debt outside this patch, so the focused lint command ignores those pre-existing categories for that file while the new regression tests are validated directly.

Changed files

  • litellm/router.py (modified, +20/-0)
  • litellm/router_utils/clientside_credential_handler.py (modified, +9/-4)
  • tests/test_litellm/test_router.py (modified, +108/-0)

Code Example

import asyncio, os
  from dataclasses import dataclass, field
  from typing import Awaitable, Callable

  import litellm
  from litellm import Router
  from langchain_litellm import ChatLiteLLM
  from langchain_core.tools import tool
  from langchain.agents import create_agent
  from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse

  litellm.drop_params = True

  router = Router(model_list=[{
      "model_name": "GPT-5.4",
      "litellm_params": {
          "model": "azure/gpt-5.4",
          "api_base": "https://<resource>.cognitiveservices.azure.com",
          "api_key": "<key>",
          "api_version": "2025-04-01-preview",
          "base_model": "gpt-5.4",
      },
  }])
  llm = ChatLiteLLM(model="GPT-5.4", streaming=True)
  llm.client = router

  @tool
  def search(query: str) -> str:
      """Search for information."""
      return "no results"

  @dataclass
  class Ctx:
      params: dict = field(default_factory=dict)

  class ParamMiddleware(AgentMiddleware):
      async def awrap_model_call(self, request: ModelRequest, handler):
          ctx = request.runtime.context
          if not ctx or not ctx.params:
              return await handler(request)
          extra = dict(request.model_settings or {})
          extra.update(ctx.params)
          return await handler(request.override(model_settings=extra))

  async def main():
      agent = create_agent(model=llm, tools=[search], middleware=[ParamMiddleware()], context_schema=Ctx)
      # Fails:
      await agent.ainvoke(
          {"messages": [{"role": "user", "content": "hi"}]},
          context=Ctx(params={"reasoning_effort": "medium"}),
      )

  asyncio.run(main())

---

litellm.APIConnectionError: AzureException APIConnectionError - api_base is required for Azure AI Studio. Please set the api_base parameter. Passed 
  api_base=None
  Traceback (most recent call last):
    File ".../litellm/responses/main.py", line 558, in aresponses
      response = await init_response
    File ".../litellm/llms/custom_httpx/llm_http_handler.py", line 2261, in async_response_api_handler
      api_base = responses_api_provider_config.get_complete_url(
    File ".../litellm/llms/azure/responses/transformation.py", line 180, in get_complete_url
      return BaseAzureLLM._get_base_azure_url(
    File ".../litellm/llms/azure/common_utils.py", line 756, in _get_base_azure_url
      raise ValueError(
  ValueError: api_base is required for Azure AI Studio. Please set the api_base parameter. Passed api_base=None
RAW_BUFFERClick to expand / collapse

Check for existing issues

  • I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using langchain.agents.create_agent with langchain_litellm.ChatLiteLLM (backed by litellm.Router for an Azure GPT-5.4 deployment on a
*.cognitiveservices.azure.com endpoint), passing reasoning_effort together with tools causes every call to fail with:

litellm.APIConnectionError: AzureException APIConnectionError - api_base is required for Azure AI Studio. Please set the api_base parameter. Passed api_base=None

Removing reasoning_effort makes the same agent succeed. Calling litellm.Router.acompletion directly with the same model, tools, and reasoning_effort also succeeds — so the trigger is something specific to the create_agent/ChatLiteLLM path (likely bind_tools or the way
agent middleware forwards params), not raw litellm.acompletion.

The error originates from the GPT-5.4+ auto-bridge in litellm/main.py:responses_api_bridge_check (~line 978) which forces calls with tools + reasoning_effort onto the Azure Responses path, where the URL builder in litellm/llms/azure/responses/transformation.py:180 raises api_base=None.

This was not fixed by upgrading to litellm==1.83.14 (PR #13526 from issue #13523 addressed default api-version handling for Responses but did not resolve this configuration). Switching AZURE_OPENAI_API_VERSION to 2025-04-01-preview did not change the behavior.

Workaround: monkey-patch OpenAIGPT5Config.is_model_gpt_5_4_plus_model to return False, which keeps the call on Chat Completions and the
request goes through.

Steps to Reproduce

Minimal reproduction (verified — direct Router call does NOT reproduce, must go through create_agent + ChatLiteLLM):

import asyncio, os
from dataclasses import dataclass, field
from typing import Awaitable, Callable

import litellm
from litellm import Router
from langchain_litellm import ChatLiteLLM
from langchain_core.tools import tool
from langchain.agents import create_agent
from langchain.agents.middleware import AgentMiddleware, ModelRequest, ModelResponse

litellm.drop_params = True

router = Router(model_list=[{
    "model_name": "GPT-5.4",
    "litellm_params": {
        "model": "azure/gpt-5.4",
        "api_base": "https://<resource>.cognitiveservices.azure.com",
        "api_key": "<key>",
        "api_version": "2025-04-01-preview",
        "base_model": "gpt-5.4",
    },
}])
llm = ChatLiteLLM(model="GPT-5.4", streaming=True)
llm.client = router

@tool
def search(query: str) -> str:
    """Search for information."""
    return "no results"

@dataclass
class Ctx:
    params: dict = field(default_factory=dict)

class ParamMiddleware(AgentMiddleware):
    async def awrap_model_call(self, request: ModelRequest, handler):
        ctx = request.runtime.context
        if not ctx or not ctx.params:
            return await handler(request)
        extra = dict(request.model_settings or {})
        extra.update(ctx.params)
        return await handler(request.override(model_settings=extra))

async def main():
    agent = create_agent(model=llm, tools=[search], middleware=[ParamMiddleware()], context_schema=Ctx)
    # Fails:
    await agent.ainvoke(
        {"messages": [{"role": "user", "content": "hi"}]},
        context=Ctx(params={"reasoning_effort": "medium"}),
    )

asyncio.run(main())

Removing the reasoning_effort param from Ctx makes the same call succeed.

Relevant log output

litellm.APIConnectionError: AzureException APIConnectionError - api_base is required for Azure AI Studio. Please set the api_base parameter. Passed 
  api_base=None
  Traceback (most recent call last):
    File ".../litellm/responses/main.py", line 558, in aresponses
      response = await init_response
    File ".../litellm/llms/custom_httpx/llm_http_handler.py", line 2261, in async_response_api_handler
      api_base = responses_api_provider_config.get_complete_url(
    File ".../litellm/llms/azure/responses/transformation.py", line 180, in get_complete_url
      return BaseAzureLLM._get_base_azure_url(
    File ".../litellm/llms/azure/common_utils.py", line 756, in _get_base_azure_url
      raise ValueError(
  ValueError: api_base is required for Azure AI Studio. Please set the api_base parameter. Passed api_base=None

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on ?

v1.83.14

Twitter / LinkedIn details

www.linkedin.com/in/andrej-bárta-98a710261

extent analysis

TL;DR

The most likely fix is to set the api_base parameter when using reasoning_effort with tools in langchain.agents.create_agent with langchain_litellm.ChatLiteLLM.

Guidance

  • The error is caused by the responses_api_bridge_check in litellm/main.py forcing calls with tools + reasoning_effort onto the Azure Responses path, which requires the api_base parameter.
  • The provided workaround is to monkey-patch OpenAIGPT5Config.is_model_gpt_5_4_plus_model to return False, keeping the call on Chat Completions.
  • To mitigate the issue, ensure that the api_base parameter is set when using reasoning_effort with tools.
  • Verify that the api_base parameter is correctly set by checking the litellm_params dictionary in the Router instance.

Example

router = Router(model_list=[{
    "model_name": "GPT-5.4",
    "litellm_params": {
        "model": "azure/gpt-5.4",
        "api_base": "https://<resource>.cognitiveservices.azure.com",  # Ensure this is set
        "api_key": "<key>",
        "api_version": "2025-04-01-preview",
        "base_model": "gpt-5.4",
    },
}])

Notes

The issue is specific to the create_agent/ChatLiteLLM path and not a general issue with the litellm package. The workaround provided may not be a permanent solution and may need to be revisited in future versions of the package.

Recommendation

Apply the workaround by monkey-patching OpenAIGPT5Config.is_model_gpt_5_4_plus_model to return False, as

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

litellm - ✅(Solved) Fix [Bug]: GPT-5.4+ Responses bridge fails with "api_base=None" when calling via langchain create_agent on Azure Cognitive Services endpoint [1 pull requests, 1 participants]