litellm - 💡(How to fix) Fix [Feature]: Support Azure AI Foundry Agents v2 (Responses API with agent_reference) [1 participants]

Feature Request

What is your use case?

We are integrating with Microsoft Foundry Agent Service. We've deployed agents using the new Foundry Agents v2 experience, which uses the Responses API with agent_reference (referencing agents by name and version) rather than the deprecated Assistants API (thread/run pattern with asst_* IDs).

The current azure_ai/agents/{agent_id} route in LiteLLM implements the old Assistants API pattern, which:

1. Requires agent IDs beginning with asst_ (but the new Foundry experience does not surface these IDs)
2. Uses the thread → message → run → poll flow (but the new Foundry Agents v2 uses openai_client.responses.create() with an agent_reference in extra_body)

Microsoft has deprecated the Assistants API with retirement set for August 26, 2026.

The new API pattern

From the Azure AI Foundry getting-started code:

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project_client = AIProjectClient(
    endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)

openai_client = project_client.get_openai_client()

response = openai_client.responses.create(
    input=[{"role": "user", "content": "Tell me what you can help with."}],
    extra_body={
        "agent_reference": {
            "name": "my-agent-name",
            "version": "1",
            "type": "agent_reference",
        }
    },
)

Key differences from the current azure_ai/agents implementation:

• Endpoint: Project-level (https://<resource>.services.ai.azure.com/api/projects/<project>)
• API: Responses API (/openai/responses), not Assistants/Threads API
• Agent identification: By name + version via agent_reference in extra_body, not by asst_* ID
• Auth: Azure AD (Entra ID) bearer token — same as current implementation

Proposed model format

azure_ai/agents/<agent_name>:<version>

For example: azure_ai/agents/my-agent-name:1

Reference docs

• Agent identity concepts in Microsoft Foundry
• Authentication and authorization in Microsoft Foundry
• Assistants API deprecation notice

Are you a LiteLLM Enterprise customer?

No