litellm - 💡(How to fix) Fix [Bug]: Invoking Azure AI Foundry Agent with Open AI SDK [1 participants]

I correctly used a2a/<agent-name> prefix to call my agent and I got a ressource not found error: InternalServerError: Error code: 500 - {'error': {'message': 'litellm.APIConnectionError: A2aException - {"error":{"code":"404","message": "Resource not found"}}', 'type': None, 'param': None, 'code': '500'}} InternalServerError: Error code: 500 - {'error': {'message': 'litellm.APIConnectionError: A2aException - {"error":{"code":"404","message": "Resource not found"}}', 'type': None, 'param': None, 'code': '500'}}

Fix Plan

To resolve the issue, we need to ensure that the agent is correctly registered and the OpenAI SDK is properly configured. Here are the steps:

• Verify that the agent is registered with the correct name and prefix (a2a/<agent-name>) in the Litellm proxy server.
• Check that the base_url in the OpenAI SDK configuration matches the URL of the Litellm proxy server.
• Ensure that the api_key used in the OpenAI SDK is valid and has the necessary permissions to access the agent.

Example code to verify the agent registration:

import httpx

response = httpx.get(
    "https://xxxxxxxx:4000/a2a/agents",
    headers={
        "Authorization": "Bearer sk-xxxxxxxx",
    }
)

print(response.json())

This should return a list of registered agents, including the one with the name a2a/snowflake-agent.

Example code to create a chat completion using the OpenAI SDK:

import openai

client = openai.OpenAI(
    api_key="sk-xxxxxx", 
    base_url="https://xxxxxxxx:4000"
)

stream = client.chat.completions.create(
    model="a2a/snowflake-agent", 
    messages=[
        {"role": "user", "content": "Tell me a short story"}
    ],
    stream=True
)

for message in stream:
    print(message)

Note that we've added the stream=True parameter to enable streaming of the response.

Verification

To verify that the fix worked, run the example code to create a chat completion using the OpenAI SDK. If the agent is correctly registered and the OpenAI SDK is properly configured, you should receive a response from the agent.

Extra Tips

• Ensure that the Litellm proxy server is running and accessible.
• Check the Litellm documentation for any specific requirements or restrictions on agent registration and OpenAI SDK usage.
• If you're still experiencing issues, try enabling debug logging in the OpenAI SDK to get more detailed error messages.