crewai - ✅(Solved) Fix [BUG] LLM call does not adhere to pydantic response_model fails for "gpt5nano" [2 pull requests, 4 comments, 3 participants]

Q: Expected behavior

LLM response adheres to the provided `response_model ` no matter the model choice.

crewai2026-02-13 12:21:56

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

crewAIInc/crewAI#4478•Fetched 2026-04-08 00:41:50

View on GitHub

Comments

Participants

Timeline

Reactions

Author

Participants

Timeline (top)

commented ×4cross-referenced ×3referenced ×3labeled ×2

LLM call will not adhere to the response_model for model="gpt5nano".

From crewai/src/crewai/llms/providers/azure/completion.py:

self.is_openai_model = any(
            prefix in model.lower() for prefix in ["gpt-", "o1-", "text-"]
        )

and if self.is_openai_model=False, it will not use my response_model, when formatting the response (for some reason).

self.is_openai_model will be false for models like "gpt5nano"; as far as I can see the model referred to in the above is in fact the deployment name, and as such the name could be anything.

Root Cause

LLM call will not adhere to the response_model for model="gpt5nano".

From crewai/src/crewai/llms/providers/azure/completion.py:

self.is_openai_model = any(
            prefix in model.lower() for prefix in ["gpt-", "o1-", "text-"]
        )

and if self.is_openai_model=False, it will not use my response_model, when formatting the response (for some reason).

self.is_openai_model will be false for models like "gpt5nano"; as far as I can see the model referred to in the above is in fact the deployment name, and as such the name could be anything.

Fix Action

Fixed

Fixed by PR: Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name (https://github.com/crewAIInc/crewAI/pull/4480)
Fixed by PR: fix: handle non-hyphenated GPT-5 model names in detection logic (https://github.com/crewAIInc/crewAI/pull/4535)

PR fix notes

PR #4480: Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name

Repository: crewAIInc/crewAI
Author: devin-ai-integration[bot]
State: closed | merged: False
Link: https://github.com/crewAIInc/crewAI/pull/4480

Description (problem / solution / changelog)

Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name

Summary

AzureCompletion.is_openai_model was determined solely by checking model name prefixes (gpt-, o1-, text-). Since Azure deployment names can be arbitrary (e.g. gpt5nano), this caused is_openai_model to be False for custom-named deployments, which silently broke:

response_model / structured output (JSON schema not sent to API, validation skipped)
Tool/function calling (tools not included in request params)
supports_function_calling() returning False

Fix: Compute is_azure_openai_endpoint first, then use it as an additional signal — if the endpoint matches openai.azure.com + /openai/deployments/, it's an OpenAI model regardless of deployment name. Non-OpenAI models on Azure AI Inference endpoints (e.g. models.inference.ai.azure.com) are unaffected.

The production code change is 6 lines in AzureCompletion.__init__(). 7 new regression tests added covering detection, response_model parsing, params preparation, tool inclusion, streaming, and negative cases.

Review & Testing Checklist for Human

Verify the assumption that openai.azure.com/openai/deployments/ always implies an OpenAI model — could a non-OpenAI model ever be deployed at this endpoint pattern?
Ideally test end-to-end with an actual Azure OpenAI deployment using a custom name (e.g. gpt5nano) and a response_model to confirm structured output works
Confirm existing test_azure_deepseek_model_support and test_azure_mistral_and_other_models still correctly identify non-OpenAI models as is_openai_model=False (they pass in CI, but worth a glance)

Notes

All 60 existing Azure tests pass, plus 7 new tests
Requested by: João
Link to Devin run

Changed files

lib/crewai/src/crewai/llms/providers/azure/completion.py (modified, +5/-4)
lib/crewai/tests/llms/azure/test_azure.py (modified, +216/-0)

PR #4535: fix: handle non-hyphenated GPT-5 model names in detection logic

Repository: crewAIInc/crewAI
Author: hztBUAA
State: open | merged: False
Link: https://github.com/crewAIInc/crewAI/pull/4535

Description (problem / solution / changelog)

Summary

Fixes #4478

Add "gpt5" prefix to model detection logic alongside "gpt-" so non-hyphenated GPT-5 model names (e.g. gpt5, gpt5nano, gpt5mini) are correctly recognized as OpenAI models
Update is_openai_model check in AzureCompletion.__init__ to detect gpt5* variants
Update supports_stop_words() to treat gpt5* models the same as gpt-5* models
Update _is_model_from_provider() for both openai and azure providers

Context

When using Azure OpenAI with a deployment named gpt5nano, gpt5, etc., the model detection logic only checked for the "gpt-" prefix. This caused is_openai_model to be False, which in turn caused response_model (Pydantic structured output) to be ignored.

Test plan

Added test_azure_gpt5_non_hyphenated_model_detection — verifies is_openai_model and supports_function_calling() for gpt5, gpt5nano, gpt5mini
Added test_azure_gpt5_non_hyphenated_models_do_not_support_stop_words — verifies stop words are correctly disabled for non-hyphenated GPT-5 names
All 59 Azure tests pass

Changed files

lib/crewai/src/crewai/llm.py (modified, +2/-2)
lib/crewai/src/crewai/llms/providers/azure/completion.py (modified, +3/-2)
lib/crewai/tests/llms/azure/test_azure.py (modified, +42/-0)

Code Example

self.is_openai_model = any(
            prefix in model.lower() for prefix in ["gpt-", "o1-", "text-"]
        )

---

from pydantic import BaseModel, Field
from dotenv import load_dotenv
import os
from azure.identity import AzureCliCredential
from crewai import LLM

load_dotenv()
api_key = os.getenv("AZURE_OPENAI_KEY")
endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
deployment = os.getenv("AZURE_OPENAI_MODEL_DEPLOYMENT_NAME")
api_version = os.getenv("AZURE_OPENAI_API_VERSION")

credential = AzureCliCredential()
token_obj = credential.get_token("https://cognitiveservices.azure.com/.default")
access_token = token_obj.token

llm = LLM(
    model=f"azure/{deployment}",
    api_key=access_token,
    base_url=f'{endpoint}/openai/deployments/{deployment}',
    api_version=api_version,
    additional_drop_params=['stop'],
)

class SimpleResponse(BaseModel):
    message: str = Field(description="A primary message or response")
    reason: str = Field(description="Reasoning or explanation for the response")

def test_llm_does_not_adhere_to_response_model():
    prompt = [
        {"role": "user",
         "content": "What is the capital of France and why is it significant?"}
    ]
    
    response = llm.call(messages=prompt, response_model=SimpleResponse)
    assert isinstance(response, SimpleResponse), "Response is not an instance of SimpleResponse"

    print(response)


def test_llm_adheres_to_response_model_when_explicitly_setting_is_openai_model():
    prompt = [
        {"role": "user",
         "content": "What is the capital of France and why is it significant?"}
    ]
    
    response = llm.call(messages=prompt, response_model=SimpleResponse)
    llm.is_openai_model = True  # Ensure it will use response_model parsing
    assert isinstance(response, SimpleResponse), "Response is not an instance of SimpleResponse"

    print(response)

def test_without_response_model():
    prompt = [
        {"role": "user",
         "content": "What is the largest planet in our solar system?"}
    ]
    
    response = llm.call(messages=prompt)
    assert isinstance(response, str), "Expected a string response when no response_model is provided"

    print(response)

RAW_BUFFERClick to expand / collapse

Description

LLM call will not adhere to the response_model for model="gpt5nano".

From crewai/src/crewai/llms/providers/azure/completion.py:

self.is_openai_model = any(
            prefix in model.lower() for prefix in ["gpt-", "o1-", "text-"]
        )

and if self.is_openai_model=False, it will not use my response_model, when formatting the response (for some reason).

self.is_openai_model will be false for models like "gpt5nano"; as far as I can see the model referred to in the above is in fact the deployment name, and as such the name could be anything.

Steps to Reproduce

Create an LLM object, call with a Pydantic class as response_model, observe class is not adhered to.

Expected behavior

LLM response adheres to the provided response_model no matter the model choice.

Screenshots/Code snippets

from pydantic import BaseModel, Field
from dotenv import load_dotenv
import os
from azure.identity import AzureCliCredential
from crewai import LLM

load_dotenv()
api_key = os.getenv("AZURE_OPENAI_KEY")
endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
deployment = os.getenv("AZURE_OPENAI_MODEL_DEPLOYMENT_NAME")
api_version = os.getenv("AZURE_OPENAI_API_VERSION")

credential = AzureCliCredential()
token_obj = credential.get_token("https://cognitiveservices.azure.com/.default")
access_token = token_obj.token

llm = LLM(
    model=f"azure/{deployment}",
    api_key=access_token,
    base_url=f'{endpoint}/openai/deployments/{deployment}',
    api_version=api_version,
    additional_drop_params=['stop'],
)

class SimpleResponse(BaseModel):
    message: str = Field(description="A primary message or response")
    reason: str = Field(description="Reasoning or explanation for the response")

def test_llm_does_not_adhere_to_response_model():
    prompt = [
        {"role": "user",
         "content": "What is the capital of France and why is it significant?"}
    ]
    
    response = llm.call(messages=prompt, response_model=SimpleResponse)
    assert isinstance(response, SimpleResponse), "Response is not an instance of SimpleResponse"

    print(response)


def test_llm_adheres_to_response_model_when_explicitly_setting_is_openai_model():
    prompt = [
        {"role": "user",
         "content": "What is the capital of France and why is it significant?"}
    ]
    
    response = llm.call(messages=prompt, response_model=SimpleResponse)
    llm.is_openai_model = True  # Ensure it will use response_model parsing
    assert isinstance(response, SimpleResponse), "Response is not an instance of SimpleResponse"

    print(response)

def test_without_response_model():
    prompt = [
        {"role": "user",
         "content": "What is the largest planet in our solar system?"}
    ]
    
    response = llm.call(messages=prompt)
    assert isinstance(response, str), "Expected a string response when no response_model is provided"

    print(response)

Operating System

Windows 11

Python Version

3.12

crewAI Version

1.9.3

crewAI Tools Version

Virtual Environment

Venv

Evidence

Possible Solution

Redefine is_openai_model
Don't let is_openai_model affect whether or not to adhere to the response_model

Additional context

None

extent analysis

Fix Plan

To fix the issue, we need to modify the is_openai_model check to not affect the usage of the response_model. We can achieve this by removing the is_openai_model check when formatting the response.

Here are the steps:

Modify the completion.py file in the crewai library to remove the is_openai_model check.
Ensure that the response_model is always used when provided.

Example code:

# Remove the is_openai_model check
# if not self.is_openai_model:
#     # code that doesn't use response_model

# Always use response_model if provided
if response_model:
    # code that uses response_model
    pass

Alternatively, you can modify the is_openai_model check to include the "gpt5nano" model:

self.is_openai_model = any(
    prefix in model.lower() for prefix in ["gpt-", "o1-", "text-", "gpt5nano"]
)

Verification

To verify that the fix worked, you can run the test_llm_does_not_adhere_to_response_model test case again. The response should now be an instance of the SimpleResponse model.

Extra Tips

Make sure to update the crewai library to the latest version to ensure that any future updates do not overwrite the fix.
Consider submitting a pull request to the crewai library to include the fix in the main codebase.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

FAQ

Expected behavior

LLM response adheres to the provided response_model no matter the model choice.

#api #ssr #installation #tensor shape #autograd error #model save/load #optimization #LLM response

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

crewai - ✅(Solved) Fix [BUG] LLM call does not adhere to pydantic response_model fails for "gpt5nano" [2 pull requests, 4 comments, 3 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #4480: Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name

Description (problem / solution / changelog)

Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name

Summary

Review & Testing Checklist for Human

Notes

Changed files

PR #4535: fix: handle non-hyphenated GPT-5 model names in detection logic

Description (problem / solution / changelog)

Summary

Context

Test plan

Changed files

Code Example

Description

Steps to Reproduce

Expected behavior

Screenshots/Code snippets

Operating System

Python Version

crewAI Version

crewAI Tools Version

Virtual Environment

Evidence

Possible Solution

Additional context

extent analysis

Fix Plan

Verification

Extra Tips

FAQ

Expected behavior

Still need to ship something?

RELATED_DISCOVERY

TRENDING