Issue Station
Sign In
Tools hereToolbox

crewai - ✅(Solved) Fix [BUG] When supports_function_calling() is True, only the output_pydantic model is sent to LiteLLM (the custom agent tools are discard) [3 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
crewAIInc/crewAI#4697•Fetched 2026-04-08 00:40:37
View on GitHub
Comments
1
Participants
2
Timeline
13
Reactions
0
Author
Killian-falKillian-fal
Participants
hidai25hidai25Killian-falKillian-fal
Timeline (top)
referenced ×6cross-referenced ×5commented ×1labeled ×1

When supports_function_calling() returns True (either natively or via override) and a output_pydantic is set, the custom agent tools are skip and only the output_pydantic model is send as a tool.

The result: the LLM only sees the output_pydantic Pydantic model as a native tool, not the agent's actual tools. The agent is forced to return structured output immediately without calling any of its tools.

Note: I believe CrewAI 1.10.0 introduced a major regression on Ollama + LiteLLM that renders CrewAI unusable. Perhaps more tests should be written for this specific feature to prevent this from happening in the future.

Root Cause

When supports_function_calling() returns True (either natively or via override) and a output_pydantic is set, the custom agent tools are skip and only the output_pydantic model is send as a tool.

The result: the LLM only sees the output_pydantic Pydantic model as a native tool, not the agent's actual tools. The agent is forced to return structured output immediately without calling any of its tools.

Note: I believe CrewAI 1.10.0 introduced a major regression on Ollama + LiteLLM that renders CrewAI unusable. Perhaps more tests should be written for this specific feature to prevent this from happening in the future.

Fix Action

Fixed

PR fix notes

PR #4699: fix: prevent tools from being discarded when response_model is set (#4697)

Description (problem / solution / changelog)

fix: prevent tools from being discarded when response_model is set (#4697)

Summary

Fixes #4697. When both native function calling (tools) and output_pydantic (response_model) are enabled, tools were being silently discarded through two mechanisms:

  1. InternalInstructor early return: When response_model was set, InternalInstructor intercepted the LLM call and created its own completion that ignored the tools parameter entirely.
  2. response_model forwarded to litellm: Even after skipping InternalInstructor, the code still passed response_model to litellm.completion(**params). Litellm uses instructor internally when response_model is provided, which also overrides the tools parameter.
  3. Streaming handler dropped tool_calls: _handle_streaming_response lacked an explicit path for tool_calls present + available_functions absent, causing it to return an empty string instead of the tool_calls (unlike the non-streaming handler which already had this path).

Changes

  • Added a has_tools guard to the three response handlers (_handle_non_streaming_response, _ahandle_non_streaming_response, _handle_streaming_response). When tools are present, InternalInstructor is skipped.
  • When tools are present, response_model is not added to params before calling litellm.completion / litellm.acompletion, preventing litellm's own instructor from overriding tools.
  • Added an explicit if tool_calls and not available_functions: return tool_calls path in _handle_streaming_response, mirroring the equivalent path in _handle_non_streaming_response.
  • InternalInstructor is still used when no tools are present (backward compatible).
  • Eight new unit tests cover the fix across sync/async, tools/no-tools, and response_model-not-forwarded scenarios.

Known limitation (pre-existing, not addressed here)

_ahandle_streaming_response has no InternalInstructor / response_model handling at all — neither before nor after this PR. This is a separate inconsistency that should be addressed in a follow-up.

Review & Testing Checklist for Human

  • Verify end-to-end behavior: The unit tests mock litellm.completion entirely, so they validate the LLM layer's branching logic but not real litellm behavior. Manually verify that an agent with both tools and output_pydantic can (1) call its tools, and (2) still return structured output in the expected Pydantic format after tool execution completes.
  • Check structured output enforcement when LLM returns text with tools present: When tools exist but the LLM returns plain text (no tool_calls), both InternalInstructor and response_model forwarding are now bypassed. Confirm that CrewAgentExecutor or the converter layer still enforces structured output on the final answer in this path.
  • Streaming handler new early-return: The new if tool_calls and not available_functions: return tool_calls path (line ~973) runs before the existing if not tool_calls or not available_functions: block. Verify this doesn't break any caller that expects a string response from the streaming handler when tool_calls are present.

Notes

Changed files

  • lib/crewai/src/crewai/llm.py (modified, +28/-5)
  • lib/crewai/tests/test_llm.py (modified, +253/-0)

PR #4717: fix: preserve agent tools when output_pydantic is set

Description (problem / solution / changelog)

Summary

Fixes #4697

When output_pydantic (response_model) is set on a task and the LLM supports native function calling, InternalInstructor intercepts the LLM call and creates its own litellm.completion that only contains the Pydantic schema as a tool -- all actual agent tools are silently discarded. The agent is forced to return structured output immediately without being able to call any of its tools first.

Root cause: the response_model guard in the four response handlers (_handle_non_streaming_response, _ahandle_non_streaming_response, _handle_streaming_response, _ahandle_streaming_response) did not check whether agent tools were also present. Two independent paths caused the issue:

  1. InternalInstructor early-return -- creates a brand-new completion() call with only the Pydantic schema, ignoring params["tools"] entirely.
  2. response_model forwarded to litellm.completion -- litellm's internal instructor overrides tools when response_model is in the kwargs.

Fix: add a has_tools = bool(params.get("tools")) guard. When tools are present:

  • InternalInstructor is skipped (the agent needs to see and call its tools first).
  • response_model is not added to params before calling litellm.completion / litellm.acompletion.
  • Streaming handlers correctly return tool_calls when available_functions is None (executor-managed tool execution), mirroring the existing non-streaming behavior.

When no tools are present, behavior is completely unchanged -- InternalInstructor and response_model work as before for structured output.

Changes

Single file: lib/crewai/src/crewai/llm.py (31 additions, 6 deletions)

Test Plan

  • Verified syntax correctness via ast.parse
  • Verified boolean logic covers all four cases: (response_model + tools), (response_model + no tools), (no response_model + tools), (no response_model + no tools)
  • Manual review of all four handler paths confirms the guard is applied consistently
  • Existing tests remain unaffected since the no-tools path is unchanged
<!-- CURSOR_SUMMARY -->

[!NOTE] Medium Risk Touches core LLM response handling paths (sync/async, streaming/non-streaming), so regressions could affect tool-calling and structured output behavior. Logic is gated on presence of params["tools"], reducing impact to tool-enabled calls.

Overview Fixes LiteLLM calls where providing response_model could silently override/discard agent tools.

When params["tools"] is present, the handlers now skip InternalInstructor and avoid forwarding response_model into litellm.completion/litellm.acompletion, allowing the model to invoke tools before structured output.

Streaming paths additionally return tool_calls directly when available_functions is None (executor-managed tool execution), mirroring non-streaming behavior.

<sup>Written by Cursor Bugbot for commit 5c9cf0ba2e0d875b678eb14d82f2da17ef7c3b42. This will update automatically on new commits. Configure here.</sup>

<!-- /CURSOR_SUMMARY -->

Changed files

  • lib/crewai/src/crewai/llm.py (modified, +31/-6)

PR #4819: Fix LiteLLM response_model and native tool call handling

Description (problem / solution / changelog)

This pull request fixes two related bugs in the LiteLLM integration used by the LLM wrapper:

  • When supports_function_calling() is true and a response_model is provided, the internal instructor path was used even when native tools were also passed. This caused custom agent tools to be ignored in favor of the structured output model.
  • Native tool calls could be dropped when the LLM returned both text and tool calls in the same response. In that case, the text branch won and the tool calls were never surfaced back to the executor.

Key changes:

  1. Only route through InternalInstructor when no tools are being passed. This ensures that when tools are provided, the call behaves like a normal tool-enabled completion and agent tools are preserved.

  2. Prefer tool calls over plain text when both are present. The non-streaming response handlers now return tool calls first when they exist, and only fall back to text when there are no tool calls.

These changes are applied to both the synchronous and asynchronous non-streaming handlers so that behavior is consistent across code paths.

Closes #4697. Closes #4788.

<!-- CURSOR_SUMMARY -->

[!NOTE] Medium Risk Changes LiteLLM non-streaming return semantics around response_model and tool-calling, which can affect downstream executors that previously received plain text instead of tool call objects. Risk is moderate due to altered control flow in a core LLM wrapper, but scoped to LiteLLM non-streaming sync/async handlers.

Overview Fixes LiteLLM non-streaming handling so response_model routing through InternalInstructor is skipped when native tools are provided, preserving tool-enabled completions.

Adjusts both sync and async non-streaming response handlers to prefer returning tool_calls when present but available_functions isn’t supplied (so callers can execute tools), and only fall back to returning text when no tool calls exist.

<sup>Written by Cursor Bugbot for commit f647049aa545f74414ca9e6e1475fab0aaa99728. This will update automatically on new commits. Configure here.</sup>

<!-- /CURSOR_SUMMARY -->

Changed files

  • lib/crewai/src/crewai/llm.py (modified, +16/-15)

Code Example

from crewai import LLM, Agent, Task, Crew
from crewai.tools import BaseTool
from pydantic import BaseModel

class MyOutput(BaseModel):
    name: str
    value: str

class MySearchTool(BaseTool):
    name: str = "my_search_tool"
    description: str = "Search for data"
    def _run(self, query: str) -> str:
        return '{"name": "Alice", "value": "42"}'

llm = LLM(model="ollama_chat/mistral-small3.2:24b", api_base="https://my-remote-ollama.example.com")

agent = Agent(role="Researcher", goal="Find data", backstory="...", llm=llm, tools=[MySearchTool()])
task = Task(
    description="Find the name and value",
    expected_output="Structured result",
    agent=agent,
    output_pydantic=MyOutput,
)
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff() # Or with async
RAW_BUFFERClick to expand / collapse

Description

When supports_function_calling() returns True (either natively or via override) and a output_pydantic is set, the custom agent tools are skip and only the output_pydantic model is send as a tool.

The result: the LLM only sees the output_pydantic Pydantic model as a native tool, not the agent's actual tools. The agent is forced to return structured output immediately without calling any of its tools.

Note: I believe CrewAI 1.10.0 introduced a major regression on Ollama + LiteLLM that renders CrewAI unusable. Perhaps more tests should be written for this specific feature to prevent this from happening in the future.

Steps to Reproduce

from crewai import LLM, Agent, Task, Crew
from crewai.tools import BaseTool
from pydantic import BaseModel

class MyOutput(BaseModel):
    name: str
    value: str

class MySearchTool(BaseTool):
    name: str = "my_search_tool"
    description: str = "Search for data"
    def _run(self, query: str) -> str:
        return '{"name": "Alice", "value": "42"}'

llm = LLM(model="ollama_chat/mistral-small3.2:24b", api_base="https://my-remote-ollama.example.com")

agent = Agent(role="Researcher", goal="Find data", backstory="...", llm=llm, tools=[MySearchTool()])
task = Task(
    description="Find the name and value",
    expected_output="Structured result",
    agent=agent,
    output_pydantic=MyOutput,
)
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff() # Or with async
  1. Set supports_function_calling() to True.
  2. Add a tool to the agent and set output_pydantic on the task.
  3. Run the crew.
  4. Observe that LiteLLM only receives the output_pydantic model as a tool — the agent's my_search_tool is absent.

Expected behavior

  • When supports_function_calling() is True and the agent has tools, all agent tools should be forwarded to LiteLLM.
  • The agent should be able to call its tools (e.g. my_search_tool) before returning the final structured output.

Screenshots/Code snippets

X

Operating System

Ubuntu 20.04

Python Version

3.10

crewAI Version

1.10.0

crewAI Tools Version

1.10.0

Virtual Environment

Venv

Evidence

I've nothing to show sorry (with mlflow we can see that the tools are not send to litellm)

Possible Solution

I tried to debug this issue and found that InternalInstructor intercepts even when params["tools"] is populated. And InternalInstructor create his own instance of litellm to call the provider. Maybe the bug is here ?

Additional context

X

extent analysis

Fix Plan

To fix the issue, we need to modify the InternalInstructor to forward the agent's tools to LiteLLM when supports_function_calling() is True. Here are the steps:

  • Modify the InternalInstructor to check if supports_function_calling() is True and if the agent has tools.
  • If both conditions are met, forward the agent's tools to LiteLLM.
  • Update the crewai library to include this fix.

Example code:

class InternalInstructor:
    # ...

    def _create_litellm_instance(self, params):
        if supports_function_calling() and params["tools"]:
            # Forward agent's tools to LiteLLM
            litellm_tools = [tool.name for tool in params["tools"]]
            params["litellm_tools"] = litellm_tools
        # ...

    # ...

Verification

To verify that the fix worked, you can:

  • Set supports_function_calling() to True.
  • Add a tool to the agent and set output_pydantic on the task.
  • Run the crew.
  • Check that LiteLLM receives the agent's tools and that the agent can call its tools before returning the final structured output.

Example code:

llm = LLM(model="ollama_chat/mistral-small3.2:24b", api_base="https://my-remote-ollama.example.com")

agent = Agent(role="Researcher", goal="Find data", backstory="...", llm=llm, tools=[MySearchTool()])
task = Task(
    description="Find the name and value",
    expected_output="Structured result",
    agent=agent,
    output_pydantic=MyOutput,
)
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff()  # Or with async

# Check that LiteLLM receives the agent's tools
print(crew.llm_tools)  # Should include MySearchTool

Extra Tips

  • Make sure to update the crewai library to include the fix.
  • Test the fix thoroughly to ensure that it works as expected.
  • Consider adding more tests to prevent similar regressions in the future.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

  • When supports_function_calling() is True and the agent has tools, all agent tools should be forwarded to LiteLLM.
  • The agent should be able to call its tools (e.g. my_search_tool) before returning the final structured output.
#api#ssr#installation#tensor shape#autograd error#runtime error#dependency conflict#environment setup#docker error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

crewai - ✅(Solved) Fix [BUG] When supports_function_calling() is True, only the output_pydantic model is sent to LiteLLM (the custom agent tools are discard) [3 pull requests, 1 comments, 2 participants]