llamaIndex - ✅(Solved) Fix [Bug]: `ContextVar` not propagated on non-`async` tool execution. [2 pull requests, 3 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
run-llama/llama_index#21555Fetched 2026-05-06 06:14:51
View on GitHub
Comments
3
Participants
2
Timeline
14
Reactions
0
Timeline (top)
commented ×3mentioned ×3subscribed ×3cross-referenced ×2

Fix Action

Fix / Workaround

The LLM being used shouldn't matter much: please try it on multiple.

def _make_nova_micro_llm() -> "BedrockConverse": # noqa: F821 """Construct a BedrockConverse LLM for amazon.nova-micro-v1:0. """ # import llama_index.llms.bedrock_converse.utils as _bedrock_utils # noqa: F401 (ensure patches applied) from llama_index.llms.bedrock_converse import BedrockConverse

PR fix notes

PR #21558: fix: propagate contextvars in sync_to_async for FunctionTool

Description (problem / solution / changelog)

Description

When a sync function is wrapped for async execution via sync_to_async, the default executor thread does not inherit the caller's contextvars context. This breaks ContextVar propagation (e.g. OpenTelemetry spans) for sync tools invoked through acall().

The fix snapshots contextvars.copy_context() and runs fn inside ctx.run(...) within the executor thread, mirroring the existing fix already present in async_utils.py's asyncio_run().

Fixes #21555

New Package?

  • Yes
  • No

Version Bump?

  • Yes
  • No

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Added test_function_tool_contextvar_propagation to tests/tools/test_base.py which sets a ContextVar and asserts it is propagated into a sync FunctionTool called via acall().
  • Existing test suite (tests/tools/test_base.py) passes locally (24/24).

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Changed files

  • llama-index-core/llama_index/core/tools/function_tool.py (modified, +5/-1)
  • llama-index-core/tests/tools/test_base.py (modified, +15/-0)

PR #21560: fix(tools): propagate ContextVars in sync_to_async for non-async tools

Description (problem / solution / changelog)

Summary

Fixes #21555

Bug

When using sync functions as tools, ContextVar values are not propagated to the tool execution. This is because sync_to_async uses loop.run_in_executor() which runs the function in a separate thread, losing the current thread's context.

Async tools work correctly because they execute in the same async context.

Fix

Use contextvars.copy_context() to capture the current context and ctx.run() to execute the function with that context in the executor thread:

from contextvars import copy_context

def sync_to_async(fn: Callable[..., Any]) -> AsyncCallable:
    async def _async_wrapped_fn(*args: Any, **kwargs: Any) -> Any:
        loop = asyncio.get_running_loop()
        ctx = copy_context()
        return await loop.run_in_executor(None, lambda: ctx.run(fn, *args, **kwargs))
    return _async_wrapped_fn

Impact

  • Sync tools now correctly receive ContextVar values from the calling context
  • Behavior matches async tools
  • No breaking changes for tools that don't use ContextVars

Changed files

  • llama-index-core/llama_index/core/tools/function_tool.py (modified, +4/-1)

Code Example

from contextvars import ContextVar

import pytest
from llama_index.core.tools import FunctionTool
from llama_index.core.workflow import Context


# The LLM being used shouldn't matter much: please try it on multiple.
def _make_nova_micro_llm() -> "BedrockConverse":  # noqa: F821
    """Construct a BedrockConverse LLM for amazon.nova-micro-v1:0.
    """
    # import llama_index.llms.bedrock_converse.utils as _bedrock_utils  # noqa: F401  (ensure patches applied)
    from llama_index.llms.bedrock_converse import BedrockConverse

    AWS_BEDROCK_GEO = "us"

    model = f"{AWS_BEDROCK_GEO}.amazon.nova-micro-v1:0"
    return BedrockConverse(model=model, max_tokens=5_120, **{"region_name": "us-east-1", "timeout": 600})



@pytest.mark.asyncio
@pytest.mark.parametrize("use_async_fn", [False, True], ids=["sync_fn", "async_fn"])
async def test_call_tool_reads_context_var_with_nova_micro(use_async_fn: bool) -> None:
    """FunctionTool (sync and async) should be able to read a ContextVar.

    When a FunctionAgent drives tool execution the Python contextvars context
    must be propagated into the tool call so that ContextVar.get() returns the
    value set on the calling context, not the default.

    Args:
        use_async_fn: When True the tool is registered as an async function;
            when False as a sync function.
    """
    from llama_index.core.agent.workflow import FunctionAgent

    some_context_var: ContextVar[list[str]] = ContextVar("some_context_var", default=["starting value"])

    # Particularly, it seems like llama_index has some custom event loop handling
    # that causes non-async tool calls to fail to correctly read `ContextVar`s. This
    # problem can be worked around by consumers who have ownership of those `ContextVar`s,
    # but cannot be if those ContextVars are owned by another package.
    # This becomes a problem particularly in OpenTelemetry, which uses a ContextVar to 
    # correctly parent a span. Currently, all non-async tool calls have spans that appear
    # as orphans in Otel.
    def read_context_var(ctx: Context) -> str:
        """Read the secret value stored in the some_context_var and return it."""
        return str(some_context_var.get())

    async def read_context_var_async(ctx: Context) -> str:
        """Read the secret value stored in the some_context_var and return it."""
        return str(some_context_var.get())

    if use_async_fn:
        tool = FunctionTool.from_defaults(
            async_fn=read_context_var_async,
            name="read_context_var",
            description="Always call this, just once.",
        )
    else:
        tool = FunctionTool.from_defaults(
            fn=read_context_var,
            name="read_context_var",
            description="Always call this, just once.",
        )

    llm = _make_nova_micro_llm()

    updated_value = f"updated value: {use_async_fn}"
    some_context_var.set([updated_value])

    agent = FunctionAgent(
        llm=llm,
        tools=[tool],
        system_prompt="You MUST call the given tool; you are testing its behavior.",
    )

    handler = agent.run(user_msg="What is the context var?")
    output = await handler

    assert updated_value in str(output), (
        f"Expected '{updated_value}' to appear in the agent output, but got: {output!r}\n"
        "This may be the llama-index contextvars propagation bug."
    )

---
RAW_BUFFERClick to expand / collapse

Bug Description

When reading ContextVars within FunctionTools, they may or may not be correct, depending on if the tool is executing async_fn or fn. When using the async_fn input, ContextVars are fine. When using fn as input, ContextVars are not propagated correctly. I suspect that there's some custom event loop iteration happening (but I'm not a Python expert).

Here's the input args I'm mentioning:

https://github.com/run-llama/llama_index/blob/d601b0f36af4f6362375eefeda509d1070340652/llama-index-core/llama_index/core/tools/function_tool.py#L78-L80

Version

llama-index-core>=0.14.18

Steps to Reproduce

Here's a unit test that reproduces what I'm observing:

from contextvars import ContextVar

import pytest
from llama_index.core.tools import FunctionTool
from llama_index.core.workflow import Context


# The LLM being used shouldn't matter much: please try it on multiple.
def _make_nova_micro_llm() -> "BedrockConverse":  # noqa: F821
    """Construct a BedrockConverse LLM for amazon.nova-micro-v1:0.
    """
    # import llama_index.llms.bedrock_converse.utils as _bedrock_utils  # noqa: F401  (ensure patches applied)
    from llama_index.llms.bedrock_converse import BedrockConverse

    AWS_BEDROCK_GEO = "us"

    model = f"{AWS_BEDROCK_GEO}.amazon.nova-micro-v1:0"
    return BedrockConverse(model=model, max_tokens=5_120, **{"region_name": "us-east-1", "timeout": 600})



@pytest.mark.asyncio
@pytest.mark.parametrize("use_async_fn", [False, True], ids=["sync_fn", "async_fn"])
async def test_call_tool_reads_context_var_with_nova_micro(use_async_fn: bool) -> None:
    """FunctionTool (sync and async) should be able to read a ContextVar.

    When a FunctionAgent drives tool execution the Python contextvars context
    must be propagated into the tool call so that ContextVar.get() returns the
    value set on the calling context, not the default.

    Args:
        use_async_fn: When True the tool is registered as an async function;
            when False as a sync function.
    """
    from llama_index.core.agent.workflow import FunctionAgent

    some_context_var: ContextVar[list[str]] = ContextVar("some_context_var", default=["starting value"])

    # Particularly, it seems like llama_index has some custom event loop handling
    # that causes non-async tool calls to fail to correctly read `ContextVar`s. This
    # problem can be worked around by consumers who have ownership of those `ContextVar`s,
    # but cannot be if those ContextVars are owned by another package.
    # This becomes a problem particularly in OpenTelemetry, which uses a ContextVar to 
    # correctly parent a span. Currently, all non-async tool calls have spans that appear
    # as orphans in Otel.
    def read_context_var(ctx: Context) -> str:
        """Read the secret value stored in the some_context_var and return it."""
        return str(some_context_var.get())

    async def read_context_var_async(ctx: Context) -> str:
        """Read the secret value stored in the some_context_var and return it."""
        return str(some_context_var.get())

    if use_async_fn:
        tool = FunctionTool.from_defaults(
            async_fn=read_context_var_async,
            name="read_context_var",
            description="Always call this, just once.",
        )
    else:
        tool = FunctionTool.from_defaults(
            fn=read_context_var,
            name="read_context_var",
            description="Always call this, just once.",
        )

    llm = _make_nova_micro_llm()

    updated_value = f"updated value: {use_async_fn}"
    some_context_var.set([updated_value])

    agent = FunctionAgent(
        llm=llm,
        tools=[tool],
        system_prompt="You MUST call the given tool; you are testing its behavior.",
    )

    handler = agent.run(user_msg="What is the context var?")
    output = await handler

    assert updated_value in str(output), (
        f"Expected '{updated_value}' to appear in the agent output, but got: {output!r}\n"
        "This may be the llama-index contextvars propagation bug."
    )

Relevant Logs/Tracebacks

extent analysis

TL;DR

The issue can be worked around by registering tools as async functions instead of sync functions to ensure correct propagation of ContextVars.

Guidance

  • The problem seems to be related to custom event loop handling in llama_index that causes non-async tool calls to fail to correctly read ContextVars.
  • To verify the issue, run the provided unit test test_call_tool_reads_context_var_with_nova_micro with use_async_fn set to False and True to see the difference in behavior.
  • Consider registering tools as async functions using FunctionTool.from_defaults with async_fn parameter to ensure correct propagation of ContextVars.
  • If using sync functions is necessary, investigate the custom event loop handling in llama_index to see if there's a way to propagate ContextVars correctly.

Example

tool = FunctionTool.from_defaults(
    async_fn=read_context_var_async,
    name="read_context_var",
    description="Always call this, just once.",
)

Notes

The issue seems to be specific to the llama_index library and its custom event loop handling. The provided workaround may not be applicable in all cases, and further investigation may be needed to find a more robust solution.

Recommendation

Apply the workaround by registering tools as async functions using FunctionTool.from_defaults with async_fn parameter, as it ensures correct propagation of ContextVars.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

llamaIndex - ✅(Solved) Fix [Bug]: `ContextVar` not propagated on non-`async` tool execution. [2 pull requests, 3 comments, 2 participants]