litellm - 💡(How to fix) Fix [Bug]: langfuse_otel callback drops OTEL spans under asyncio.gather() concurrency [1 participants]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When multiple litellm.acompletion() calls run concurrently via asyncio.gather(), the langfuse_otel callback consistently drops ~1 in 3 OTEL spans. The spans are silently lost — no errors, no warnings. This means Langfuse traces are incomplete and cost tracking is unreliable for batch LLM operations.

This is related to #25978 (which covers the teardown/drain variant), but this is the concurrent execution variant — spans are dropped while the event loop is still alive, not during shutdown.

Root cause

LiteLLM dispatches success callbacks via asyncio.create_task() through GLOBAL_LOGGING_WORKER. When multiple acompletion() calls complete near-simultaneously under asyncio.gather(), the callback tasks that create OTEL spans (_handle_success → _start_primary_span) interfere with each other. The result: some spans are never created or are lost before reaching the BatchSpanProcessor.

Steps to Reproduce

import asyncio
import litellm
from langfuse import observe, get_client

litellm.success_callback = ["langfuse_otel"]
os.environ["USE_OTEL_LITELLM_REQUEST_SPAN"] = "true"

async def classify(text):
    return await litellm.acompletion(
        model="claude-haiku-4-5-20251001",
        messages=[
            {"role": "system", "content": "Classify as: positive/negative. One word."},
            {"role": "user", "content": text},
        ],
        max_tokens=10,
    )

@observe(name="batch_operation")
def run_batch():
    async def _run():
        tasks = [
            classify("I love this product"),
            classify("Terrible experience"),
            classify("It was okay"),
        ]
        return await asyncio.gather(*tasks)
    return asyncio.run(_run())

run_batch()
get_client().flush()

# Force flush OTEL exporter
from opentelemetry import trace
trace.get_tracer_provider().force_flush(timeout_millis=5000)

Expected: 3 litellm_request GENERATION spans nested under batch_operation Actual: Only 1-2 litellm_request spans appear. The missing span(s) change between runs (race condition).

Verification that it's concurrency-specific

Threading works because each thread gets its own context copy. The issue is specific to asyncio task concurrency.

Impact

• Cost tracking: Langfuse cannot compute cost for dropped spans → batch operation costs are underreported
• Observability: Incomplete traces make debugging LLM pipelines unreliable
• Workaround required: Users must switch from asyncio to threading for concurrent LLM calls, losing asyncio's efficiency benefits

Environment

• LiteLLM version: latest (installed via pip)
• Langfuse SDK: 4.5.0
• Python: 3.11
• OpenTelemetry SDK: 1.41.0
• Runtime: AWS Lambda (512MB) and local

Suggested fix

The callback dispatch in _client_async_logging_helper uses asyncio.create_task() which is not safe for concurrent span creation. Consider:

1. Creating OTEL spans synchronously within the callback (before dispatching to background)
2. Or providing a mechanism to ensure all callback tasks complete before the caller proceeds

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response