litellm - 💡(How to fix) Fix [Bug]: Docker image still bundles ddtrace 2.19.0 — breaks /embeddings on Python 3.13 with APM (v1.86.2)

Description

The LiteLLM Docker image (including latest v1.86.2) still ships with ddtrace 2.19.0, which breaks the `/embeddings` endpoint entirely when APM tracing is enabled on Python 3.13.

This was originally reported in #8744, but that issue was auto-closed as stale without a fix.

Root Cause

`ddtrace==2.19.0` has a bug in its function bytecode rewriting that causes a `coroutine already executing` error when `AsyncOpenAI` client is reused on Python 3.13. This affects all Azure OpenAI and OpenAI embedding models going through `/embeddings`.

The fix was released in ddtrace==2.19.2 (DataDog issue: https://github.com/DataDog/dd-trace-py/issues/11994).

Affected Versions

Confirmed broken: v1.83.7-stable through v1.86.2 (latest as of 2026-05-29)

# Verify ddtrace version in current image:
docker run --rm --entrypoint /bin/sh ghcr.io/berriai/litellm:v1.86.2 \
  -c "/app/.venv/bin/python -c 'import ddtrace; print(ddtrace.__version__)'"
# Output: 2.19.0

Steps to Reproduce

1. Run LiteLLM v1.86.2 with USE_DDTRACE=true and DD_APM_ENABLED=true
2. Send any request to POST /v1/embeddings with an Azure OpenAI or OpenAI embedding model
3. Observe coroutine already executing error

Workaround

We're currently using this in our Kubernetes deployment:

/app/.venv/bin/python -m ensurepip && \
/app/.venv/bin/python -m pip install 'ddtrace>=2.19.2' -q && \
ddtrace-run litellm --config ...

Note: pip binary is not available in v1.84.0+ images, so ensurepip is required first.

Requested Fix

Please upgrade ddtrace to >=2.19.2 in the Docker image build. The fix has been available since ddtrace v2.19.2.

Related

• Original report (auto-closed as stale): #8744
• DataDog fix: https://github.com/DataDog/dd-trace-py/issues/11994