litellm - 💡(How to fix) Fix [Feature]: Auto-tag generic pass-through requests by upstream URL for observability (Langfuse / Spend Logs)

What happened?

For pass-through endpoints registered via general_settings.pass_through_endpoints[] (i.e. custom user-defined paths, not the built-in provider-specific handlers /anthropic/v1/messages, /openai/v1/*, /vertex_ai/*, /gemini/*, Cursor), every request is logged to Langfuse and LiteLLM_SpendLogs indistinguishably:

• observation.model = "unknown" — pass_through_endpoints.py reads model from request body (_parsed_body.get("model") or "unknown"), but many non-OpenAI providers (fal.ai, Modal, ComfyUI-style workflows, image-gen APIs) do not carry model in body — the model identity lives in the URL path itself.
• No tag identifies the upstream endpoint anywhere in trace.tags or LiteLLM_SpendLogs.request_tags.
• The full URL is captured in PassthroughStandardLoggingPayload.url but is not propagated to either standard_logging_object.metadata or standard_logging_object.request_tags, so downstream Langfuse / Spend Logs see nothing.

Net effect: e.g. all of /my-fal/fal-ai/nano-banana-2/edit, /my-fal/fal-ai/flux/schnell, /my-fal/openai/gpt-image-2/edit produce trace records that are byte-identical except for timestamp and request body — impossible to filter, group, or cost-attribute by model in Langfuse UI / Spend Logs UI / API consumers.

Why existing/related upstream work doesn't cover this

• #26120 (fix(pass-through): propagate x-litellm-spend-logs-metadata header, merged into the dev branch litellm_ishaan_april20, not main) covers only the provider-specific logging handlers (anthropic_passthrough_logging_handler.py, openai_*, vertex_*, gemini_*, cohere_*) and only writes to StandardLoggingPayload.metadata.spend_logs_metadata. It requires the client to send x-litellm-spend-logs-metadata: {...} per request and does not touch Langfuse trace.tags/observation.model. Generic custom pass-through (general_settings.pass_through_endpoints[]) is not in scope. Backport #26149 has been stalled since April.
• #25250 (Anthropic passthrough Prometheus metrics with model name missing provider prefix, open) — same root cause symptom (kwargs["model"] wrong for passthrough) but only provider-specific.
• #28228 (Anthropic /v1/messages cost tracking ignores router model_id, open) — likewise scoped to provider-specific.
• #28444 (Post API hook not enabled for passthrough endpoints, open) — adjacent, about hooks not tagging.

Searched issues and PRs with various query combinations (passthrough model unknown, pass_through_endpoint custom URL, langfuse passthrough tag, generic metadata, recent passthrough activity ≤90d) — no existing report for the generic-custom + URL-based + observability angle.

How to reproduce

# values.yaml fragment
general_settings:
  pass_through_endpoints:
    - path: "/my-fal"
      target: "https://fal.run"
      include_subpath: true
      headers:
        Authorization: "Key os.environ/FAL_API_KEY"

litellm_settings:
  callbacks: ["langfuse"]

curl -X POST "http://localhost:4000/my-fal/fal-ai/flux/schnell" \
  -H "Authorization: Bearer sk-..." \
  -d '{"prompt": "test", "num_images": 1}'

curl -X POST "http://localhost:4000/my-fal/fal-ai/nano-banana-2/edit" \
  -H "Authorization: Bearer sk-..." \
  -d '{"prompt": "test", "image_urls": [...], "num_images": 1}'

Both Langfuse traces will show observation.model = "unknown" and no tag identifying which fal endpoint was actually called.

Expected behavior

The upstream URL (already captured in PassthroughStandardLoggingPayload.url) should be used to populate at least observation.model and ideally a structured tag (e.g. <provider>_model:<path>), so Langfuse / Spend Logs can group, filter, and break down by actual upstream endpoint without client-side cooperation.

Suggested directions

Two non-exclusive options for the API surface:

Option A — automatic URL-derived identifier In pass_through_endpoints.py (the generic handler, around line 800 in v1.88.x where passthrough_model = _parsed_body.get("model") or "unknown" is set), fall back to the URL path when the body doesn't carry a model field. This is a 5-line change with minimal risk — if body has model, behavior unchanged; if not, model becomes the URL path component (currently "unknown").

Option B — declarative extractor per endpoint Allow pass_through_endpoints[] to declare a model extractor:

- path: "/my-fal"
  target: "https://fal.run"
  include_subpath: true
  model_from_url: "regex"        # or "path_suffix" / "host_segment[-2]"
  model_url_pattern: "https?://fal\\.run/(.+?)(?:\\?|$)"

More configurable but heavier. Option A covers ~90% of cases out of the box.

Either way, the resulting model identifier should flow into kwargs["model"], kwargs["standard_logging_object"]["metadata"], and kwargs["standard_logging_object"]["request_tags"] — so all existing callbacks (Langfuse, Prometheus, Spend Logs) pick it up uniformly. The same hole closed by #26120 for provider-specific paths needs to be closed for generic paths too.

Workaround we use today

A custom CustomLogger that hooks async_logging_hook (runs before async_log_success_event in the same async_success_handler, so mutations propagate to Langfuse / Prometheus / Spend Logs), extracts provider + identifier from the URL, and writes them into the same fields the upstream code is expected to populate:

"""Generic passthrough URL → model tag bridge for LiteLLM CustomLogger."""
import re
from typing import Any, Tuple
from urllib.parse import urlparse
from litellm.integrations.custom_logger import CustomLogger


def _provider_from_host(host: str) -> str:
    # "fal.run" → "fal"; "x-y--z.modal.run" → "modal"; "api.openai.com" → "openai"
    if not host:
        return "unknown"
    parts = host.split(".")
    return parts[-2] if len(parts) >= 2 else host


class PassthroughModelTagger(CustomLogger):
    # Async-marker so coroutine_checker.is_async_callable routes this instance
    # into litellm._async_success_callback (the list async_success_handler iterates).
    async def __call__(self, *args, **kwargs):
        pass

    async def async_logging_hook(
        self, kwargs: dict, result: Any, call_type: str
    ) -> Tuple[dict, Any]:
        if call_type != "pass_through_endpoint":
            return kwargs, result

        payload = kwargs.get("passthrough_logging_payload")
        url = (
            payload.get("url") if isinstance(payload, dict)
            else getattr(payload, "url", None)
        )
        if not url:
            return kwargs, result

        parsed = urlparse(str(url))
        host = parsed.hostname or ""
        path = (parsed.path or "").lstrip("/")
        if not host:
            return kwargs, result

        provider = _provider_from_host(host)
        identifier = path or host
        key = f"{provider}_model"
        tag = f"{key}:{identifier}"

        kwargs["model"] = identifier  # observation.model

        slo = kwargs.get("standard_logging_object")  # Langfuse trace + observation
        if isinstance(slo, dict):
            meta = slo.setdefault("metadata", {})
            if isinstance(meta, dict):
                meta[key] = identifier
            tags = slo.setdefault("request_tags", [])
            if isinstance(tags, list) and tag not in tags:
                tags.append(tag)

        litellm_params = kwargs.get("litellm_params")  # LiteLLM_SpendLogs.request_tags
        if isinstance(litellm_params, dict):
            lp_meta = litellm_params.setdefault("metadata", {})
            if isinstance(lp_meta, dict):
                lp_meta[key] = identifier
                lp_tags = lp_meta.setdefault("tags", [])
                if isinstance(lp_tags, list) and tag not in lp_tags:
                    lp_tags.append(tag)

        return kwargs, result


proxy_handler_instance = PassthroughModelTagger()

Registered via litellm_settings.success_callback: ["passthrough_model_tagger.proxy_handler_instance"] — not litellm_settings.callbacks:, because the latter routes the instance into litellm.callbacks only, while async_success_handler iterates litellm._async_success_callback. The async def __call__ is a marker for coroutine_checker.is_async_callable (which inspects iscoroutinefunction(instance.__call__)) — without it the callback would be classified as sync and routed to litellm.success_callback, never firing in the async Loop 1. Both surprises took a non-trivial amount of code-archaeology to figure out and seem worth documenting upstream.

Tested on v1.86.1 and v1.88.1 against fal.ai endpoints (/my-fal/fal-ai/flux/schnell, /my-fal/fal-ai/nano-banana-2/edit, /my-fal/openai/gpt-image-2/edit) — Langfuse trace.tags now show fal_model:<endpoint>, observation.model shows the real upstream identifier, and LiteLLM Spend Logs UI shows the tag in Request Tags column.

Twitter / LinkedIn handle

n/a

Relevant log output

2026-06-09T11:24:46Z LiteLLM:INFO langfuse.py:379 - Langfuse Layer Logging - logging success
INFO:     "POST /my-fal/fal-ai/flux/schnell HTTP/1.1" 200 OK

(No error — the issue is silent loss of model identity, not a crash.)