langchain - 💡(How to fix) Fix Add deterministic middleware orchestration with priority, conditional activation, and safe parallel execution

Submission checklist

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Feature Description

LangChain middleware is already a strong abstraction for cross-cutting agent concerns like guardrails, prompt transforms, tool handling, logging, and observability.

The recent dependency-management work around middleware is a strong step forward, especially because dependency declaration and resolution make larger middleware stacks easier to manage.

That said, there still seems to be a gap for production-scale middleware orchestration in three areas:

• Explicit priority-based ordering
• Conditional activation based on runtime context
• Safe parallel execution for read-only middleware, with deterministic merge after execution

Today, middleware can be stacked, but there is no first-class orchestration contract for:

1. Which middleware must run before others
2. Which middleware should activate only under specific runtime conditions
3. Which middleware is safe to run concurrently
4. How concurrent results should be merged deterministically for correctness and debugging

Use Case

This would help when building production agents that combine multiple middleware concerns on the same agent, for example:

• safety / policy middleware
• PII redaction
• prompt/context transforms
• logging / tracing
• metrics / analytics
• provider- or tool-specific request shaping

In these setups, some middleware must run before others, some should only activate under certain runtime conditions, and some are effectively read-only and could run concurrently.

Right now, larger middleware stacks become harder to reason about because ordering relies mostly on stack composition and dependency declarations, while concurrency and deterministic merge semantics are not first-class.

This is especially relevant because AgentMiddleware hooks can run around important points in the agent loop, including before/after agent execution, before/after model execution, and wrapping model/tool calls.

Proposed Solution

Introduce a lightweight orchestration wrapper around middleware stacks.

Example shape:

from langchain.agents import create_agent
from langchain.middleware import MiddlewareSpec, MiddlewareStack

stack = MiddlewareStack(
    [
        MiddlewareSpec(
            middleware=SafetyMiddleware(),
            name="safety",
            priority=100,
            mode="sequential",
        ),
        MiddlewareSpec(
            middleware=PIIRedactionMiddleware(),
            name="pii",
            priority=90,
            depends_on=["safety"],
            mode="sequential",
        ),
        MiddlewareSpec(
            middleware=LoggingMiddleware(),
            name="logging",
            priority=10,
            mode="parallel_readonly",
        ),
        MiddlewareSpec(
            middleware=MetricsMiddleware(),
            name="metrics",
            priority=10,
            mode="parallel_readonly",
        ),
    ],
    deterministic_merge=True,
)

agent = create_agent(
    model=model,
    tools=tools,
    middleware=stack,
)

Suggested concepts:

• priority: int Explicit execution ordering without relying only on list position.

• depends_on: list[str] Dependency constraints between middleware components.

• condition: Callable[[RuntimeContext], bool] | None Activate middleware only for certain tools, models, tenants, requests, or state conditions.

• mode: Literal["sequential", "parallel_readonly", "batched"] Execution policy per middleware.

• deterministic_merge: bool = True Preserve stable ordering and traceability after concurrent execution.

A possible safety model could classify middleware as:

• readonly: inspects state, emits logs/metrics, attaches metadata
• transform: mutates request/response/state, sequential by default
• wrap: wraps model/tool execution and preserves nesting order
• terminal: can short-circuit/block execution and stays highest-priority and sequential

Only middleware explicitly marked safe/read-only would be eligible for concurrent execution.

Alternatives Considered

The current workaround is to keep middleware fully sequential and manually control ordering through stack composition and dependency declarations.

Another alternative is to move more logic out of middleware and into LangGraph-level workflow orchestration. That works for workflow-level parallelism, but it is less natural for cross-cutting concerns like tracing, redaction, request shaping, and guardrails, which fit better as middleware concerns.

Another option is to combine multiple concerns into one custom middleware implementation, but that reduces modularity and reusability.

Additional Context

This proposal is not intended to replace LangGraph for workflow orchestration or fan-out/fan-in patterns.

The goal is to make middleware stacks more scalable and explicit for production use while preserving a clean abstraction boundary:

• middleware handles cross-cutting concerns around agent execution
• LangGraph handles broader workflow-level parallelism

A narrow MVP could be:

1. MiddlewareSpec
2. MiddlewareStack
3. validation for dependency cycles and unsafe concurrent candidates
4. concurrent execution only for explicitly read-only middleware
5. deterministic merge plus debug tracing of actual execution order

Happy to help with an RFC, API sketch, or initial implementation if this direction seems aligned.