n8n - 💡(How to fix) Fix Webhook HTTP response blocks on concurrency queue even in `onReceived` response mode — causes deadlock with self-calling workflows [1 comments, 2 participants]

Bug Description

When N8N_CONCURRENCY_PRODUCTION_LIMIT is reached, incoming webhook requests block indefinitely without sending an HTTP response — even when the webhook node's responseMode is set to onReceived (which should respond immediately upon receiving the request).

This is a regression from n8n v1, where webhook executions would always respond immediately in onReceived mode and run the workflow asynchronously. In v2, the concurrency throttle (ConcurrencyCapacityReservation.reserve()) was introduced into the execution path before the HTTP response is sent, creating a blocking dependency that did not exist in v1.

This causes a complete system deadlock when workflows call other workflows on the same instance via webhook HTTP requests (a common orchestration pattern). Parent workflows occupy execution slots while waiting for child workflow HTTP responses that can never come — because the children are queued behind the parents.

Root Cause Location

File: packages/cli/src/webhooks/webhook-helpers.ts, function executeWebhook()

// Line ~461: THIS BLOCKS when concurrency queue is full
executionId = await Container.get(WorkflowRunner).run(runData, true, !didSendResponse && !shouldDeferOnReceivedResponse, executionId, responsePromise);

// Line ~462+: THIS NEVER EXECUTES because the await above never resolves
if (shouldDeferOnReceivedResponse) {
    // ... builds response body
    responseCallback(null, webhookResponse);  // <-- HTTP response to caller
    didSendResponse = true;
}

The call chain that blocks:

1. WorkflowRunner.run() → activeExecutions.add()
2. activeExecutions.add() → capacityReservation.reserve()
3. capacityReservation.reserve() → concurrencyControl.throttle()
4. concurrencyControl.throttle() → queue.enqueue(executionId)
5. queue.enqueue() → returns new Promise((resolve) => this.queue.push({ executionId, resolve })) — blocks forever until a slot is freed

Since the HTTP response (responseCallback) is located after the blocking await, the webhook caller never receives a response, even though shouldDeferOnReceivedResponse === true.

Contrast with v1 Behavior

In n8n v1, activeExecutions.add() did not have any concurrency throttle — executions were simply registered and started immediately. The onReceived response mode correctly sent the HTTP response right away, and the workflow executed in the background without blocking the webhook connection.

To Reproduce

1. Set N8N_CONCURRENCY_PRODUCTION_LIMIT=5 (or any small number)
2. Create Workflow A (webhook trigger, responseMode: onReceived):
   • Webhook node (POST, path: /test-parent)
   • HTTP Request node that POSTs to the same n8n instance's webhook: http://localhost:5678/webhook/test-child
   • Set timeout to 60 seconds
3. Create Workflow B (webhook trigger, responseMode: onReceived):
   • Webhook node (POST, path: /test-child)
   • Wait node (5 seconds)
   • Respond with { "ok": true }
4. Activate both workflows
5. Send 5 simultaneous POST requests to /webhook/test-parent
6. Result: All 5 slots are consumed by Workflow A waiting for Workflow B responses. Workflow B instances are queued (capacity = 0) and their webhook connection is held open indefinitely. The HTTP response (which should have been sent in onReceived mode) is never sent.
7. After the HTTP Request node timeout (60s), Workflow A instances fail, freeing slots. In production with longer timeouts (e.g., 3 hours), this becomes a multi-hour deadlock.

Simpler reproduction (no self-calling needed):

1. Set N8N_CONCURRENCY_PRODUCTION_LIMIT=2
2. Create a workflow with a Webhook trigger (responseMode: onReceived) that does a 30-second Wait
3. Activate the workflow
4. Send 2 POST requests to the webhook (fills both slots)
5. Send a 3rd POST request
6. Expected: 3rd request gets an immediate HTTP 200 response (because responseMode is onReceived) and the execution is queued
7. Actual: 3rd request hangs indefinitely — no HTTP response is sent until a slot frees up (30 seconds later when one of the first two finishes)

Expected behavior

When a webhook workflow has responseMode: 'onReceived':

• The HTTP response should be sent immediately upon receiving the request, regardless of concurrency queue state
• The execution should be enqueued and start when capacity is available
• The webhook caller should NOT be blocked waiting for execution capacity

This was the behavior in n8n v1 and is what onReceived semantically promises.

Debug Info

Additional Context

• This issue affects any self-hosted n8n v2 instance using regular execution mode with a concurrency limit
• The deadlock is especially severe for orchestration patterns where workflows trigger other workflows on the same instance via HTTP webhook calls
• Health checks (/healthz) continue to pass during the deadlock since they don't require an execution slot, making the issue invisible to load balancers and monitoring
• In n8n v1, concurrency was unlimited (no ConcurrencyControlService existed), so this class of bug was impossible
• The mode === 'queue' check in the constructor (this.isEnabled = false) means this only affects regular mode — queue mode (with Redis/BullMQ workers) is not affected

Operating System

Linux (Alpine 3.22 — Docker container on AWS ECS Fargate)

n8n Version

2.15.0

Node.js Version

24.15.0

Database

PostgreSQL

Execution mode

main (default)

Hosting

self hosted