openclaw - 💡(How to fix) Fix Model fallback chain interrupted by race condition — 6 fallback models configured but task terminates before all are tried [1 comments, 2 participants]

OpenClaw version: v2026.4.26 (npm @latest) Platform: Windows Server, NVIDIA API provider Config: gents.defaults.timeoutSeconds = 30 (also observed at 120) Fallback chain: 6 models configured: deepseek-v4-flash → glm-5.1 → minimax-m2.7 → qwen-397b → kimi-k2.6 → OR.intelligence → openrouter/free

Observed Behavior

When the primary model times out, the fallback mechanism correctly logs model-fallback/decision: next=<model>, but the next model is never actually called because the lane task error propagates upward at the same millisecond, killing the entire request.

The core issue: 6 fallback models are configured, but the task is terminated before any model returns a successful response and before all fallback models are tried in order.

Log Evidence

Occurrence 1 (runId=212953a6, timeoutSeconds=120 at time)

18:37:24 [agent/embedded] embedded run timeout: timeoutMs=120000 from=nVidia/minimaxai/minimax-m2.7 18:37:24 [embedded run failover decision] decision=fallback_model from=nVidia/minimaxai/minimax-m2.7 18:37:24 [lane task error] lane=main durationMs=133963 18:37:24 [model-fallback/decision] candidate_failed ... next=nvidia/z-ai/glm-5.1

Two models had already failed. The system decided to try the 3rd model (glm-5.1), but the task was terminated at the same moment. Remaining fallback models were never called.

Occurrence 2 (runId=551bbcd9, timeoutSeconds=30 at time)

22:16:18 [agent/embedded] embedded run timeout: timeoutMs=30000 from=nVidia/deepseek-ai/deepseek-v4-flash 22:16:18 [embedded run failover decision] decision=fallback_model from=nVidia/deepseek-ai/deepseek-v4-flash 22:16:18 [lane task error] lane=main durationMs=85570 error=""FailoverError: LLM request timed out."" 22:16:18 [model-fallback/decision] next=nvidia/z-ai/glm-5.1

Only the primary model failed. The system decided to try the 2nd model (glm-5.1), but the task was terminated. The remaining 5 fallback models were never called.

Expected Behavior

When a model times out, the system should try each fallback model in configured order until:

1. A model returns a successful response, OR
2. ALL fallback models have been exhausted (all failed)

The task should not be terminated early when no model has succeeded and not all fallback models have been tried.

Analysis

The model-fallback/decision log shows the system correctly decides which model to fail over to ( ext=...). However, FailoverError: LLM request timed out propagates through [diagnostic] lane task error → session task error before the new model request can actually start. This appears to be a race condition: error propagation completes before the fallback sub-task is spawned.

Steps to Reproduce

1. Configure multiple fallback models (6 in this case)
2. Set gents.defaults.timeoutSeconds to a low value (30-120)
3. Wait for the primary model to time out
4. Observe: model-fallback/decision: next=xxx appears but the model is never executed, and remaining fallback models are never attempted

Reported from OpenClaw WebChat