openclaw - 💡(How to fix) Fix feat: add timeoutMs to ModelProviderConfig for per-provider request timeout [1 participants]

Problem

ModelProviderConfig (in src/config/types.models.ts) has no timeoutMs field, making it impossible to configure a per-provider request timeout for local LLM providers (llamacpp, vllm, ollama, etc.).

This is a real pain point for operators running large local models (80–120GB) on unified-memory hardware (DGX Spark / Strix Halo). After a period of idle, the OS evicts model pages from cache and the first request after reload can take 10–30 seconds of TTFT before --mlock kicks in or pages are re-faulted. The default HTTP timeout at the network layer often fires before the model finishes the cold reload, killing the request before a single token is generated.

There is currently no way to set a longer timeout for local providers without patching OpenClaw itself.

Expected behavior

{
  models: {
    providers: {
      llamacpp: {
        baseUrl: "http://localhost:8080",
        apiKey: "local",
        api: "openai-completions",
        timeoutMs: 120000,  // 2 min for cold-start recovery
        models: [...]
      }
    }
  }
}

Current ModelProviderConfig type

export type ModelProviderConfig = {
  baseUrl: string;
  apiKey?: SecretInput;
  auth?: ModelProviderAuthMode;
  api?: ModelApi;
  injectNumCtxForOpenAICompat?: boolean;
  headers?: Record<string, SecretInput>;
  authHeader?: boolean;
  models: ModelDefinitionConfig[];
  // ← no timeoutMs
};

Proposed change

Add optional timeoutMs?: number to ModelProviderConfig and thread it through to the fetch/stream layer when making LLM requests against that provider.

Use case

• Local models on unified-memory hardware (DGX Spark GB10, Strix Halo Ryzen AI Max+) where cold-start page-fault reloads cause slow TTFT after idle
• Self-hosted LiteLLM / vLLM / sglang deployments with slow startup or high load
• Any local provider where the operator knows their p99 TTFT and wants to set an appropriate timeout rather than relying on a global default

Workaround

Currently none at the config level. The only mitigations are:

• --no-mmap --mlock on the llama-server side to eliminate cold starts
• Setting timeoutSeconds on individual cron jobs (works for scheduled runs, not interactive sessions)

A per-provider timeoutMs config option would be a clean, targeted fix.