Summary

Add a --all flag to models status --probe that sends a real completion request to every configured model (primary + fallbacks) and reports per-model health, not just per-provider auth.

Problem to solve

models status --probe currently validates provider authentication, not individual model IDs. This means:

1. Deprecated/removed models go undetected — A model like openai/gpt-5.4 can sit in config returning 400s, and --probe will still report the provider as healthy because the API key is valid.

2. Fallback chain health is invisible — If your primary model fails and OpenClaw falls back to a model that's deprecated or rate-limited, the entire chain breaks silently. There's no way to pre-validate the fallback sequence.

3. Multi-model configs are increasingly common — Users on OpenRouter routinely configure 10-15+ models across tiers (primary, fallbacks, image models). The current probe tests one model per provider, leaving the rest unchecked.

The only workaround today is manually curling each model endpoint or writing a bash loop — which requires knowing each provider's API format and isn't integrated with OpenClaw's config resolution.

Proposed solution

Add --all flag to openclaw models status --probe:

openclaw models status --probe --all

Behavior:

• Iterate over every model in agents.defaults.model.primary, agents.defaults.model.fallbacks, and agents.defaults.imageModel (plus per-agent overrides if --agent is specified)
• Send a minimal completion request (e.g., max_tokens: 1, prompt: "hi") to each model ID through the configured provider
• Report per-model status:

✅ anthropic/claude-opus-4-6 ok (1.2s) ✅ openrouter/deepseek/deepseek-v3 ok (0.8s) ⚠️ openrouter/google/gemini-3.1-pro rate_limited (429, retry: 30s) ❌ openrouter/openai/gpt-5.4 error (400: model not found) ❌ openrouter/moonshot/kimi-k2.5 timeout (10.0s)

JSON output (--json): Each entry should include: model, provider, status (ok | rate_limited | error | timeout | no_auth), latencyMs, httpStatus, message.

Scope options:

• --probe --all — every configured model
• --probe --all --provider openrouter — only models under a specific provider
• --probe (current behavior unchanged) — primary model / auth check only

Timeout: Respect existing probe timeout, or allow --probe-timeout <seconds>.

Alternatives considered

1. Bash loop with curl — Works but requires per-provider API knowledge, doesn't read from OpenClaw config, and isn't portable across setups. Not realistic for most users.

2. openclaw models scan --probe — This exists for OpenRouter's free catalog discovery, but it probes available models on the provider, not your configured models. Different use case.

3. External monitoring (UptimeRobot, cron scripts) — Overkill for config validation. Users just want a pre-flight check, not ongoing monitoring.

4. Probe on first use (lazy validation) — Would catch broken models at runtime, but by then the user is already mid-conversation and hitting errors. Pre-flight is better UX.

Impact

Affected users: Anyone running multiple models — especially OpenRouter users who configure 5-15+ models across providers and tiers. Also affects multi-agent setups where different agents use different models.

Severity: Blocks workflow silently. A broken fallback model means the entire failover chain is compromised without any warning. Users discover it only when their primary goes down and the fallback also fails.

Frequency: Every time a provider deprecates or renames a model — which on OpenRouter happens regularly. Also relevant after config changes, migrations, or upgrades.

Consequence:

• Silent downtime when fallbacks are needed most
• Wasted debugging time tracing 400/404 errors back to a stale model ID
• False confidence in config health after --probe passes

Evidence/examples

Related issues:

• #12215 — "Model Ping / Health Check" feature request (same pain, broader scope)
• #42401 — Expose rate-limit headroom in probe results
• #41874 — --probe returning "LLM request timed out" with unclear errors

Real scenario (today):

• 12 models configured via OpenRouter
• openclaw models status --probe → all green (auth valid)
• Manual curl to openai/gpt-5.4 → 400 (model deprecated on OpenRouter)
• Model was silently broken in config for unknown duration

Workaround used:

for model in $(grep -oP 'openrouter/\S+' config.yaml); do
  curl -s -o /dev/null -w "%{http_code}" \
    -H "Authorization: Bearer $OPENROUTER_API_KEY" \
    -d '{"model":"'$model'","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
    https://openrouter.ai/api/v1/chat/completions
  echo " $model"
done

This works but shouldn't be necessary.

Additional information

• Happy to help test a PR if someone picks this up
• The openclaw models scan probe logic already sends real completion requests to validate tool/image support — similar infrastructure could be reused here for --probe --all
• Would pair well with #42401 (rate-limit headers) if probe responses also surfaced rate-limit headroom
• OpenClaw version: 2026.2.26