Summary

tools.media appears to accept a LiteLLM tier alias like provider: "litellm", model: "vision", but in practice this is brittle / broken for image preprocessing. Using a direct provider (provider: "openai", model: "gpt-4o-mini") fixed the issue immediately in the same environment.

This is especially confusing because the OpenClaw app wrote litellm/vision-style config, so the resulting config looked valid.

Environment

• OpenClaw: 2026.3.24
• macOS
• Main model: litellm/claude-sonnet-4-6
• LiteLLM proxy configured with a vision model group that resolves to gpt-4o-mini

Problem

With this config:

{
  "tools": {
    "media": {
      "models": [{
        "provider": "litellm",
        "model": "vision",
        "capabilities": ["image"],
        "type": "provider",
        "maxBytes": 10485760,
        "timeoutSeconds": 60
      }]
    }
  },
  "agents": {
    "defaults": {
      "imageModel": {
        "primary": "litellm/vision"
      }
    }
  }
}

image understanding failed / was unreliable.

Switching only tools.media to a direct provider fixed it:

{
  "tools": {
    "media": {
      "models": [{
        "provider": "openai",
        "model": "gpt-4o-mini",
        "capabilities": ["image"],
        "type": "provider",
        "maxBytes": 10485760,
        "timeoutSeconds": 60
      }]
    }
  },
  "agents": {
    "defaults": {
      "imageModel": {
        "primary": "litellm/gpt-4o-mini"
      }
    }
  }
}

Why this seems like a product bug

Even if LiteLLM vision resolves to gpt-4o-mini, these are not equivalent paths:

• provider=openai model=gpt-4o-mini → direct provider call
• provider=litellm model=gpt-4o-mini → LiteLLM exact model target
• provider=litellm model=vision → LiteLLM model-group alias + fallback machinery

For tools.media, that alias/group path seems unsafe or unsupported.

Expected behavior

One of these should happen:

1. tools.media with LiteLLM tier aliases should work reliably, or
2. OpenClaw should reject / warn on configs like:
   • provider=litellm model=vision
   • provider=litellm model=simple|medium|complex

Suggested fixes

Option A: validation / warning

Reject or warn when tools.media uses LiteLLM tier aliases instead of direct providers or concrete models.

Option B: better error text

Emit a clear message that tools.media should use a direct provider or a concrete LiteLLM model, not a tier alias.

Option C: app-side config generation

If the macOS/iOS app is generating this config, it should avoid writing provider=litellm model=vision into tools.media.

Notes

In our debugging session, replacing:

• tools.media = litellm/vision

with:

• tools.media = openai/gpt-4o-mini

resolved the problem immediately while keeping:

• imageModel = litellm/gpt-4o-mini

So the problem appears specifically tied to tools.media + LiteLLM tier alias usage, not LiteLLM in general.