Issue Station
Sign In
Tools hereToolbox

openclaw - ✅(Solved) Fix Custom providers with input: ["image"] not recognized by media-understanding registry [2 pull requests, 1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
openclaw/openclaw#56266•Fetched 2026-04-08 01:42:59
View on GitHub
Comments
1
Participants
1
Timeline
6
Reactions
0
Author
Ezio0Ezio0
Participants
Ezio0Ezio0
Timeline (top)
cross-referenced ×2closed ×1commented ×1locked ×1

When configuring a custom provider (e.g., bailian) with models that support image input via input: [\"text\", \"image\"], the image tool fails with:

No media-understanding provider registered for bailian

Root Cause

The media-understanding capability appears to be tied to an internal provider registry. Only built-in providers (like google) are registered. Custom providers are not recognized even when their models explicitly declare input: [\"image\"].

Fix Action

Workaround

Using built-in providers (e.g., google/gemini-3-flash-preview) works correctly.

PR fix notes

PR #56279: fix: auto-register custom providers with media-capable models

Description (problem / solution / changelog)

Problem

Custom providers defined in config (e.g., bailian) with models having input: ["image"] would fail with:

No media-understanding provider registered for bailian

Root Cause

The media-understanding provider registry only included providers registered through the plugin system. Custom providers were not recognized.

Solution

This PR modifies buildMediaUnderstandingRegistry to:

  1. Scan cfg.models.providers for models with media input types
  2. Auto-register providers that have models supporting image/audio/video
  3. Not override plugin-registered providers

Changes

  • src/media-understanding/provider-registry.ts: Added detectCapabilitiesFromConfig function and auto-registration logic
  • src/media-understanding/provider-registry.test.ts: Added 4 new tests for custom provider auto-registration

Testing

All 7 tests pass:

  • 3 existing tests (unchanged)
  • 4 new tests for custom provider auto-registration

Example Config

After this fix, the following config will work for image understanding:

{
  "models": {
    "providers": {
      "bailian": {
        "baseUrl": "https://coding.dashscope.aliyuncs.com/v1",
        "api": "openai-completions",
        "models": [
          {
            "id": "qwen3.5-plus",
            "input": ["text", "image"]
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "imageModel": {
        "primary": "bailian/qwen3.5-plus"
      }
    }
  }
}

Fixes #56266

Changed files

  • src/media-understanding/provider-registry.test.ts (modified, +52/-0)
  • src/media-understanding/provider-registry.ts (modified, +8/-2)

PR #56678: fix: normalize openai-codex and github-copilot provider IDs for media-understanding

Description (problem / solution / changelog)

Summary

When using openai-codex (Codex OAuth) or github-copilot as a provider, the image/media-understanding tool fails with:

No media-understanding provider registered for openai-codex

Both providers route through OpenAI models that fully support vision, but normalizeMediaProviderId() did not map them to "openai", so the registry lookup couldn't find the OpenAI media-understanding adapter.

Changes

  • src/media-understanding/provider-id.ts — Added normalization rules: openai-codex → openai and github-copilot → openai, matching the existing gemini → google pattern.
  • src/media-understanding/provider-registry.test.ts — Added two tests confirming both provider variants resolve to the openai media-understanding provider.

Test plan

  • All 5 existing + new media-understanding tests pass (pnpm vitest src/media-understanding)
  • Pre-commit hooks (lint, typecheck) pass
  • Manual verification: configure openai-codex as primary provider → image tool should work without fallback

Changed files

  • src/media-understanding/provider-id.ts (modified, +14/-0)
  • src/media-understanding/provider-registry.test.ts (modified, +43/-0)

Code Example

No media-understanding provider registered for bailian

---

{
  \"models\": {
    \"providers\": {
      \"bailian\": {
        \"baseUrl\": \"https://coding.dashscope.aliyuncs.com/v1\",
        \"api\": \"openai-completions\",
        \"models\": [
          {
            \"id\": \"qwen3.5-plus\",
            \"name\": \"qwen3.5-plus\",
            \"input\": [\"text\", \"image\"],
            \"contextWindow\": 1000000,
            \"maxTokens\": 65536
          }
        ]
      }
    }
  },
  \"agents\": {
    \"defaults\": {
      \"imageModel\": {
        \"primary\": \"bailian/qwen3.5-plus\"
      }
    }
  }
}

---

All image models failed: bailian/qwen3.5-plus: No media-understanding provider registered for bailian
RAW_BUFFERClick to expand / collapse

Description

When configuring a custom provider (e.g., bailian) with models that support image input via input: [\"text\", \"image\"], the image tool fails with:

No media-understanding provider registered for bailian

Configuration

{
  \"models\": {
    \"providers\": {
      \"bailian\": {
        \"baseUrl\": \"https://coding.dashscope.aliyuncs.com/v1\",
        \"api\": \"openai-completions\",
        \"models\": [
          {
            \"id\": \"qwen3.5-plus\",
            \"name\": \"qwen3.5-plus\",
            \"input\": [\"text\", \"image\"],
            \"contextWindow\": 1000000,
            \"maxTokens\": 65536
          }
        ]
      }
    }
  },
  \"agents\": {
    \"defaults\": {
      \"imageModel\": {
        \"primary\": \"bailian/qwen3.5-plus\"
      }
    }
  }
}

Expected Behavior

Models configured with input: [\"image\"] should be usable by the image tool for media understanding.

Actual Behavior

All image models failed: bailian/qwen3.5-plus: No media-understanding provider registered for bailian

Analysis

The media-understanding capability appears to be tied to an internal provider registry. Only built-in providers (like google) are registered. Custom providers are not recognized even when their models explicitly declare input: [\"image\"].

Workaround

Using built-in providers (e.g., google/gemini-3-flash-preview) works correctly.

Request

Please allow custom providers with input: [\"image\"] models to be used by the image tool, either by:

  1. Automatically registering providers that have models with image input
  2. Providing a config option to mark providers as media-capable

extent analysis

Fix Plan

To fix the issue, we need to register custom providers with media-understanding capability. We can achieve this by adding a configuration option to mark providers as media-capable.

Step-by-Step Solution

  • Add a new configuration option mediaCapable to the provider configuration:
{
  "models": {
    "providers": {
      "bailian": {
        "baseUrl": "https://coding.dashscope.aliyuncs.com/v1",
        "api": "openai-completions",
        "mediaCapable": true, // Add this option
        "models": [
          {
            "id": "qwen3.5-plus",
            "name": "qwen3.5-plus",
            "input": ["text", "image"],
            "contextWindow": 1000000,
            "maxTokens": 65536
          }
        ]
      }
    }
  }
}
  • Update the provider registry to automatically register providers with mediaCapable set to true.

Example Code

# provider_registry.py
class ProviderRegistry:
    def __init__(self):
        self.providers = {}

    def register_provider(self, provider_id, media_capable):
        if media_capable:
            self.providers[provider_id] = {"media_understanding": True}
        else:
            self.providers[provider_id] = {"media_understanding": False}

    def is_media_capable(self, provider_id):
        return self.providers.get(provider_id, {}).get("media_understanding", False)

# usage
registry = ProviderRegistry()
registry.register_provider("bailian", True)
print(registry.is_media_capable("bailian"))  # Output: True

Verification

To verify that the fix worked, try using the image tool with the custom provider bailian. The tool should now recognize the provider as media-capable and use it for media understanding.

Extra Tips

  • Make sure to update the provider registry whenever a new provider is added or an existing provider is updated.
  • Consider adding a default value for the mediaCapable option to avoid having to explicitly set it for each provider.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#api#ssr#installation#tensor shape#autograd error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

openclaw - ✅(Solved) Fix Custom providers with input: ["image"] not recognized by media-understanding registry [2 pull requests, 1 comments, 1 participants]