PR #66275: fix(gateway): include configured custom provider models

• Repository: openclaw/openclaw
• Author: neeravmakwana
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/66275

Description (problem / solution / changelog)

Summary

• Problem: gateway attachment capability checks only consulted the runtime model catalog, and that catalog could omit config-defined custom provider models even when models.json preserved them.
• Why it matters: multimodal OpenAI-compatible custom models like bigmodel/glm-5v-turbo were treated as text-only, so image attachments were dropped before the model ever saw them.
• What changed: loadModelCatalog() now merges configured provider models into the runtime catalog before gateway capability checks consume it, and a regression test locks in the custom-provider omission case.
• What did NOT change (scope boundary): this does not change attachment parsing semantics, provider auth flows, or Pi discovery behavior outside catalog assembly.

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #66253
• Related #65211
• This PR fixes a bug or regression

Root Cause (if applicable)

• Root cause: resolveGatewayModelSupportsImages() fails closed when the selected provider/model pair is missing from loadGatewayModelCatalog(), and config-defined custom provider models were not being merged into that catalog.
• Missing detection / guardrail: there was no catalog-level regression test covering a custom provider model that exists in config but is omitted by Pi discovery.
• Contributing context (if known): Pi discovery can omit custom provider models, while ensureOpenClawModelsJson() still writes them into models.json, so the gateway catalog and the actual configured runtime diverged.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target test or file: src/agents/model-catalog.test.ts
• Scenario the test should lock in: a configured custom provider model with input: ["text", "image"] is present in the runtime catalog even when Pi discovery omits it.
• Why this is the smallest reliable guardrail: the regression happens at catalog assembly time, before attachment parsing or gateway RPC handling.
• Existing test that already covers this (if any): src/gateway/session-utils.test.ts already covers the image-capability helper once a matching catalog row exists.
• If no new test is added, why not: N/A

User-visible / Behavior Changes

Custom OpenAI-compatible provider models configured with image input now stay image-capable during gateway attachment checks, so image attachments are no longer dropped just because the runtime catalog omitted that provider/model row.

Diagram (if applicable)

Before:
[configured custom vision model] -> [missing from runtime catalog] -> [supportsImages=false] -> [gateway drops image attachments]

After:
[configured custom vision model] -> [merged into runtime catalog] -> [supportsImages=true] -> [gateway keeps image attachments]

Security Impact (required)

• New permissions/capabilities? (Yes/No): No
• Secrets/tokens handling changed? (Yes/No): No
• New/changed network calls? (Yes/No): No
• Command/tool execution surface changed? (Yes/No): No
• Data access scope changed? (Yes/No): No
• If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

• OS: macOS 25.3.0 (arm64)
• Runtime/container: local dev checkout
• Model/provider: bigmodel/glm-5v-turbo
• Integration/channel (if any): gateway / Control UI-style attachment path
• Relevant config (redacted): custom models.providers.bigmodel with api: "openai-completions", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", and model input: ["text", "image"]

Steps

1. Configure a custom provider model such as bigmodel/glm-5v-turbo with input: ["text", "image"].
2. Send an image through a gateway path that calls resolveGatewayModelSupportsImages() before parseMessageWithAttachments().
3. Observe whether the selected model exists in the gateway runtime catalog.

Expected

• The configured model appears in the runtime catalog.
• supportsImages resolves to true.
• The gateway keeps the image attachment.

Actual

• Before this change, the configured model could be absent from the runtime catalog even though models.json contained it.
• The gateway then resolved supportsImages to false and dropped the image attachment.

Evidence

• Failing test/log before + passing after
• Trace/log snippets
• Screenshot/recording
• Perf numbers (if relevant)

Human Verification (required)

• Verified scenarios:
  • Ran pnpm test src/agents/model-catalog.test.ts -t "includes configured custom provider models even when Pi discovery omits them".
  • Ran pnpm test src/gateway/session-utils.test.ts -t "resolveGatewayModelSupportsImages".
  • Ran pnpm check.
  • Reproduced the exact catalog lookup path locally with a temporary bigmodel/glm-5v-turbo config and auth profile; before the fix the catalog probe returned found: false, and after the fix it returned found: true with supportsImages: true.
• Edge cases checked:
  • Existing registry and provider-plugin dedupe still go through the same normalized provider/id key path.
• What you did not verify:
  • I did not run a full end-to-end webchat/manual UI attachment flow in this branch.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? (Yes/No): Yes
• Config/env changes? (Yes/No): No
• Migration needed? (Yes/No): No
• If yes, exact upgrade steps: N/A

Risks and Mitigations

• Risk: a config-defined provider model could collide with an already discovered provider/id pair.
  • Mitigation: catalog assembly now uses the same normalized provider/id dedupe key across discovered, configured, and supplemental entries, preserving existing discovered rows as authoritative when they already exist.

AI assistance: used for implementation and drafting, with local verification by targeted tests plus a direct runtime repro.

Made with Cursor

Changed files

• CHANGELOG.md (modified, +1/-0)
• src/agents/model-catalog.test.ts (modified, +84/-0)
• src/agents/model-catalog.ts (modified, +62/-13)