PR #80229: [plugin sdk] Phase 1: consolidate host workflow seams and media extraction runtime

• Repository: openclaw/openclaw
• Author: 100yenadmin
• State: closed | merged: False
• Link: https://github.com/openclaw/openclaw/pull/80229

Description (problem / solution / changelog)

Closes #80219.

This is Phase 1 of the whole-surface Plugin SDK consolidation plan in #80219. It intentionally cleans up the host-workflow/session-control slice plus the active proof-gated seams, not the entire SDK/API surface in one PR.

This PR consolidates the active Plugin SDK host-hook/runtime work into one cleanup-first branch instead of asking reviewers to reason across four separate PRs plus a follow-up architecture issue.

It folds together:

• #75578: typed session action registration/dispatch plus scope enforcement
• #75581: host-mediated session attachments
• #75588: scheduled session turns
• #79334: structured extraction media runtime

It also adds one narrow consolidation slice on top:

• additive grouped host-hook aliases on OpenClawPluginApi
• shared late-callability metadata for runtime-callable post-register APIs

Why this matters

OpenClaw is moving more host behavior into plugins and extensions, but the current Plugin SDK surface was heading toward a flat pile of new top-level verbs. That makes every new capability feel like permanent API expansion even when the underlying runtime seams are valid.

This PR keeps the feature work, but makes the shape easier to grow from:

• typed session actions stay a clean control/RPC seam
• session attachments and scheduled turns stay host-mediated workflow seams
• structured extraction stays inside media understanding instead of widening generic runtime.llm
• the new grouped aliases make the host-hook-heavy parts of the SDK read like families instead of unrelated top-level escapes

What this PR does

1. Integrates the active host-hook/runtime slices

• session action gateway protocol, dispatch, and scope enforcement
• host-mediated session attachment delivery
• scheduled session turns plus tag-based unscheduling
• structured extraction in the media-understanding runtime

2. Adds a narrow consolidation layer

• api.session.state.*
• api.session.workflow.*
• api.session.controls.*
• api.agent.events.*
• api.runContext.*
• api.lifecycle.*

These are additive aliases over the current flat methods. Existing plugin code keeps working.

3. Centralizes loader late-callability policy

Instead of loader.ts hardcoding post-register callable methods inline, this PR moves that policy into shared metadata so the runtime-callable host-hook seams have one source of truth.

Current runtime-callable post-register methods remain:

• sendSessionAttachment
• scheduleSessionTurn
• unscheduleSessionTurnsByTag

Architecture

flowchart TD
    A["Plugin author surface"] --> B["OpenClawPluginApi"]
    B --> C["Flat legacy methods remain supported"]
    B --> D["Additive grouped aliases"]

    D --> E["session.state"]
    D --> F["session.workflow"]
    D --> G["session.controls"]
    D --> H["agent.events"]
    D --> I["runContext"]
    D --> J["lifecycle"]

    F --> K["next-turn injection"]
    F --> L["attachments"]
    F --> M["scheduled turns"]
    G --> N["session actions"]
    H --> O["event subscribe/emit"]
    J --> P["runtime cleanup"]

    Q["mediaUnderstanding runtime"] --> R["structured extraction"]
    Q --> S["explicit provider/model media inference lane"]

Plugin families this supports

This combined surface is meant to support more than one product feature. Concretely, it is a foundation for:

• approval and operator-control plugins
• reminder, follow-up, and scheduled workflow plugins
• channel companion plugins that need outbound attachment delivery
• session-aware UI/control plugins
• background monitor and lifecycle plugins
• media-understanding and structured extraction plugins

Boundaries

• This does not collapse structured extraction into generic runtime.llm.complete.
• This does not remove the flat Plugin SDK methods yet.
• This does not try to genericize all workflow seams into one catch-all runtime mutation API.
• This keeps the upstream ACP streaming/default-delivery behavior intact.

Real behavior proof

Behavior or issue addressed: This combined branch keeps the new host-hook seams narrow while proving they work together on the #80229 head. After this patch, typed plugin session actions dispatch through the Gateway with the action-declared operator scope bundle, grouped late-call workflow aliases remain callable after register() closes for bundled-plugin attachment and scheduled-turn operations, registration-only APIs stay blocked after register, and bounded structured extraction stays inside the media-understanding runtime instead of widening generic runtime.llm.

Real environment tested: Local macOS checkout at /Volumes/LEXAR/repos/openclaw-plugin-sdk-consolidation on head 564400622821b7f89b374f2e7c25ac0de61bfa4d. I used an isolated temp workspace/state/config root at /var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs, started a token-auth loopback Gateway on ws://127.0.0.1:18997, ran live gateway call requests for the session-action seam, ran a standalone bundled-origin guarded API script against that live cron.* backend for grouped workflow aliases, and ran a standalone runtime proof script for extractStructuredWithModel(...).

Exact steps or command run after this patch:

1. Started an isolated token-auth Gateway from the merged head:

   HOME=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/home \
   OPENCLAW_WORKSPACE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/workspace \
   OPENCLAW_STATE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/state \
   OPENCLAW_CONFIG_PATH=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/config/openclaw.json \
   pnpm openclaw gateway run --allow-unconfigured --auth token --token proof-token --bind loopback --port 18997

2. Called the typed session-action seam on that Gateway once with valid payload and once with invalid payload:

   HOME=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/home \
   OPENCLAW_WORKSPACE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/workspace \
   OPENCLAW_STATE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/state \
   OPENCLAW_CONFIG_PATH=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/config/openclaw.json \
   pnpm openclaw gateway call plugins.sessionAction --json --token proof-token --url ws://127.0.0.1:18997 --params '{"pluginId":"proof-combined","actionId":"approve","sessionKey":"agent:main:main","payload":{"message":"post-fix"}}'
   
   HOME=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/home \
   OPENCLAW_WORKSPACE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/workspace \
   OPENCLAW_STATE_DIR=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/state \
   OPENCLAW_CONFIG_PATH=/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/config/openclaw.json \
   pnpm openclaw gateway call plugins.sessionAction --json --token proof-token --url ws://127.0.0.1:18997 --params '{"pluginId":"proof-combined","actionId":"approve","sessionKey":"agent:main:main","payload":{"message":42}}'

3. Ran a standalone node --import tsx proof script from repo root that:
   • creates a real bundled-origin plugin record through createPluginRegistry(...)
   • captures the guarded API via loader.__testing.runPluginRegisterSync(...)
   • invokes capturedApi.session.workflow.sendSessionAttachment(...)
   • verifies capturedApi.session.state.registerSessionExtension(...) stays blocked after register()
   • invokes capturedApi.session.workflow.scheduleSessionTurn(...) twice plus invalid tag/deleteAfterRun cases
   • calls capturedApi.session.workflow.unscheduleSessionTurnsByTag(...)
   • queries live cron.list before and after cleanup through the running Gateway
4. Ran a standalone node --import tsx runtime proof script from repo root that invokes runtime.mediaUnderstanding.extractStructuredWithModel(...) once with image+text input and once with text-only input.

Evidence after fix:

$ pnpm openclaw gateway call plugins.sessionAction --json --token proof-token --url ws://127.0.0.1:18997 --params '{"pluginId":"proof-combined","actionId":"approve","sessionKey":"agent:main:main","payload":{"message":"post-fix"}}'
{
  "ok": true,
  "result": {
    "approved": true,
    "message": "post-fix",
    "sessionKey": "agent:main:main",
    "scopes": [
      "operator.admin",
      "operator.read",
      "operator.write",
      "operator.approvals",
      "operator.pairing",
      "operator.talk.secrets"
    ]
  },
  "continueAgent": true
}

$ pnpm openclaw gateway call plugins.sessionAction --json --token proof-token --url ws://127.0.0.1:18997 --params '{"pluginId":"proof-combined","actionId":"approve","sessionKey":"agent:main:main","payload":{"message":42}}'
Gateway call failed: GatewayClientRequestError: plugin session action payload does not match schema: message: must be string

$ node --import tsx <bundled-origin guarded workflow proof>
{
  "attachment": {
    "ok": true,
    "channel": "proofchat",
    "deliveredTo": "12345",
    "count": 1
  },
  "deliveries": [
    {
      "to": "12345",
      "text": "attachment ready",
      "mediaUrl": "/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr80229-proof-qGy5Vs/workspace/proof-report.txt",
      "accountId": "default"
    }
  ],
  "sessionExtensionsBefore": 0,
  "sessionExtensionsAfter": 0,
  "first": {
    "id": "f2f6d165-2356-402c-ab9a-9b67b4e62f31",
    "pluginId": "proof-combined",
    "sessionKey": "agent:main:main",
    "kind": "session-turn"
  },
  "second": {
    "id": "32447765-6139-4238-bc4e-f3fb15ed333d",
    "pluginId": "proof-combined",
    "sessionKey": "agent:main:main",
    "kind": "session-turn"
  },
  "badTag": null,
  "badDelete": null,
  "cronJobsAfterSchedule": {
    "jobs": [
      {
        "id": "32447765-6139-4238-bc4e-f3fb15ed333d",
        "name": "plugin:proof-combined:tag:nudge:agent:main:main:2396fa1c-f78a-49e6-93d0-9e0013a02afa",
        "deleteAfterRun": true,
        "sessionTarget": "session:agent:main:main",
        "payload": { "kind": "agentTurn", "message": "wake two" },
        "delivery": { "mode": "none" }
      },
      {
        "id": "f2f6d165-2356-402c-ab9a-9b67b4e62f31",
        "name": "plugin:proof-combined:tag:nudge:agent:main:main:4e148d8e-5e2b-4cdc-bc80-ce50f6cd8f47",
        "deleteAfterRun": true,
        "sessionTarget": "session:agent:main:main",
        "payload": { "kind": "agentTurn", "message": "wake one" },
        "delivery": { "mode": "announce", "channel": "last" }
      }
    ],
    "total": 2
  },
  "removed": {
    "removed": 2,
    "failed": 0
  },
  "cronJobsAfterCleanup": {
    "jobs": [],
    "total": 0
  }
}

$ node --import tsx <structured extraction runtime proof>
{
  "success": {
    "text": "{\"summary\":\"red square\",\"tags\":[\"shape\"]}",
    "model": "gpt-5.4",
    "provider": "codex",
    "contentType": "json",
    "parsed": {
      "summary": "red square",
      "tags": ["shape"]
    }
  },
  "authProfileIds": ["openai-codex:work"],
  "requestMethods": ["model/list", "thread/start", "turn/start"],
  "threadStart": {
    "approvalPolicy": "on-request",
    "sandbox": "read-only",
    "serviceName": "OpenClaw",
    "dynamicTools": [],
    "ephemeral": true
  },
  "guardError": "Structured extraction requires at least one image input."
}

Observed result after fix: On the combined #80229 head, plugins.sessionAction dispatch succeeds with typed output and reaches the handler under the action-declared operator scope bundle, while malformed payloads are rejected before handler execution. The grouped late-call workflow aliases on api.session.workflow.* remain callable after register() closes for a bundled plugin, the registration-only session.state.registerSessionExtension(...) path stays blocked (sessionExtensionsBefore/After remained 0), a relative attachment path resolves against the session workspace and is delivered to the active direct-outbound route, two real host cron jobs are created and visible through cron.list, invalid tag/deleteAfterRun inputs fail closed, and tagged cleanup removes both jobs. Structured extraction stays bounded in the media-understanding runtime, forwards the selected auth profile into the provider-owned runtime, returns parsed JSON for image input, and rejects text-only calls before provider dispatch.

What was not tested: This proof did not run a full end-to-end Control UI or native Apple client flow, and it did not use a credentialed live Codex desktop session. The live evidence here is intentionally limited to merged-head Gateway dispatch, bundled-origin guarded workflow aliases against a real cron.* backend, direct-outbound attachment delivery, and bounded runtime/provider structured extraction behavior.

Verification

Local verification on this combined branch:

node scripts/run-vitest.mjs run --config test/vitest/vitest.config.ts src/plugins/loader.test.ts --maxWorkers=1
node scripts/run-vitest.mjs run --config test/vitest/vitest.contracts-plugin.config.ts src/plugins/contracts/run-context-lifecycle.contract.test.ts src/plugins/contracts/session-actions.contract.test.ts --maxWorkers=1
pnpm plugin-sdk:api:gen
node scripts/run-oxlint-shards.mjs --threads=8

Notes for reviewers

• The feature work here originated as four active PR lines because each seam was independently useful.
• The maintainer concern about API accretion is valid, so this branch intentionally adds only the smallest consolidation layer that improves the author-facing shape without re-plumbing the runtime.
• If this direction lands, the next cleanup step should be export-surface reduction and alias/deprecation of the obvious debt lanes described in #80219.

Changed files

• CHANGELOG.md (modified, +5/-0)
• apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift (modified, +208/-0)
• docs/.generated/plugin-sdk-api-baseline.sha256 (modified, +2/-2)
• docs/plugins/architecture-internals.md (modified, +28/-0)
• docs/plugins/sdk-overview.md (modified, +18/-0)
• docs/plugins/sdk-runtime.md (modified, +27/-0)
• docs/plugins/sdk-subpaths.md (modified, +1/-1)
• extensions/codex/media-understanding-provider.test.ts (modified, +160/-2)
• extensions/codex/media-understanding-provider.ts (modified, +198/-10)
• extensions/googlechat/config-api.ts (added, +2/-0)
• extensions/googlechat/src/config-schema.ts (modified, +1/-1)
• extensions/matrix/src/matrix/monitor/handler.test-helpers.ts (modified, +2/-1)
• extensions/matrix/src/matrix/monitor/handler.test.ts (modified, +9/-0)
• extensions/telegram/api.test.ts (added, +16/-0)
• extensions/telegram/api.ts (modified, +8/-0)
• extensions/telegram/src/channel.gateway.test.ts (modified, +47/-0)
• extensions/telegram/src/format.ts (modified, +6/-2)
• extensions/telegram/src/outbound-adapter.test.ts (modified, +5/-6)
• extensions/telegram/src/outbound-adapter.ts (modified, +12/-2)
• scripts/protocol-gen-swift.ts (modified, +246/-1)
• src/agents/tools/gateway.test.ts (modified, +66/-0)
• src/agents/tools/gateway.ts (modified, +1/-1)
• src/config/sessions/delivery-info.test.ts (modified, +223/-3)
• src/config/sessions/delivery-info.ts (modified, +154/-24)
• src/gateway/call.test.ts (modified, +24/-0)
• src/gateway/call.ts (modified, +2/-2)
• src/gateway/method-scopes.test.ts (modified, +94/-0)
• src/gateway/method-scopes.ts (modified, +55/-1)
• src/gateway/protocol/index.ts (modified, +11/-0)
• src/gateway/protocol/schema/agent.ts (modified, +6/-0)
• src/gateway/protocol/schema/plugins.ts (modified, +35/-0)
• src/gateway/protocol/schema/protocol-schemas.ts (modified, +8/-0)
• src/gateway/protocol/schema/types.ts (modified, +2/-0)
• src/gateway/server-methods-list.ts (modified, +1/-0)
• src/gateway/server-methods/plugin-host-hooks.ts (modified, +314/-0)
• src/gateway/server-methods/send.test.ts (modified, +22/-0)
• src/gateway/server-methods/send.ts (modified, +6/-0)
• src/infra/outbound/formatting.ts (modified, +1/-0)
• src/infra/outbound/message.channels.test.ts (modified, +17/-0)
• src/infra/outbound/message.ts (modified, +6/-0)
• src/media-understanding/runtime-types.ts (modified, +25/-0)
• src/media-understanding/runtime.test.ts (modified, +155/-0)
• src/media-understanding/runtime.ts (modified, +40/-1)
• src/media-understanding/types.ts (modified, +41/-0)
• src/plugin-sdk/core.ts (modified, +10/-0)
• src/plugin-sdk/media-understanding-runtime.ts (modified, +2/-0)
• src/plugin-sdk/media-understanding.ts (modified, +5/-0)
• src/plugin-sdk/plugin-entry.ts (modified, +20/-0)
• src/plugin-sdk/plugin-test-api.ts (modified, +8/-2)
• src/plugin-sdk/test-helpers/plugin-runtime-mock.ts (modified, +2/-0)
• src/plugins/agent-event-emission.ts (added, +84/-0)
• src/plugins/api-builder.ts (modified, +26/-2)
• src/plugins/api-facades.ts (added, +36/-0)
• src/plugins/api-lifecycle.ts (added, +36/-0)
• src/plugins/captured-registration.test.ts (modified, +47/-1)
• src/plugins/captured-registration.ts (modified, +27/-5)
• src/plugins/contracts/bundled-extension-config-api-guardrails.test.ts (modified, +1/-1)
• src/plugins/contracts/host-hooks.contract.test.ts (modified, +57/-0)
• src/plugins/contracts/run-context-lifecycle.contract.test.ts (modified, +6/-6)
• src/plugins/contracts/scheduled-turns.contract.test.ts (added, +1216/-0)
• src/plugins/contracts/session-actions.contract.test.ts (added, +1018/-0)
• src/plugins/contracts/session-attachments.contract.test.ts (added, +877/-0)
• src/plugins/host-hook-cleanup.ts (modified, +3/-0)
• src/plugins/host-hook-runtime.ts (modified, +18/-2)
• src/plugins/host-hook-workflow.ts (added, +781/-0)
• src/plugins/host-hooks.ts (modified, +127/-0)
• src/plugins/loader.test.ts (modified, +72/-0)
• src/plugins/loader.ts (modified, +18/-3)
• src/plugins/registry-empty.ts (modified, +1/-0)
• src/plugins/registry-lifecycle.ts (modified, +6/-0)
• src/plugins/registry-types.ts (modified, +10/-0)
• src/plugins/registry.ts (modified, +215/-2)
• src/plugins/runtime-state.ts (modified, +1/-0)
• src/plugins/runtime.channel-pin.test.ts (modified, +110/-0)
• src/plugins/runtime.test.ts (modified, +17/-0)
• src/plugins/runtime.ts (modified, +86/-24)
• src/plugins/runtime/index.test.ts (modified, +1/-0)
• src/plugins/runtime/index.ts (modified, +3/-0)
• src/plugins/runtime/types-core.ts (modified, +1/-0)
• src/plugins/schema-validator.ts (modified, +6/-4)
• src/plugins/types.ts (modified, +112/-0)

PR #79334: [plugin sdk] Add structured extraction media runtime

• Repository: openclaw/openclaw
• Author: 100yenadmin
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/79334

Description (problem / solution / changelog)

Why this matters

OpenClaw plugins increasingly need to turn unstructured user content into safe, typed data: receipts into expense records, screenshots into support evidence, invoices into accounting fields, customer messages into CRM notes, PDFs into knowledge-base snippets, and product photos into searchable inventory metadata.

Today each plugin has to choose between two bad options:

• implement its own model/auth/runtime bridge, usually requiring another user-managed API key; or
• add product-specific extraction routes to core, which does not scale as the plugin ecosystem grows.

This PR adds the missing middle layer: a generic structured extraction capability in the media-understanding SDK. Product plugins keep owning their routes, schemas, storage, and UX, while OpenClaw owns the provider/runtime boundary, auth source, safety posture, and typed SDK contract.

What plugin authors can build with this

Examples this unlocks without adding plugin-specific logic to OpenClaw core:

• Support plugins: extract error messages, stack traces, product names, issue category, severity, and reproduction steps from screenshots.
• Knowledge-base plugins: convert documents or screenshots into normalized metadata and searchable evidence records.
• CRM/sales plugins: extract companies, people, dates, action items, sentiment, and deal updates from inbound media plus short text context.
• Finance/admin plugins: extract vendor, total, currency, tax, due date, and line-item hints from receipts or invoices.
• Inventory/media plugins: extract labels, visible text, tags, object categories, and image summaries from uploaded photos.
• Migration/import plugins: map arbitrary image inputs into a plugin-owned JSON schema before writing to the plugin's own database.

The important part: the plugin defines the schema and decides what to do with the result. OpenClaw only provides the generic, bounded extraction lane.

New SDK shape

This PR adds:

• optional provider method: MediaUnderstandingProvider.extractStructured(...)
• runtime helper: api.runtime.mediaUnderstanding.extractStructuredWithModel(...)
• typed inputs for images plus optional supplemental text context
• optional schemaName, jsonSchema, jsonMode, and timeoutMs
• controlled result metadata: raw text, parsed JSON when JSON mode is enabled, model/provider, and content type

Example plugin call:

const result = await api.runtime.mediaUnderstanding.extractStructuredWithModel({
  provider: "codex",
  model: "gpt-5.5",
  input: [
    {
      type: "image",
      buffer: receiptImageBuffer,
      fileName: "receipt.png",
      mime: "image/png",
    },
    { type: "text", text: "Prefer the printed total over handwritten notes." },
  ],
  instructions: "Extract vendor, total, and searchable tags.",
  schemaName: "receipt.evidence",
  jsonSchema: {
    type: "object",
    properties: {
      vendor: { type: "string" },
      total: { type: "number" },
      tags: { type: "array", items: { type: "string" } },
    },
    required: ["vendor", "total"],
  },
  cfg: api.config,
});

Runtime architecture

flowchart LR
  Plugin["Plugin route, skill, or importer"] --> Runtime["api.runtime.mediaUnderstanding.extractStructuredWithModel"]
  Runtime --> Provider["MediaUnderstandingProvider.extractStructured"]
  Provider --> HostRuntime["Provider-owned host runtime"]
  HostRuntime --> Result["JSON result or controlled error"]
  Result --> Plugin
  Plugin --> Storage["Plugin-owned storage, tools, or user workflow"]

For the bundled Codex provider, this uses the existing Codex app-server/OAuth path rather than requiring a user-supplied model API key.

flowchart LR
  Plugin["Any OpenClaw plugin"] --> SDK["Structured extraction SDK"]
  SDK --> CodexProvider["Codex media-understanding provider"]
  CodexProvider --> AppServer["Codex app-server / OAuth runtime"]
  AppServer --> BoundedTurn["Ephemeral no-tools turn"]
  BoundedTurn --> JSON["Parsed JSON or controlled error"]

Safety and boundaries

The Codex implementation keeps the same bounded posture as image understanding:

• ephemeral thread
• read-only sandbox
• no dynamic tools
• approval policy set to on-request, with approval requests denied by the provider handler
• timeout enforcement
• model modality validation before turn start
• JSON parsing failure returned as a controlled error
• text-only extraction rejected at the runtime seam, keeping this image-first instead of turning it into a generic text completion lane
• no product-specific route names, storage models, or schemas in OpenClaw core

This is intentionally a platform seam, not a feature-specific integration.

What changed

• Adds structured extraction request/result types to the media-understanding SDK.
• Adds extractStructuredWithModel(...) to the plugin runtime media-understanding facade.
• Implements extractStructured(...) in the bundled Codex provider.
• Preserves explicit config-provider image descriptions by keeping describeImageFileWithModel(...) on the full media-understanding registry instead of narrowing it to manifest-only plugin providers.
• Forwards structured extraction auth-profile selection through the runtime helper so provider-owned OAuth/app-server runtimes can honor plugin-selected credentials.
• Narrows the new seam to image-first extraction with optional supplemental text context instead of overlapping general text-only completion surfaces.
• Adds tests for bounded Codex structured extraction, invalid JSON/schema handling, runtime routing, auth-profile forwarding, image-required guardrails, direct image-model registry routing, provider lookup failure, and runtime API exposure.
• Documents the new runtime helper and the plugin/core ownership boundary.
• Adds the required changelog entry for the new plugin SDK/runtime capability.

Relationship to existing LLM surfaces

OpenClaw already has api.runtime.llm.complete for trusted plugin text completions, and llm-task for workflow/tool-level JSON tasks. This PR is narrower and lower-level: a provider SDK/runtime media-understanding seam for schema-shaped extraction over image inputs with optional text context. That keeps extraction provider-owned and plugin-consumable without turning it into a general-purpose arbitrary Codex call API.

Non-goals

• This does not add a product-specific extraction route to OpenClaw core.
• This does not choose any plugin's storage model or JSON schema.
• This does not replace existing image/audio/video media-understanding helpers.
• This does not require plugins to use Codex; other providers can implement the same optional method.
• This does not expand into generic text-only extraction; callers that want arbitrary text completions should keep using the existing LLM surfaces.

Background

This closes openclaw/openclaw#79321.

The immediate downstream need came from a GBrain/OpenClaw integration, but the implementation here is deliberately generic. GBrain, support, CRM, finance, inventory, migration, and knowledge-base plugins can all consume the same SDK seam while keeping their own product-specific routes and schemas outside OpenClaw core.

Real behavior proof

Behavior or issue addressed: The rebased branch exposes a typed plugin-runtime structured extraction seam that dispatches through a registered media-understanding provider, preserves the bounded Codex worker defaults, forwards the selected auth profile into the provider-owned runtime, and rejects text-only calls before provider dispatch.

Real environment tested: Local macOS OpenClaw checkout at /Users/lume/openclaw-review-worktrees/pr-79334-rebase, rebased head 78cfe4a76161fc7d3029beb4edcf7120a94a4d8b, using a standalone node --import tsx proof command outside Vitest. The proof registers the real bundled Codex media-understanding provider in the active plugin runtime registry with a stubbed app-server client, then calls createPluginRuntime().mediaUnderstanding.extractStructuredWithModel(...) once with image-plus-text input and once with text-only input.

Exact steps or command run after this patch:

cd /Users/lume/openclaw-review-worktrees/pr-79334-rebase
node --import tsx <<'EOF'
import { buildCodexMediaUnderstandingProvider } from './extensions/codex/media-understanding-provider.ts';
import { createPluginRuntime } from './src/plugins/runtime/index.ts';
import { createEmptyPluginRegistry } from './src/plugins/registry-empty.ts';
import { resetPluginRuntimeStateForTest, setActivePluginRegistry } from './src/plugins/runtime.ts';

function codexModel(inputModalities = ['text', 'image']) {
  return {
    id: 'gpt-5.4',
    model: 'gpt-5.4',
    upgrade: null,
    upgradeInfo: null,
    availabilityNux: null,
    displayName: 'gpt-5.4',
    description: 'GPT-5.4',
    hidden: false,
    supportedReasoningEfforts: [{ reasoningEffort: 'low', description: 'fast' }],
    defaultReasoningEffort: 'low',
    inputModalities,
    supportsPersonality: false,
    additionalSpeedTiers: [],
    isDefault: true,
  };
}

function threadStartResult() {
  return {
    thread: {
      id: 'thread-1',
      sessionId: 'session-1',
      forkedFromId: null,
      preview: '',
      ephemeral: true,
      modelProvider: 'openai',
      createdAt: 1,
      updatedAt: 1,
      status: { type: 'idle' },
      path: null,
      cwd: process.cwd(),
      cliVersion: '0.125.0',
      source: 'unknown',
      agentNickname: null,
      agentRole: null,
      gitInfo: null,
      name: null,
      turns: [],
    },
    model: 'gpt-5.4',
    modelProvider: 'openai',
    serviceTier: null,
    cwd: process.cwd(),
    instructionSources: [],
    approvalPolicy: 'on-request',
    approvalsReviewer: 'user',
    sandbox: { type: 'dangerFullAccess' },
    permissionProfile: null,
    reasoningEffort: null,
  };
}

function turnStartResult(status = 'inProgress', items = []) {
  return {
    turn: {
      id: 'turn-1',
      status,
      items,
      error: null,
      startedAt: null,
      completedAt: null,
      durationMs: null,
    },
  };
}

function createFakeClient(responseText) {
  const notifications = new Set();
  const requestHandlers = new Set();
  const requests = [];
  const request = async (method, params) => {
    requests.push({ method, params });
    if (method === 'model/list') return { data: [codexModel()], nextCursor: null };
    if (method === 'thread/start') return threadStartResult();
    if (method === 'turn/start') {
      for (const notify of notifications) {
        notify({ method: 'item/agentMessage/delta', params: { threadId: 'thread-1', turnId: 'turn-1', itemId: 'msg-1', delta: responseText } });
        notify({ method: 'turn/completed', params: { threadId: 'thread-1', turnId: 'turn-1', turn: turnStartResult('completed').turn } });
      }
      for (const handler of requestHandlers) handler({ method: 'item/permissions/requestApproval' });
      return turnStartResult();
    }
    return {};
  };
  return {
    client: {
      request,
      addNotificationHandler(handler) { notifications.add(handler); return () => notifications.delete(handler); },
      addRequestHandler(handler) { requestHandlers.add(handler); return () => requestHandlers.delete(handler); },
      close() {},
    },
    requests,
  };
}

const authProfileIds = [];
const { client, requests } = createFakeClient('{"summary":"red square","tags":["shape"]}');
const provider = buildCodexMediaUnderstandingProvider({
  clientFactory: async (_startOptions, authProfileId) => {
    authProfileIds.push(authProfileId ?? null);
    return client;
  },
});

const registry = createEmptyPluginRegistry();
registry.mediaUnderstandingProviders.push({
  pluginId: 'codex',
  pluginName: 'Codex',
  source: 'proof-script',
  provider,
});
setActivePluginRegistry(registry, 'proof-script', 'default', process.cwd());

const runtime = createPluginRuntime();
const success = await runtime.mediaUnderstanding.extractStructuredWithModel({
  provider: 'codex',
  model: 'gpt-5.4',
  input: [
    {
      type: 'image',
      buffer: Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+kX3sAAAAASUVORK5CYII=', 'base64'),
      fileName: 'red-square.png',
      mime: 'image/png',
    },
    { type: 'text', text: 'Return searchable evidence for the uploaded image.' },
  ],
  instructions: 'Return JSON with summary and tags.',
  schemaName: 'proof.red-square',
  jsonSchema: {
    type: 'object',
    properties: {
      summary: { type: 'string' },
      tags: { type: 'array', items: { type: 'string' } },
    },
    required: ['summary'],
  },
  profile: 'openai-codex:work',
  cfg: {},
  agentDir: process.cwd(),
});

let guardError = null;
try {
  await runtime.mediaUnderstanding.extractStructuredWithModel({
    provider: 'codex',
    model: 'gpt-5.4',
    input: [{ type: 'text', text: 'No image present.' }],
    instructions: 'Return JSON.',
    cfg: {},
    agentDir: process.cwd(),
  });
} catch (error) {
  guardError = error instanceof Error ? error.message : String(error);
}

console.log(JSON.stringify({
  success,
  authProfileIds,
  requestMethods: requests.map((entry) => entry.method),
  threadStart: requests.find((entry) => entry.method === 'thread/start')?.params,
  turnInput: requests.find((entry) => entry.method === 'turn/start')?.params?.input,
  guardError,
}, null, 2));

resetPluginRuntimeStateForTest();
EOF

Evidence after fix:

{
  "success": {
    "text": "{\"summary\":\"red square\",\"tags\":[\"shape\"]}",
    "model": "gpt-5.4",
    "provider": "codex",
    "contentType": "json",
    "parsed": {
      "summary": "red square",
      "tags": [
        "shape"
      ]
    }
  },
  "authProfileIds": [
    "openai-codex:work"
  ],
  "requestMethods": [
    "model/list",
    "thread/start",
    "turn/start"
  ],
  "threadStart": {
    "model": "gpt-5.4",
    "modelProvider": "openai",
    "cwd": "/Users/lume/openclaw-review-worktrees/pr-79334-rebase",
    "approvalPolicy": "on-request",
    "sandbox": "read-only",
    "serviceName": "OpenClaw",
    "developerInstructions": "You are OpenClaw's bounded structured-extraction worker. Return only the requested extraction. Do not call tools, edit files, ask follow-up questions, or include secrets.",
    "dynamicTools": [],
    "experimentalRawEvents": true,
    "persistExtendedHistory": false,
    "ephemeral": true
  },
  "turnInput": [
    {
      "type": "text",
      "text": "Return JSON with summary and tags.\n\nSchema name: proof.red-square\n\nJSON schema:\n{\"type\":\"object\",\"properties\":{\"summary\":{\"type\":\"string\"},\"tags\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}}},\"required\":[\"summary\"]}\n\nReturn valid JSON only. Do not wrap the JSON in Markdown fences.",
      "text_elements": []
    },
    {
      "type": "image",
      "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+kX3sAAAAASUVORK5CYII="
    },
    {
      "type": "text",
      "text": "Return searchable evidence for the uploaded image.",
      "text_elements": []
    }
  ],
  "guardError": "Structured extraction requires at least one image input."
}

Observed result after fix: The plugin runtime facade dispatched extractStructuredWithModel(...) through the registered Codex media-understanding provider, the provider returned parsed JSON on the bounded app-server path, the selected auth profile reached the provider-owned runtime, and the text-only call failed early with the intended image-required guard instead of widening this seam into general text extraction.

What was not tested: This proof intentionally uses a stubbed app-server client so it can exercise the real runtime/provider dispatch path deterministically in a local checkout without requiring a desktop-bound live OAuth session. The PR does not include a credentialed live Codex desktop turn artifact because that would require shipping private local auth/session material into public review evidence.

Validation

• pnpm install --frozen-lockfile
• pnpm plugin-sdk:api:gen
• pnpm plugin-sdk:api:check
• pnpm test src/media-understanding/runtime.test.ts src/media-understanding/provider-registry.test.ts extensions/codex/media-understanding-provider.test.ts src/plugins/runtime/index.test.ts
• pnpm check:changed

Changed files

• CHANGELOG.md (modified, +1/-0)
• docs/.generated/plugin-sdk-api-baseline.sha256 (modified, +2/-2)
• docs/plugins/architecture-internals.md (modified, +28/-0)
• docs/plugins/sdk-runtime.md (modified, +27/-0)
• docs/plugins/sdk-subpaths.md (modified, +1/-1)
• extensions/codex/media-understanding-provider.test.ts (modified, +160/-2)
• extensions/codex/media-understanding-provider.ts (modified, +198/-10)
• src/media-understanding/runtime-types.ts (modified, +25/-0)
• src/media-understanding/runtime.test.ts (modified, +155/-0)
• src/media-understanding/runtime.ts (modified, +40/-1)
• src/media-understanding/types.ts (modified, +41/-0)
• src/plugin-sdk/media-understanding-runtime.ts (modified, +2/-0)
• src/plugin-sdk/media-understanding.ts (modified, +5/-0)
• src/plugin-sdk/test-helpers/plugin-runtime-mock.ts (modified, +2/-0)
• src/plugins/runtime/index.test.ts (modified, +1/-0)
• src/plugins/runtime/index.ts (modified, +3/-0)
• src/plugins/runtime/types-core.ts (modified, +1/-0)

PR #75578: [plugin sdk] Add session action gateway protocol

• Repository: openclaw/openclaw
• Author: 100yenadmin
• State: closed | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75578

Description (problem / solution / changelog)

Why this matters

OpenClaw plugins increasingly need typed actions that the host can expose and dispatch: approve a workflow, reopen a case, accept extracted data, retry a run, continue an agent after a human decision, or branch a workflow from a structured operator choice.

Without a generic seam, plugin authors end up with two bad options:

• invent bespoke gateway methods and one-off UI contracts for every product feature; or
• push product-specific action routers into OpenClaw core.

This PR adds the missing middle layer: plugins register typed session actions once, the Gateway exposes one stable dispatch method, and OpenClaw keeps ownership of schema validation, scope enforcement, stale-plugin safety, and client protocol/codegen.

What kinds of plugins this could support

This seam is intentionally generic. It can support many plugin families without adding plugin-specific action routes to core:

• Approval and review plugins: approve, reject, request changes, or continue the agent with a typed reason.
• Support and case-management plugins: escalate, close, reopen, request follow-up, or attach a structured disposition.
• CRM and sales plugins: mark contacted, accept extracted updates, hand off ownership, or snooze the next step.
• Task and workflow plugins: claim, retry, confirm completion, or branch the workflow from an operator choice.
• Ops and incident plugins: suppress an alert, rerun diagnostics, confirm mitigation, or start escalation.
• UI companion plugins: render action buttons or menus in Control UI, native Apple clients, CLI helpers, or external companion surfaces that all call back through the same Gateway seam.

The important part: the plugin defines the action semantics. OpenClaw owns dispatch, auth, and transport-level enforcement.

New SDK seam

This PR adds:

• OpenClawPluginApi.registerSessionAction(...)
• typed action context, payload schema, success/failure results, and optional continueAgent
• plugins.sessionAction on the Gateway protocol plus regenerated Swift models
• runtime scope classification that honors action-declared requiredScopes

Example plugin usage:

api.registerSessionAction({
  id: "approve",
  description: "Approve the current workflow",
  requiredScopes: ["operator.approvals"],
  schema: {
    type: "object",
    properties: {
      message: { type: "string" },
    },
  },
  handler: ({ payload, sessionKey }) => ({
    data: {
      approved: true,
      message: payload?.message,
      ...(sessionKey ? { sessionKey } : {}),
    },
    continueAgent: true,
  }),
});

Runtime architecture

flowchart LR
  Plugin["Plugin register()"] --> Registry["Typed session action registry"]
  Client["Gateway client / Control UI / native app / CLI"] --> RPC["plugins.sessionAction"]
  RPC --> Guard["Schema validation + scope classification"]
  Guard --> Dispatch["Registered plugin action handler"]
  Dispatch --> Result["Typed success or failure payload"]
  Result --> Client

The action is registered once inside the plugin runtime and can then be dispatched from any client surface that speaks the Gateway protocol.

Safety and boundaries

This stays intentionally narrow:

• action ids, schemas, and duplicate registrations are validated at registration time
• payloads are validated before handler execution and malformed traffic is rejected
• dispatch enforces action-declared requiredScopes instead of treating every action like the same write lane
• unresolved out-of-process callers fall back to the standard operator scope bundle instead of under-scoping to operator.write
• stale or unloaded plugins are blocked instead of dispatching into dead registry state
• the upstream ACP live-delivery / streaming defaults already on upstream/main stay preserved in this slice

This is a bounded typed action protocol, not arbitrary plugin-owned gateway method injection.

What changed

• Added OpenClawPluginApi.registerSessionAction plus SDK exports for action registration, context, and result types.
• Added registry storage, validation, duplicate detection, loader snapshot/restore support, and plugin test API/captured registration support.
• Added Gateway protocol schemas/types, generated Swift models, and plugins.sessionAction dispatch.
• Added dynamic Gateway scope classification so known actions keep their declared least-privilege scopes, while unresolved out-of-process callers fall back to the standard operator scope bundle instead of under-scoping to operator.write.
• Fixed the shared agent-event bridge teardown so duplicate runtime.ts module instances do not double-dispatch pinned/active registry subscriptions.
• Preserved the upstream ACP live-delivery / streaming default changes already on upstream/main; this slice stays narrowly focused on typed session-action registration, dispatch, and scope enforcement.
• Added focused contract coverage for registration validation, payload schema validation, typed success/failure result validation, dispatch auth, stale-plugin blocking, pinned-registry event fanout, and method-scope classification.

Relationship to adjacent seams

This is narrower than attachments, scheduled turns, SessionEntry projection, or run-context lifecycle. It only adds the typed session-action registration/dispatch protocol plus the host scope enforcement that makes it safe to call.

Non-goals

• Does not render UI on its own; clients still choose how to present actions.
• Does not replace generic plugin gateway methods or open arbitrary new RPC namespaces.
• Does not include host-mediated attachments, scheduled turns, derived path facts, SessionEntry projection, finalize retry/run-context lifecycle, or advanced workflow composition fixtures.
• Does not reopen ACP streaming/live-delivery defaults; it preserves the upstream behavior already on main.

Stack context

• Predecessors: #73384, #74483
• Docs split: #74853
• Foundation: #72287

Real behavior proof

Behavior or issue addressed: This fixes the under-scoped least-privilege path for plugins.sessionAction without broadening the seam beyond the session-action protocol. Before this patch, out-of-process callers that could not resolve a local plugin registry entry fell back to operator.write, so actions that correctly declared scopes like operator.approvals failed before reaching the handler. After this patch, known actions still use their declared scopes, unresolved callers fall back to the standard operator scope bundle instead of under-scoping, dispatch still enforces requiredScopes, and typed payload/result validation still rejects malformed traffic.

Real environment tested: Fresh loopback Gateway started from /Users/lume/openclaw-review-worktrees/pr-75578-rebase with a workspace-loaded proof plugin at /tmp/openclaw-pr75578-proof.4VP2Yu/workspace/.openclaw/extensions/session-action-proof. The runtime used HOME=/tmp/openclaw-pr75578-proof.4VP2Yu/home, OPENCLAW_WORKSPACE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/workspace, OPENCLAW_STATE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/state, and OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr75578-proof.4VP2Yu/config/openclaw.json.

Exact steps or command run after this patch:

HOME=/tmp/openclaw-pr75578-proof.4VP2Yu/home OPENCLAW_WORKSPACE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/workspace OPENCLAW_STATE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/state OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr75578-proof.4VP2Yu/config/openclaw.json pnpm openclaw plugins inspect session-action-proof --runtime --json

HOME=/tmp/openclaw-pr75578-proof.4VP2Yu/home OPENCLAW_WORKSPACE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/workspace OPENCLAW_STATE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/state OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr75578-proof.4VP2Yu/config/openclaw.json pnpm openclaw gateway call plugins.sessionAction --json --params '{"pluginId":"session-action-proof","actionId":"approve","payload":{"message":"post-fix"}}'

HOME=/tmp/openclaw-pr75578-proof.4VP2Yu/home OPENCLAW_WORKSPACE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/workspace OPENCLAW_STATE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/state OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr75578-proof.4VP2Yu/config/openclaw.json node --import tsx <<'NODE'
import { callGateway } from './src/gateway/call.ts';
import { GATEWAY_CLIENT_MODES, GATEWAY_CLIENT_NAMES } from './src/utils/message-channel.ts';
const result = await callGateway({
  method: 'plugins.sessionAction',
  params: {
    pluginId: 'session-action-proof',
    actionId: 'approve',
    payload: { message: 'post-fix' },
  },
  token: 'proof-token',
  clientName: GATEWAY_CLIENT_NAMES.GATEWAY_CLIENT,
  mode: GATEWAY_CLIENT_MODES.BACKEND,
});
console.log(JSON.stringify(result, null, 2));
NODE

HOME=/tmp/openclaw-pr75578-proof.4VP2Yu/home OPENCLAW_WORKSPACE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/workspace OPENCLAW_STATE_DIR=/tmp/openclaw-pr75578-proof.4VP2Yu/state OPENCLAW_CONFIG_PATH=/tmp/openclaw-pr75578-proof.4VP2Yu/config/openclaw.json node --import tsx <<'NODE'
import { callGateway } from './src/gateway/call.ts';
import { GATEWAY_CLIENT_MODES, GATEWAY_CLIENT_NAMES } from './src/utils/message-channel.ts';
try {
  await callGateway({
    method: 'plugins.sessionAction',
    params: {
      pluginId: 'session-action-proof',
      actionId: 'approve',
      payload: { message: 42 },
    },
    token: 'proof-token',
    clientName: GATEWAY_CLIENT_NAMES.GATEWAY_CLIENT,
    mode: GATEWAY_CLIENT_MODES.BACKEND,
  });
  console.log('unexpected success');
} catch (error) {
  console.error(String(error));
  process.exitCode = 1;
}
NODE

Evidence after fix:

$ pnpm openclaw gateway call plugins.sessionAction --json --params '{"pluginId":"session-action-proof","actionId":"approve","payload":{"message":"post-fix"}}'
gateway connect failed: GatewayClientRequestError: scope upgrade pending approval (requestId: f97fd499-4729-4cd9-b7e0-d7dc90334595)
Gateway call failed: GatewayTransportError: gateway closed (1008): pairing required: device is asking for more scopes than currently approved (requestId: f97fd499-4729-4cd9-b7e0-d7dc90334595)

$ node --import tsx <authorized backend proof>
{
  "ok": true,
  "result": {
    "approved": true,
    "message": "post-fix",
    "sessionKey": null,
    "scopes": [
      "operator.admin",
      "operator.read",
      "operator.write",
      "operator.approvals",
      "operator.pairing",
      "operator.talk.secrets"
    ]
  }
}

$ node --import tsx <invalid payload proof>
GatewayClientRequestError: plugin session action payload does not match schema: message: must be string

Observed result after fix: The paired CLI flow no longer under-scopes plugins.sessionAction to operator.write; it now requests the broader operator scope bundle and hits the existing scope-upgrade approval gate, which preserves pairing/approval enforcement instead of failing with a misleading handler-level missing-scope error. An authorized backend/token caller successfully dispatches the typed session action and receives the typed success payload. A malformed payload is rejected before handler execution with the expected schema error.

What was not tested: This proof did not run a full end-to-end Control UI or native Apple client invocation against the regenerated Swift models. The Swift surface was regenerated and covered at the protocol/codegen layer in this patch, but the live proof here is intentionally limited to real Gateway registration/dispatch/scope enforcement and typed payload validation.

Validation

• pnpm protocol:gen
• pnpm protocol:gen:swift
• pnpm test src/gateway/method-scopes.test.ts src/gateway/call.test.ts src/agents/tools/gateway.test.ts src/plugins/contracts/session-actions.contract.test.ts
• pnpm test src/plugins/runtime.channel-pin.test.ts
• OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test src/plugins/runtime.test.ts src/plugins/runtime.channel-pin.test.ts
• git diff --check

Changed files

• CHANGELOG.md (modified, +1/-0)
• apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift (modified, +196/-0)
• docs/.generated/plugin-sdk-api-baseline.sha256 (modified, +2/-2)
• scripts/protocol-gen-swift.ts (modified, +246/-1)
• src/agents/tools/gateway.test.ts (modified, +66/-0)
• src/agents/tools/gateway.ts (modified, +1/-1)
• src/gateway/call.test.ts (modified, +24/-0)
• src/gateway/call.ts (modified, +2/-2)
• src/gateway/method-scopes.test.ts (modified, +94/-0)
• src/gateway/method-scopes.ts (modified, +55/-1)
• src/gateway/protocol/index.ts (modified, +11/-0)
• src/gateway/protocol/schema/plugins.ts (modified, +35/-0)
• src/gateway/protocol/schema/protocol-schemas.ts (modified, +8/-0)
• src/gateway/protocol/schema/types.ts (modified, +2/-0)
• src/gateway/server-methods-list.ts (modified, +1/-0)
• src/gateway/server-methods/plugin-host-hooks.ts (modified, +314/-0)
• src/plugin-sdk/core.ts (modified, +5/-0)
• src/plugin-sdk/plugin-entry.ts (modified, +10/-0)
• src/plugin-sdk/plugin-test-api.ts (modified, +2/-0)
• src/plugins/agent-event-emission.ts (added, +84/-0)
• src/plugins/api-builder.ts (modified, +9/-0)
• src/plugins/captured-registration.ts (modified, +8/-0)
• src/plugins/contracts/session-actions.contract.test.ts (added, +1018/-0)
• src/plugins/host-hooks.ts (modified, +46/-0)
• src/plugins/loader.ts (modified, +3/-0)
• src/plugins/registry-empty.ts (modified, +1/-0)
• src/plugins/registry-lifecycle.ts (modified, +6/-0)
• src/plugins/registry-types.ts (modified, +10/-0)
• src/plugins/registry.ts (modified, +137/-2)
• src/plugins/runtime-state.ts (modified, +1/-0)
• src/plugins/runtime.channel-pin.test.ts (modified, +110/-0)
• src/plugins/runtime.test.ts (modified, +17/-0)
• src/plugins/runtime.ts (modified, +86/-24)
• src/plugins/schema-validator.ts (modified, +6/-4)
• src/plugins/types.ts (modified, +12/-0)

PR #75581: [plugin sdk] Add host-mediated session attachments

• Repository: openclaw/openclaw
• Author: 100yenadmin
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75581

Description (problem / solution / changelog)

Why this matters

OpenClaw plugins increasingly need to send real files back to the user in the same session lane: support artifacts, generated reports, approval docs, exports, screenshots, and workflow evidence.

Without a generic seam, plugin authors end up with two bad options:

• own Telegram/Slack/Discord delivery logic themselves, including route lookup and transport quirks; or
• push product-specific attachment routes into OpenClaw core.

This PR adds the missing middle layer: a bundled plugin can ask the host to validate local files and deliver them through the active direct-outbound session route, while OpenClaw keeps ownership of channel credentials, delivery routing, formatting quirks, and transport behavior.

What kinds of plugins this could support

This seam is intentionally generic. It can support many plugin families without adding plugin-specific delivery logic to core:

• Support and incident plugins: send logs, screenshots, repro bundles, or diagnostics back into the live support conversation.
• Approval and review plugins: attach generated PDFs, redlines, or diffs into the same operator thread that requested them.
• CRM and case-management plugins: deliver quote files, customer evidence, or case exports to the active session route.
• Field ops and inspection plugins: return photos, signed forms, or inspection reports to the same conversation lane.
• Migration and import plugins: send rejected-record CSVs or validation reports back to a human operator.
• Research and reporting plugins: attach exported summaries or evidence files instead of pasting large blobs into chat.

The important part: the plugin decides what artifact to send. OpenClaw decides whether the file is safe and where the active session route actually goes.

New SDK seam

This PR adds:

• OpenClawPluginApi.sendSessionAttachment(...)
• typed attachment params and result types in the SDK
• host-side file validation, size/count limits, session-route lookup, and stale-registry protection
• workspace-relative resolution against the session agent workspace
• channel hints for caption formatting, Telegram delivery behavior, and Slack thread targeting

Example plugin usage:

const result = await api.sendSessionAttachment({
  sessionKey: "agent:main:main",
  files: [{ path: "./proof-report.txt" }],
  text: "attachment ready",
  captionFormat: "plain",
});

Runtime architecture

flowchart LR
  Plugin["Plugin callback, handler, or workflow"] --> SDK["sendSessionAttachment(...)"]
  SDK --> Validate["Host file validation and workspace resolution"]
  Validate --> Route["Active session route lookup"]
  Route --> Channel["Direct outbound channel adapter"]
  Channel --> User["Delivered attachment in the live session lane"]

The plugin never talks directly to Telegram, Slack, or another channel transport. It asks the host to deliver a validated file to the already-active session route.

Safety and boundaries

This stays intentionally narrow:

• bundled-only: this seam depends on host-managed session and channel integrations
• the host validates file count, size, path, and optional delivery hints before calling the outbound adapter
• relative paths resolve against the session agent workspace instead of whatever host cwd happens to be active
• delivery stays bound to the active session route; plugins do not get arbitrary channel send power
• stale or unloaded plugin registries are rejected instead of dispatching through dead state
• channel hints stay hints; plugins do not take raw provider or credential ownership

This is a host-mediated attachment seam, not a generic channel SDK.

What changed

• Added OpenClawPluginApi.sendSessionAttachment plus SDK exports for attachment params and results.
• Added host-side file validation, delivery-route lookup hardening, bundled-plugin enforcement, and stale-registry protection.
• Resolved relative file paths against the session agent workspace so real plugin callbacks do not fall back to the default host workspace.
• Added channel attachment hints from #74483: caption-format precedence, Telegram silent/parse-mode/force-document hints, and Slack thread targeting.
• Plumbed parseMode through outbound message formatting and the Telegram direct/gateway adapters.
• Re-exported Telegram formatting helpers for plugin authors that need Telegram-safe captions.
• Preserved the loader's closed-after-register guard for registration-only APIs, but kept sendSessionAttachment callable after register() so a real bundled plugin can capture it during registration and invoke it later from a live callback or handler.
• Added focused contract and loader regression coverage for stale-registry protection, workspace-relative resolution, direct route dispatch, and post-registration late calls.

Relationship to adjacent seams

This does not try to solve scheduling, session actions, session projection, or run-context lifecycle. It is deliberately narrower: one host-mediated path for delivering plugin-owned files through the active session route.

Non-goals

• Does not give plugins direct Telegram, Slack, or Discord delivery ownership.
• Does not add arbitrary channel broadcast or free-form outbound routing.
• Does not include scheduled turns, session actions, derived path facts, SessionEntry projection, finalize retry/run-context lifecycle, or advanced workflow composition fixtures.
• Does not claim to close broader Control UI/native-page RFC scope; this only adds the generic attachment delivery seam.

Stack context

• Predecessors: #73384, #74483
• Docs split: #74853
• Foundation: #72287

Real behavior proof

Behavior or issue addressed: This fixes the narrow host-mediated attachment seam without broadening it into direct channel ownership. Before these follow-up fixes, a captured sendSessionAttachment closure could be closed accidentally by the loader guard, relative attachment paths could resolve against the wrong workspace, and Telegram media captions could be forced through the HTML path even when HTML was not explicitly requested. After this patch, the loader keeps sendSessionAttachment callable after register(), the host resolves relative paths against the session agent workspace, and the active direct-outbound route receives the validated file plus caption text.

Real environment tested: Standalone runtime proof from /Users/lume/openclaw-review-worktrees/pr-75581-rebase using a live session store, an agent workspace rooted in a fresh temp state directory, and a direct outbound proof channel registered in the active plugin registry.

Exact steps or command run after this patch:

node --import tsx <<'NODE'
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';
import { updateSessionStore } from './src/config/sessions.js';
import { sendPluginSessionAttachment } from './src/plugins/host-hook-workflow.js';
import { setActivePluginRegistry, getActivePluginRegistry } from './src/plugins/runtime.js';
import { createTestRegistry } from './src/test-utils/channel-plugins.js';

const previousRegistry = getActivePluginRegistry();
const stateDir = await fs.mkdtemp(path.join(os.tmpdir(), 'openclaw-pr75581-proof-'));
const workspaceDir = path.join(stateDir, 'workspace');
const storePath = path.join(stateDir, 'sessions.json');
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, 'proof-report.txt'), 'proof attachment body\n', 'utf8');
await updateSessionStore(storePath, (store) => {
  store['agent:main:main'] = {
    sessionId: 'session-id',
    updatedAt: Date.now(),
    deliveryContext: { channel: 'proofchat', to: '12345', accountId: 'default' },
  };
  return undefined;
});
const deliveries = [];
setActivePluginRegistry(
  createTestRegistry([
    {
      pluginId: 'proofchat',
      source: 'proof',
      plugin: {
        id: 'proofchat',
        meta: {
          id: 'proofchat',
          label: 'Proof Chat',
          selectionLabel: 'Proof Chat',
          docsPath: '/channels/proofchat',
          blurb: 'proof direct channel',
        },
        capabilities: { chatTypes: ['direct'] },
        config: {
          listAccountIds: () => ['default'],
          resolveAccount: () => ({ accountId: 'default' }),
        },
        outbound: {
          deliveryMode: 'direct',
          sendText: async ({ to, text, accountId }) => {
            deliveries.push({ to, text, mediaUrl: null, accountId: accountId ?? null });
            return { channel: 'proofchat', messageId: 'proof-text-1' };
          },
          sendMedia: async ({ to, text, mediaUrl, accountId }) => {
            deliveries.push({ to, text, mediaUrl: mediaUrl ?? null, accountId: accountId ?? null });
            return { channel: 'proofchat', messageId: 'proof-media-1' };
          },
        },
      },
    },
  ]),
);

const result = await sendPluginSessionAttachment({
  origin: 'bundled',
  sessionKey: 'agent:main:main',
  files: [{ path: './proof-report.txt' }],
  text: 'attachment ready',
  config: {
    session: { store: storePath },
    agents: { list: [{ id: 'main', workspace: workspaceDir }] },
  },
});
console.log(JSON.stringify({ result, deliveries }, null, 2));
setActivePluginRegistry(previousRegistry);
NODE

Evidence after fix:

{
  "result": {
    "ok": true,
    "channel": "proofchat",
    "deliveredTo": "12345",
    "count": 1
  },
  "deliveries": [
    {
      "to": "12345",
      "text": "attachment ready",
      "mediaUrl": "/var/folders/l7/tyvn8qfs50v57w__75s3k0f00000gp/T/openclaw-pr75581-proof-pHeAVB/workspace/proof-report.txt",
      "accountId": "default"
    }
  ]
}

Observed result after fix: The bundled-only attachment dispatch succeeds, the relative attachment path resolves against the session agent workspace instead of falling back to the default host workspace, and the active direct-outbound route receives the resolved file plus caption text. The loader regression is also covered on this head by pnpm test src/plugins/loader.test.ts, which proves sendSessionAttachment stays callable after register() while registration-only APIs still close.

What was not tested: This proof did not exercise a full external Telegram/Slack/Discord live account or an end-to-end GUI flow. The runtime proof here is intentionally limited to real host validation, session-route lookup, workspace-relative path resolution, and direct outbound attachment dispatch inside a live OpenClaw runtime.

Validation

• pnpm test src/plugins/loader.test.ts
• pnpm test src/config/sessions/delivery-info.test.ts src/plugins/contracts/session-attachments.contract.test.ts extensions/telegram/api.test.ts extensions/telegram/src/outbound-adapter.test.ts extensions/telegram/src/channel.gateway.test.ts
• pnpm plugin-sdk:api:check
• pnpm lint:core
• pnpm lint:extensions
• git diff --check

Changed files

• CHANGELOG.md (modified, +1/-0)
• apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift (modified, +12/-0)
• docs/.generated/plugin-sdk-api-baseline.sha256 (modified, +2/-2)
• docs/plugins/sdk-overview.md (modified, +1/-0)
• extensions/googlechat/config-api.ts (added, +2/-0)
• extensions/googlechat/src/config-schema.ts (modified, +1/-1)
• extensions/matrix/src/matrix/monitor/handler.test-helpers.ts (modified, +2/-1)
• extensions/matrix/src/matrix/monitor/handler.test.ts (modified, +2/-0)
• extensions/telegram/api.test.ts (added, +16/-0)
• extensions/telegram/api.ts (modified, +8/-0)
• extensions/telegram/src/channel.gateway.test.ts (modified, +47/-0)
• extensions/telegram/src/format.ts (modified, +6/-2)
• extensions/telegram/src/outbound-adapter.test.ts (modified, +5/-6)
• extensions/telegram/src/outbound-adapter.ts (modified, +12/-2)
• src/config/sessions/delivery-info.test.ts (modified, +223/-3)
• src/config/sessions/delivery-info.ts (modified, +154/-24)
• src/gateway/protocol/schema/agent.ts (modified, +6/-0)
• src/gateway/server-methods/send.test.ts (modified, +22/-0)
• src/gateway/server-methods/send.ts (modified, +6/-0)
• src/infra/outbound/formatting.ts (modified, +1/-0)
• src/infra/outbound/message.channels.test.ts (modified, +17/-0)
• src/infra/outbound/message.ts (modified, +6/-0)
• src/plugin-sdk/core.ts (modified, +2/-0)
• src/plugin-sdk/plugin-entry.ts (modified, +4/-0)
• src/plugin-sdk/plugin-test-api.ts (modified, +1/-0)
• src/plugins/api-builder.ts (modified, +6/-0)
• src/plugins/captured-registration.ts (modified, +1/-0)
• src/plugins/contracts/bundled-extension-config-api-guardrails.test.ts (modified, +1/-1)
• src/plugins/contracts/session-attachments.contract.test.ts (added, +877/-0)
• src/plugins/host-hook-workflow.ts (added, +305/-0)
• src/plugins/host-hooks.ts (modified, +41/-0)
• src/plugins/loader.test.ts (modified, +61/-0)
• src/plugins/loader.ts (modified, +9/-1)
• src/plugins/registry.ts (modified, +32/-0)
• src/plugins/types.ts (modified, +15/-0)

PR #75588: [plugin sdk] Add scheduled session turns

• Repository: openclaw/openclaw
• Author: 100yenadmin
• State: open | merged: False
• Link: https://github.com/openclaw/openclaw/pull/75588

Description (problem / solution / changelog)

Why this matters

OpenClaw plugins increasingly need to create follow-up work that happens later, in the same session lane the user is already operating in: remind me in an hour, reopen this investigation tomorrow, schedule a follow-up agent turn after an approval window, or clean up a family of pending nudges when the workflow resolves.

Without a generic seam, plugin authors end up with two bad options:

• push product-specific scheduling logic into OpenClaw core; or
• own host cron details themselves, which leaks infrastructure concerns into every plugin.

This PR adds the narrow host-scheduler seam in the plugin SDK. Plugins can ask the host to schedule a future session turn and later clean up plugin-owned scheduled jobs by tag, while OpenClaw keeps ownership of cron naming, lifecycle, delivery routing, and cleanup semantics.

What kinds of plugins this could support

This seam is intentionally generic. It can support many plugin families without adding plugin-specific scheduling routes to core:

• Support and ops plugins: reopen or re-check incidents after a delay, schedule escalation nudges, or queue a follow-up diagnostic turn.
• CRM and sales plugins: create reminder turns for deal follow-ups, pending replies, or renewal check-ins.
• Approval and workflow plugins: schedule reminders while waiting on a human decision, then clean up all related nudges when the workflow resolves.
• Digest and assistant plugins: schedule a future turn that produces a recap, check-in, or status refresh in the same session lane.
• Import, migration, and reconciliation plugins: retry or re-check long-running work later without inventing a parallel scheduler stack.
• Research and monitoring plugins: schedule a revisit turn after some cooling-off period when an external dependency is expected to change.

The important part: the plugin decides why the follow-up exists. OpenClaw only provides the bounded host seam for when it should happen and how it is cleaned up.

New SDK seam

This PR adds:

• OpenClawPluginApi.scheduleSessionTurn(...)
• OpenClawPluginApi.unscheduleSessionTurnsByTag(...)
• typed schedule / cleanup params and result types in the SDK
• host-side support for at, delayMs, and cron schedules
• plugin-prefixed cron naming plus plugin-owned tag cleanup
• a narrow post-registration carve-out so only these two helpers remain callable after register()

Example plugin usage:

const reminder = await api.scheduleSessionTurn({
  sessionKey,
  delayMs: 60 * 60 * 1000,
  text: "Re-check whether the customer replied.",
  tag: "followup",
  deliveryMode: "announce/last",
  deleteAfterRun: true,
});

await api.unscheduleSessionTurnsByTag({
  sessionKey,
  tag: "followup",
});

Runtime architecture

flowchart LR
  Plugin["Plugin callback, handler, or workflow"] --> SDK["scheduleSessionTurn / unscheduleSessionTurnsByTag"]
  SDK --> HostHooks["Host hook workflow"]
  HostHooks --> Cron["Host cron naming and persistence"]
  Cron --> FutureTurn["Future agent turn in the same session lane"]
  Plugin --> Cleanup["Plugin-owned tag cleanup"]
  Cleanup --> Cron

The plugin owns the follow-up intent and tag taxonomy. OpenClaw owns the cron contract, namespacing, and actual future-turn delivery.

Safety and boundaries

This stays intentionally narrow:

• plugins do not get generic cron ownership or arbitrary background-job control
• only scheduleSessionTurn and unscheduleSessionTurnsByTag stay callable after register()
• cron job names are namespaced by plugin id so short tags do not clobber each other
• reserved cleanup tags are rejected instead of silently widening semantics
• invalid cron + deleteAfterRun: true combinations are rejected instead of guessing
• scheduled-turn records stay plugin-owned until lifecycle cleanup confirms the host job is gone

This is a scheduler seam for future session turns, not a generic host automation API.

What changed

• Added OpenClawPluginApi.scheduleSessionTurn and unscheduleSessionTurnsByTag plus SDK exports for schedule/cleanup params and results.
• Added host cron payload construction for at, delayMs, and cron schedules, with optional delivery mode.
• Added plugin-prefixed cron names and tag cleanup so two plugins can reuse short tags without clobbering each other.
• Rejected reserved : cleanup tags and rejected cron + deleteAfterRun: true instead of silently widening the host contract.
• Preserved plugin-owned scheduled-turn records until lifecycle cleanup confirms the host job is gone, instead of time-pruning those records early.
• Preserved the loader's closed-after-register guard for normal registration mutations, but let only scheduleSessionTurn and unscheduleSessionTurnsByTag remain callable after register() so a real bundled plugin can capture them during register() and invoke them later from a registered callback/handler.
• Added real-loader regression coverage for post-registration gateway-handler dispatch plus the invalid-tag / invalid-deleteAfterRun cases.
• Moved the changelog entry back under ## Unreleased and regenerated the plugin-sdk API baseline after the rebase.

Relationship to adjacent seams

This does not try to solve every workflow problem in one slice. It is intentionally narrower than session actions, run-context lifecycle, attachments, or SessionEntry projection. Those remain split into follow-up PRs so maintainers can review one host seam at a time.

Non-goals

• Does not widen the generic post-registration plugin API beyond these two operational helpers.
• Does not give plugins arbitrary cron ownership or a general background-job runner.
• Does not include session actions, host-mediated attachments, finalization retry/run-context lifecycle, tool-derived paths, SessionEntry projection, or advanced workflow composition fixtures.
• Does not claim to close broader Control UI/native-page RFC scope; this only adds the generic scheduled-turn seam.

Real behavior proof

Behavior or issue addressed: This keeps the post-registration seam narrow while proving the scheduled-turn helpers work in a real host runtime. After this patch, a bundled plugin can capture scheduleSessionTurn and unscheduleSessionTurnsByTag during register(), invoke them later from a registered callback/handler, create real host cron jobs, reject invalid cleanup inputs without widening the contract, and clean up the tagged jobs deterministically.

Real environment tested: On 2026-05-09, I started a real loopback gateway from /Users/lume/openclaw-review-worktrees/pr-75588-rebase and loaded a real bundled proof-scheduler plugin through loadOpenClawPlugins.

Exact steps or command run after this patch:

1. Loaded the bundled proof-scheduler plugin and let it capture api.scheduleSessionTurn plus api.unscheduleSessionTurnsByTag during register().
2. Invoked the registered plugin gateway handler proof-scheduler.exercise so the plugin exercised those captured helpers after registration.
3. Called cron.list immediately after scheduling to inspect the real host jobs.
4. Ran the cleanup phase through unscheduleSessionTurnsByTag, then called cron.list again to verify that the tagged jobs were gone.

Evidence after fix:

• The schedule phase returned two real handles:
  • 7d2d8bf2-f028-4a44-ad08-c75fd966b965
  • ca448912-25a0-4aec-9e87-88587464e0dc
• The invalid tag: "bad:tag" case returned null.
• The invalid cron + deleteAfterRun: true case returned null.
• cron.list immediately after schedule showed two real host jobs named plugin:proof-scheduler:tag:proof:agent:main:main:*, both targeting session:agent:main:main, both deleteAfterRun: true, with delivery modes announce/last and none, and with scheduled at timestamps 2026-05-09T17:08:00.334Z and 2026-05-09T17:09:00.356Z.
• The cleanup phase returned { removed: 2, failed: 0 }.
• cron.list after cleanup returned [].

Copied live output after fix:

$ proof-scheduler.exercise
first.id=7d2d8bf2-f028-4a44-ad08-c75fd966b965
second.id=ca448912-25a0-4aec-9e87-88587464e0dc
badTag=null
badDelete=null

$ cron.list
plugin:proof-scheduler:tag:proof:agent:main:main:* deleteAfterRun=true delivery=announce/last at=2026-05-09T17:08:00.334Z
plugin:proof-scheduler:tag:proof:agent:main:main:* deleteAfterRun=true delivery=none at=2026-05-09T17:09:00.356Z

$ unscheduleSessionTurnsByTag
{ removed: 2, failed: 0 }

$ cron.list
[]

Observed result after fix: The real bundled plugin was able to use the two late-call helpers after register() without reopening the rest of the plugin API. Real host cron jobs were created, visible in cron.list, and then removed cleanly by the tagged cleanup path. Invalid cleanup tags and invalid cron + deleteAfterRun: true combinations were rejected instead of silently widening the contract.

What was not tested: This proof did not cover a full user-facing UI flow or a long-lived production cron lifecycle across process restarts. The live proof here is intentionally limited to real post-registration helper capture, real host cron job creation/listing, invalid-input rejection, and tagged cleanup in a live OpenClaw runtime.

Validation

• pnpm test src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/contracts/host-hooks.contract.test.ts
• pnpm plugin-sdk:api:gen
• pnpm plugin-sdk:api:check
• pnpm lint:core
• git diff --check
• npm run check:no-conflict-markers

Changed files

• CHANGELOG.md (modified, +1/-0)
• docs/.generated/plugin-sdk-api-baseline.sha256 (modified, +2/-2)
• src/plugin-sdk/core.ts (modified, +3/-0)
• src/plugin-sdk/plugin-entry.ts (modified, +6/-0)
• src/plugin-sdk/plugin-test-api.ts (modified, +2/-0)
• src/plugins/api-builder.ts (modified, +8/-0)
• src/plugins/captured-registration.test.ts (modified, +47/-1)
• src/plugins/captured-registration.ts (modified, +18/-5)
• src/plugins/contracts/host-hooks.contract.test.ts (modified, +57/-0)
• src/plugins/contracts/scheduled-turns.contract.test.ts (added, +1216/-0)
• src/plugins/host-hook-cleanup.ts (modified, +3/-0)
• src/plugins/host-hook-runtime.ts (modified, +18/-2)
• src/plugins/host-hook-workflow.ts (added, +481/-0)
• src/plugins/host-hooks.ts (modified, +40/-0)
• src/plugins/loader.ts (modified, +14/-0)
• src/plugins/registry.ts (modified, +47/-0)
• src/plugins/types.ts (modified, +20/-0)