openclaw - 💡(How to fix) Fix [Feature]: OpenClaw trace emission should include captureContent

Summary

diagonistics.otel.captureContent fields don't actually seem to populate with anything, and despite the plugin being pretty accurate, the core gateway never populates captrueContent fields so nothing changes in exported traces.

Problem to solve

Operators using OTEL-compatible backends (or even just dumping locally) cannot get model or tool content into their exported traces, even when explicitly enabling captureContent. The config is accepted, the plugin loads cleanly, and traces export, but the content fields are always absent with no error or warning. There is no way to debug agent behavior, audit tool usage, or build dashboards tied to actual prompt content through the OpenClaw gateway today.

Proposed solution

he fix is intentionally minimal. The diagnostics-otel plugin already implements captureContent filtering correctly — resolveContentCapturePolicy, assignOtelModelContentAttributes, and assignOtelToolContentAttributes are all wired up and working. The only reason the feature is broken is that the content fields are never populated on the diagnostic events before the plugin receives them. The entire fix lives in the gateway core.

1. Extend diagnostic event types (src/infra/diagnostic-events.ts)

Add optional content fields to two existing event types — purely additive, no existing fields touched:

DiagnosticModelCallCompletedEvent — add:

• inputMessages?: string | string[] — the messages array sent to the model for this call
• outputMessages?: string | string[] — the assembled text content from the model response stream
• systemPrompt?: string — the system prompt active for this call

DiagnosticToolExecutionCompletedEvent — add:

• toolInput?: Record<string, unknown> — the params passed to the tool
• toolOutput?: string | string[] — the serialized result returned by the tool

All fields are optional. Existing consumers that don't read them are completely unaffected.

2. Populate content at the tool execution emission site (src/agents/pi-tools.before-tool-call.ts)

At the tool.execution.completed emit site, outcome.params (tool input) and result (tool output) are already in scope. Simply attach them:

• toolInput ← outcome.params (already a plain object)
• toolOutput ← result, serialized to string if not already

No config check, no gating — always populate. The plugin handles filtering.

3. Populate content at the model call emission site (src/agents/pi-embedded-runner/run/attempt.model-diagnostic-events.ts)

Two pieces of content need to be captured here:

Input side: The messages array and system prompt are available in the stream context passed into wrapStreamFnWithDiagnosticModelCallEvents. Extract them and attach to eventBase so they're available when emitModelCallCompleted fires.

Output side: The response stream is already being iterated chunk-by-chunk in observeModelCallIterator for byte counting. Accumulate the text content from response chunks into the existing ModelCallObservationState struct (add an outputChunks: string[] field), then join and attach as outputMessages when emitModelCallCompleted is called.

Again — no gating, always populate, let the plugin filter.

4. No plugin changes needed

The diagnostics-otel plugin already does the right thing once the fields are present. captureContent config will start working as documented the moment the gateway emits the data.

Alternatives considered

Comet Opik has its own handling for OpenClaw traces, but it's a home-brew and works around the official plugin. Langsmith has an option as well, community-sourced. Both of these force you to use a specific provider. It also makes using the official plugin useless, which is a shame since it's the most seamless and simple of the bunch.

Impact

Affected: Anyone trying to get OpenClaw observability Severity: Medium (not small, not huge, but not delivering on a promised feature seems pretty genuine of a problem) Frequency: Always Consequence: Either switch off the OpenClaw plugin and lock into a vendor, or don't get full trace data

Evidence/examples

1. Documented as a working feature (docs/gateway/opentelemetry.md)

The docs describe each field as a functioning opt-in:

Set diagnostics.otel.captureContent.* to true only when your collector and
retention policy are approved for prompt, response, tool, or system-prompt text.

- inputMessages  — user prompt content
- outputMessages — model response content
- toolInputs     — tool argument payloads
- toolOutputs    — tool result payloads
- systemPrompt   — assembled system/developer prompt

The feature is presented as fully operational.

2. Plugin handler is correctly implemented (diagnostics-otel/dist/index.js)

The plugin reads the fields and maps them to span attributes exactly as expected:

function assignOtelModelContentAttributes(attributes, event, policy) {
  if (policy.inputMessages) assignOtelContentAttribute(attributes, "openclaw.content.input_messages", event.inputMessages);
  if (policy.outputMessages) assignOtelContentAttribute(attributes, "openclaw.content.output_messages", event.outputMessages);
  if (policy.systemPrompt)   assignOtelContentAttribute(attributes, "openclaw.content.system_prompt",  event.systemPrompt);
}
function assignOtelToolContentAttributes(attributes, event, policy) {
  if (policy.toolInputs)  assignOtelContentAttribute(attributes, "openclaw.content.tool_input",  event.toolInput);
  if (policy.toolOutputs) assignOtelContentAttribute(attributes, "openclaw.content.tool_output", event.toolOutput);
}

3. Core event types never define the fields (plugin-sdk/src/infra/diagnostic-events.d.ts)

The fields the plugin expects simply don't exist on the events:

// Missing: inputMessages, outputMessages, systemPrompt
export type DiagnosticModelCallCompletedEvent = DiagnosticModelCallBaseEvent & {
  type: "model.call.completed";
  durationMs: number;
  requestPayloadBytes?: number;
  responseStreamBytes?: number;
  timeToFirstByteMs?: number;
};

// Missing: toolInput, toolOutput
export type DiagnosticToolExecutionCompletedEvent = DiagnosticToolExecutionBaseEvent & {
  type: "tool.execution.completed";
  durationMs: number;
};

4. Live OTLP trace dump confirms zero content keys exported

With captureContent fully enabled, exported traces contain no openclaw.content.* attributes — only timing, sizing, and provider metadata:

gen_ai.operation.name, gen_ai.request.model, gen_ai.system,
openclaw.model, openclaw.model.call, openclaw.model_call.request_bytes,
openclaw.model_call.response_bytes, openclaw.tool.execution ...
→ openclaw.content.*: NONE

Additional information

Privacy / Security

This is the most significant concern. The existing model guarantees content is never exposed without explicit opt-in — but "always emit" upholds this only at the OTEL export layer, not at the event layer. Any plugin subscribing to model.call.completed or tool.execution.completed will now receive full message content unconditionally, regardless of captureContent config. Third-party plugins that log or serialize events could silently start capturing prompt content.

Recommended mitigation: Add a single lightweight isDiagnosticContentCaptureEnabled() gate at the core emission sites that checks diagnostics.otel.captureContent.enabled. Content is only attached to the event when the feature is explicitly on. No config threading needed — one flag, one check per site. The plugin's per-field filtering remains unchanged.

Logging consumers

diagnostic-run-activity.ts and diagnostic-stability.ts consume these event types for TUI display and structured logging. If either file spreads or serializes the full event object, tool params, tool outputs, and message content could start appearing in logging.file output — which users may ship to external log aggregators.

agent-changelog hook

The agent-changelog-capture hook is enabled in this instance and observes agent activity. If it subscribes to model.call.completed or tool.execution.completed, it may start persisting content it was never designed to handle. Worth auditing before merge.

Event payload size at scale

inputMessages on model.call.completed can be substantial for long sessions with large context windows — potentially hundreds of KB per event. In-process and short-lived so usually fine, but plugins that hold event references for batching or correlation could see increased memory pressure.

Plugin SDK contract

The diagnostic event types are part of the public plugin SDK API. Adding optional fields is non-breaking in TypeScript, but it's a meaningful behavioral change for plugins that serialize or log events. A clear changelog callout is warranted, and the privacy implications above may merit a semver note.