openclaw - 💡(How to fix) Fix Telegram inbound images use describer-only path; never attached as native vision blocks (model receives `<media:image>` placeholder text) [1 participants]

Summary

When a user sends an image to a Claude-backed agent via Telegram, the model never receives the image as a native vision content block. Instead, the prompt contains either:

1. The literal placeholder string <media:image> (when tools.media.image is disabled or fails), or
2. A text caption produced by the image-describer (when tools.media.image.enabled: true).

The same image sent via the webchat / Control UI works perfectly — the model sees and analyzes the actual image bytes.

Result: Telegram users on a vision-capable model effectively cannot send images, even though the gateway successfully downloads and resizes them.

Environment

• OpenClaw 2026.4.5 (commit 3e72c03)
• Node v22.22.1
• OS: Linux x86_64
• Channel: Telegram (single bot, DM)
• Model: claude-cli/claude-opus-4-6 (vision-capable)
• Gateway: loopback, systemd
• Config: tools.media.image.enabled: true

Reproduction

1. Configure an OpenClaw agent with a vision-capable Claude model and Telegram channel.
2. Send an image to the bot via Telegram DM with the caption "what do you see?"
3. Observe in logs: gateway downloads the image successfully, resizes it, sets MediaPaths on the inbound context (agents/tool-images processing visible).
4. Model reply describes the placeholder text or returns generic confusion ("I can't see any image"), confirming it never received the image as a content block.
5. Send the same image to the same agent via webchat (http://127.0.0.1:18789/).
6. Model accurately describes the image. ✅

Root Cause

OpenClaw has two separate inbound image pipelines, and Telegram is wired to the wrong one:

Pipeline trace (Telegram, broken)

1. extensions/telegram/src/bot-message-context.ts → buildTelegramInboundContextPayload() collects media into contextMedia (bot-message-context-ID6QScNo.js, ~line 343).
2. The payload is built with MediaPath, MediaPaths, MediaTypes, MediaUrls set to local file paths from the Telegram download (~lines 383–390).
3. Initial bodyText is set to the literal placeholder <media:image> when there is no user-supplied text (bot-message-context-ID6QScNo.js:136).
4. The agent runner reads ctx.MediaPaths via normalizeAttachments() in runner-Bo7fJw79.js (line 51), classifies as image via isImageAttachment() (line 121).
5. Runner imports describeImageWithModel from image-runtime-C6QbxTR_.js (line 17) and uses it to generate a text caption.
6. No code path on the Telegram side ever calls parseMessageWithAttachments or constructs a native vision content block.
7. The model receives the prompt text with either the describer's caption or the raw <media:image> placeholder. Image bytes are never sent.

Pipeline trace (Webchat, working)

1. server-Cv5hzFG4.js (gateway server) calls parseMessageWithAttachments(message, normalizedAttachments, ...) at lines 10759 and 12932.
2. parseMessageWithAttachments (in attachment-normalize-CnmzeVAm.js, ~line 893) returns { images: [{type: "image", data, mimeType}], ... }.
3. Those images are passed downstream to the model adapter as native content blocks. ✅

Why the runner-side path exists

The describer pipeline appears to predate native vision support and is what tools.media.image was originally designed for: it enables a separate lightweight model to caption an image so that text-only or non-vision models can still get some signal from images. Useful as a fallback, but the wrong default for vision-capable models.

Expected Behavior

When the configured agent model supports native vision (modelSupportsImages(model) === true), Telegram inbound images should be loaded from ctx.MediaPaths, base64-encoded, and attached to the model call as native {type: "image", source: {...}} content blocks — the same way webchat does it via parseMessageWithAttachments.

The describer pipeline should remain available as a fallback for text-only models, or as an opt-in.

Suggested Fix

In the agent runner (TypeScript source corresponding to runner-Bo7fJw79.js), where image attachments are detected:

// Pseudocode
const attachments = normalizeAttachments(ctx);
const imageAttachments = attachments.filter(isImageAttachment);

if (imageAttachments.length > 0) {
  if (modelSupportsImages(model)) {
    // NEW: native vision path — read files, base64, attach as content blocks
    const imageBlocks = await Promise.all(imageAttachments.map(loadAsNativeImageBlock));
    appendNativeImageBlocksToPrompt(imageBlocks);
  } else {
    // EXISTING: describer fallback for text-only models
    const captions = await Promise.all(imageAttachments.map(describeImageWithModel));
    appendCaptionsToPromptText(captions);
  }
}

loadAsNativeImageBlock would mirror the behavior of parseMessageWithAttachments for the offload-or-inline decision (OFFLOAD_THRESHOLD_BYTES, SUPPORTED_OFFLOAD_MIMES), but starting from a file path instead of a base64 buffer.

Ideally, the two pipelines would be unified into a single inbound media handler shared by all channels (webchat, Telegram, WhatsApp/Watson, Signal, etc.) so this class of bug doesn't recur per-channel.

Workaround

Currently the only workaround for vision use is to ask the user to send the image via webchat instead of Telegram, OR have them upload the file to the workspace and use the image tool from there.

The describer (tools.media.image.enabled: true) does provide some understanding of Telegram-attached images, but only as third-party text — the model cannot perform any actual vision tasks (OCR, layout reasoning, comparing images, fine detail) because it never sees the bytes.

Related

• File: dist/runner-Bo7fJw79.js (agent runner, media-understanding/attachments.normalize.ts namespace)
• File: dist/bot-message-context-ID6QScNo.js (Telegram inbound context builder)
• File: dist/attachment-normalize-CnmzeVAm.js (working webchat path, parseMessageWithAttachments)
• File: dist/server-Cv5hzFG4.js (calls webchat path at lines 10759, 12932)

If a similar gap exists for other inbound channels (Signal, WhatsApp via Watson, etc.), it would be worth auditing them in the same fix.

Reporter context

Found while debugging Dani (a personal OpenClaw agent on 2026.4.5). Telegram users are sending images to the bot expecting the model to see them; the model receives only text. Webchat works for the same agent and model.

Happy to test a fix or PR, and can provide gateway logs from a repro session if helpful.