openclaw - ✅(Solved) Fix Image Tool returns "no image" when called from agent session — image data lost in tool invocation pipeline [1 pull requests, 1 participants]

zhengriu · 2026-04-09T01:37:17Z

[openclaw] 当 agent 在对话中调用 image 工具识别图片时，视觉模型（如 zai/glm-4.6v）回复"没有收到图片"。但直接调用 describeImageWithModel 函数完全正常（3秒返回正确结果）。问题出在 agent 框架的 tool 调用管线中，图片 buffer 数据在传递过… 当 agent 在对话中调用 image 工具识别图片时，视觉模型（如 zai/glm-4.6v）回复"没有收到图片"。但直接调用 describeImageWithModel() 函数完全正常（3秒返回正确结果）。问题出在 agent 框架的 tool 调用管线中，图片 buffer 数据在传递过程中丢失。 # PR #63497: fix(agents): support media://inbound URIs in image tool #63476 - Repository: openclaw/openclaw - Author: jepson-liu - State: open | merged: False - Link: https://github.com/openclaw/openclaw/pull/63497 ## Description (problem / solution / changelog) ## Summary - **Problem:** With a text-only primary model (e.g. `zai/glm-5-turbo`), inbound images are not passed as multimodal input; agents rely on the `image` tool. Gateway claim-check references use `media://inbound/ `, but the tool either rejected them as unsupported schemes or corrupted them when normalizing `MEDIA:`-style prefixes (stripping `media:` from `media://` produced `//inbound/...` and broke local resolution). - **Why it matters:** Users hitting Telegram (and similar) flows could see vision failures or “no image” behavior even though files exist under the media store. - **What changed:** Resolve `media://inbound/ ` via `resolveMediaBufferPath` + `loadWebMedia` with `localRoots: [getMediaDir()]`, aligned with native prompt image loading. Only strip the `MEDIA:` tag when the value is not already a `media://` gateway URI. Added Vitest coverage with `OPENCLAW_STATE_DIR`. - **What did NOT change:** Sandbox/workspace rules for ordinary paths and URLs; no changes to gateway offload or channel parsers. ## Change Type (select all) - [x] Bug fix - [ ] Feature - [ ] Refactor required for the fix - [ ] Docs - [ ] Security hardening - [ ] Chore/infra ## Scope (select all touched areas) - [ ] Gateway / orchestration - [x] Skills / tool execution - [ ] Auth / tokens - [ ] Memory / storage - [ ] Integrations - [ ] API / contracts - [ ] UI / DX - [ ] CI/CD / infra ## Linked Issue/PR - Fixes #63476 ## User-visible / Behavior Changes The `image` tool now accepts gateway claim-check URIs `media://inbound/ ` (and `MEDIA:media://inbound/ `), resolving them to on-disk inbound media the same way native prompt image loading does. ## Security Impact (required) - New permissions/capabilities? **No** - Secrets/tokens handling changed? **No** - New/changed network calls? **No** - Command/tool execution surface changed? **No** (same tool; additional allowed URI form, still constrained by `resolveMediaBufferPath` and media dir roots) - Data access scope changed? **No** (only resolves IDs already under the configured inbound media store) ## Repro + Verification ### Environment - OS: Linux (CI-style); issue originally macOS + Telegram + ZAI - Runtime: Node 22+ / Vitest - Model/provider: N/A for automated test (stubbed media-understanding + fetch) - Integration/channel: N/A in unit tests - Relevant config (redacted): `OPENCLAW_STATE_DIR` pointing at a temp dir containing `media/inbound/ ` ### Steps 1. Set `OPENCLAW_STATE_DIR` to a temp directory; create `media/inbound/claim-check-test.png` with a tiny PNG payload. 2. Run `pnpm test -- src/agents/tools/image-tool.test.ts -t "gateway media"`. 3. (Optional) Full file: `pnpm test -- src/agents/tools/image-tool.test.ts`. ### Expected - Tool executes successfully and returns text result; fetch stub invoked once for the generic image path. ### Actual - Both tests pass locally; `pnpm build` passes for the tree that includes this change. ## Evidence - [x] Failing test/log before + passing after (behavior covered by new tests; prior behavior was wrong path / unsupported URI) - [ ] Trace/log snippets - [ ] Screenshot/recording - [ ] Perf numbers (if relevant) ## Human Verification (required) - Verified scenarios: Vitest `image tool gateway media://inbound URIs`; full `src/agents/tools/image-tool.test.ts`; `pnpm build` after the change. - Edge cases checked: bare `media://inbound/...` vs `MEDIA:media://inbound/...`. - What I did **not** verify: Live Telegram + ZAI end-to-end (requires external keys and channel). ## Review Conversations - [x] I replied to or resolved every bot review conversation I addressed in this PR. - [x] I left unresolved only the conversations that still need reviewer or maintainer judgment. ## Compatibility / Migration - Backward compatible? **Yes** - Config/env changes? **No** - Migration needed? **No** ## Failure Recovery (if this breaks) - Revert the commit touching `src/agents/tools/image-tool.ts` (and tests/changelog if needed). - Symptoms: `media://inbound/...` might again be rejected or mis-resolved. ## Risks and Mitigations - **Risk:** Broader interpretation of `media://` inputs. - **Mitigation:** Only `media://inbound/ ` is handled; ID validation matches existing `resolveMediaBufferPath` guards. ## Changed files - `CHANGELOG.md` (modified, +2/-0) - `src/agents/tools/image-tool.test.ts` (modified, +66/-0) - `src/agents/tools/image-tool.ts` (modified, +60/-11) ## Fixed

openclaw2026-04-09 01:37:17

ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

GitHub issue URL

Helpful · Quick feedback

GitHub stats

openclaw/openclaw#63476•Fetched 2026-04-09 07:53:19

View on GitHub

Comments

Participants

Timeline

Reactions

Author

zhengriu

Participants

zhengriu

Timeline (top)

cross-referenced ×1referenced ×1

当 agent 在对话中调用 image 工具识别图片时，视觉模型（如 zai/glm-4.6v）回复"没有收到图片"。但直接调用 describeImageWithModel() 函数完全正常（3秒返回正确结果）。问题出在 agent 框架的 tool 调用管线中，图片 buffer 数据在传递过程中丢失。

Root Cause

buildImageContext() 中的 image.buffer 可能在 agent tool 管线传递过程中为空。函数本身没问题，是 agent 框架调用 tool 时参数传递的 bug。

RAW_BUFFERClick to expand / collapse

Bug type

Behavior bug

Summary

Environment

OS: macOS 26.2 (arm64)
OpenClaw: 2026.4.8
imageModel: zai/glm-4.6v
chat model: zai/glm-5-turbo

Steps to reproduce

配置 imageModel: "zai/glm-4.6v"
通过 Telegram 发送图片给 agent
Agent 调用 image 工具
工具返回"您似乎忘记上传图片了"

Expected vs Actual

Expected: 正确识别图片内容
Actual: 视觉模型回复"没有收到图片"

Evidence

直接调 describeImageWithModel() → ✅ 3秒返回 "TEST 123" 直接调 pi-ai complete() → ✅ 3秒返回 "TEST 123"
Agent session 内调 image tool → ❌ "您似乎忘记上传图片了" Gateway 重启 + 新 session 都不行日志显示图片被正确 resize，但没有 API 调用记录

Root cause

buildImageContext() 中的 image.buffer 可能在 agent tool 管线传递过程中为空。函数本身没问题，是 agent 框架调用 tool 时参数传递的 bug。

extent analysis

TL;DR

The issue can be fixed by ensuring that the image.buffer is properly passed through the agent framework's tool pipeline, possibly by modifying the buildImageContext() function or the pipeline itself.

Guidance

Verify that the image.buffer is not empty before passing it to the buildImageContext() function.
Check the agent framework's documentation to see if there are any specific requirements for passing image data through the tool pipeline.
Consider adding logging statements to track the value of image.buffer at different points in the pipeline to identify where it is being lost.
Review the buildImageContext() function to ensure that it is correctly handling the image data and not modifying it in a way that causes it to be lost.

Example

// Example of how to add logging to track the value of image.buffer
console.log('Image buffer before buildImageContext:', image.buffer);
const context = buildImageContext(image);
console.log('Image buffer after buildImageContext:', context.image.buffer);

Notes

The issue seems to be specific to the agent framework's tool pipeline, and the buildImageContext() function itself is not the problem. The fact that the describeImageWithModel() function works correctly when called directly suggests that the issue is with how the image data is being passed through the pipeline.

Recommendation

Apply a workaround by modifying the buildImageContext() function or the agent framework's tool pipeline to ensure that the image.buffer is properly passed through. This may involve adding additional logging or error checking to identify where the issue is occurring.

Vote matrix · Quick signals

Works

Did the solution work? Tap to confirm.

Easy Fix

Was it a quick fix?

Time Saver

Did it save you time?

Blocking

Was it severely blocking?

Common Issue

Are others likely hitting this too?

Flaky / Intermittent

Is it intermittent?

Verified / Reproducible

Can you reproduce it reliably?

#api #latency issue #model loading #dependency error #configuration error

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

openclaw - ✅(Solved) Fix Image Tool returns "no image" when called from agent session — image data lost in tool invocation pipeline [1 pull requests, 1 participants]

Recommended Tools

GitHub issue graph ai analysis

Root Cause

Fix Action

Fixed

PR fix notes

PR #63497: fix(agents): support media://inbound URIs in image tool #63476

Description (problem / solution / changelog)

Summary

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

User-visible / Behavior Changes

Security Impact (required)

Repro + Verification

Environment

Steps

Expected

Actual

Evidence

Human Verification (required)

Review Conversations

Compatibility / Migration

Failure Recovery (if this breaks)

Risks and Mitigations

Changed files

Bug type

Summary

Environment

Steps to reproduce

Expected vs Actual

Evidence

Root cause

extent analysis

TL;DR

Guidance

Example

Notes

Recommendation

Still need to ship something?

RELATED_DISCOVERY

TRENDING