hermes - 💡(How to fix) Fix MCP tool responses: image content silently dropped + large text results truncated

Summary

Two related issues with how Hermes handles MCP tool responses that affect tools returning screenshots or large data payloads.

Issue 1: Image content blocks silently dropped from MCP tool results

File: tools/mcp_tool.py, lines 2057-2062

The _call() method only extracts text content blocks from CallToolResult:

parts: List[str] = []
for block in (result.content or []):
    if hasattr(block, "text"):
        parts.append(block.text)
text_result = "\n".join(parts) if parts else ""

MCP tools can return ImageContent blocks (with .data and .mimeType attributes) alongside TextContent blocks. These image blocks are silently skipped. The agent only receives the text portion.

Impact: Tools like vm_screenshot that return [ImageContent(data=<png>, mimeType="image/png"), TextContent(text=<json metadata>)] lose the image entirely. The agent sees only the JSON metadata.

Note: The codebase already handles image blocks in _convert_messages (line 609-612) for MCP sampling/prompt messages, converting data+mimeType into image_url format. The same pattern should be applied to tool result content.

Issue 2: Large text results truncated to 1,500 char preview

Files:

• tools/budget_config.py line 18: DEFAULT_RESULT_SIZE_CHARS = 100_000
• tools/tool_result_storage.py line 116: Results exceeding 100K chars are persisted to /tmp/hermes-results/ and replaced with a ~1,500-char preview

This means any tool result over 100KB (e.g., a base64-encoded screenshot returned as text, which is typically 100-500KB) is truncated before reaching the agent. The agent gets a preview + file path, but must make a separate read_file call to access the full content.

For the screenshot use case, this creates a double barrier:

1. Image content blocks are dropped (Issue 1)
2. Even if the image is returned as base64 text (workaround), it's truncated (Issue 2)

Expected Behavior

1. ImageContent blocks in MCP tool results should be converted to inline images (e.g., image_url format) and forwarded to the LLM, similar to how _convert_messages handles them for prompts.
2. Large tool results containing image data should either not be subject to the 100K text truncation, or the truncation threshold should be configurable per-tool.

Environment

• Hermes Agent: latest main branch
• Model: Claude Sonnet 4.6 via Anthropic API

Workaround

Currently using RPA scripts with safe_screenshot_b64() that return base64 as stdout text within the JSON response, staying under the 100K limit by compressing images. This avoids both issues but limits image quality and adds complexity.