hermes - 💡(How to fix) Fix Bug: PLATFORM_HINTS missing "webui" entry — agent never uses MEDIA: in WebUI

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

The Hermes WebUI creates AIAgent(platform="webui"), but PLATFORM_HINTS in agent/prompt_builder.py has no entry for "webui". This means the LLM receives zero platform-specific guidance when responding via the WebUI, and critically, never learns it can use MEDIA:/path to display images and files inline.

Root Cause

In run_agent.py (line ~5287):

platform_key = (self.platform or "").lower().strip()  # → "webui"
if platform_key in PLATFORM_HINTS:                    # False! no "webui" key
    prompt_parts.append(PLATFORM_HINTS[platform_key])
elif platform_key:
    # Check plugin registry → not found either → nothing injected

Result: WebUI sessions get no platform hint at all.

Fix Action

Fix / Workaround

TypeRendering
Images (.png, .jpg, .webp, .gif, .svg, .avif)Inline <img>
Audio (.mp3, .ogg, .wav, etc.)Inline player with speed controls
Video (.mp4, .webm, .mov, etc.)Inline player
PDFFirst-page preview
HTMLSandboxed iframe preview
CSVTable rendering
Diff/PatchColored diff view
ExcalidrawInline embed

Code Example

platform_key = (self.platform or "").lower().strip()  # → "webui"
if platform_key in PLATFORM_HINTS:                    # False! no "webui" key
    prompt_parts.append(PLATFORM_HINTS[platform_key])
elif platform_key:
    # Check plugin registry → not found either → nothing injected

---

"webui": (
    "You are in the Hermes WebUI — a browser-based chat interface with full "
    "Markdown rendering (headings, bold, italic, code blocks, tables, math, "
    "Mermaid diagrams) and native media support. "
    "You can display images, audio, and video to the user by including "
    "MEDIA:/absolute/path/to/file in your response. Images (.png, .jpg, "
    ".webp, .gif, .svg, .avif) render inline, audio gets an inline player "
    "with speed controls, video plays inline, and PDF/HTML/CSV/diff files "
    "get rich previews. Local file paths must be absolute. "
    "You can also use MEDIA:https://... for remote images. "
    "Do NOT use Markdown image syntax like ![alt](/path) for local files — "
    "it will not render. Always use MEDIA:/path for local and "
    "MEDIA:https://... for remote media."
),
RAW_BUFFERClick to expand / collapse

Bug: PLATFORM_HINTS missing "webui" entry — agent never uses MEDIA: in WebUI

Summary

The Hermes WebUI creates AIAgent(platform="webui"), but PLATFORM_HINTS in agent/prompt_builder.py has no entry for "webui". This means the LLM receives zero platform-specific guidance when responding via the WebUI, and critically, never learns it can use MEDIA:/path to display images and files inline.

Impact

The WebUI frontend (static/ui.js lines 2008–2470) has full support for MEDIA:/absolute/path rendering:

TypeRendering
Images (.png, .jpg, .webp, .gif, .svg, .avif)Inline <img>
Audio (.mp3, .ogg, .wav, etc.)Inline player with speed controls
Video (.mp4, .webm, .mov, etc.)Inline player
PDFFirst-page preview
HTMLSandboxed iframe preview
CSVTable rendering
Diff/PatchColored diff view
ExcalidrawInline embed

The backend (api/routes.py line 5161) has a /api/media?path= endpoint with auth gating, path traversal protection, and MIME type mapping.

However, because no platform hint tells the agent it can use MEDIA:, the agent either:

  1. Ignores MEDIA: entirely — just describes files verbally
  2. Uses Markdown image syntax ![alt](/path) — which silently fails for local files in WebUI (the Markdown renderer doesn't serve local paths)

Meanwhile, the "cli" platform hint explicitly tells the agent:

"Do NOT emit MEDIA:/path tags (those are only intercepted on messaging platforms like Telegram, Discord, Slack, etc.; on the CLI they render as literal text)."

This is correct for CLI, but there's no countervailing hint for WebUI where MEDIA: does work and is the only reliable way to display local images.

Root Cause

In run_agent.py (line ~5287):

platform_key = (self.platform or "").lower().strip()  # → "webui"
if platform_key in PLATFORM_HINTS:                    # False! no "webui" key
    prompt_parts.append(PLATFORM_HINTS[platform_key])
elif platform_key:
    # Check plugin registry → not found either → nothing injected

Result: WebUI sessions get no platform hint at all.

Suggested Fix

Add a "webui" entry to PLATFORM_HINTS in agent/prompt_builder.py:

"webui": (
    "You are in the Hermes WebUI — a browser-based chat interface with full "
    "Markdown rendering (headings, bold, italic, code blocks, tables, math, "
    "Mermaid diagrams) and native media support. "
    "You can display images, audio, and video to the user by including "
    "MEDIA:/absolute/path/to/file in your response. Images (.png, .jpg, "
    ".webp, .gif, .svg, .avif) render inline, audio gets an inline player "
    "with speed controls, video plays inline, and PDF/HTML/CSV/diff files "
    "get rich previews. Local file paths must be absolute. "
    "You can also use MEDIA:https://... for remote images. "
    "Do NOT use Markdown image syntax like ![alt](/path) for local files — "
    "it will not render. Always use MEDIA:/path for local and "
    "MEDIA:https://... for remote media."
),

Architecture Context

Hermes Agent has three entry points:

Entry PointPlatform ValuePLATFORM_HINTS KeyStatus
CLI"cli""cli"✅ Has hint
Gateway (Telegram, Discord, etc.)"telegram", "discord", etc.Per-platform✅ Has hints (15 platforms)
WebUI MVP"webui""webui"Missing

The WebUI directly creates AIAgent instances (api/routes.py line 6361: platform="webui"), bypassing the Gateway entirely. This is why it needs its own platform hint.

Environment

  • hermes-agent: v0.13.x (commit 7338e5d)
  • hermes-webui: MVP (standalone, not part of main repo)
  • Bug confirmed on current main branch

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - 💡(How to fix) Fix Bug: PLATFORM_HINTS missing "webui" entry — agent never uses MEDIA: in WebUI