hermes - 💡(How to fix) Fix Expose update-safe TUI/composer hooks for voice transcript and composer events

Summary

Hermes plugins currently do not appear to have an update-safe extension point for Ink TUI composer/transcript behavior, especially around voice transcription events. Please add plugin/config hooks for frontend events such as voice.transcript, composer input mutation, and transcript/status rendering.

Motivation

Some Hermes UX changes are currently only possible by patching the installed TUI source or built bundle. That is update-fragile and conflicts with the intended plugin/config-based extension model.

A concrete case is voice dictation: users may want speech-to-text to insert text into the composer for manual editing, suppress or transform related UI messages, or choose whether transcripts auto-submit. Today the relevant behavior lives in the Ink TUI event path, not in a plugin-exposed lifecycle.

Current behavior

The plugin system exposes general Python-side plugin registration and hooks, but the Ink TUI composer/transcript path is frontend-specific. In the current code path, voice.transcript is handled directly in the TUI frontend.

Relevant current behavior in ui-tui/src/app/createGatewayEventHandler.ts:

case 'voice.transcript': {
  const text = String(ev.payload?.text ?? '').trim()

  if (process.env.HERMES_VOICE_AUTOSUBMIT === '0') {
    setInput(current => (current.trim() ? `${current.trimEnd()} ${text}` : text))
    sys(`voice transcript: ${text}`)
    return
  }

  setInput('')
  setTimeout(() => submitRef.current(text), 0)
  return
}

useSubmission.ts then sends only submitted composer text through prompt.submit, while sys(...) creates frontend system display rows. That means some UX changes are purely TUI-layer changes and cannot be implemented cleanly with current Python plugins.

Expected behavior

There should be an update-safe way for plugins or config to participate in TUI/composer events without patching the Hermes source tree.

Possible lifecycle hooks/config surfaces:

• on_voice_transcript(event) before the TUI decides whether to auto-submit, insert into composer, or display status text.
• transform_composer_input(current, incoming, source) for composer mutations.
• on_transcript_display(message) or a filter for frontend-only sys(...) display rows.
• Config options for common behavior such as dictation/no-autosubmit and transcript status visibility.

Proposed design

One reasonable shape:

ctx.register_hook("tui.voice_transcript", callback)
ctx.register_hook("tui.composer_insert", callback)
ctx.register_hook("tui.transcript_display", callback)

Callbacks could return structured decisions such as:

{
  "insert_into_composer": "...",
  "auto_submit": false,
  "display_status": null
}

If frontend plugins are not planned, a smaller config-first design would still help:

voice:
  autosubmit: false
  show_transcript_status: false
  enabled_by_default: false

Technical context

• Hermes version observed locally: Hermes Agent v0.14.0 (2026.5.16)
• Frontend: Ink TUI
• Platform: WSL launched from Windows PowerShell wrapper
• Relevant files:
  • ui-tui/src/app/createGatewayEventHandler.ts — handles voice.transcript, mutates composer via setInput, emits sys("voice transcript: ..."), or calls submitRef.current(text).
  • ui-tui/src/app/useSubmission.ts — sends composer submissions through prompt.submit.
  • tui_gateway/server.py — backend prompt.submit path runs the agent with submitted text/history.
  • hermes_cli/plugins.py — plugin discovery/registration currently lives on the Python side and does not expose this TUI event path.

Alternatives considered

• Wrapper env flags: useful for simple dictation/no-autosubmit behavior, but not enough to customize display or implement richer composer behavior.
• Local source patch: works short-term but is overwritten by hermes update and is not suitable as a durable user configuration.
• User plugin: current plugin hooks do not appear to reach this frontend/composer lifecycle.

Acceptance criteria

• A user can implement or configure voice transcript behavior without editing Hermes source.
• Common dictation behavior can be configured: transcript goes into the composer, waits for manual Enter, and optionally suppresses transcript status display.
• Existing auto-submit voice behavior remains available for users who want it.