hermes - 💡(How to fix) Fix [Feature Request] Soporte de audio nativo (speech-to-speech) con modelos multimodales

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
RAW_BUFFERClick to expand / collapse

Problema

Actualmente Hermes Agent usa un pipeline STT → texto → LLM → TTS para el modo voz. Esto agrega latencia innecesaria y pierde información (entonación, énfasis, emoción) al pasar por transcripción.

Propuesta

Agregar soporte para modelos multimodales que aceptan audio directamente (audio in → audio out), saltando el paso de transcripción:

  • openai/gpt-4o-audio-preview (OpenRouter) — full-duplex, permite interrumpir al agente
  • google/gemini-3.5-flash — full-duplex, permite interrumpir al agente

Requerimientos

  • Streaming de audio directo al modelo (sin pasar por STT)
  • Modo full-duplex: el usuario puede interrumpir (barge-in) mientras el agente habla
  • Detección de turnos automática (VAD avanzado)
  • Integración vía OpenRouter (API compatible con OpenAI Realtime)

Alternativas consideradas

  • Pipeline actual (funciona pero es lento)
  • LiveKit Agents (requiere infra propia)
  • Azure Speech Services (no open-source)

Prioridad

Media-alta. El modo voz es un diferenciador clave frente a otros agentes CLI.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - 💡(How to fix) Fix [Feature Request] Soporte de audio nativo (speech-to-speech) con modelos multimodales