hermes - 💡(How to fix) Fix [Bug]: Google Meet bot misreports Japanese caption language as English

Bug Description

Bug Description

The Google Meet bot can misreport the caption spoken-language state for Japanese meetings.

In a live Google Meet session, the visible Meet UI can show Japanese captions correctly, but the bot's status/debug output may still report the caption language as English, e.g. language 英語 / _caption_language_state() returns english.

This makes the debug/status signal unreliable: the actual live caption UI and the bot's detected caption language can diverge.

Steps to Reproduce

1. Start the Hermes Google Meet bot against a Google Meet call where Japanese speech/captions are expected.
2. Ensure captions are enabled and the Meet UI visibly shows Japanese captions.
3. Inspect the bot status/debug output, such as status.json and debugInfo.
4. Observe that the debug/status language probe may still report English even though the visible Meet UI captions are Japanese.

Expected Behavior

The bot should report the caption spoken-language state based on the actual active caption language, or return unknown when it cannot reliably determine it.

It should not report english when the visible Meet caption UI is already displaying Japanese captions correctly.

Actual Behavior

The live Meet screen can show Japanese captions correctly, while the bot status/debug output still indicates English.

This appears to be a false positive from DOM/body text probing rather than the real active caption state.

Affected Component

Google Meet plugin / Meet bot caption handling.

Relevant areas:

• plugins/google_meet/meet_bot.py
  • _caption_language_state(page)
  • _set_captions_language_japanese(page)
  • _CAPTION_OBSERVER_JS
• tests/plugins/test_google_meet_plugin.py

Operating System

macOS 26.2

Debug Report

TODO: paste output from hermes debug share or /debug.

Additional Context

There is an existing related PR for non-English Google Meet locales:

https://github.com/NousResearch/hermes-agent/pull/20025

That PR focuses on Thai locale join/admission detection. This issue is related but different: it is specifically about caption spoken-language detection and debug/status reliability for Japanese captions.

Root Cause Analysis

The current language probe appears to rely on text from page.locator("body").inner_text() and checks for strings such as:

python "language\n日本語" "言語\n日本語" "日本語\nformat_size" "language\n英語" "言語\n英語" "English\nformat_size"

This can be fragile because Google Meet's body text may include caption settings UI, hidden/stale menu text, localized UI labels, or non-active language options. As a result, the probe can report English even when the visible active captions are Japanese.

A more reliable approach may be to:

• scope language detection to the opened caption settings dialog only, not the whole page body;
• prefer explicit selected/checked menu item state if available;
• avoid treating background/body text as authoritative;
• return unknown when the active selection cannot be determined;
• verify against actual caption transcript quality or visible caption container state when possible.

Proposed Fix

Make _caption_language_state() more conservative and less dependent on whole-body text.

Potential improvements:

1. Scope detection to visible caption settings dialog/menu nodes.
2. Detect selected/active language via ARIA attributes or checked menu item state if Meet exposes them.
3. Treat conflicting body text as unknown rather than english.
4. Add regression tests where body text contains both English and Japanese settings labels but the active/visible caption state is Japanese.
5. Keep debug output explicit, e.g. captionLanguageProbe: unknown/body-conflict, so users do not mistake the probe for ground truth.

Steps to Reproduce

1. Start a Google Meet session with Japanese speakers.
2. Run the Hermes Google Meet bot against that meeting.
3. Enable captions in Google Meet.
4. Set or confirm the spoken caption language as Japanese in the Meet caption settings.
5. Confirm visually that the live Meet UI is displaying Japanese captions correctly.
6. Check the bot's status.json / debug output.
7. The bot may report the caption language as English even though Japanese captions are visibly active.

Expected Behavior

The live Google Meet screen displayed Japanese captions correctly, but Hermes still reported the caption language as English in its status/debug output.

This caused a mismatch between the actual visible caption state and the bot's detected caption language, making the debug information misleading.

Actual Behavior

The visible Google Meet UI showed Japanese captions correctly, but the Hermes bot status/debug output still indicated that the caption language was English.

For example, status.json / debugInfo reported a state consistent with language 英語, and _caption_language_state() appeared to return english, even though the active captions visible on the Meet screen were Japanese.

As a result, the bot's reported caption language did not match the actual visible caption state. This makes the debug/status output unreliable when diagnosing Japanese caption issues.

Affected Component

Tools (terminal, file ops, web, code execution, etc.)

Messaging Platform (if gateway-related)

Telegram

Debug Report

Debug report not generated yet.

I observed the issue during a live Google Meet bot session. I can provide sanitized `hermes debug share` or `hermes debug share --local` output if needed.

Operating System

macOS 26.2

Python Version

No response

Hermes Version

No response

Additional Logs / Traceback (optional)

Root Cause Analysis (optional)

No response

Proposed Fix (optional)

No response

Are you willing to submit a PR for this?

• I'd like to fix this myself and submit a PR