PR #24872: fix: read auxiliary.vision.model from config.yaml in vision/browser tools

• Repository: NousResearch/hermes-agent
• Author: vanthinh6886
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/24872

Description (problem / solution / changelog)

Problem

vision_analyze, browser_vision, and video_analyze tools only read the vision model from the AUXILIARY_VISION_MODEL environment variable, ignoring the auxiliary.vision.model setting in config.yaml.

When users configure a custom vision model via config.yaml:

auxiliary:
  vision:
    model: nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free
    api_key: sk-...

The tools still use the auto-detected default (often google/gemma-4-31b-it:free from Nous Portal free tier), making it impossible to work around rate limits by switching to a different vision model.

Fix

Each tool now reads from config.yaml as fallback when the env var is not set.

Resolution order: env var (AUXILIARY_VISION_MODEL) > auxiliary.vision.model in config.yaml > None (lets auxiliary router auto-detect)

Files changed:

• tools/browser_tool.py: _get_vision_model() now reads auxiliary.vision.model from config
• tools/vision_tools.py: _handle_vision_analyze and _handle_video_analyze now read auxiliary.vision.model / auxiliary.video.model from config

Before vs After

Tests

• py_compile passes on both modified files
• 189 vision-related tests pass (0 regressions)
• Pre-existing test failures (missing optional deps) unchanged

Fixes #24842

Changed files

• tools/browser_tool.py (modified, +21/-2)
• tools/vision_tools.py (modified, +27/-0)

PR #24875: fix(vision): respect auxiliary vision routing

• Repository: NousResearch/hermes-agent
• Author: felix-windsor
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/24875

Description (problem / solution / changelog)

Summary

• stop vision_analyze from passing stale AUXILIARY_VISION_MODEL values as explicit model overrides
• let browser_vision rely on call_llm(task="vision") so auxiliary.vision provider/model/base_url/api_key resolve together
• update regression tests around stale env bridges and browser screenshot vision routing

Fixes #24842

Tests

• scripts/run_tests.sh tests/tools/test_vision_tools.py tests/tools/test_browser_console.py tests/tools/test_vision_native_fast_path.py tests/agent/test_vision_resolved_args.py tests/agent/test_auxiliary_config_bridge.py
• .venv/bin/ruff check tools/vision_tools.py tools/browser_tool.py tests/tools/test_vision_tools.py tests/tools/test_browser_console.py tests/agent/test_auxiliary_config_bridge.py

Changed files

• tests/agent/test_auxiliary_config_bridge.py (modified, +5/-8)
• tests/tools/test_browser_console.py (modified, +23/-4)
• tests/tools/test_vision_tools.py (modified, +6/-9)
• tools/browser_tool.py (modified, +5/-12)
• tools/vision_tools.py (modified, +4/-4)