PR #8858: feat: add custom OpenAI-compatible search backend

• Repository: NousResearch/hermes-agent
• Author: yuanqingz
• State: open | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/8858

Description (problem / solution / changelog)

Add a new custom web search backend that works with any OpenAI-compatible chat completions endpoint with built-in web search (e.g. Perplexity Sonar, ChatGPT with browsing).

Configuration via config.yaml: web: backend: custom custom_base_url: https://api.example.com/v1 custom_model: model-name custom_api_key: sk-xxx

Or via environment variables: CUSTOM_SEARCH_BASE_URL, CUSTOM_SEARCH_MODEL, CUSTOM_SEARCH_API_KEY

The backend extracts structured search results from the model citations/search_results fields, falling back to the answer text. Also supports web_extract via chat completion.

What does this PR do?

Adds a fifth web_search / web_extract backend — custom — for any OpenAI-compatible /chat/completions endpoint that returns search citations inline. Concrete targets: Perplexity Sonar, ChatGPT with browsing, self-hosted LLMs with web access, LiteLLM or vLLM wrapping a search-augmented model.

Why this belongs in tools/web_tools.py next to Exa / Tavily / Firecrawl / Parallel rather than as a skill:

• web_search / web_extract are first-class tools invoked by every agent and every platform toolset — a skill would be invisible unless explicitly loaded.
• The web: config block is already the canonical place users expect to configure search; adding a skill would fragment the mental model.
• The response-parsing logic (search_results[] → citations[] → answer text) is proxy-shape normalization, which is the same responsibility the other backends already discharge inside web_tools.py.

This is deliberately distinct from #10284 / #10414 (a custom JSON backend for 4get / SearXNG) — that path uses GET with url_template + results_path field mapping, this path uses POST /chat/completions with citation extraction. Different request shape, different response parser, different extract semantics; they want to coexist.

Related Issue

#12832

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

tools/web_tools.py (+219/-4) — new Custom Chat Completions Backend section with:

• _get_custom_base_url(), _get_custom_model(), _custom_headers() — resolution order is CUSTOM_SEARCH_* env → web.custom_* config → (model only) default sonar.
• _custom_chat(prompt) — single httpx.post to {base_url}/chat/completions with OpenAI-shaped payload.
• _custom_search(query, limit) — three-tier response normalization: search_results[] (Perplexity native) → citations[] (string or dict) → choices[0].message.content as "Search Answer". Always returns the same {success, data: {web: [...]}} shape as the other backends so web_search_tool doesn't branch downstream.
• _custom_extract(urls) — one chat call per URL with per-URL exception isolation; returns the same document shape (url, title, content, raw_content, metadata) as Firecrawl/Tavily/Parallel extract.
• _get_backend(), _is_backend_available(), _web_requires_env(), check_web_api_key(), get_debug_session_info(), web_search_tool(), web_extract_tool() — threaded "custom" through every backend-aware dispatch point. _is_backend_available("custom") uniquely accepts either CUSTOM_SEARCH_API_KEY env or web.custom_api_key config, matching the helper resolution order.

hermes_cli/tools_config.py (+33/-0) — adds a "Custom (OpenAI-compatible)" entry to the web-backend provider list in hermes tools, plus a small generic extra_config mechanism in _configure_provider() that prompts for custom_base_url / custom_model (with sonar as the default) and writes them under web:. The mechanism is generic so future backends with non-env config can reuse it without a bespoke block.

tests/tools/test_web_tools_config.py (+363) — 20 new tests across three areas:

1. TestBackendSelection parity (4 tests): config → "custom", case-insensitive, env-only fallback, priority ordering when Firecrawl also has a key.
2. TestCheckWebApiKey parity (3 tests): env-only, backend: custom configured but no key returns False, web.custom_api_key via config only still returns True.
3. Custom-backend-specific classes (13 tests): TestCustomBackendHelpers covers env vs config priority, trailing-slash stripping, default model, missing-config ValueError, Bearer auth construction. TestCustomSearch covers all three response-shape paths and their priority, limit enforcement, and the empty-response branch. TestCustomExtract covers multi-URL success, per-URL exception isolation, and empty-list short-circuit.

Also extended two existing _ENV_KEYS tuples to include CUSTOM_SEARCH_API_KEY / CUSTOM_SEARCH_BASE_URL / CUSTOM_SEARCH_MODEL so setup_method cleans them — without this, Custom env vars set in one test could leak into another.

website/docs/user-guide/configuration.md, website/docs/integrations/index.md, website/docs/reference/tools-reference.md — added the fifth backend to every enumerating table / comment / env-var list. The configuration guide gets a dedicated subsection explaining the resolution order and the extraction priority.

How to Test

1. Configure Hermes against a real Perplexity Sonar account:

   # ~/.hermes/config.yaml
   web:
     backend: custom
     custom_base_url: https://api.perplexity.ai
     custom_model: sonar

   # ~/.hermes/.env
   CUSTOM_SEARCH_API_KEY=<your-perplexity-api-key>

2. Verify the debug banner picks it up:

   hermes doctor
   # → "Using custom backend: https://api.perplexity.ai (model: sonar)"

3. Exercise both tools end-to-end:

   hermes -q "Use web_search to find the latest llama.cpp release notes, then web_extract the top result."

   Expected: agent gets normalized {title, url, description, position} items from search_results[] (no Search Answer fallback, no empty list). web_extract returns a markdown document per URL with per-URL error isolation when a URL is unreachable.

4. Run the test suite for the touched area:

   cd ~/.hermes/hermes-agent && source venv/bin/activate
   pytest tests/tools/test_web_tools_config.py tests/tools/test_web_tools_tavily.py -q
   # → 93 passed (78 in test_web_tools_config.py, 15 in test_web_tools_tavily.py)

5. For contributors without a Sonar account: the added unit tests patch _custom_chat directly, so no real network call or key is needed to verify the response-shape contracts.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Ubuntu 24.04

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — updated configuration.md, integrations/index.md, tools-reference.md
• I've updated cli-config.yaml.example if I added/changed config keys — N/A (the example file does not document the web: block, so there's no existing shape to extend)
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A (no architecture change)
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — N/A (pure httpx.post + os.getenv + string processing, no termios/fcntl/os.setsid/path separators/subprocess)
• I've updated tool descriptions/schemas if I changed tool behavior — the module-level docstring enumerates Custom alongside Exa/Tavily/Firecrawl/Parallel; web_search / web_extract schemas are unchanged (backend is transparent to the tool contract)

Screenshots / Logs

$ hermes doctor | head -20
...
✅ Web Search & Extract: configured
   Using custom backend: https://api.perplexity.ai (model: sonar)
...

$ pytest tests/tools/test_web_tools_config.py -q
78 passed in 2.18s

Changed files

• hermes_cli/tools_config.py (modified, +33/-0)
• tests/tools/test_web_tools_config.py (modified, +363/-0)
• tools/web_tools.py (modified, +219/-4)
• website/docs/integrations/index.md (modified, +4/-3)
• website/docs/reference/tools-reference.md (modified, +2/-2)
• website/docs/user-guide/configuration.md (modified, +15/-2)