Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix registry.dispatch: 'Unknown tool' error for trivially-normalizable CamelCase/_tool-suffix drift from Claude family [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#14186•Fetched 2026-04-23 07:46:20
View on GitHub
Comments
0
Participants
1
Timeline
7
Reactions
0
Author
JayGwodJayGwod
Participants
JayGwodJayGwod
Timeline (top)
labeled ×3cross-referenced ×2referenced ×2

The tool dispatch path in tools/registry.py::ToolRegistry.dispatch() rejects tool calls whose name field has cosmetic variations that are trivially resolvable, producing spurious Unknown tool: X errors and forcing the model into retry loops.

Observed variations from Claude 4.5 / 4.6 / 4.7:

Emitted nameCanonical target
TodoTool_tooltodo
Todo_tooltodo
Patch_toolpatch
BrowserClick_toolbrowser_click
Terminal_toolterminal
WriteFile_toolwrite_file

The drift pattern is CamelCase + trailing _tool suffix + mixed case. These are never ambiguous with any real tool name — all registered tools use lowercase snake_case.

Error Message

  • Observed repeatedly in production gateway logs on this installation (openclaw, PID 2473729), especially after a prior dispatch error — the retry often worsens the drift. dispatch() does an exact-match lookup against the registry. A miss returns {"error": "Unknown tool: X"} immediately, with no attempt to normalize.
  • Truly unknown tools still error — e.g. NotARealTool_tool → Unknown tool: NotARealTool_tool (verified). Live-verified after gateway reload: all 5 drift patterns above route to the correct handler; a control case (NotARealTool_tool) still returns the expected error.

Root Cause

dispatch() does an exact-match lookup against the registry. A miss returns {"error": "Unknown tool: X"} immediately, with no attempt to normalize.

Fix Action

Fix / Workaround

The tool dispatch path in tools/registry.py::ToolRegistry.dispatch() rejects tool calls whose name field has cosmetic variations that are trivially resolvable, producing spurious Unknown tool: X errors and forcing the model into retry loops.

Emitted nameCanonical target
TodoTool_tooltodo
Todo_tooltodo
Patch_toolpatch
BrowserClick_toolbrowser_click
Terminal_toolterminal
WriteFile_toolwrite_file
  • mastra-ai/mastra#12581 — empirical 44-model study: Claude family fails tool-name conformance at ~30%, other families ≈ 0%.
  • anthropic/anthropic-cookbook#50235 — upstream advisory recommending normalization at the dispatch boundary.
  • Observed repeatedly in production gateway logs on this installation (openclaw, PID 2473729), especially after a prior dispatch error — the retry often worsens the drift.

PR fix notes

PR #14188: fix(registry): normalize tool names at dispatch entry to tolerate LLM drift

Description (problem / solution / changelog)

Fixes #14186.

What

Adds _normalize_tool_name() and wires it into ToolRegistry.dispatch() as a miss-only fallback to recover from cosmetic drift in the name field emitted by Claude-family models.

Why

See #14186 for full context. TL;DR: CamelCase / trailing-_tool / mixed-case variants of correct tool names are emitted at ~30% by Claude 4.5+ (per mastra-ai/mastra#12581's 44-model study), producing spurious Unknown tool: X errors and retry loops.

Changes

tools/registry.py

  • New _normalize_tool_name(name) — fixpoint loop (bounded MAX_ITER=4) that applies:
    • strip trailing _tool suffix
    • CamelCase → snake_case
    • hyphen → underscore
    • lowercase
  • ToolRegistry.dispatch() — on exact-match miss, try _normalize_tool_name() once and log a warning if it recovers a handler.

Key invariants preserved

  • Exact match always wins — normalization only fires when the registry does not contain name verbatim. Zero impact on well-formed calls.
  • Fail-closed on truly unknown tools — NotARealTool_tool still returns Unknown tool: NotARealTool_tool.
  • Drift stays observable — logger.warning("Tool name normalized: %r -> %r (dispatch recovery)", ...) so ops can monitor model behavior.

Layered drift example

TodoTool_tool requires fixpoint iteration:

pass 1: strip _tool suffix -> "TodoTool"
pass 2: camel->snake       -> "todo_tool"
pass 3: strip _tool suffix -> "todo"        ✓ registry hit

Testing

Unit tests for _normalize_tool_name:

InputExpectedPass
TodoTool_tooltodo✅
Patch_toolpatch✅
BrowserClick_toolbrowser_click✅
Terminal_toolterminal✅
WriteFile_toolwrite_file✅
todo (already canonical)todo✅

End-to-end dispatch (on this installation's live gateway, PID 2473729):

dispatch(name, args)OutcomeJudgment
TodoTool_tool, {...}routes to todo handler✅ normalized
Patch_tool, {}path required (param validation, not unknown-tool)✅ normalized
NotARealTool_tool, {}Unknown tool: NotARealTool_tool✅ no false match

Risk

Low. ~100 LOC (normalizer + tests). Fallback-only wiring preserves existing behavior for well-formed calls. No dependency changes.

Related

Companion PR #<will-be-filled-after-pr1-merges> addresses the parameter-side of the same robustness theme (string-encoded todos).

Changed files

  • run_agent.py (modified, +26/-3)
  • tools/registry.py (modified, +94/-0)

PR #2: fix: apply P1 issue hardening (todo/registry/retry)

Description (problem / solution / changelog)

결론

GitHub P1 오픈 이슈 4건의 핵심 재현 케이스를 테스트로 먼저 추가하고, 최소 수정으로 런타임 방어 로직을 반영했습니다.

반영 이슈

변경 요약

  1. todo_tool 하드닝 (#14185)
  • todos가 문자열이면 JSON 파싱 시도
  • 파싱 실패 시 구조화 에러 반환
  • list 타입 검증 추가
  • _validate, _dedupe_by_id에서 non-dict 입력 방어
  1. registry dispatch 하드닝 (#14186)
  • _normalize_tool_name() 추가
  • CamelCase, _tool suffix, 구분자 드리프트를 정규화 fallback으로 처리
  • 예: TodoTool_tool -> todo, WriteFile_tool -> write_file
  1. run_agent 분류 수정 (#14271)
  • local validation fast-fail 분류에서 json.JSONDecodeError 제외
  • transient JSON 파싱 실패가 retry 경로를 타도록 보정
  1. error classifier 보강 (#14195)
  • _extract_error_body()가 __cause__/__context__ 체인을 따라 nested body를 추출하도록 수정
  • wrapped 402 오류에서 nested message("usage limit ... try again")를 잃지 않도록 보정
  • billing 오분류를 줄이고 transient rate_limit 분류 정확도 개선

테스트

  • scripts/run_tests.sh tests/tools/test_todo_tool.py tests/tools/test_registry.py tests/run_agent/test_run_agent.py::TestRetryExhaustion::test_jsondecode_error_is_retried_not_treated_as_local_validation
  • scripts/run_tests.sh tests/agent/test_error_classifier.py tests/tools/test_todo_tool.py tests/tools/test_registry.py tests/run_agent/test_run_agent.py::TestRetryExhaustion::test_jsondecode_error_is_retried_not_treated_as_local_validation

모두 통과했습니다.

비고

  • aideautomation/aide_hermes_agent 저장소는 코드 구조가 현재 hermes-agent와 상이하여 동일 커밋 체리픽이 불가했습니다.
  • 실행 가능한 반영은 aideautomation/hermes-agent 포크 기준으로 완료했습니다.

Changed files

  • agent/error_classifier.py (modified, +24/-13)
  • run_agent.py (modified, +1/-1)
  • tests/agent/test_error_classifier.py (modified, +23/-0)
  • tests/run_agent/test_run_agent.py (modified, +20/-0)
  • tests/tools/test_registry.py (modified, +24/-0)
  • tests/tools/test_todo_tool.py (modified, +23/-0)
  • tools/registry.py (modified, +43/-1)
  • tools/todo_tool.py (modified, +16/-2)

Code Example

pass 1: strip _tool suffix -> "TodoTool"
pass 2: camel->snake        -> "todo_tool"
pass 3: strip _tool suffix  -> "todo"        ✓ hit
RAW_BUFFERClick to expand / collapse

Summary

The tool dispatch path in tools/registry.py::ToolRegistry.dispatch() rejects tool calls whose name field has cosmetic variations that are trivially resolvable, producing spurious Unknown tool: X errors and forcing the model into retry loops.

Observed variations from Claude 4.5 / 4.6 / 4.7:

Emitted nameCanonical target
TodoTool_tooltodo
Todo_tooltodo
Patch_toolpatch
BrowserClick_toolbrowser_click
Terminal_toolterminal
WriteFile_toolwrite_file

The drift pattern is CamelCase + trailing _tool suffix + mixed case. These are never ambiguous with any real tool name — all registered tools use lowercase snake_case.

Evidence of prevalence

  • mastra-ai/mastra#12581 — empirical 44-model study: Claude family fails tool-name conformance at ~30%, other families ≈ 0%.
  • anthropic/anthropic-cookbook#50235 — upstream advisory recommending normalization at the dispatch boundary.
  • Observed repeatedly in production gateway logs on this installation (openclaw, PID 2473729), especially after a prior dispatch error — the retry often worsens the drift.

Root cause

dispatch() does an exact-match lookup against the registry. A miss returns {"error": "Unknown tool: X"} immediately, with no attempt to normalize.

Proposed fix

Add a _normalize_tool_name() helper and call it as a fallback (only on miss — exact match still wins). Layered drift like TodoTool_tool needs fixpoint iteration:

pass 1: strip _tool suffix -> "TodoTool"
pass 2: camel->snake        -> "todo_tool"
pass 3: strip _tool suffix  -> "todo"        ✓ hit

Bounded MAX_ITER = 4. Logs a warning when normalization recovers a call so drift stays visible.

Safety properties:

  • Zero impact on well-formed calls — fallback is never reached when exact match succeeds.
  • No false matches — normalization produces lowercase snake_case; collisions with real tool names would already have been exact matches.
  • Truly unknown tools still error — e.g. NotARealTool_tool → Unknown tool: NotARealTool_tool (verified).

PR

Will open from JayGwod/hermes-agent:fix/registry-tool-name-normalize.

Live-verified after gateway reload: all 5 drift patterns above route to the correct handler; a control case (NotARealTool_tool) still returns the expected error.

extent analysis

TL;DR

Implement a _normalize_tool_name() helper function in ToolRegistry.dispatch() to normalize tool names with cosmetic variations before lookup.

Guidance

  • Identify and strip the _tool suffix from the tool name.
  • Convert CamelCase to snake_case to match the registered tool names.
  • Implement a bounded iteration (e.g., MAX_ITER = 4) to handle layered drift.
  • Log a warning when normalization recovers a call to monitor drift.

Example

def _normalize_tool_name(name):
    for _ in range(MAX_ITER):
        name = name.removesuffix('_tool')  # strip _tool suffix
        name = ''.join(['_'+i.lower() if i.isupper() else i for i in name]).lstrip('_')  # camel->snake
        if name in ToolRegistry:  # check if normalized name is in registry
            return name
    return None  # return None if normalization fails

Notes

This solution assumes that the registered tool names are in lowercase snake_case and that the _tool suffix is only present in drifted tool names.

Recommendation

Apply the proposed fix by implementing the _normalize_tool_name() helper function as a fallback in ToolRegistry.dispatch() to handle cosmetic variations in tool names. This approach ensures zero impact on well-formed calls and prevents false matches.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#installation#conversation history#tool integration#LLM response#prompt template

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING