hermes - 💡(How to fix) Fix [Bug] pre_api_request hook passes messages but test asserts it should not

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…

Root Cause

The test asserts "response" not in c, which is satisfied because the key is assistant_message, not response. However, passing a raw SDK object to plugins may be unintended — it leaks the internal response representation and makes the hook API harder to stabilize. Consider passing a serializable dict (e.g. {"content": ..., "tool_calls": ..., "role": ...}) instead.

Fix Action

Fix / Workaround

  • Agent Core (conversation loop / hooks dispatch)

Code Example

_invoke_hook(
    "pre_api_request",
    task_id=effective_task_id,
    session_id=self.session_id or "",
    platform=self.platform or "",
    model=self.model,
    provider=self.provider,
    base_url=self.base_url,
    api_mode=self.api_mode,
    api_call_count=api_call_count,
    messages=api_messages,                 # <-- full message list is passed
    message_count=len(api_messages),
    tool_count=len(self.tools or []),
    approx_input_tokens=approx_tokens,
    request_char_count=total_chars,
    max_tokens=self.max_tokens,
)

---

assert all("message_count" in c and "messages" not in c for c in pre_request_calls)

---

_invoke_hook(
    "post_api_request",
    ...
    assistant_message=assistant_message,
    ...
)
RAW_BUFFERClick to expand / collapse

Problem Description

There is an inconsistency between the implementation of the pre_api_request lifecycle hook in run_agent.py and the corresponding test in tests/run_agent/test_run_agent.py.

Current Implementation (run_agent.py ~L11786)

_invoke_hook(
    "pre_api_request",
    task_id=effective_task_id,
    session_id=self.session_id or "",
    platform=self.platform or "",
    model=self.model,
    provider=self.provider,
    base_url=self.base_url,
    api_mode=self.api_mode,
    api_call_count=api_call_count,
    messages=api_messages,                 # <-- full message list is passed
    message_count=len(api_messages),
    tool_count=len(self.tools or []),
    approx_input_tokens=approx_tokens,
    request_char_count=total_chars,
    max_tokens=self.max_tokens,
)

Current Test Assertion (tests/run_agent/test_run_agent.py ~L2491)

assert all("message_count" in c and "messages" not in c for c in pre_request_calls)

The test explicitly expects messages NOT to be present in the hook kwargs, but the implementation passes the full api_messages list.

Impact

  • The test test_request_scoped_api_hooks_fire_for_each_api_call will fail if run against current main.
  • Passing the full messages list to every pre_api_request hook can be expensive:
    • Large conversation histories get serialized for every plugin callback.
    • Plugins that only need metadata (message count, token estimates) pay an unnecessary penalty.

Affected Component

  • Agent Core (conversation loop / hooks dispatch)

Possible Fixes

Option A — Remove messages from hook kwargs (align with test)

  • Delete messages=api_messages from the invoke_hook("pre_api_request", ...) call site.
  • Plugins that need the actual message content can rely on message_count + approx_input_tokens + request_char_count.
  • This keeps the hook payload lightweight and matches the existing test contract.

Option B — Update the test to expect messages

  • Change the assertion to allow "messages" in c.
  • Document that pre_api_request receives the full message list.
  • Consider whether this has performance implications for plugins.

Option C — Pass a truncated / summary version of messages

  • Instead of the full message objects, pass a lightweight summary (e.g. last message role + content preview).
  • Gives plugins more context than pure metadata without the full serialization cost.

Related Code

  • run_agent.py: pre_api_request invocation (~L11774-L11794)
  • run_agent.py: post_api_request invocation (~L13653-L13677)
  • tests/run_agent/test_run_agent.py: test_request_scoped_api_hooks_fire_for_each_api_call (~L2461-L2492)
  • hermes_cli/hooks.py: _DEFAULT_PAYLOADS["pre_api_request"] does not include messages, which is consistent with the test but inconsistent with the implementation.

Additional Note on post_api_request

The post_api_request hook currently passes the raw assistant_message object:

_invoke_hook(
    "post_api_request",
    ...
    assistant_message=assistant_message,
    ...
)

The test asserts "response" not in c, which is satisfied because the key is assistant_message, not response. However, passing a raw SDK object to plugins may be unintended — it leaks the internal response representation and makes the hook API harder to stabilize. Consider passing a serializable dict (e.g. {"content": ..., "tool_calls": ..., "role": ...}) instead.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - 💡(How to fix) Fix [Bug] pre_api_request hook passes messages but test asserts it should not