PR #11471: fix(dingtalk): support dingtalk-stream 0.24+ SDK (async process, CallbackMessage, oapi webhooks, TextContent)

• Repository: NousResearch/hermes-agent
• Author: teknium1
• State: closed | merged: True
• Link: https://github.com/NousResearch/hermes-agent/pull/11471

Description (problem / solution / changelog)

Summary

Salvages #11257 (kevinskysunny — authorship preserved) plus a follow-up fix discovered during E2E testing against the real dingtalk-stream 0.24.3 SDK.

What's broken on main

Our gateway/platforms/dingtalk.py was written against pre-0.20 dingtalk-stream. Four incompatibilities break DingTalk in production:

1. ChatbotHandler.process() is now async — ours was sync and used run_coroutine_threadsafe, so it never fires on the new SDK.
2. process() now receives a CallbackMessage envelope with a .data dict — ours expected a ChatbotMessage directly.
3. DingTalkStreamClient.start() is now a coroutine — asyncio.to_thread(self._stream_client.start) never awaits it.
4. Reply webhooks now come from oapi.dingtalk.com — our regex only allowed api.dingtalk.com, so every reply was silently rejected by the origin allowlist.

Items 1-4 close out a pile of duplicate PRs reporting the same root cause: #5038, #8477, #8954, #9131, #9764, #9828, #10153, #10369, #10820, #11257, plus issues #5037, #6986, #8811, #8816, #9149, #9752.

What kevinskysunny's commit fixes (cherry-picked as-is)

• _DINGTALK_WEBHOOK_RE → ^https://(?:api|oapi)\.dingtalk\.com/
• _run_stream awaits self._stream_client.start() directly
• _IncomingHandler.process becomes async and parses callback_message.data via ChatbotMessage.from_dict

Follow-up fix added on top (825b0fe5)

E2E testing against real dingtalk-stream==0.24.3 revealed _extract_text() was also broken by the SDK change:

Every text message received by the agent was coming in as the string TextContent(content=...) instead of the actual user message. Fix handles both shapes via hasattr(text, 'content') and falls back through legacy paths.

Tests

• Adds 13 new tests covering the webhook allowlist, async process(), and _extract_text() against the current SDK, the legacy SDK, and edge cases.
• Full tests/gateway/test_dingtalk.py: 29 passed.
• Full tests/gateway/: 3042 passed, 6 pre-existing failures in signal/telegram (unrelated to this PR).

E2E verification

Ran the adapter end-to-end against real dingtalk-stream==0.24.3:

PASS: _extract_text(real-SDK text msg) = 'hello world'
PASS: process() → _on_message → _extract_text = 'hello world'
PASS: oapi.dingtalk.com webhook passes origin validation
PASS: legacy dict-shaped text still extracted correctly

Authorship

Original commit preserved with [email protected] authorship — will merge with --rebase to keep attribution.

Supersedes / closes

Once merged, the following can be closed with credit:

• PRs: #5038, #8477, #8954, #9131, #9764, #9828, #10153, #10369, #10820, #11257 (this one), #8957, #9608, #10002, #9609, #10003, #7231 (origin validation — already on main)
• Issues: #5037, #6986, #8811, #8816, #9149, #9752, #11463 (pending confirmation)

Changed files

• gateway/platforms/dingtalk.py (modified, +38/-24)
• scripts/release.py (modified, +1/-0)
• tests/gateway/test_dingtalk.py (modified, +130/-0)

PR #9003: fix(dingtalk): fix multiple bugs in DingTalk platform adapter

• Repository: NousResearch/hermes-agent
• Author: yyq4193
• State: closed | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/9003

Description (problem / solution / changelog)

Summary

This PR fixes multiple bugs in the DingTalk platform adapter that prevented it from working correctly.

Bugs Fixed

1. Fix async stream client: Replace asyncio.to_thread(start) with await start() — the SDK's start() is an async method, not a blocking function. Using asyncio.to_thread() creates a coroutine that is never awaited, so the WebSocket loop never starts.

2. Make _IncomingHandler.process an async method: The SDK's raw_process() does await self.process(). A sync method returning a tuple cannot be awaited, causing object tuple can't be used in 'await' expression.

3. Convert CallbackMessage.data to ChatbotMessage: The SDK passes a CallbackMessage wrapper. The actual chatbot payload is in callback_message.data as a dict that must be converted via ChatbotMessage.from_dict().

4. Use fire-and-forget pattern for message processing: process() must return quickly so the SDK can ACK to DingTalk server. Blocking on _on_message causes the SDK to miss server heartbeats and disconnect.

5. Fix session_webhook regex to match oapi.dingtalk.com: DingTalk uses oapi.dingtalk.com for session webhooks, not api.dingtalk.com.

6. Fix log level for inbound messages: Changed from debug to info for easier tracing.

7. Fix platform_toolsets None AttributeError in tools_config.py: When platform_toolsets is an empty YAML key, config.get() returns None instead of empty dict, causing AttributeError on .get(platform).

Testing

All fixes have been tested and verified — DingTalk now connects stably and correctly processes/replies to messages.

Changed files

• gateway/platforms/dingtalk.py (modified, +19/-17)
• hermes_cli/tools_config.py (modified, +1/-1)

PR #11509: fix(dingtalk): convert callback payloads before dispatch

• Repository: NousResearch/hermes-agent
• Author: LeonSGP43
• State: closed | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/11509

Description (problem / solution / changelog)

Summary

• convert DingTalk callback wrappers into ChatbotMessage instances before handing them to the async adapter
• keep already-converted messages untouched so existing SDK flows still work
• add regression coverage for callback conversion and session webhook reply routing

Testing

• pytest -q -o addopts= tests/gateway/test_dingtalk.py

Closes #11463

Changed files

• gateway/platforms/dingtalk.py (modified, +24/-1)
• tests/gateway/test_dingtalk.py (modified, +132/-0)

PR #11518: fix(dingtalk): fire-and-forget message processing & session_webhook fallback

• Repository: NousResearch/hermes-agent
• Author: kagura-agent
• State: closed | merged: False
• Link: https://github.com/NousResearch/hermes-agent/pull/11518

Description (problem / solution / changelog)

Summary

Fixes #11463: DingTalk channel receives messages but fails to reply with No session_webhook available.

Changes

1. Fire-and-forget message processing

_IncomingHandler.process() now dispatches _on_message as a background task via asyncio.create_task() instead of awaiting it directly. This ensures the SDK ACK is returned immediately, preventing heartbeat timeouts and WebSocket disconnections when message processing takes longer than the SDK's ACK deadline.

2. session_webhook extraction fallback

If ChatbotMessage.from_dict() fails to map the sessionWebhook field (possible across SDK versions where the camelCase key name differs), the handler now falls back to extracting it directly from the raw callback data dict using both sessionWebhook and session_webhook key variants.

Testing

Added 3 new tests:

• test_process_extracts_session_webhook — verifies webhook is populated from callback data
• test_process_fallback_session_webhook_when_from_dict_misses_it — verifies fallback extraction
• test_process_returns_ack_immediately — verifies ACK returns before message processing completes

All 31 DingTalk tests pass.

Changed files

• gateway/platforms/dingtalk.py (modified, +31/-5)
• tests/gateway/test_dingtalk.py (modified, +106/-0)

PR #11605: fix(dingtalk): get_connected_platforms + fire-and-forget processing + null-toolsets guard

• Repository: NousResearch/hermes-agent
• Author: teknium1
• State: closed | merged: True
• Link: https://github.com/NousResearch/hermes-agent/pull/11605

Description (problem / solution / changelog)

Summary

Three DingTalk follow-up fixes in one PR — all from external contributors, all cherry-picked with authorship preserved:

1. #11500 @youngDoo — GatewayConfig.get_connected_platforms() was missing a DingTalk branch entirely. A DingTalk-configured gateway (via YAML extra: or env vars) never appeared in the connected-platforms list, so status displays and iteration callers silently omitted it.

2. #11518 @kagura-agent — _IncomingHandler.process() currently awaits _on_message directly, which blocks the SDK's recv loop for the full duration of agent processing. For a chat agent responding in 10-30s, this breaks the SDK's heartbeat deadline and causes WebSocket disconnects. Fix: dispatch via asyncio.create_task() so ACK returns immediately. Also adds a defensive session_webhook fallback (raw dict lookup for both sessionWebhook and session_webhook keys) in case a future SDK revision changes the field name. Resolves issue #11463 (@sgjeff's "No session_webhook available" report).

3. #9003 @yyq4193 (one-liner cherry-picked with Co-authored-by trailer) — hermes_cli/tools_config.py was calling config.get("platform_toolsets", {}), which returns None when the YAML key is explicitly null (common with platform_toolsets: and no value below). The next line's .get(platform) then crashed with AttributeError. Changed to config.get(...) or {}.

The rest of #9003 is redundant with #11471 that landed earlier today (webhook regex, async start(), async process(), CallbackMessage → ChatbotMessage) and includes one security regression (https?:// allowing plain HTTP on the webhook allowlist) that I deliberately did not carry over. The tools_config.py one-liner is the only net-new legitimate change from #9003 and I'm landing it here with credit.

Commits (all authorship preserved)

726bea34  Teknium (Co-authored-by: yyq4193)
          test(dingtalk): cover get_connected_platforms + null platform_toolsets
973e0128  kagura-agent
          fix(dingtalk): fire-and-forget message processing & session_webhook fallback
0d2a845f  youngDoo
          gateway cant add DingTalk platform

Merge with --rebase to preserve per-commit authorship.

What I cleaned up

• gateway/config.py: stripped ~140 trailing whitespace characters on the new DingTalk branch line from @youngDoo's diff.
• Resolved a cherry-pick conflict in tests/gateway/test_dingtalk.py — kagura-agent's branch predated the group-gating test classes we merged earlier today via #11564, and both sides appended new test classes at EOF. Kept both.

What I deliberately dropped from #9003

• r'^https?://(api|oapi)\.dingtalk\.com/' — this would re-allow plain HTTP on the webhook URL allowlist. Current main enforces ^https:// only and this is the right security posture. Rejected as a regression.
• Pre-#11471 SDK compat changes (async start(), async process(), CallbackMessage → ChatbotMessage, webhook regex oapi accept) — already on main.
• logger.debug → logger.info for inbound messages — debatable noise; every received message would land in agent.log at INFO. Not carrying over.

Tests

• tests/gateway/test_dingtalk.py — 50 passed (includes kagura-agent's 3 new TestIncomingHandlerProcess tests + our earlier regression tests)
• tests/gateway/test_config.py — 28 passed (includes 4 new TestGetConnectedPlatforms::test_dingtalk_* tests I added)
• tests/hermes_cli/test_tools_config.py — 31 passed (includes test_get_platform_tools_handles_null_platform_toolsets regression test)
• Combined: 109 passed

Closes

Closes #11500, #11518, #9003 on merge. Should also resolve #11463 (pending @sgjeff pulling latest main).

Changed files

• gateway/config.py (modified, +8/-0)
• gateway/platforms/dingtalk.py (modified, +31/-5)
• hermes_cli/tools_config.py (modified, +1/-1)
• scripts/release.py (modified, +2/-0)
• tests/gateway/test_config.py (modified, +45/-0)
• tests/gateway/test_dingtalk.py (modified, +106/-0)
• tests/hermes_cli/test_tools_config.py (modified, +13/-0)