Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix fix(curator/cron): detect stale skill references in cron prompt text after consolidation [1 pull requests, 1 comments, 2 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#23398•Fetched 2026-05-11 03:29:41
View on GitHub
Comments
1
Participants
2
Timeline
6
Reactions
0
Author
SleepyBanditSleepyBandit
Participants
konsisumerkonsisumerSleepyBanditSleepyBandit
Timeline (top)
labeled ×4commented ×1cross-referenced ×1

After curator consolidation, Hermes rewrites cron skills / skill fields but can leave stale skill-name references inside the cron job's freeform prompt.

This produces a partial migration:

  • loaded skill metadata points to the new umbrella skill
  • prompt text can still instruct the agent to follow the old skill name

That can silently change cron behavior without causing an obvious failure.

Error Message

  1. Warn when a cron prompt still references a renamed or unloaded skill after consolidation

Root Cause

This is a silent consistency bug:

  • the cron job still runs
  • the new umbrella skill may still load
  • but the prompt can remain semantically stale

That makes the resulting behavior drift hard to notice and hard to debug.

Fix Action

Fixed

PR fix notes

PR #23433: fix(cron): warn on stale skill refs in cron prompt text after consolidation

Description (problem / solution / changelog)

After curator consolidation the cron rewrite path updated structured skills / skill fields but left freeform prompt text untouched, so a job's loaded skill could silently disagree with what its prompt named.

What changed and why

  • cron.jobs.rewrite_skill_refs now scans each job's prompt for token-level mentions of any skill listed in consolidated or pruned. Matches are recorded as prompt_warnings on each rewrite entry (with the forwarding target, or None when the skill was pruned). The prompt itself is not auto-rewritten — natural language is too ambiguous to munge safely, so the safer path is to surface the drift.
  • The report gains a top-level prompt_warning_jobs count; warning-only entries are included in rewrites so jobs whose structured fields were already clean but whose prompts still mention an old name still get flagged.
  • A custom boundary check (letters, digits, -, _ as continuation) keeps the match conservative — e.g. legacy is flagged in Follow `legacy` or Use legacy. but not inside my-legacy-skill or legacy_v2.
  • agent.curator._write_run_report now surfaces these warnings in REPORT.md and writes cron_rewrites.json when warnings exist even if no job was structurally rewritten; the run payload exposes counts.cron_prompt_warning_jobs.

How to test

  • pytest tests/cron/test_rewrite_skill_refs.py tests/agent/test_curator_reports.py tests/agent/test_curator_backup.py -q --timeout=60 — 57 pass locally.
  • New tests live in tests/cron/test_rewrite_skill_refs.py::TestRewriteSkillRefsPromptWarnings, covering: consolidated mention emits a warning with target; pruned mention emits a warning with target=None; structural-clean but prompt-dirty jobs are still surfaced; substring/embedded-token false positives are rejected; multiple stale refs in one prompt are listed in stable order.

What platforms tested on

  • macOS on darwin-arm64 (local)

Fixes #23398

<!-- autocontrib:worker-id=issue-new-776082bc kind=pr-open -->

Changed files

  • agent/curator.py (modified, +45/-6)
  • cron/jobs.py (modified, +96/-11)
  • tests/cron/test_rewrite_skill_refs.py (modified, +137/-1)

Code Example

{
  "skills": ["old-skill"],
  "prompt": "Follow `old-skill` as the canonical spec."
}

---

{
  "skills": ["umbrella-skill"],
  "prompt": "Follow `old-skill` as the canonical spec."
}
RAW_BUFFERClick to expand / collapse

Summary

After curator consolidation, Hermes rewrites cron skills / skill fields but can leave stale skill-name references inside the cron job's freeform prompt.

This produces a partial migration:

  • loaded skill metadata points to the new umbrella skill
  • prompt text can still instruct the agent to follow the old skill name

That can silently change cron behavior without causing an obvious failure.

Not the same as #18671

This is related to, but distinct from, #18671.

  • #18671: stale structured cron skill references (skills[] / skill) after consolidation
  • This issue: stale textual skill references inside job["prompt"] after the structured fields were already rewritten

So this is not a duplicate. It is the remaining prompt-text gap after the metadata-side fix.

Current behavior

Example before consolidation:

{
  "skills": ["old-skill"],
  "prompt": "Follow `old-skill` as the canonical spec."
}

If curator consolidates old-skill into umbrella-skill, the rewritten job can become:

{
  "skills": ["umbrella-skill"],
  "prompt": "Follow `old-skill` as the canonical spec."
}

The job now loads one skill while the prompt still names another.

Expected behavior

Hermes should do at least one of these:

  1. Safely rewrite exact prompt references to renamed skills when unambiguous
  2. Warn when a cron prompt still references a renamed or unloaded skill after consolidation
  3. Surface that warning in the curator / cron rewrite report

Why this matters

This is a silent consistency bug:

  • the cron job still runs
  • the new umbrella skill may still load
  • but the prompt can remain semantically stale

That makes the resulting behavior drift hard to notice and hard to debug.

Likely root cause

The cron rewrite path updates structured skill fields but does not inspect or validate job["prompt"].

Suggested regression test

Add a test where:

  • a cron job starts with skills=["old-skill"]
  • the prompt contains Follow \old-skill` as the canonical spec.`
  • old-skill is consolidated into umbrella-skill

Assert that Hermes either:

  • rewrites the prompt reference, or
  • emits an explicit warning/report about the stale prompt reference

Related

  • #18671
  • follow-up fixes for that issue handled structured cron skill metadata, but not freeform prompt text

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Hermes should do at least one of these:

  1. Safely rewrite exact prompt references to renamed skills when unambiguous
  2. Warn when a cron prompt still references a renamed or unloaded skill after consolidation
  3. Surface that warning in the curator / cron rewrite report
#memory management#API rate limit#retriever error#indexing error#inference speed

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix fix(curator/cron): detect stale skill references in cron prompt text after consolidation [1 pull requests, 1 comments, 2 participants]