hermes - 💡(How to fix) Fix [Feature]: Skill Management — Usage Tracking, Conflict Detection, and Pre-Creation Validation [1 participants]

[Feature]: Skill Management — Usage Tracking, Conflict Detection, and Pre-Creation Validation

Background

After managing 146+ skills in a production hermes-agent setup, we identified several systemic issues with the current skill system that lead to skill proliferation, wasted context tokens, and difficulty maintaining a clean skill library.

Current Problems

1. No Usage Tracking

Skills have no built-in invocation counter or last-used timestamp. There's no way to know:

• How many times a skill has been loaded via skill_view
• When a skill was last used
• Which skills are "dead weight" (never used since installation)

Impact: Accumulated unused skills waste available_skills block tokens on every turn.

2. No Conflict/Dedup Detection

There's no mechanism to detect:

• Name conflicts (two skills with the same name field in different directories)
• Description overlap (skills covering the same use case with different implementations)
• Ecosystem clustering (many skills around the same tool, e.g., 6 skills for Obsidian, 6 for OCR)

Real example from our setup: superdesign had name: frontend-design — same as another skill. This was only discovered during manual audit.

3. No Pre-Creation Validation

When skill_manage(action='create') is called, there's no built-in check for:

• Does a skill with this name already exist?
• Does the description overlap significantly with existing skills?
• Could this be a patch to an existing skill instead of a new creation?

Result: Agents create skills freely without any guardrails, leading to duplicates.

4. Linear Token Growth

The available_skills block (name + description for every skill) is injected into the system prompt on every turn. Each skill costs ~120-200 characters (~30-50 tokens).

Without cleanup or merging, this grows indefinitely as agents auto-create skills.

5. No Tool Call Hooks

There's no way to intercept skill_manage(action='create') calls for pre-processing. The only enforcement mechanism is relying on the agent's SOUL.md instructions — which is fragile and depends on agent compliance.

Proposed Solutions

A. Skill Usage Metadata

Add optional tracking to the skill system:

// ~/.hermes/skill-usage.json (auto-maintained)
{
  "skill-name": {
    "total_uses": 15,
    "last_used": "2026-04-21T19:00:00+08:00",
    "first_used": "2026-03-01T10:00:00+08:00"
  }
}

Updated automatically when skill_view loads a skill. Exposed via hermes skills stats command.

B. Pre-Creation Validation Hook

Add an optional pre-flight check when skill_manage(action='create') is called:

• Check for name collisions
• Compute description similarity (TF-IDF or embedding-based)
• Return warnings/blocks to the agent before creation
• Configurable via skills.creation_requires_approval: true in config.yaml

C. hermes skills audit Enhancement

Extend the existing hermes skills audit command to:

• Detect name conflicts and description overlaps
• Report unused skills (based on usage metadata)
• Suggest merges for highly overlapping skills
• Output JSON for programmatic consumption

D. Lazy Loading Option

Option to load available_skills on-demand rather than every turn:

skills:
  loading: lazy  # or "eager" (current default)

With lazy loading, skills are only discoverable via hermes skills search rather than being in the system prompt. Reduces baseline token usage significantly.

Workaround (Currently Implemented)

We built external scripts as a workaround:

• skill-conflict-detector.py — scans SKILL.md files for conflicts
• skill-usage-tracker.py — manual usage tracking
• skill-gatekeeper.py — pre-creation check script

These work but require agent compliance (calling them before skill_manage). Native support would be far more reliable.

Environment

• hermes-agent: latest (2026-04)
• Skills installed: 134 (after manual cleanup from 146)
• Platform: Ubuntu 25.10, headless server
• Usage: Multi-agent setup (3 worker profiles) with heavy skill usage