hermes - 💡(How to fix) Fix Add schema-bound LLM classifier for Gmail candidates [1 comments, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#16837Fetched 2026-04-29 06:38:38
View on GitHub
Comments
1
Participants
1
Timeline
2
Reactions
0
Author
Participants
Timeline (top)
closed ×1commented ×1
RAW_BUFFERClick to expand / collapse

Issue: Add schema-bound LLM classifier for Gmail candidates

Feature Description

Add an internal classifier boundary that lets a Gmail domain extractor ask an LLM whether a bounded candidate message or thread is relevant to the user's request. The classifier must return strict JSON and include grounded evidence.

Motivation

Deterministic scoring is useful for candidate pruning, but the LLM should make semantic relevance decisions. This is required for ambiguous requests such as "emails related to closing" where keyword matching alone confuses real estate closings with account closing, deal closing, or unrelated business language.

Proposed Solution

  • Define a GmailCandidateClassification JSON schema.
  • Add an internal LLM call boundary that does not recursively start a full agent loop.
  • Classify candidates after metadata ranking and before ledger writes.
  • Validate model output against source text.
  • Preserve weak matches separately from accepted records.
  • Add benchmark fixtures for travel and real estate false positives.

Acceptance Criteria

  • Classifier returns JSON with is_relevant, confidence, reason, evidence, entities, dates, amounts, identifiers, and next_actions.
  • Non-JSON model output is retried or rejected.
  • Evidence must be grounded in candidate content.
  • Local-model benchmark reports precision, recall, latency, refusal rate, and token usage where available.

extent analysis

TL;DR

Implement a schema-bound LLM classifier for Gmail candidates to improve semantic relevance decisions.

Guidance

  • Define the GmailCandidateClassification JSON schema to ensure structured output from the LLM classifier.
  • Integrate the LLM call boundary after metadata ranking and before ledger writes to classify candidates effectively.
  • Validate model output against source text to ensure accuracy and relevance.
  • Implement benchmark fixtures for common false positive scenarios, such as travel and real estate, to test the classifier's performance.

Example

No code snippet is provided as the issue focuses on the design and implementation of the classifier rather than specific code details.

Notes

The success of this implementation depends on the quality of the LLM model and the accuracy of the GmailCandidateClassification schema. Thorough testing and validation are necessary to ensure the classifier meets the acceptance criteria.

Recommendation

Apply the proposed solution to implement the schema-bound LLM classifier, as it addresses the need for semantic relevance decisions and provides a structured approach to candidate classification.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING