hermes - 💡(How to fix) Fix [Feature]: In batch processing for dataset generation, support using pre-exiting configured instances or sharing the instance with a set of prompts

Problem or Use Case

While reading batch-processing I noticed a few things.

1. The dataset doesn't support chaining multiple prompt in the same environment.

2. A docker runtime image is possible but it's not possible to re-use a existing Hermes configured environment.

Looking on HF, there are a few datasets generated with the current implementation. So it works. But if we want to enable a model to have enhanced capabilities running Hermes-Agent, we should provide it with a less vanilla flavored environment.

The dataset generation is missing a phase 2 : running Hermes-Agent with customization.

1. Also, hermes-agent has other features that run in background and uses the llm, but they are not captured.

I am submitting this as an idea as I really liked Hermes models and hope to see a Hermes-V5-Agent dataset + new V5 models. It's a pity that the NousResearch portal doesn't host a Hermes model for Hermes-Agent as API endpoint. Something like a fine tune of Qwen3.5 120B or Nemotron-3-Super specific for Hermes agent at a low price would be great ! And a A3B for local small r/LocalLLaMA while the 120B would be for Strix/DGX.

Proposed Solution

1. supporting prompts instead of prompt could be a solution. Providing a /goal and /subgoal that produce a finalized action (like sending a compilation of news to a messaging number OR writing a file at a certain place) could also work and move more to a IFeval type where the expected final action is checked against expectations (or llm judge).

2. the ~/.hermes could be passed as a copy to the docker container or it could be passed as a git repository (+commit_hash|branch).

• if it's a git repository, it would be possible to commit to the repository at the end of the task. That way the environment would evolve : new skills, new capabilities, new cron jobs, kanban use, different memory systems, communicating with bots on messaging platforms ...

• Building on the above, it would be possible to build and maintain many different hermes environment centered around different persona/interest/goals/use case. It would result in more diversification of the training data.

• Special keys, url for setting up services could be passed as a .env key + instructions about it. The git repository should encrypt/decrypt with git smudge filter.

• Could include giving nudges/hints when execution failed; or providing a more complete implementation plan when the task is complex. A task specification plan could be provided to a model (Hermes 4 70B?) and it would have to monitor Hermes responses and answer any question it has by giving it the answer in the plan but in very short sentences (lazy human mode) or be very critique on one feature if what was made doesn't look like what is in the plan (Nitpicker human mode) or ask for complete rework (unsatisfied human mode).

1. If possible, it could also be useful to capture additional communications for smaller models, if 2) is implemented and an environment is running in batch mode for a long time, it could help training small models for compression.

HINT: If the git repository is public (aka NousResearch/hermes-agent-dev-configs) users could submit /goal and /subgoal as issues and provide expected result. An agent could look opened issues and run safe and non problematic /goals. When the task is done, have the user review (or an other) his branch and put comments. Users could use it to test things or see how it could be implemented and we are getting traces + evaluations. The comments could be used to evaluate what didn't go as expected. Only one issue per long time git user; triage for similarity or duplication -> close issue by agent. Good user feedback -> +1 issue slot for him (at the end of queue). The git could host different configurations (vanilla, memory plugin enabled, sport enthusiast, stock exchange news aggregator...) and have the user select the desired base Hermes config for their prompt. Their issue could also create a new configuration; long lived configurations that evolve (keep selected /goal artifacts in the configured Herme profile) could simulate real use cases that accumulate skills and has curation process running. Also keeping the profile in github could enable the user to inspect the final state of the task, even generating correcting prompts for any issues on his end then have a /resume-my-fix command that would print a detailed and annotated report that can be pasted in the issue to enable the dataset creation to continue with the fix proposed by the user. Also this could be automated at a later stage and use Hermes 405b as judge of task competition and propose follow up commands...

Alternatives Considered

Run Hermes-Agent behind a logging proxy.

You should host one or more free model (rotating stealth?) on the portal and log usage for dataset generation ! Many would use it for non sensitive information. You could always anonymize with a PII model the conversations before training/publishing it. Control API usage by giving special credits keys to users on the service for 1 month : uncapped usage of the free models at 2-4 requests active (give enough key to fill a server, lower concurrency when pressure is too much for server).

Feature Type

Other

Scope

Medium (few files, < 300 lines)

Contribution

• I'd like to implement this myself and submit a PR

Debug Report (optional)