hermes - 💡(How to fix) Fix [Feature]: the deadly 84k tokens [1 participants]

Problem or Use Case

i want to unpack some observations and questions that landed in support

1. [where] does hermes go quadratic in python?
2. glm has had a bad run since v9
3. non-zero-temperature and the models that crumble: i anecdotally determined that playing with temperature has been the cause of seeing preview models unravel into poisoned degradation.

Proposed Solution

Hermes already has some related behavior here around fallback providers and model-specific handling,

suggesting — broader temperature policy, repeat/stall mitigation strategy, provider-health-driven switching

...so i recently hammered on hermes to tweak the model defaults to zero based on a lot of faint signals over time. my friend never had cracking preview models and I did, having set temps on kimi and others.

this was to spark a conversation about what amounts to assigning tokens to goal seeking performance metrics autotuning. a simple dht gossip could mapreduce to the interesting features building confidence scores and directing analysis of model covariance anomalies for example. broadcasting the mean on dht should cost nothing with a simple codec predictor logic.

Alternatives Considered

dumping cores to analyze catastrophes is not in the same zipcode. git is probably instrumental for that.

A simple dht doesn't need large user-base to profile the general performance norms across multiple dimensions. has security feature overlap as well. Happy to help!

Feature Type

Performance / reliability

Scope

Small (single file, < 50 lines)

Contribution

• I'd like to implement this myself and submit a PR

Debug Report (optional)