hermes - 💡(How to fix) Fix Proposal: dynamic pricing source instead of hand-maintained _OFFICIAL_DOCS_PRICING snapshots [1 pull requests]

Problem

agent/usage_pricing.py:83 (_OFFICIAL_DOCS_PRICING) is a static dict that has to be updated by hand every time:

• A new provider gets used by anyone (Fireworks, MiniMax, Bedrock — recent PRs reflect this).
• A new model from an existing provider becomes the recommended default.
• A provider changes pricing (Anthropic and OpenAI have done this multiple times since hermes shipped).

Each entry carries its own pricing_version="provider-pricing-YYYY-MM-DD" string, which makes it visible that the snapshots drift independently and require manual refresh PRs. The result is a long tail of "Estimated Cost: N/A" sessions for anyone running a provider that hasn't yet been PR'd in, and silent drift for providers that have been but haven't been refreshed.

Filed PR #29860 adds Fireworks entries as the latest stopgap. This issue proposes a permanent fix to the underlying PR-per-model cycle.

Proposal

Add a pluggable dynamic pricing source layer that supplements (not replaces) _OFFICIAL_DOCS_PRICING. The static dict stays as the trusted-overrides layer; dynamic sources fill in the long tail.

Sources worth considering, ranked by maintenance burden / coverage:

1. LiteLLM's model_prices_and_context_window.json — MIT-licensed, community-maintained, covers most major providers including Fireworks-hosted models, updated very frequently. Single HTTP GET to fetch, easy to cache locally. Schema differs from PricingEntry but the mapping is mechanical.
2. OpenRouter /models endpoint — already integrated in hermes-agent via _openrouter_pricing_entry for provider="openrouter" flows. Could be reused for non-OpenRouter providers if we add a name-mapping layer (e.g. accounts/fireworks/models/kimi-k2p6 → moonshotai/kimi-k2.6). Most invasive but most "official."
3. Provider /v1/models endpoints — only works for providers whose responses include pricing (some custom OpenAI-compat endpoints do; Fireworks, Anthropic, OpenAI do NOT, per my checks). Already supported for provider="custom" via fetch_endpoint_model_metadata + _pricing_entry_from_metadata. Generalize to other providers where applicable.

Suggested architecture

def get_pricing_entry(...):
    # 1. Hard-coded snapshot (trusted overrides) — same as today
    entry = _lookup_official_docs_pricing(route)
    if entry: return entry

    # 2. Configured dynamic sources, tried in order
    for source in _dynamic_pricing_sources():
        entry = source(route)
        if entry: return entry

    # 3. Existing fallbacks (openrouter, custom-endpoint /models) — same as today
    ...

Configuration could live in ~/.hermes/config.yaml:

pricing:
  sources:
    - kind: litellm_json
      url: https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
      cache_ttl_hours: 24
    - kind: openrouter_models_api  # reuse existing _openrouter_pricing_entry

…with sensible defaults so most users don't have to configure anything. The static dict becomes the explicit overrides layer for cases where the dynamic source is wrong or missing.

Migration path

• Phase 1: add the dynamic-source plumbing; no behavior change unless a user opts in via config.
• Phase 2: ship a default config that enables LiteLLM as a low-priority source. Existing static entries continue to win.
• Phase 3 (optional): prune the static dict for entries that LiteLLM tracks reliably, keeping the dict for genuine overrides / providers without community coverage.

Why this matters

Three of the last ~10 merged PRs to usage_pricing.py (by my count of recent commits) have been pricing-snapshot additions or refreshes. None of those PRs improved the underlying code — they just kept the dict from going stale. Dynamic sourcing would let the project absorb new providers without code review overhead, and would prevent the silent-drift problem on existing entries.

Happy to take a first cut at this if maintainers think it's a direction they'd accept — would appreciate guidance on (a) preferred dynamic source for the default config and (b) whether the static dict should remain as overrides vs. be deprecated entirely.

Refs: PR #29860 (latest snapshot-add; intended as a stopgap until something like this lands).