hermes - 💡(How to fix) Fix model.max_tokens from config.yaml is ignored in both Gateway and CLI modes [1 comments, 1 participants]

Bug Description

model.max_tokens set in ~/.hermes/config.yaml has no effect — it is never read or passed to AIAgent in either Gateway or CLI mode.

Steps to Reproduce

1. Set model.max_tokens in config.yaml:

model:
  default: aws.claude-sonnet-4.6
  provider: custom:friday
  max_tokens: 128000

1. Use Hermes via Gateway (API server / messaging platforms) or CLI (hermes chat)

2. The upstream API receives no max_tokens parameter in the request

Expected Behavior

The configured model.max_tokens should be passed to the AIAgent constructor and included in API requests as max_tokens (or max_completion_tokens for direct OpenAI).

Actual Behavior

• Gateway mode (gateway/run.py): _resolve_runtime_agent_kwargs() only extracts provider-related fields (api_key, base_url, provider, api_mode, etc.) — max_tokens is not included.
• CLI mode (cli.py): AIAgent(...) is constructed without max_tokens at line ~2872. No code reads model.max_tokens from config.
• AIAgent.__init__ defaults to max_tokens=None (line 799 of run_agent.py)
• When max_tokens is None, _build_api_kwargs() skips adding it to the request (line ~6563)

Impact

Most providers default to a reasonable output limit when max_tokens is omitted. However, some providers (e.g., AWS Bedrock proxied through custom endpoints) default to very low values like 1024 tokens, causing:

• Tool call arguments getting truncated (finish_reason: length)
• Long responses cut short (e.g., multi-step analysis, structured JSON output)
• The agent entering a truncation → retry → give up loop, returning "Response truncated due to output length limit"

Affected Code Paths

1. gateway/run.py:_resolve_runtime_agent_kwargs() (line ~319) — does not read model.max_tokens from config
2. gateway/run.py:_resolve_turn_agent_config() (line ~967) — primary dict and route["runtime"] do not include max_tokens
3. cli.py (line ~2872) — AIAgent(...) constructor call missing max_tokens=
4. All other AIAgent(...) instantiation points in gateway/run.py (lines ~5690, ~5871, ~8572)

Suggested Fix

Read model.max_tokens from config and pass it through to AIAgent:

# In _resolve_runtime_agent_kwargs() or wherever config is loaded:
cfg = _load_gateway_config()
model_cfg = cfg.get("model", {})
if isinstance(model_cfg, dict):
    max_tokens = model_cfg.get("max_tokens")
    if max_tokens is not None:
        result["max_tokens"] = int(max_tokens)

And ensure max_tokens flows through resolve_turn_route() → route["runtime"] → AIAgent(max_tokens=...).

Environment

• Hermes v0.9.0 (2026.4.13)
• Provider: custom:friday (AWS Bedrock proxy via https://aigc.sankuai.com/v1/openai/native)
• Model: aws.claude-sonnet-4.6
• OS: macOS (arm64)

Workaround

Manually patch gateway/run.py to bridge model.max_tokens from config into runtime_kwargs and turn_route["runtime"]. This survives until the next hermes update.