Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix [Bug]: fails on documented short form context_length_exceeded: 131072 [2 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#17983•Fetched 2026-05-01 05:54:37
View on GitHub
Comments
0
Participants
1
Timeline
5
Reactions
0
Author
SimbaKingjoeSimbaKingjoe
Participants
SimbaKingjoeSimbaKingjoe
Timeline (top)
labeled ×3cross-referenced ×2

Error Message

agent/model_metadata.py::parse_context_limit_from_error lists "context_length_exceeded: 131072" as a supported format in its own docstring, but the function returns None for that exact input. As a result, when a provider surfaces the OpenAI-style short error code, the runtime cannot recover the real context window and falls back to unnecessary context halving / probing in run_agent.py.

Additional Logs / Traceback (optional)

Root Cause

Root Cause Analysis (optional)

Fix Action

Fixed

PR fix notes

PR #17985: fix(model_metadata): parse_context_limit_from_error matches underscore short form

Description (problem / solution / changelog)

What does this PR do?

agent/model_metadata.py::parse_context_limit_from_error advertises "context_length_exceeded: 131072" as a supported format in its docstring, but actually returned None for that input. Two regex patterns failed: pattern 1 had no max/limit keyword to anchor on, and pattern 2 used \s* between context and length (so _ did not match) and required an is/of/: connector immediately after the keyword (so length_exceeded: did not match either).

This PR widens both patterns just enough to accept the underscore short form while keeping every existing negative case rejected. The wider pattern still requires an explicit connector (: / is / of) so adversarial strings like "context window exceeds limit (2013)" continue to return None, and the 1024 <= limit <= 10_000_000 sanity gate is unchanged.

Related Issue

Fixes #17983

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

How to Test

  1. Reproduce the bug on main: 
    python3 -c "from agent.model_metadata import parse_context_limit_from_error as f; print(f('context_length_exceeded: 131072'))"
# → None  (BUG)

Changed files

  • agent/model_metadata.py (modified, +2/-2)
  • tests/agent/test_model_metadata.py (modified, +7/-0)

PR #18144: fix(model_metadata): parse_context_limit_from_error matches underscore short form

Description (problem / solution / changelog)

Problem

Fixes #17983

parse_context_limit_from_error in agent/model_metadata.py advertises "context_length_exceeded: 131072" as a supported format in its docstring, but actually returned None for that input.

Two regex patterns failed:

  • Pattern 1 had no max/limit keyword to anchor on
  • Pattern 2 used \s* between context and length (so _ did not match) and required an is/of/: connector immediately after the keyword (so length_exceeded: did not match either)

Fix

Widen pattern 2 in parse_context_limit_from_error:

  • \s* → [\s_]* between context and (length|size|window) to accept underscore separators
  • [\w]* after the keyword to allow trailing text like _exceeded

This accepts the underscore short form while keeping every existing negative case rejected. The adversarial string "context window exceeds limit (2013)" continues to return None.

Before vs After

InputBeforeAfter
"context_length_exceeded: 131072"None131072
"maximum context length is 32768 tokens"3276832768 (unchanged)
"Maximum context size 32768 exceeded"3276832768 (unchanged)
"context window exceeds limit (2013)"NoneNone (unchanged)

Tests

  • tests/agent/test_model_metadata.py — added test_context_length_exceeded_short_form verifying the underscore short form is now parsed correctly

Changed files

  • agent/model_metadata.py (modified, +1/-1)
  • tests/agent/test_model_metadata.py (modified, +5/-0)

Code Example

context\s*(?:length|size|window)\s*(?:is|of|:)?\s*(\d{4,}) — \s* between context and length cannot match _

---
RAW_BUFFERClick to expand / collapse

Bug Description

agent/model_metadata.py::parse_context_limit_from_error lists "context_length_exceeded: 131072" as a supported format in its own docstring, but the function returns None for that exact input. As a result, when a provider surfaces the OpenAI-style short error code, the runtime cannot recover the real context window and falls back to unnecessary context halving / probing in run_agent.py.

Steps to Reproduce

cd hermes-agent python3 -c "from agent.model_metadata import parse_context_limit_from_error as f; print(f('context_length_exceeded: 131072'))"

Expected Behavior

Prints 131072, matching the contract in the docstring at model_metadata.py:782.

Actual Behavior

Prints None.

Affected Component

CLI (interactive chat)

Messaging Platform (if gateway-related)

No response

Debug Report

context\s*(?:length|size|window)\s*(?:is|of|:)?\s*(\d{4,}) — \s* between context and length cannot match _

Operating System

Ubuntu 24.04

Python Version

No response

Hermes Version

No response

Additional Logs / Traceback (optional)

Root Cause Analysis (optional)

No response

Proposed Fix (optional)

No response

Are you willing to submit a PR for this?

  • I'd like to fix this myself and submit a PR

extent analysis

TL;DR

The parse_context_limit_from_error function in agent/model_metadata.py should be updated to correctly parse the "context_length_exceeded: 131072" error format.

Guidance

  • Review the regular expression in the parse_context_limit_from_error function to ensure it correctly matches the "context_length_exceeded: 131072" format.
  • Verify that the function returns the expected value (131072) for the given input by running the provided test case.
  • Check the docstring in model_metadata.py at line 782 to ensure it accurately reflects the supported error formats.
  • Consider adding additional test cases to cover other possible error formats.

Example

import re

def parse_context_limit_from_error(error):
    # Updated regular expression to match the "context_length_exceeded: 131072" format
    match = re.search(r'context_length_exceeded: (\d+)', error)
    if match:
        return int(match.group(1))
    # ... existing code ...

Notes

The provided debug report suggests a regular expression that may not correctly match the "context_length_exceeded: 131072" format. The proposed fix should focus on updating this regular expression to ensure correct parsing.

Recommendation

Apply workaround: Update the parse_context_limit_from_error function to correctly parse the "context_length_exceeded: 131072" error format, as this will allow the runtime to recover the real context window and avoid unnecessary context halving/probing.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…
#environment setup#docker error#permission error#memory optimization#batch processing

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix [Bug]: fails on documented short form context_length_exceeded: 131072 [2 pull requests, 1 participants]