Issue Station
Sign In
Tools hereToolbox

hermes - ✅(Solved) Fix Cron scheduler tests call real ~/.hermes/.tick.lock and fail whenever gateway is running [1 pull requests, 1 participants]

Official PRs (…)
ON THIS PAGE

Recommended Tools

×6

Utilities matched from this issue’s tags and category — try them while you read without losing context.

GitHub issue graph ai analysis

Paste a GitHub issue URL. We fetch that issue, discover linked issues from bodies/comments/timeline, collect linked pull requests, and produce a structured English report.

The report is written in English Markdown for sharing and archival.

Helpful · Quick feedback

Loading…
GitHub stats
NousResearch/hermes-agent#11474•Fetched 2026-04-18 06:00:53
View on GitHub
Comments
0
Participants
1
Timeline
1
Reactions
0
Author
NewTurn2017NewTurn2017
Participants
NewTurn2017NewTurn2017
Timeline (top)
cross-referenced ×1

tests/cron/test_scheduler.py calls cron.scheduler.tick() directly, but tick() always acquires the real global lock at ~/.hermes/cron/.tick.lock. In a normal dev environment with hermes gateway run already running, these tests short-circuit before any mocked cron behavior executes.

Root Cause

Running the affected tests then fails because tick() returns early:

Fix Action

Fix / Workaround

These tests patch get_due_jobs, run_job, save_job_output, etc., but they do not patch _LOCK_FILE / _LOCK_DIR or the locking call itself.

Expected behavior

Scheduler tests should isolate the lock file (e.g. temp path / monkeypatch) or stub locking so assertions exercise scheduler behavior, not the host machine's gateway state.

Suggested investigation

  • Monkeypatch _LOCK_DIR / _LOCK_FILE to a tmp path in affected tests, or
  • Add a small test helper/fixture that forces an isolated lock file for tick() tests, or
  • Refactor tick() so lock acquisition can be injected/stubbed in tests.

PR fix notes

PR #11501: fix(tests): isolate cron scheduler tick lock per test

Description (problem / solution / changelog)

Fixes #11474.

Summary

  • isolate cron.scheduler.tick() lock state in tests/cron/test_scheduler.py by redirecting _LOCK_DIR and _LOCK_FILE to a per-test temporary path
  • remove dependence on the real ~/.hermes/cron/.tick.lock held by a local gateway process
  • keep scheduler assertions focused on mocked cron behavior instead of host machine state

Testing

  • pytest -q -o addopts= tests/cron/test_scheduler.py

Changed files

  • tests/cron/test_scheduler.py (modified, +8/-0)

Code Example

lsof ~/.hermes/cron/.tick.lock

---

Python  12143 genie   31w   REG ... /Users/genie/.hermes/cron/.tick.lock
/opt/homebrew/.../Python -m hermes_cli.main gateway run --replace

---

source venv/bin/activate
pytest -q -n0 \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_normal_response_delivers \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_silent_response_suppresses_delivery \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_output_saved_even_when_delivery_suppressed \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_failed_job_always_delivers \
  tests/cron/test_scheduler.py::TestTickAdvanceBeforeRun::test_advance_called_before_run_job
RAW_BUFFERClick to expand / collapse

Summary

tests/cron/test_scheduler.py calls cron.scheduler.tick() directly, but tick() always acquires the real global lock at ~/.hermes/cron/.tick.lock. In a normal dev environment with hermes gateway run already running, these tests short-circuit before any mocked cron behavior executes.

Evidence

Production code

  • cron/scheduler.py:912-926 acquires _LOCK_FILE = ~/.hermes/cron/.tick.lock and returns 0 immediately if another process holds it.
  • cron/scheduler.py:938-971 only reaches run_job(), save_job_output(), and _deliver_result() after the lock succeeds.

Tests that currently depend on the real lock file

  • tests/cron/test_scheduler.py:980-1055 (TestSilentDelivery)
  • tests/cron/test_scheduler.py:1132-1164 (TestTickAdvanceBeforeRun)

These tests patch get_due_jobs, run_job, save_job_output, etc., but they do not patch _LOCK_FILE / _LOCK_DIR or the locking call itself.

Reproduction

On this machine, the gateway is already holding the lock:

lsof ~/.hermes/cron/.tick.lock

Output during review:

Python  12143 genie   31w   REG ... /Users/genie/.hermes/cron/.tick.lock
/opt/homebrew/.../Python -m hermes_cli.main gateway run --replace

Running the affected tests then fails because tick() returns early:

source venv/bin/activate
pytest -q -n0 \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_normal_response_delivers \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_silent_response_suppresses_delivery \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_output_saved_even_when_delivery_suppressed \
  tests/cron/test_scheduler.py::TestSilentDelivery::test_failed_job_always_delivers \
  tests/cron/test_scheduler.py::TestTickAdvanceBeforeRun::test_advance_called_before_run_job

Observed failures include:

  • deliver_mock.assert_called_once() failing because _deliver_result() was never reached
  • save_job_output.assert_called_once_with(...) failing because tick() exited before saving output
  • assert executed == 1 failing because tick() returned 0

Why this is a bug

These are unit-style scheduler tests, but they depend on external process state in ~/.hermes/cron/.tick.lock. That makes them non-hermetic and flaky in exactly the environment developers are likely to use (gateway running locally). The same pattern can also create worker-to-worker contention under parallel pytest runs.

Expected behavior

Scheduler tests should isolate the lock file (e.g. temp path / monkeypatch) or stub locking so assertions exercise scheduler behavior, not the host machine's gateway state.

Actual behavior

A live gateway process causes tick() to return before the mocked job flow runs, so unrelated assertions fail.

Suggested investigation

  • Monkeypatch _LOCK_DIR / _LOCK_FILE to a tmp path in affected tests, or
  • Add a small test helper/fixture that forces an isolated lock file for tick() tests, or
  • Refactor tick() so lock acquisition can be injected/stubbed in tests.

extent analysis

TL;DR

The most likely fix is to isolate the lock file in the scheduler tests by monkeypatching _LOCK_DIR / _LOCK_FILE to a temporary path.

Guidance

  • Identify the tests that depend on the real lock file (tests/cron/test_scheduler.py) and refactor them to use a temporary lock file or stub the locking mechanism.
  • Use a test helper or fixture to create an isolated lock file for tick() tests, ensuring that the tests do not interfere with the live gateway process.
  • Consider refactoring tick() to allow for injection or stubbing of the lock acquisition mechanism, making it easier to test in isolation.
  • Verify that the tests pass after applying the fix by running them in an environment where the gateway is running and holding the lock.

Example

import tempfile
import pytest

@pytest.fixture
def isolated_lock_file():
    with tempfile.TemporaryDirectory() as tmp_dir:
        yield tmp_dir + '/.tick.lock'

def test_tick(isolated_lock_file):
    # Monkeypatch _LOCK_FILE to use the isolated lock file
    with pytest.monkeypatch.context() as m:
        m.setattr(cron.scheduler, '_LOCK_FILE', isolated_lock_file)
        # Run the test
        cron.scheduler.tick()

Notes

The suggested fix assumes that the issue is caused by the tests depending on the real lock file. If the issue is more complex, additional investigation may be required.

Recommendation

Apply a workaround by monkeypatching _LOCK_DIR / _LOCK_FILE to a temporary path in the affected tests. This will allow the tests to run in isolation and avoid interference from the live gateway process.

Vote matrix · Quick signals

Works
Did the solution work? Tap to confirm.
Easy Fix
Was it a quick fix?
Time Saver
Did it save you time?
Blocking
Was it severely blocking?
Common Issue
Are others likely hitting this too?
Flaky / Intermittent
Is it intermittent?
Verified / Reproducible
Can you reproduce it reliably?
Loading…

FAQ

Expected behavior

Scheduler tests should isolate the lock file (e.g. temp path / monkeypatch) or stub locking so assertions exercise scheduler behavior, not the host machine's gateway state.

#autograd error#model save/load#optimization#mixed precision#training loop

Still need to ship something?

×6

Another batch ranked right after the header list — different links, same matching logic.

Back to top recommendations

TRENDING

hermes - ✅(Solved) Fix Cron scheduler tests call real ~/.hermes/.tick.lock and fail whenever gateway is running [1 pull requests, 1 participants]