hermes - 💡(How to fix) Fix Docker sandbox fails with exit 125 when container_persistent=true and credential files are registered

Bug

When container_persistent: true is set in config.yaml and a skill declares required_credential_files (e.g. google-workspace), the Docker sandbox container fails to start with exit code 125.

Error

docker: Error response from daemon: ... error mounting "/opt/data/google_client_secret.json" to rootfs at "/root/.hermes/google_client_secret.json": not a directory: Are you trying to mount a directory onto a file (or vice-versa)?

Root Cause (confirmed via forensic investigation)

This is a Docker-in-Docker path resolution problem combined with sandbox state corruption.

Path resolution: hermes runs inside a container where HERMES_HOME=/opt/data. When hermes spawns a sandbox container and passes -v /opt/data/google_token.json:/root/.hermes/google_token.json:ro, the Docker daemon resolves /opt/data/ against the host filesystem, not the hermes container's filesystem. If /opt/data/ exists on the host (as a separate directory structure), Docker mounts from there — not from HERMES_HOME inside hermes.

State corruption: On failed sandbox runs (e.g. from a previously broken container_persistent state or earlier credential setup attempts), docker run with an invalid source path causes Docker to auto-create the source as a directory rather than failing cleanly. This leaves /opt/data/google_token.json on the host as a directory, not a file. Subsequent runs then fail trying to bind-mount a directory as a file destination.

Compounding factor: The persistent sandbox home volume (-v /opt/data/sandboxes/.../home:/root) can also accumulate corrupted state — if google_token.json was ever created as a directory inside the sandbox home, Docker cannot bind-mount the real credential file over it.

How to reproduce

1. Run hermes inside a container where /opt/data/ also exists on the host
2. Have a failed sandbox attempt (e.g. any run before container_persistent was enabled)
3. Docker will have auto-created /opt/data/google_token.json as a directory on the host
4. All subsequent sandbox launches fail with exit 125

Fix

Immediate (manual cleanup):

# On the HOST (not inside the hermes container)
rm -rf /opt/data/google_token.json /opt/data/google_client_secret.json
cp $HERMES_DATA_PATH/google_token.json /opt/data/google_token.json
cp $HERMES_DATA_PATH/google_client_secret.json /opt/data/google_client_secret.json

# Also clean the sandbox home volume
rm -rf /opt/data/sandboxes/docker/default/home/.hermes
mkdir -p /opt/data/sandboxes/docker/default/home/.hermes
touch /opt/data/sandboxes/docker/default/home/.hermes/google_token.json
touch /opt/data/sandboxes/docker/default/home/.hermes/google_client_secret.json

Proper fix (in hermes): The credential_files.py mount logic should validate that source credential paths are files before adding them as -v args. If a path resolves to a directory, log a warning and skip rather than letting Docker create corrupted state. Additionally, docker.py should guard against creating sandbox directories with names that conflict with credential filenames.

Architectural fix: Pass the host-resolvable credential paths explicitly via config rather than relying on container-internal HERMES_HOME paths that diverge from what the Docker daemon resolves.

Environment

• hermes-agent: nousresearch/hermes-agent:latest
• Sandbox image: nikolaik/python-nodejs:python3.11-nodejs20
• Host OS: Ubuntu (OpenMediaVault 7), Docker 27.x
• container_persistent: true, terminal backend: docker
• /opt/data/ exists on the host as a separate directory structure