Plain type=completion events built in _run_process_watcher carried only
session_key (chat/thread routing) with no spawning-session stamp, so after
/new (or a session switch) a completion notification from the OLD session
was injected into the chat's NEW session. Main already solved this exact
class for async delegations via the _classify_completion_target pre-flight
(_USER_BOUNDARY_END_REASONS drop on user-closed sessions, deliver on
idle-ends, follow the compression-tip chain), but the gate only ran for
type=async_delegation events.
Kernel salvage of #16455:
- Stamp the spawning conversation's session-db id (HERMES_SESSION_ID via
session-scoped env) on the ProcessSession and the pending_watchers entry
at spawn time in tools/terminal_tool.py; persist it through the process
registry checkpoint/restore so recovered watchers keep the stamp.
- Thread the stamp into the completion_evt built by _run_process_watcher
(watcher entry first, ProcessSession fallback for recovered watchers).
- In _deliver_completion_notification, run the SAME pre-flight classifier
for stamped type=completion events: terminal -> drop with a log (output
stays available via process(action='log')), retry -> False so the
watcher re-polls, deliver -> proceed. The policy has exactly one owner
(_classify_completion_target); nothing is forked. Unstamped legacy
events keep today's deliver-always behavior, and the async-delegation
path is untouched.
Based on the session-boundary approach from #16455 by @Tosko4 (original PR
was over-scoped across adapters/slash-commands/cron; this lands the kernel
only).
Tests: completion from a /new-closed session is dropped; completion after
an idle-end still delivers; unstamped legacy event delivers; retry verdict
returns retryable False without adapter injection; async_delegation gate
unchanged; stamp survives checkpoint recovery.
The command wrapper prints the cwd marker after the command returns. A
killed or timed-out command emits no marker, so ``env.cwd`` still holds the
directory of the last command to FINISH. One local environment serves every
session, because ``_resolve_container_task_id`` collapses cwd-only overrides
to ``"default"``. That leftover directory is therefore routinely another
session's.
The post-command dual-write copied ``env.cwd`` into the interrupted session's
durable record. Every later command in that session then ran in the foreign
directory, and the cwd echo told the model it had moved there. A desktop chat
silently re-homed into a worktree that another chat had opened.
Report the observation instead of inferring it. The marker parse now sets
``result["cwd_observed"]``, and both the record write and the echo read that
flag. The local override clears the flag when it rolls back a path that does
not exist, because the restored value is also unobserved. When a command
reports no cwd, the session keeps the directory it already had.
This needs no second session to be wrong: a lone session that interrupts a
command re-adopts a stale value too. A second session only makes the wrong
directory belong to somebody else.
The same class of write exists in the file-tools rescue for a reaped
environment (#26211). That rescue copied the cached snapshot of the shared
``env.cwd`` into the session record. The rescue is now fill-only: it writes
the snapshot when the session has no record, and it never overwrites a
record that the session wrote for itself.
The tests drive ``terminal_tool`` itself through an interrupt, not a copy of
its gate. Review found that a revert of either call site passed the first
version of the tests. Each gate now has a test that fails when the gate is
removed (verified by mutation).
Two exact-dict assertions in the Vercel sandbox tests now assert the two
fields they care about, so a new result key does not fail them.
Spill files (terminal overflow, hook context, web_extract full text,
subagent summaries) were written with plain open()/write_text into
predictable directories. A pre-planted symlink at any of those paths
redirected the write onto an arbitrary user-owned file, and raw
pre-redaction terminal/hook spills landed world-readable under the
default umask.
New tools/spill_safety.py helpers create files with
O_CREAT|O_EXCL|O_NOFOLLOW (a link-shaped path fails the write instead of
following it) and overwrite via lstat-checked unlink + exclusive
re-create, so even the redaction rewrite cannot be diverted. Private
tier (0o700 dir / 0o600 file) covers raw terminal and hook spills;
cache/web and cache/delegation keep umask perms because those dirs are
bind-mounted into remote backends that must read them.
Pattern borrowed from DeepSeek Harness dsh-spill-local (MIT):
private root + exclusive owner-only opens for spill artifacts.
Whole-bug-class sibling of the execute_code fix: terminal(code=...) —
the reverse confusion — fell through to command=None and failed with
'Invalid command: expected string, got NoneType', naming neither the
stray 'code' argument nor execute_code as the right tool. Mirror the
guard in _handle_terminal (verified live: the opaque NoneType error
reproduces on main). Mutation-checked: removing the guard fails the
new regression test.
The previous commit's regex-based stripper removed EVERY heredoc body,
which review flagged as bypassable: a fake '<<EOF' marker inside a
comment or quoted string enters the unterminated path and swallows a
later REAL background operator, and unquoted ('cat <<EOF' — expansion
runs) or shell-consumed ('bash <<'EOF'' — body IS shell) bodies are
executable content that must stay visible to the guard.
Replace it with tools/shell_heredoc.strip_inert_heredoc_bodies(), a
conservative shell-state scanner: a body is masked ONLY when every
delimiter on the opener is quoted (no expansion), every heredoc is
terminated by an exact delimiter line, the opener composes a single
command (no list/pipeline operators, no nested $()/backtick/process-
substitution scope), and the consumer is an allowlisted non-shell
interpreter (python/osascript/cat). Anything ambiguous is returned
unchanged — a false positive on exotic syntax is acceptable; hiding a
real background operator is not. Masked bodies become newlines so line
structure is preserved for MULTILINE regexes.
The helper is a standalone stdlib-only module (precedent:
tools/ansi_strip.py) because the same heredoc-as-data false-positive
class exists in the blocked-command regex checks (#83104) and the
gateway lifecycle guard (#81721/#79835, cron/lifecycle_guard.py) —
which must not import the terminal-tool module graph.
Adapted from Wolfram Ravenwolf's security-hardened rework of #63788
(69c7663c6de6b6cb05bf99203fa39673efe01ccf); test scenarios for the
bypass cases derive from his suite.
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
_strip_quotes documented that it stripped heredoc bodies but only handled
single/double/backtick quotes. As a result _foreground_background_guidance
scanned heredoc body text for a backgrounding '&' and wrongly rejected valid
foreground commands whose heredoc body contained a spaced ampersand — e.g.
AppleScript string concat (osascript <<'EOF' ... "a" & b ... EOF), Python
bitwise-and, or literal UI text like 'FaceTime & Privacy'.
Add a _strip_heredocs pass (runs before quote-stripping, since a heredoc
delimiter may itself be quoted) covering <<EOF, <<-EOF, <<'EOF', <<"EOF".
The same-line tail after the opener (redirects/args) is preserved and the
opener token is blanked so a real backgrounding '&' after the heredoc is
still detected.
Adds tests/tools/test_terminal_heredoc_background_guard.py.
Two bugs reported on the docker terminal backend (desktop app, sandboxed
profiles with container_persistent: false):
1. A NEW chat's container inherited the PREVIOUS session's workspace,
bind-mounted rw at /workspace, because the mount source was the
process-global TERMINAL_CWD env var (written by the workspace picker,
outliving its session) and all sessions shared one 'default' container.
2. Every command failed with exit 126 because the desktop gateway recorded
the HOST launch directory as the session cwd, and each command was
prefixed with 'cd /Users/<user>/...' inside the container.
Fixes (class-wide, single owners):
- container_persistent: false + docker now keys containers PER SESSION:
fresh container per chat, removed at session close/idle. delegate_task
children share the parent's container via an explicit alias registry.
container_persistent: true keeps the documented ONE-long-lived-container
contract unchanged.
- _resolve_task_host_cwd() is the single owner of the cwd->/workspace mount
policy across all four env-creation sites; under isolation it refuses
process-global cwd sources and mounts only the session's own attached
workspace (tui_gateway now tags overrides with cwd_source).
- _resolve_command_cwd() gains the same host-path guard the env-creation
sites already had (#50636/#54447 sibling site): a recorded host cwd is
discarded on container backends instead of cd-ing every command into a
nonexistent path.
E2E-tested against real Docker: distinct containers per session, no stale
mount in a fresh session, no exit 126 from host cwd records, containers
removed at session teardown.
Force-redact every terminal exception and traceback field before JSON serialization, including environment creation, background startup, exhausted foreground retries, and the outer catch-all. Preserve the current command-aware, opt-out-respecting redact_terminal_output(output, command) behavior for successful output.
Closes the last two emission gaps from #77484:
- tools/terminal_tool.py: both exception paths (generic except and
TERMINAL_DEGRADED_MODE=fail) returned raw str(e) + traceback.format_exc()
to the model — only the logger copy was redacted. Exception text can
embed the failing command line and any secrets inline in it; both fields
now pass through redact_sensitive_text.
- acp_adapter/entry.py: _setup_logging cleared root handlers and installed
a plain logging.Formatter, bypassing redaction entirely on ACP stderr.
Now uses RedactingFormatter like every other logging surface.
The other three gaps from #77484 (process(list), *_KEY regex variants,
control-char splits) were fixed in #80964/#80965.
Wire the self-repo guard in next to the gateway lifecycle hard-block,
before the force check — force=True cannot make the command safe, only
delay the crash. Local backend only: sandboxed backends cannot reach the
host checkout. The block message explains the version-skew mechanism and
redirects to git worktree add / a temp clone, or running the command
outside hermes with a restart after.
vision_analyze reads container-only images by exec-reading them inside the
sandbox, but unlike terminal_tool it never triggered environment creation. Under
a non-local backend (ssh, docker, ...), a session whose first action was
vision_analyze on a remote path failed with 'no active sandbox session' until an
unrelated terminal command happened to establish the connection.
Add terminal_tool.ensure_task_env(task_id), a public lazy get-or-create that
reuses the terminal tool's own creation machinery, and call it from
image_source._resolve_container_fallback before the in-sandbox read. Extract the
ssh/container config-dict builders so both paths derive settings identically
(no duplication). Best-effort and fail-closed: a failed bring-up leaves the
existing 'no active sandbox' error intact, never a host read.
Fixes#62825
Connection-class infrastructure failures on remote terminal backends (SSH
host unreachable/timed out, Docker daemon down or missing, remote file
sync failing on a dead link) previously surfaced to the model as raised
RuntimeError tracebacks. The model got a stack blob with no guidance and
the failure was indistinguishable from a tool bug.
Now:
- New EnvironmentConnectionError(RuntimeError) in tools/environments/base.py
carrying a reason + retry_hint. Subclassing RuntimeError keeps every
existing catcher working.
- ssh.py classifies connect-refused, connect-timeout, scp, remote mkdir,
bulk upload/download, and remote rm failures as connection errors.
- docker.py classifies all four _ensure_docker_available() failure paths
(missing exe, non-executable exe, daemon timeout, `docker version`
failure).
- terminal_tool catches EnvironmentConnectionError and returns a
structured tool result the model can act on:
{"status": "degraded", "reason": ..., "retry_hint": ..., "exit_code": -1}
The failed backend is evicted from the environment cache so a later
call retries from scratch — recovery is automatic once the backend is
reachable again.
- Config gate terminal.degraded_mode: warn|fail (default warn) in
config.yaml, bridged as TERMINAL_DEGRADED_MODE across all four bridge
sites (cli.py env_mappings, gateway/run.py _terminal_env_map,
TERMINAL_CONFIG_ENV_MAP, DEFAULT_CONFIG). "fail" preserves the
historical error+traceback tool result.
- Command failures (nonzero exit, command-not-found) are NOT touched —
only infrastructure failures classify as degraded.
Tests: tests/tools/test_terminal_degraded_mode.py (15 tests) covering
exception classification for ssh+docker, structured degraded results,
no-caching of degraded envs, recovery after the backend returns,
nonzero-exit results unaffected, fail-mode preservation, invalid-mode
fallback to warn, and the four-site config bridge invariant.
Inspired by: Claude Cowork degraded-backend behavior (idea-level,
docs-only evidence).
3-reviewer simplify pass (reuse/quality/efficiency) findings:
- cron/scheduler.py _run_job_script: the ORIGINAL that
lifecycle_guard._resolve_script_path documents mirroring had the exact
same unguarded expanduser() — a NUL-bearing script value survives
creation (the guard treats it as nothing-to-scan) and crashed the
scheduler at fire time with ValueError instead of a clean job failure.
Same ingestion contract applied; regression test added.
- lifecycle_guard._resolve_script_path: get_hermes_home() -> Path.home()
raises RuntimeError when neither HERMES_HOME nor HOME resolves
(arbitrary-UID containers); the cron entry point called it bare.
Caught -> None; totality test added.
- terminal_tool: stale 'cat ...' docstring updated to the bounded
head -c form.
- lifecycle_guard: dead 'script_text and' condition dropped (guarded by
'if not script_text: continue' directly above).
Efficiency reviewer: no material findings (measured — encode/expand
costs negligible vs walk I/O, no timing regression vs base).
- _sanitize_remote_script_text: compare re-encoded BYTES against the cap,
not characters — a >1MiB multibyte file truncated at the head -c byte
bound decodes to fewer chars than bytes and would have scanned the
truncated text instead of failing closed (the exact local/remote
divergence this PR closes).
- terminal_tool: replace the three hardcoded 1MiB literals with
lifecycle_guard._MAX_REFERENCED_SCRIPT_BYTES so the budget cannot
drift; use the redirect-safe 'head -c N < path' form from
tools/image_source.py so leading-dash paths stay out of argv.
- Public guard wrapper: drop the duplicate depth-0 direct scan — the
walk already runs it; the except-path now falls back to the pure
string scans, preserving the direct verdict when the walk crashes.
The guard feeds untrusted byte streams (tokenized binaries, remote cat
output) into OS-path and shell-text operations; every incident so far
(#76762, #77703, #77780, #78256, #77729) was hot-fixed with an except at
whichever frame crashed that week. tilllt's regression suite on #79454
showed 4 members of the class still open on merged main. Close the class
at three boundaries instead:
- _expand_candidate_path(): single ingestion chokepoint for path
candidates — reject NUL/empty tokens before any Path OS call and
tolerate ValueError/RuntimeError/OSError from expanduser (T1/T2, plus
the HOME-unset launchd crash). Both _resolve_terminal_script_path and
_resolve_script_path now go through it.
- _sanitize_remote_script_text(): apply the local-read contract (NUL =
binary = nothing to scan; >1MiB = fail closed) to whatever any
read_remote_script callback returns, at the recursion boundary — the
guard stops trusting its callbacks (T3/T4).
- contains_gateway_lifecycle_command_or_referenced_script() is now total
by construction: direct regex scans (pure string ops) run first; the
best-effort filesystem walk is wrapped so an unexpected failure logs a
warning and falls back to the direct-scan verdict instead of killing
every terminal command until gateway restart.
terminal_tool's remote fallback also bounds the read at the source
(head -c 1MiB+1 instead of cat), so a 166MB ELF never crosses the wire —
the superlinear-shlex 30-minute stall from #79838's field report drops
to a 0.02s fail-closed verdict.
Regression tests: tilllt's T1-T4 adopted verbatim, plus an adversarial
never-raises sweep (NUL paths, unset HOME, over-long paths) and a
walk-crash fallback test.
The gateway terminal guard crashed with 'ValueError: embedded null byte'
(command never ran, exit_code -1) when a command invoked an ELF binary by
full path. _read_referenced_script correctly rejects the binary locally
(NUL in first chunk), but the read_remote_script fallback
(_read_script_in_env) then re-read the SAME file's bytes without a NUL
guard, decoded them, and fed machine code back into the scanner, which
re-tokenized it into a bogus NUL-bearing path and crashed at os.open.
- _read_script_in_env: skip content containing a NUL byte on both the
local-read and remote-cat branches (mirrors _read_referenced_script:
a binary is nothing to scan), so binary never re-enters the guard.
- _read_referenced_script: tolerate ValueError from os.open on a
NUL-in-path, alongside the existing OSError guard, so the guard can
never crash the terminal tool regardless of input.
Extends the #76762 NUL-safety fix (local path only) to the gateway's
remote-read fallback path.
The workdir allowlist regex was ASCII-only, so perfectly normal
non-ASCII workdirs (Chinese Obsidian vault paths, accented dirnames)
were rejected with 'disallowed character'. Replace the regex with a
per-character check that accepts Unicode letters/digits (str.isalnum)
plus the same safe ASCII punctuation set, while still rejecting shell
metacharacters, control characters (newlines/tabs), and NUL.
Salvaged from PR #54314.
Co-authored-by: kshitij <82637225+kshitijk4poor@users.noreply.github.com>
Every tool schema ships on every API call. The terminal schema was
5,641 chars (~1,410 tokens) and execute_code 2,842 (~710) — the two
largest core tools, padded with repeated war stories and triple-stated
rules. Schema token audit across 88 tools: ~33k tokens total.
This trims prose while preserving every hard rule (each still stated
exactly once):
- terminal description 2,324 -> 1,233 chars: tool-redirect lines
collapsed to one sentence; background/notify guidance deduplicated
(was stated in desc + 2 params); PTY/pager rules merged.
- background/notify_on_complete/watch_patterns params 692/508/1,114 ->
~330/250/490 chars: kept the mutual-exclusion contracts, the
rate-limit consequence, and the bounded-vs-long-lived distinction;
dropped narrative repetition.
- execute_code description tightened (helper docs inlined to one line
each; when-to-use kept).
Net: terminal schema 5,641 -> 3,386 chars, execute_code 2,842 -> 2,522
— ~700 tokens saved on EVERY request with the terminal+code toolsets.
One test updated (pinned a removed phrase; now pins the rule's new
phrasing).
Truncated terminal output was information LOSS: the middle was gone
and the only recovery was re-running the command (data: 1,394
truncation markers in a 250k-call window, with re-runs and grep
retries chained behind the big ones).
Truncation is now deferred retrieval (opencode/goose/qwen-code
pattern, codex's original_token_count idea):
- tools/environments/base.py: _BoundedOutputCollector gains an
optional spill tee — when foreground output overflows the capture
window, the FULL stream is teed to
~/.hermes/cache/terminal-output/out-*.log (lazy file creation with
backlog backfill, 5MB hard cap, 7-day opportunistic cleanup,
disk errors never break execution). All three _wait_for_process
returns attach {output_total_chars, full_output_path} via a shared
finalizer.
- tools/terminal_tool.py: redacts the spill with the same
redact_terminal_output pass as the visible output (no secret
persists unmasked), then surfaces output_total_chars,
full_output_path, and a truncation_note pointing at
search_files/read_file instead of a re-run.
Non-truncated results are byte-identical; internal unbounded
consumers (file-ops cat reads, RPC reads) are untouched (spill only
arms with bounded_capture=true).
Two block classes from production mining (250k-call window) that
models answered with blind rephrase-retries:
1. Parser-limit / malformed-payload hardline blocks (198x): these fire
on oversized inline payloads (heredocs, giant one-liners), not on a
forbidden operation - but the message read like a permanent ban.
The block now appends: 'RECOVERY: ... write the script to a file
with write_file, then run bash /path/script.sh - do not retry
inline.' Genuine hardline blocks (destructive filesystem
operations) are unchanged.
2. Backgrounding-wrapper blocks (200x): the guidance now spells out
the exact corrected call shape - 're-send WITHOUT the wrapper as
terminal(command="<cmd>", background=true,
notify_on_complete=true)' - instead of describing the feature
abstractly.
Production mining (state.db, 400k-msg window): 60.2% of 104k terminal
calls carry a defensive 'cd X && ' prefix (~925k tokens of pure prefix)
and 2,462 failed calls led with cd — the model cannot see cwd state, so
it re-asserts it on every call and runs pwd/ls diagnostics after
directory changes.
The result dict now includes a 'cwd' field whenever the session cwd
after the command differs from the cwd it started in (cd, pushd,
chained cd). Stable-cwd commands are unchanged (no field, no noise).
Per-command workdir overrides stay transient by contract and never
echo. Schema note added so models learn to trust session cwd instead
of prefixing. Pattern borrowed from crush's <cwd> injection.
realpath comparison avoids false echoes through symlinks; the echo is
wrapped defensively so a backend without .cwd can never break the
result path.
When a command exits non-zero, scan the first 4KB of output for
well-known failure shapes and attach one short, actionable recovery
hint to the tool result ('hint' field):
- gh 'Unknown JSON field' (9.2k occurrences in a 250k-call window)
- git merge conflicts (1.2k) — stop verbatim retries
- command not found (1.0k), incl. python->python3 and pip->pip3
- ModuleNotFoundError (739) — venv activation guidance
- 'already exists' (633), gh rate limits (133), permission denied
- exit-code-only tier: 124 timeout, 126 not-executable, 137 SIGKILL
Hints are suppressed when the existing exit_code_meaning tier already
explains the code (grep=1 etc). Pattern order = production frequency
from state.db mining; first match wins; pure function, no I/O.
NOTION_API_KEY/LINEAR_API_KEY defaults, DEEPINFRA_API_KEY model gate, FAL_KEY via the file's own _scoped_credential helper, XAI/VERCEL/DAYTONA presence checks, GITHUB_TOKEN/GH_TOKEN + GitHub App creds in tirith/skills_hub, and the masked HERMES_API_KEY display in tui_gateway config.show all route through get_secret.
Three lower-severity core-tool robustness fixes from a targeted audit, each
reproduced live:
1. terminal_tool did not validate non-positive timeouts. 'timeout or default'
silently coerced 0 to the config default (0 can't mean 'no timeout'), and a
negative value is truthy so it flowed into 'deadline = now + timeout' and
fired an immediate '-Ns' timeout. Reject timeout <= 0 with a clear message.
2. fuzzy_find_and_replace accepted a whitespace-only old_string, which matches
trivially (blank line / run of spaces) and mass-replaces under replace_all
or raises an opaque ambiguity error. Reject it alongside the empty check.
3. The '/private/var/' sensitive-path prefix over-blocked ALL macOS temp-file
writes: , /tmp, and /var/folders realpath into /private/var/folders
on macOS (and paths are resolved through symlinks), and /private/var/tmp is
a normal temp dir. Narrowed to the genuinely-sensitive subtrees
(/private/var/db, /private/var/root); /etc and /private/etc stay blocked.
All verified with sabotage-checked regression tests. 85 terminal/fuzzy/file
tests pass; normal timeouts, legit replacements, and /var + /boot + /etc
blocking are unaffected.
Two independent HIGH-severity correctness bugs found in a core-tools audit,
each reproduced live against current main:
1. Read-dedup was never evicted after a write on non-default tasks.
_invalidate_dedup_for_path looked up the read-tracker under the correct
task_id but resolved the path with _resolve_path(filepath) — which
DEFAULTS task_id='default'. The dedup cache is keyed by the task-resolved
absolute path, so for any task whose workspace cwd differs from the process
cwd (every -w worktree / Desktop / ACP session using relative paths) the
computed key never matched and the stale entry was never removed. A
read_file after a write_file/patch could then return the OLD content stub
when mtime coincided. Fix: pass task_id through.
2. A per-command workdir override permanently hijacked the session cwd.
The post-command dual-write unconditionally recorded env.cwd (stamped to
the transient workdir) into the durable session-cwd store, so every later
command that omitted workdir inherited the one-off directory — contradicting
the documented 'Working directory for this command' contract. Fix: skip the
session-cwd record when workdir was explicitly supplied.
Both verified with sabotage-checked regression tests (fail without the fix).
Extends the shared _GATEWAY_LIFECYCLE_PATTERN (used by BOTH the cron
creation-time guard in cron/lifecycle_guard.py and the terminal
execution-time hard-block in tools/terminal_tool.py) so Branch B covers
launchctl submit and bootstrap alongside kickstart/unload/load/stop/
restart, and normalizes POSIX shell line continuations before matching
so the exact multi-line reported shape in #62891 cannot slip past.
Also extends the execution-aware, label-independent detector
(contains_launchctl_submit_command, cherry-picked from #63272) to cover
launchctl bootstrap, since a neutral label like ai.hermes.svc-reload-tmp
defeats any label-anchored regex — the second production reproduction.
Regression tests cover both sites, including
`launchctl submit -l com.foo -- /path/gateway` and the bootstrap
variant, plus outside-gateway pass-through.
Branch B regex extension and continuation normalization drawn from
PR #62896; bootstrap coverage and test shapes drawn from PR #51003.
Co-authored-by: JackJin <1037461232@qq.com>
Co-authored-by: joelbrilliant <joelbrilliant1@gmail.com>
- Resolve guard cwd against get_session_cwd(session_key); fall back to env.cwd
when no session record exists yet, matching current main's per-session cwd
architecture.
- Make referenced-script reads backend-aware: local read first; if missing,
fall back to env.execute('cat ...') for SSH/Modal/Daytona backends.
- Reuse the recursive scanner in check_gateway_lifecycle so nested cron
wrapper scripts are caught, and resolve relative refs inside a script
against that script's directory.
- Add regression tests for remote-backend reads, two-session cwd, and nested
cron wrappers.
Verification: 80 passed tests/hermes_cli/test_gateway_restart_loop.py;
694 passed tests/cron; ruff + git diff --check clean.
Let terminal keys explicitly present in config.yaml override matching stale TERMINAL_* values while preserving environment values for omitted keys. Merged defaults remain backfill-only.
Exercise the real config.yaml to _get_env_config path for backend selection, partial terminal sections, matching-key overrides, environment fallback, one-shot bridging, and config read failures.
Closes#71137
Port from nanocoai/nanoclaw#2748: Docker's built-in 64 MB /dev/shm silently
breaks shared-memory-hungry workloads inside the sandbox — Chromium/Playwright
renderers crash tabs, and PyTorch DataLoader workers die with 'bus error' /
'insufficient shared memory'. tmpfs is lazily allocated, so the higher ceiling
costs nothing until actually used, and usage still counts against the
container's --memory cgroup limit.
- tools/environments/docker.py: --shm-size 1g in resource args (not
cgroup-gated; tmpfs mount option). Skipped when docker_extra_args already
sets --shm-size, or when configured empty/'0' (Docker default).
- terminal.docker_shm_size config key (DEFAULT_CONFIG + all three
config->TERMINAL_DOCKER_SHM_SIZE env bridges: CLI, gateway, config.py map)
- tests: default emit, custom value, opt-out, extra_args precedence,
helper edge cases (sabotage-verified: default/custom tests fail without
the emit)
Dispatcher-spawned Kanban workers are finite one-shot processes, so detached delegation completions can outlive their only consumer. Mark that runtime as unable to deliver async completions and reuse the synchronous delegation fallback, returning required child results before the worker exits.\n\nAlso make unsupported-session notes runtime-generic and cover the delayed-child lifecycle regression.\n\nRefs #63169
run_agent._dispatch_delegate_task forces background=True for every top-level
delegation, and async_delivery_supported() returns True for any session that
never binds the capability. On runners that cannot receive a completion after
their turn ends, that combination silently discards every subagent result: the
model gets a dispatch handle, ends its turn, and reports 'waiting for results'.
Two such runners never bind the capability:
* hermes -z (one-shot) prints one final response and exits. It bypasses cli.py,
so nothing drains process_registry.completion_queue (only the interactive
process_loop and the gateway watchers do).
* cron run_job clears the HERMES_SESSION_* routing keys, so a completion event
carries session_key="" — _enrich_async_delegation_routing cannot resolve it
and _inject_watch_notification drops it ("no routing metadata"). By then
run_job has already shipped the job's final response via _deliver_result;
there is no turn left to re-enter. Worse, get_current_session_key() can fall
back to the ambient os.environ HERMES_SESSION_KEY, so a cron subagent's output
can be routed into an unrelated user chat rather than merely dropped.
Add declare_stateless_channel() and bind it in both runners, routing
delegate_task to its existing inline/synchronous path — the same fallback the
stateless HTTP adapter already relies on, and the fix suggested in #63142. The
helper binds only the capability: set_session_vars() would also latch
_session_context_engaged, which a pure single-process one-shot must not trigger.
Also correct two agent-facing strings that hardcoded 'stateless HTTP API' as the
only channel without async delivery (delegate_tool, terminal_tool); they now name
the actual condition.
Repro (before): hermes -z 'Use delegate_task to spawn a subagent that replies
BANANA. Report its reply.' -> "Waiting for the subagent's response...", exit 0,
no BANANA. After: BANANA is returned in-turn.
Fixes#53027Fixes#63142
terminal_tool reads all settings from TERMINAL_* env vars, bridged from
config.yaml by the CLI, gateway, and TUI-PTY launchers. Processes that
skip every launcher bridge — hermes serve / the Desktop app backend's
in-process agents, the desktop cron ticker — saw an unset TERMINAL_ENV
and silently ran every command on the host even when config.yaml selects
terminal.backend: docker. A user who configured Docker isolation got
unsandboxed host execution with no warning.
Two layers:
- _ensure_terminal_env_bridged() in _get_env_config(): when TERMINAL_ENV
is unset, backfill TERMINAL_* from config.yaml via
apply_terminal_config_to_env(override=False). Explicit env always wins
(honor explicit choice; only fix the accidental fallback). One-shot,
fail-open to the historical local default.
- cmd_dashboard/serve: run the same bridge at startup so every consumer
in the backend process (in-process agents, desktop cron ticker,
tui_gateway cwd resolution) sees the bridged env directly.
Fixes#63141, #54449, #61115, #65696.
The per-session record store is now the ONLY cwd mechanism. Deleted:
- env.cwd_owner stamping + prev_owner threading (terminal_tool): the
shared env no longer carries ownership metadata at all
- _resolve_command_cwd's env/prev_owner params: resolution is
workdir > session record > config/override default
- file_tools._live_cwd_if_owned + _get_live_tracking_cwd: path
resolution never consults the shared env's live cwd
- file_tools._last_known_cwd + _remember_last_known_cwd +
_last_known_cwd_for: the #26211 preserved-anchor registry is
subsumed by the session record, which never lived on the env and
therefore cannot be lost to env cleanup. The _get_file_ops
stale-cache rescue now writes the record instead.
- env recreation (both _get_file_ops and terminal_tool) seeds the
fresh env from override > session record > config
Why no transition fallback: the legacy state was process-local and
in-memory exactly like the record store — after a restart both start
empty, and within a running process every legacy write site has been
dual-writing the record since step 1. There is no populated-legacy/
empty-record state to fall back for.
Tests updated to drive the record store instead of the deleted
mechanism; the cross-session isolation suite now asserts the same
behavior contracts (no leak, cd isolation, #26211 persistence)
against the new architecture, plus a new "session C inherits nothing"
case that the old ownership guard could not express.
Third step of the cwd rearchitecture: _resolve_command_cwd now prefers
the session's own cwd record over the shared env's live cwd.
New resolution order: workdir > session record > legacy env.cwd
(ownership-gated, transition-only) > config/override default.
The record is written after every completed command for the session, so
it IS the session's cd state — another session's cd lands in another
record and cannot affect this session's commands. The legacy env.cwd
branch only fires for a session with no record yet (no command has
completed since this code loaded); it keeps the prev_owner ownership
guard for that transition window and is deleted in step 4 along with
env.cwd_owner stamping and file_tools' _last_known_cwd machinery.
Adds command-path regression tests including the terminal sibling of
the leak-A scenario (unowned shared env cwd vs session record) and an
E2E cd round-trip through terminal_tool.
First step of the cwd rearchitecture (see PR #65185 for the targeted
leak fixes this will eventually supersede, and
.hermes/plans/cwd-rearch-audit.md for the full audit + sequencing).
The root cause of the wrong-worktree bug class is that cwd lives on the
SHARED terminal env — a global mutable timeshared between sessions.
env.cwd_owner stamping, _last_known_cwd, and file_tools' ownership
ladder are all patches over that misplacement.
This adds the replacement store: _session_cwd, keyed by the raw
session/task key, with record/get/clear accessors. Step 1 is dual-write
only — every site that learns a session's live cwd also records it:
- terminal_tool foreground path: after env.execute() the env's own
post-command tracking has updated env.cwd; mirror it under the
session key that drove the command
- register_task_env_overrides: a registered workspace cwd (ACP/TUI/
desktop) seeds the session record
- clear_task_env_overrides: drops the record on teardown
Readers are untouched — behavior is identical. Later steps flip
file_tools resolution and _resolve_command_cwd to read this store,
then delete env-side tracking, cwd_owner, and _last_known_cwd.
Also hardens terminal_tool's env acquisition with an explicit
env-is-None guard (previously implicitly unbound on an unreachable
branch, flagged by pyright once the dual-write read env post-loop).
Review finding on the salvaged collector: _wait_for_process is the shared
drain for EVERY env.execute() consumer, not just the terminal tool. Applying
tool_output.max_bytes there silently truncated file-operation cat reads
(read_file_raw feeds the patch engine — read-modify-write on any file >50KB
would corrupt it), paginated read_file, code-execution RPC reads, and log
reads.
bounded_capture is now an explicit opt-in on execute()/_wait_for_process,
set only by the foreground terminal tool. Default preserves the historical
full-fidelity capture via an effectively-unbounded collector (single code
path). Modal transports accept the kwarg for signature parity.
New regression test: default execute() returns a 200KB payload complete and
untruncated. E2E: 20MB internal read intact; ShellFileOperations
read_file_raw round-trips byte-exact; terminal path still bounded at 50KB.
Teknium
ethernet
kshitij
Taylor Mingos
Coy Geek
Erosika
Gille
PRATHAMESH75
kshitij
f1aggo_macair
AMATH
John Lussier
ajzrva-sys
Alex Fournier
Jeffrey Quesnelle
Mustafa
Clifford Garwood