Inspired by Claude Code v2.1.223: 'Fixed permission prompts so commands
padded with tabs or invisible Unicode can no longer hide part of the
command from the approval dialog.'
A dangerous command rendered into an approval prompt could previously
lie to the human approver three ways:
- invisible/format Unicode (zero-width, bidi overrides/isolates,
variation selectors, U+E0000 tag block) rendered as nothing
- raw control bytes (ANSI/OSC escapes, bare CR) could erase or
overwrite the just-printed prompt line in the terminal
- long whitespace padding runs pushed the dangerous tail out of view
or past platform preview truncation (~200 chars on gateway)
New agent.redact.sanitize_command_for_display() replaces hidden chars
with visible escape markers (\u202e, \x1b) and collapses padding runs
to explicit markers, preserving literal IOCs instead of deleting them.
Wired at every approval display-mint site: CLI prompt, gateway
dangerous-command + execute_code + tool-approval payloads, pending
fallbacks, and gateway _redact_approval_command. Display-only — the
executed command and pattern-key persistence are untouched.
25 new tests; redact (97), approval (103), gateway approval-format
suites green; E2E with real imports across CLI + gateway paths.
The windows-compat change-detector checked for the literal string
'start_new_session=True', but the systemd scope isolation path
conditionally uses start_new_session=False (the scope creates its own
session/cgroup). Assert 'start_new_session=' instead — the value may
now be a variable.
The entry_data/lock_held path (used by mark_turn_active/clear_turn_active)
was eagerly constructing a full O(n) snapshot of all routing entries under
the lock on every turn, even when the DB upsert would succeed. Defer the
fallback_data construction to the except branch where it's actually needed,
and respect lock_held to avoid re-acquiring a non-reentrant _lock.
When Hermes runs as a systemd gateway with MemoryHigh/MemoryMax limits,
local background terminal commands (terminal(background=true)) inherit the
gateway's cgroup. A memory-heavy executor (Codex, tests, Node) can push
the whole cgroup past MemoryMax and trigger systemd-oomd to kill the
ENTIRE gateway — taking down the messaging control plane and silently
losing the active turn.
Root cause: tools/process_registry.py::spawn_local() uses
start_new_session=True (creates a process session/group, NOT a resource
cgroup). The spawned process tree stays in the gateway's systemd cgroup.
Fix: when running under a service manager (detected via the existing
is_gateway_supervisor_process() helper), wrap the pipe-mode spawn command
in 'systemd-run --user --scope --unit=hermes-worker-<id>' so the worker
gets its own transient cgroup. An OOM in the worker then kills only the
worker, not the gateway.
The systemd-run availability is probed once (a no-op /bin/true in a
transient scope) and cached, because the binary can exist on PATH while
the user D-Bus session is unavailable (system services, containers). If
unavailable, fall back to the current start_new_session=True behavior
with a debug log.
Scope: this covers the common background pipe-mode path. PTY mode
(PtyProcess.spawn) is left as future work — it uses a different spawn
mechanism and is used for interactive CLI tools where cgroup isolation
has additional considerations.
- Fix _usage_audit_path() to use _get_hermes_home() instead of hardcoded
Path.home() / '.hermes' (profile-safe resolution, sweeper finding)
- Rewrite skip_background_review tests to exercise finalize_turn() directly
instead of duplicating the guard expression (sweeper finding)
- Fix response_silent audit field to use _is_cron_silence_response()
instead of the buggy SILENT_MARKER substring check it was meant to
replace (simplify-code review finding)
- Remove dead 'model' in locals() guard — model is always in scope
before the try block (simplify-code review finding)
- Extract _stub_agent_for_finalize() helper to eliminate ~40 lines of
copy-pasted agent stubbing in tests (simplify-code review finding)
- Clean up 'Phase 0.5' instrumentation comments
Phase 0.5 of the Hermes Agent token leak mitigation plan: append a single
JSONL line to ~/.hermes/cron/usage_audit.jsonl after every cron LLM
invocation, capturing prompt/completion/total tokens, model, duration_ms,
deliver target, and error (when raised). Read from agent.session_*_tokens
which run_conversation already returns in its result dict.
Without this, we have no measured baseline to attribute token deltas to
subsequent mitigation phases. The plan's hard gate: observability lands
before any mitigation phase.
Writer NEVER raises — wrapped in a single try/except that logs a warning
on any json.dumps / mkdir / open failure so an audit-log bug cannot
break a cron job. Failure-path audit guard via locals() check covers
exceptions that fire before the fire_id is assigned.
No new dependencies, no new env vars (the plan rejected one in v2).
Tests: 7 new unit tests in tests/cron/test_usage_audit_logger.py covering
the success path, missing token info, swallowed writer exception, parent
dir creation, multiple appends, and unicode preservation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 8 wire-in + Vector 8 doc comment from
ralplan-hermes-token-leaks.md.
(1) Phase 8 wire-in: cron AIAgent construction now passes
skip_background_review=True. This suppresses the end-of-turn
skill/memory review fork (~30K tokens/event, ≤30K typical and
≤150K worst-case daily on bluenode) which has no human-in-the-loop
value for cron sessions.
(2) Vector 8 doc comment: a one-line comment immediately above the
AIAgent(...) construction documenting the verified-negative
finding that title generation does not run on the cron path
(maybe_auto_title is gateway/CLI-side only). Future contributors
won't accidentally introduce title-gen here without realizing it
would add ~600-1000 tokens/fire on a path that explicitly opts out
of memory/review/title overhead.
No new tests required for the doc comment (no behavior change). The
skip_background_review wiring is covered by the existing source-text
assertion in tests/agent/test_skip_background_review.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 8 of the Hermes Agent token leak mitigation plan
(ralplan-hermes-token-leaks.md §3.9). Adds a boolean kwarg
`skip_background_review` (default False) to AIAgent.__init__ that
suppresses the end-of-turn _spawn_background_review fork.
Each background review fork instantiates a new AIAgent with its own
~15K input tokens + up to 8 LLM iterations, accumulating ~30K tokens
per event in the worst case. On cron sessions there is no
human-in-the-loop benefit from the review (no skill-creation pressure,
nobody curating MEMORY.md), so the cost is pure waste.
The end-of-turn guard now reads:
if (final_response and not interrupted
and not getattr(self, "skip_background_review", False)
and (_should_review_memory or _should_review_skills)):
skip_memory=True already disables the memory-review trigger; this
flag is the explicit single-switch off for both review paths.
Defaults to False, so behavior is unchanged for gateway/CLI callers
that omit the kwarg.
Tests: 5 new unit tests in tests/agent/test_skip_background_review.py
covering the default value, flag persistence, the gate short-circuit,
the gate fall-through, and a source-text assertion that the cron
scheduler sets the flag to True (separate commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Table-driven across every session-scoped RPC that can hit a dead runtime
id, plus the invariants the scattered copies disagreed on: resume targets
the session-owning profile (never forks into the active one), the recovered
id publishes exactly once, drift during the resume aborts instead of
retrying, a resume that itself 404s rethrows the ORIGINAL error, and
recovery is bounded to a single retry.
These run without touching the REST layer because profile resolution is
injected — the pre-consolidation helper reached through resolveStoredSession
-> getSession(), which made its coverage depend on store state left behind
by whichever test file ran first (passed alone, timed out at 15s alongside
index.test.tsx).
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
Co-authored-by: rapsealk <rapsealk@users.noreply.github.com>
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Same bug class, remaining call paths. runRewindSubmit handled only
"session busy" — a rewind runs right after cancelRun, and interrupting can
drop the gateway's session, so "Restore checkpoint" after a stop hit a dead
runtime id and surfaced the raw error. The session-tile delegate's
interrupt/submit had no recovery either, so a tile left open across sleep
was dead until reopened.
Both now use the shared resolver. The tile delegate resolves its durable id
by reversing the stored->runtime cache and repoints that mapping on
recovery, so later tile actions use the live binding instead of recovering
on every call.
Co-authored-by: JonthanaHanh <JonthanaHanh@users.noreply.github.com>
Co-authored-by: akivavh <akivavh@users.noreply.github.com>
After sleep/wake, a long idle, or a remote backend restart, the gateway
drops its in-memory runtime session while the desktop still holds the old
id. prompt.submit already recovered, so plain text kept working and the
failure looked selective: attaching an image or running /compress died
with a bare "session not found" and a new chat was the only workaround.
Attach runs BEFORE prompt.submit, so submit's recovery never got a
chance — image.attach_bytes / image.attach / file.attach failed first.
Route them (and session.compress) through the shared resolver, and thread
the recovered id back to the caller so the follow-up submit targets the
live session instead of the dead one.
Attachment bytes are read once, outside the retry, so recovering a large
upload doesn't re-read it. /compress deliberately does NOT opt into
timeout recovery: it is legitimately LLM-slow and retrying a timeout
would double a minutes-long call.
Reported-by: bapemonkey (Discord)
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
Co-authored-by: zzz163519 <zzz163519@users.noreply.github.com>
Co-authored-by: luxles <luxles@users.noreply.github.com>
main carried three hand-rolled copies of the same recovery policy —
prompt.submit (submit.ts), session.interrupt (cancelRun) and
session.redirect (steering) each open-coded
isSessionNotFoundError -> resolveSessionProfile -> session.resume ->
retry. Three copies of one policy is how call sites drift apart, and it
is why the RPCs that were never given a copy (attach, /compress,
checkpoint restore) still surface a raw "session not found" after
sleep/wake while plain text silently recovers.
Introduce withSessionNotFoundResume() as the single resolver and move
all three existing copies onto it. Profile resolution is injected rather
than imported so the helper is unit-testable without reaching through
resolveStoredSession -> getSession(); a drift callback lets each caller
keep its own abort semantics via SessionRecoveryAborted.
Co-authored-by: xxxigm <xxxigm@users.noreply.github.com>
Eight cases across the whole class, not just the reported path:
- cold resume rebinds from the selected row before resume settles
- an empty runtime cwd releases ownership (the permanent-staleness half)
- releasing leaves the PATH intact, so panes don't collapse and the persisted
workspace survives
- a session row outside the loaded sidebar page doesn't blank the pane
- a non-git workspace with a null git_repo_root still uses its row cwd
- the branch label clears so the previous project doesn't leak
Verified as a real barrier: reverting utils.ts fails 5 of these.
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
Co-authored-by: ZHJay <ZHJay@users.noreply.github.com>
Co-authored-by: worlldz <worlldz@users.noreply.github.com>
`_fallback_session_info` returned `_default_session_cwd()` — the directory the
gateway process happened to start in — so a session resumed without a built
agent told its client the wrong workspace, and the desktop Files pane painted
the wrong project even after the renderer rebound correctly.
Return the session's own cwd and always emit `branch` ("" outside a git repo)
so a client can clear a stale label instead of retaining it. This matches the
contract `_lazy_session_info` already follows a few hundred lines above.
Co-authored-by: ZHJay <ZHJay@users.noreply.github.com>
`session.info` claimed the cwd for whatever the selection happened to be, so a
background tile's payload could re-point a fresh draft at the tile's workspace.
Treat a nonempty `stored_session_id` as non-matching when no primary session is
selected; only an ABSENT id uses the selected-session fallback (the backend
omits it on a lazy session, and refusing there would leave the workspace
un-owned for the rest of the conversation).
Matching goes through the lineage rather than raw string equality: the backend
id is the live session_key, which auto-compression rotates to the continuation
tip, while a selection made from a pinned row holds the stable lineage root.
Comparing those literally reads one conversation as two.
Co-authored-by: ZHJay <ZHJay@users.noreply.github.com>
Two defects left the Files pane showing the previous project's tree:
- `applyStoredSessionPreviewRuntimeInfo` reset every composer atom EXCEPT cwd,
and runs before the `session.resume` RPC. The sidebar row already knows the
conversation's workspace (`cwd` is in the compact row projection), so mirror
it on the same tick the selection changes.
- `if (info.cwd)` was truthy-only, so a detached session reporting `cwd: ''`
never cleared and the pane stayed pinned to the last project for the rest of
the session — the "not always" in the report. Empty is now authoritative.
Empty routes through ownership release rather than a persisted `''`:
`setCurrentCwd` writes to localStorage and seeds `$currentCwd` on next boot, so
blanking would also wipe the remembered workspace.
Only `cwd` is consulted, never `git_repo_root` — the latter is documented null
for non-git workspaces and not-yet-backfilled rows, so falling back to it reads
as "no workspace" and blanks a pane that was correct. A session outside the
loaded sidebar page (no row at all) releases ownership instead of blanking, for
the same reason.
Also claims ownership on the warm-cache path (its missing-RPC compat branch
returns before `applyRuntimeInfo`) and for a center tile, whose Project "+"
create left the right rail on the previous session's folder.
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
Co-authored-by: worlldz <worlldz@users.noreply.github.com>
Co-authored-by: ZHJay <ZHJay@users.noreply.github.com>
`$currentCwd` is a global singleton, but a conversation switch publishes the
new stored session id immediately while the new workspace only arrives when
`session.resume` settles. For that whole window the path still names the
PREVIOUS conversation, and every workspace-derived surface treats it as
authoritative.
Track WHICH conversation the live path describes instead of trying to keep the
path itself in lockstep. Ownership — not emptiness — is what makes the switch
atomic: clearing the path would collapse the workspace/review panes and drop
file-tree state on every switch, so the path stays put and is simply marked
not-yet-owned.
The released marker is deliberately not `null`: `null` MATCHES a fresh draft
(whose selected id is also null), so releasing to it would hand a leftover path
to the draft as its own workspace.
Co-authored-by: ZHJay <ZHJay@users.noreply.github.com>
The main-checkout test compared the two probe roots with raw string
equality. When they differed only in separator spelling, the repo's own
checkout was misclassified as a linked worktree: it fell through to the
worktree branch and was labeled by directory basename. The sidebar then
showed one checkout twice — a dir-labeled lane plus the branch-labeled
`main` lane built from the same sessions.
Compare with `_path_key` so platform path identity decides, matching how
every other path comparison in this module is already keyed.
Tests cover the single-checkout case and the main + linked-worktree case;
both fail before this change (the lane comes back labeled `repo`, not
`main`).
`common_repo_root` derives its answer via `os.path.realpath` +
`os.path.dirname`, which rewrite separators to the platform-native `\` on
Windows, while `repo_root` returns raw `--show-toplevel` output (always
forward slashes). The same directory therefore came back spelled two ways
from a single `resolve()` call, so callers comparing the two roots for
identity could not see that a repo's own checkout IS its common root.
Normalize the derived path back to git's forward-slash spelling so both
probes agree byte-for-byte.
Fleet audit showed these task skills are commonly needed across users;
shipping bundled per Teknium's direction. Docs and tests follow the
bundled paths.
Surfaced during the post-merge review pass on our own #81113 follow-up:
cached_fetch_api_models gained _cache_entry_valid (numeric-'at'
validation) but its sibling cached_provider_model_ids still did
float(entry.get('at', 0)), which raises ValueError/TypeError on a
hand-edited or corrupted provider_models_cache.json row and propagates
uncaught into the /model picker call sites. Same fix, same helper:
corrupt rows are now a cache miss (live fetch), never an exception.
Both wrappers now share the identical validity predicate, closing the
divergence the 'mirrors' docstring promised away.
Also two test nits from the same review: unused OrderedDict import
dropped and the drain-order assertion strengthened to pin LRU-first
FIFO order in tests/gateway/test_agent_cache_pressure.py.
Mutation-checked: restoring the raising float() form makes the new
corrupt-at tests fail.
Two correctness holes left by the session-switch perf work (#72504 / #72524):
1. MessageRenderBoundary only cleared a swallowed transient useClientLookup
error when the structural resetKey changed. Mid-turn, ids/roles/count are
stable, so a lookup race during a stream left the boundary rendering null
for the rest of the turn. The boundary now self-retries on a 0ms timer
(rAF never fires in a parked renderer), bounded to 5 consecutive
transient catches with the budget reset on recovery; the structural
resetKey path is unchanged, and non-transient errors still re-throw.
2. cwd / gateway / sessionId were removed from the messageComponents memo
deps and read through a render-time ref so session switches stop
reminting the component types. But a mounted UserEditComposer only
reads that ref when it renders, and a same-session change (cwd remap,
gateway reconnect) leaves every ThreadMessageList prop referentially
equal, so the memo'd list bails out and the open composer keeps stale
values: @-completions, slash completions, and OS-drop uploads target
the old cwd / gateway / session. Thread now provides the three values
through a memoized ThreadEditContext; context propagates through the
bail-out, the component type identity is untouched, and the transcript
never remounts.
Rescope: hermes verify fills only the runtime-smoke gap and plugs into
the pieces Hermes already has instead of standing beside them.
- agent/verification_evidence.py: record_verify_run() — explicit ledger
write for hermes verify results (shared _insert_evidence factored out
of record_terminal_result). Passing runs mark the workspace passed
like scripts/run_tests.sh; failures are recorded; --phase/--skip-start
runs are recorded as targeted scope.
- hermes_cli/verify_cmd.py: record results into the ledger on completion
(fail-silent, HERMES_SESSION_ID attribution); on the detect path merge
detect_project_facts verify commands the recipe missed into the
recipe's test list (never applied to a saved manifest).
- agent/verification_stop.py: recipe-aware nudge — when the workspace
has a runnable recipe (start command or .hermes/environment.json),
suggest hermes verify --json as the preferred full check; cheap,
try/except-guarded detection that can never break the nudge path.
- agent/verify/recipes.py: document layer ownership (coding_context =
cheap prompt facts; verify/recipes = deep runtime recipe).
- tests/verify/test_ledger_and_nudge_integration.py: 17 tests covering
ledger pass/fail recording, the closed edit->nudge->verify->satisfied
loop, recipe-aware nudge wording + fail-silence, and the facts merge.
Teknium
rob-maron
kshitij
Dominic Bejar
toprakeker
hermes-seaeye[bot]
brooklyn!
0xarkstar
bb
Ben Barclay
Jakub Wolniewicz
Adolanium
Jeffrey Quesnelle