Three mechanisms to detect and notify when gateway sessions stall silently:
1. Mid-turn activity heartbeats stamped to SessionDB so hermes sessions list
and hermes status show progress during long turns without new message rows.
2. Stall watchdog: when a busy session has pending inbound and the shared
activity clock is idle past agent.session_stall_timeout (default 300),
log a WARNING and notify the user once to try /new. Notify-only; does
not kill the turn.
3. Compaction timeout: fenceless compress_context callers get a progress-aware
host budget (compression.context_timeout_seconds default 120 idle,
compression.context_total_ceiling_seconds default 600 ceiling). On timeout,
cancel via commit fence, skip compaction without dropping messages, and
continue the turn.
Closes#72016 (slices 1-3; slice 4 cumulative SSE stream-retry deadline
remains a follow-up).
Cherry-picked from PR #72424 by @fangliquanflq.
Every tool schema ships on every API call. The terminal schema was
5,641 chars (~1,410 tokens) and execute_code 2,842 (~710) — the two
largest core tools, padded with repeated war stories and triple-stated
rules. Schema token audit across 88 tools: ~33k tokens total.
This trims prose while preserving every hard rule (each still stated
exactly once):
- terminal description 2,324 -> 1,233 chars: tool-redirect lines
collapsed to one sentence; background/notify guidance deduplicated
(was stated in desc + 2 params); PTY/pager rules merged.
- background/notify_on_complete/watch_patterns params 692/508/1,114 ->
~330/250/490 chars: kept the mutual-exclusion contracts, the
rate-limit consequence, and the bounded-vs-long-lived distinction;
dropped narrative repetition.
- execute_code description tightened (helper docs inlined to one line
each; when-to-use kept).
Net: terminal schema 5,641 -> 3,386 chars, execute_code 2,842 -> 2,522
— ~700 tokens saved on EVERY request with the terminal+code toolsets.
One test updated (pinned a removed phrase; now pins the rule's new
phrasing).
Truncated terminal output was information LOSS: the middle was gone
and the only recovery was re-running the command (data: 1,394
truncation markers in a 250k-call window, with re-runs and grep
retries chained behind the big ones).
Truncation is now deferred retrieval (opencode/goose/qwen-code
pattern, codex's original_token_count idea):
- tools/environments/base.py: _BoundedOutputCollector gains an
optional spill tee — when foreground output overflows the capture
window, the FULL stream is teed to
~/.hermes/cache/terminal-output/out-*.log (lazy file creation with
backlog backfill, 5MB hard cap, 7-day opportunistic cleanup,
disk errors never break execution). All three _wait_for_process
returns attach {output_total_chars, full_output_path} via a shared
finalizer.
- tools/terminal_tool.py: redacts the spill with the same
redact_terminal_output pass as the visible output (no secret
persists unmasked), then surfaces output_total_chars,
full_output_path, and a truncation_note pointing at
search_files/read_file instead of a re-run.
Non-truncated results are byte-identical; internal unbounded
consumers (file-ops cat reads, RPC reads) are untouched (spill only
arms with bounded_capture=true).
'Found N matches for old_string' (190+ occurrences in a 250k-window)
previously reported only the count, forcing a re-read of the file to
find the occurrences before retrying. The error now appends up to 5
'L<line>: <snippet>' rows (80-char cap per snippet, overflow noted),
so the model can disambiguate in ONE follow-up — add neighboring
context or choose replace_all — without the intermediate read.
Applies to both patch modes (replace + V4A) since they share
fuzzy_find_and_replace.
#77128 fixed the orphaned zero-match probe but shipped no tests, so the
same class of break can recur: an early return anywhere in the rg branch
silently makes the whole steering tier unreachable.
Asserts hint wiring per search engine with the probe stubbed to a
sentinel (the probe itself needs rg, so a real-text parity assertion
fails on the grep leg for an unrelated reason), plus the negative case
and the rg newline-warning skip that the early return originally
existed to preserve.
Sabotage-verified against 794d6c434e: restoring the early return turns
4 red; a naive fix that also drops the rg newline guard turns 1 red;
attaching the hint when matches exist turns 2 red.
Integration regression between two merged PRs: #77102's rg-path early
return (added to skip the grep-era line-oriented warning) also skipped
the #77011 zero-match steering probes (case-insensitive, hidden-file,
literal-vs-regex), silencing them on the primary engine. Caught by
#77001's rebased CI run (its branch carried both features together for
the first time).
_search_content now runs the zero-match probe block for BOTH engines
and only exempts the rg path from the legacy line-oriented \n warning
(rg auto-enables --multiline; the grep fallback keeps the explanation).
The check-attribution CI gate kept bouncing salvage PRs because mapping
contributor emails was a manual, easy-to-forget step (bare
<login>@users.noreply.github.com emails don't auto-resolve like the
<id>+<login> form).
- scripts/audit_pr_attribution.py: mirrors the CI gate's logic exactly
(merge-base scan, same skip rules). Report mode for pre-push checks;
--fix auto-resolves via the bare-noreply local part (verified against
the GitHub users API) or GitHub email search, then writes
contributors/emails/<email> files via add_contributor.py. Prints a
confirm-the-human warning on bare-noreply resolution since the local
part is user-controlled (the bryan->hydraxman case).
- contributor-check.yml: failure output + review_status how_to_fix now
lead with the one-command fix instead of hand-editing instructions
(also drops the stale 'edit AUTHOR_MAP' guidance — AUTHOR_MAP is
frozen).
The top execute_code failure shapes in production (state.db mining)
are sandbox-contract confusions, not logic bugs: importing tools that
aren't in the sandbox from hermes_tools (23x in one window, incl.
importing the built-in helpers json_parse/shell_quote/retry), importing
third-party packages absent from the sandbox interpreter (matplotlib
6x), and indexing tool-result dicts as strings. The stderr traceback
alone sends models into re-diagnosis loops.
Failed scripts (exit != 0) now carry one actionable 'hint' field:
- unavailable hermes_tools import -> lists the tools that ARE
importable in this session + points to normal tool calls otherwise;
- built-in helper import -> 'no import needed, call it directly';
- ModuleNotFoundError -> 'sandbox has stdlib only; use terminal() with
the project venv for third-party packages';
- string-indexing errors -> 'tool functions return dicts, do not
json.loads them'.
Bounded 4KB stderr scan, first match wins, never raises; successful
scripts and unknown failures are untouched.
A regex \n (or a raw newline) in a search_files content pattern cannot
match in rg's default line-oriented mode. It previously either
hard-errored ('the literal "\n" is not allowed in a regex' — 17
occurrences in the production window) or, after the newline-warning
patch, returned 0 matches with an explanation — either way the model's
cross-line search intent required a manual workaround.
The rg engine now detects the pattern shape (_pattern_has_regex_newline,
already used by the warning path: odd-backslash \n escape or raw
newline; escaped \\n literals excluded) and enables -U/--multiline up
front, noting the mode switch in the result warning. Plain patterns are
untouched; the old line-oriented explanation is retained for the grep
fallback engine, which has no multiline mode.
skill_view re-sent full skill content on every call: ~286k tokens of
verbatim repeat views in a 400k-msg production window (one session
loaded the same skill 9 times), and a single repeat view of a large
skill costs ~25k tokens.
Mirrors read_file's proven unchanged-stub pattern: a per-task cache
keyed on (resolved name, file_path) with an mtime+size fingerprint of
the served file. On a repeat view of an UNCHANGED file, return a short
stub pointing at the earlier result. This does NOT violate the
skills-are-loaded-fully rule — the stub only ever replaces content
that is already fully present earlier in the same conversation, and:
- any on-disk change (patch, external edit) invalidates the entry;
- context compression clears the cache (wired next to
reset_file_dedup in conversation_compression.py) so post-compression
re-views return full content;
- setup-needed views are never deduped (readiness can change without
the file changing);
- no task_id -> no dedup; caches are task-isolated; 200-entry cap.
Live E2E: repeat view of hermes-agent-dev 99,739 chars -> 374-char
stub.
write_file confirmed only SIZE (wc -c) after writing — never content.
Models compensated by re-reading files immediately after writing them
(154 verify-reads in a 400k-msg production window), and a corrupted
write (truncated pipe, backend FS oddity) could silently pass.
The write path now compares the on-disk sha256 against the intended
content (one shell call). Three outcomes:
- match -> result carries verified: true; the schema tells the model
an explicit contract: do NOT re-read to check the write landed.
- mismatch -> hard error ('The write did not persist correctly'),
mirroring patch_replace's existing post-write verification.
- backend can't hash (no sha256sum) -> flag omitted, write unaffected.
Hashes the shim-adjusted content (after CRLF/BOM preservation) so
Windows-line-ending and BOM round-trips verify correctly; surrogatepass
encoding matches the rest of the codebase's hashing of model text.
When old_string survives all 9 fuzzy strategies without a match but
the closest candidate line matches after stripping whitespace, the
failure is whitespace-shaped (tabs vs spaces, indent depth). The
did-you-mean hint now appends a two-line diagnosis with leading
whitespace made visible:
Whitespace difference detected (→ = tab, · = space):
file has: →def start(self):
you sent: ····def start(self):
Use the exact whitespace shown in 'file has'.
Pattern ported from crush's diagnoseMismatch (agent-codebase survey) —
it converts the residual dead-end error into a one-turn fix. Only the
leading run is visualized (interior spacing stays readable); content-
shaped misses and raw-exact candidates are unchanged.
Unify @null-runner's selector approach with the merged #77019 keepalive:
the reapers now match live delegations by origin UI sid AND (when the TUI
owns the durable lifecycle, never for gateway-viewer tabs, #60609) by the
durable session_key, so a delegation dispatched from an earlier tab of
the same resumed session still keeps it alive. has_live_for_session also
counts 'stalling' to match the #77019 live-state set. Explicit teardown
(_finalize_session) interrupt semantics unchanged.
Follow-up to the recovery-recipe commit on this branch, per review:
instead of only TELLING the model to re-author the payload via
write_file (2 turns), materialize the blocked command to
~/.hermes/cache/blocked-scripts/blocked-*.sh and point the recovery
at it directly: 'saved to <path> - review it, then run
terminal(command="bash <path>")' (1 turn).
Safety posture is unchanged or better:
- Nothing is executed here; the file is only written.
- The bash <path> follow-up goes through the normal execution
pipeline, including the referenced-script content guard, which
inspects script files named in commands - the payload is MORE
visible to policy than it was inline.
- Genuine hardline blocks (destructive ops) never save anything
(test-asserted).
- Save failures fall back to the previous manual write_file recipe.
- 7-day opportunistic cleanup of saved payloads.
Two block classes from production mining (250k-call window) that
models answered with blind rephrase-retries:
1. Parser-limit / malformed-payload hardline blocks (198x): these fire
on oversized inline payloads (heredocs, giant one-liners), not on a
forbidden operation - but the message read like a permanent ban.
The block now appends: 'RECOVERY: ... write the script to a file
with write_file, then run bash /path/script.sh - do not retry
inline.' Genuine hardline blocks (destructive filesystem
operations) are unchanged.
2. Backgrounding-wrapper blocks (200x): the guidance now spells out
the exact corrected call shape - 're-send WITHOUT the wrapper as
terminal(command="<cmd>", background=true,
notify_on_complete=true)' - instead of describing the feature
abstractly.
process(action='wait') hitting its window returned status='timeout'
with a terse note — models read it as an error and re-issued identical
waits (process is the #1 exact-duplicate tool call in production: 511
dupes in a 400k-msg window; wait is 57% of all process actions).
The timeout result now carries:
- process_running: true — machine-readable 'this is a status, not a
failure'
- an explicit note: 'Wait window of Ns elapsed — the process is still
running. This is not an error. Uptime: Ms.' plus the right next step:
when notify_on_complete is set, 'you will be notified on exit — do
more work instead of waiting again'; otherwise a pointer to
notify_on_complete for next time.
- the clamp note (requested > max) now composes with the status note
instead of replacing it.
Exited/interrupted results are unchanged.
Third zero-match steering tier: when a content search finds nothing in
visible files, probe once with rg --hidden --no-ignore --count-matches.
If the pattern exists only in dotdirs or gitignored files, the result
says so and tells the model to search the hidden path explicitly.
Found live by the benchmark battery: a task with a match inside
.hidden/ returned a bare 0 and the model missed the file entirely in
2/3 baseline runs.
Two dead-turn classes from production mining (state.db, recent window):
1. 13.9% of 19.6k content searches return 0 matches with no steering.
Now a 0-match content search runs one cheap rg -i --count-matches
probe (plus an rg -F probe when the pattern has regex metachars) and
attaches what it found: '0 exact matches, but N case-insensitive
matches — casing may be wrong' / 'N literal matches — metacharacters
need escaping'. True zero-match results stay clean (no noise).
2. 122 'Path not found' failures came from models passing several paths
in ONE path string ('dir1 dir2 dir3', comma lists). Instead of
failing wholesale, split the string, search every path that exists,
merge results, and report skipped parts in a warning. Single-path
misses keep the existing Similar-paths hint; all-missing multi-path
strings still error.
Both probes are bounded (count-only rg, 30s timeout, max 2 invocations)
and wrapped so a probe failure can never break the search result.
- New website/docs/user-guide/messaging/a2a.md: when/where to use A2A
(cross-machine, specialist peers, being callable) vs delegation/kanban
for same-machine multi-agent; enable, outbound tools, inbound surface,
security model, env reference, quick test, troubleshooting. Registered
in sidebars.ts and the messaging index.
- README/DESIGN/plugin.yaml/protocol.py prose updated to name the A2A
v1.0 canonical discovery path /.well-known/agent-card.json (the code
already served both; only the docs lagged).
A2A authenticates every inbound request via bearer token in do_POST
(401 before dispatch). Without overriding authorization_is_upstream=True,
the gateway's per-platform user allow-list ({PLATFORM}_ALLOWED_USERS)
rejects A2A peers because their identity is a token-derived name or
pod IP, not a platform account in any configured allow-list. Messages
never reach the agent and callers get empty replies.
This is authorization delegated to the bearer-token transport, not a
fail-open: every request is 401'd if the credential is wrong.
Reported by kuangmi-bit (PR #41711 comment, Jun 27).
Attribution: gfdsa's a2a-hermes repro fixture (LOCAL PATCH triad-hermoperator).
The other two patches from gfdsa's fixture were already in our branch:
- k8s Agent Card URL derivation (_request_public_url, commit ea59b85c2)
- send() gating on metadata['notify'] (reply-capture fix, commit ea59b85c2)
All 168 tests pass (151 unit + 17 integration).
Two bugs reported by gfdsa (PR #41711 comment, Jul 12) that break the
official a2a-sdk 1.1.0 Python client:
1. Task objects serialized non-spec createdAt/lastModified fields.
The A2A v1.0 Task proto (lf.a2a.v1.Task) only has id, contextId,
status, artifacts, history, metadata. Strict ProtoJSON parsers
reject unknown fields with ParseError. Removed both fields from
build_task(); created_at param kept for call-site compatibility.
2. SSE streaming frames were not JSON-RPC wrapped. A2A v1.0 §9.4
requires data: {"jsonrpc":"2.0","id":...,"result":{StreamResponse}}.
sse_data() now accepts req_id and wraps in JSON-RPC envelope.
sse_done() changed from 'data: {}' to SSE comment ': done' so
SDK doesn't try to parse an empty JSON-RPC response.
All call sites in adapter.py (_emit_terminal, _rpc_message_stream,
_rpc_tasks_subscribe) updated to thread req_id through.
Tests updated: 153 pass (151 unit + 17 integration, including 2 new
tests for JSON-RPC envelope wrapping and fallback behavior).
Refs: gfdsa/a2a-hermes reproduction repo
## File/data Parts (v1.0 unified Part)
- file_part(url=, raw=, filename=, media_type=) builds v1.0 file Parts
- data_part(data, media_type=) builds v1.0 data Parts
- message_with_parts(role, parts, context_id=) builds Messages with mixed Part types
- extract_text now renders file/data Parts into the text stream:
- File with URL: '[file: name] https://url (mediaType)'
- File with raw: '[file: name] N bytes base64-encoded (mediaType)'
- Data: '[data (mediaType)]\n{json}'
- v0.3 file (file.fileWithUri) and data (kind=data) still accepted
- Outbound replies stay text-only (agent produces text)
## Push notification config full CRUD
- get_push_config(task_id, config_id) — retrieve by task, optionally by configId
- list_push_configs(task_id) — list all configs for a task (max 1 per task)
- delete_push_config(task_id, config_id) — remove a config
- New JSON-RPC methods: tasks/pushNotificationConfig/get, /list, /delete
- New adapter handlers: _rpc_push_config_get, _list, _delete
- All return spec-shaped PushNotificationConfig with configId + createdAt
## Tests
- 6 new unit tests for Part builders + extract_text with file/data
- 13 new unit tests for push config get/list/delete (happy + error paths)
- 2 new integration tests over real HTTP:
- test_mixed_parts_delivered_to_agent: file URL + data JSON reach agent
- test_push_config_crud_over_http: full create→get→list→delete cycle
- Old test_extract_text_skips_non_text_parts replaced (now renders, not skips)
Total: 151 tests (134 unit + 17 integration), 0 failed.
DESIGN.md updated: file/data Parts and push config CRUD removed from
out-of-scope list.
Consolidates 5 follow-up PRs onto the a2a-work branch:
1. Reply-capture fix (#56437): adapter.send() now only resolves the
blocked RPC Future when metadata['notify'] is True (the gateway's
final-reply marker). Interim sends no longer short-circuit the
response. Also accepts **kwargs in connect() for reconnect compat.
2. Slash command passthrough (#53743): wrap_inbound() passes /-prefixed
text through unwrapped so the gateway command processor sees it.
Fixes /sethome deadlock during A2A onboarding. Documented security
trade-off (bearer auth at network layer compensates).
3. Routable URL in Agent Card (#53736): _build_card() now derives URL
from A2A_PUBLIC_URL env > X-Forwarded-Host/Host header > bind host.
Fixes k8s bug where Agent Card advertised 0.0.0.0.
4. contextId multi-turn memory (#53756): _handle_inbound_task() now
checks top-level params.contextId first (A2A spec), falls back to
params.message.contextId (legacy). Outbound a2a_call also sends
contextId at both top-level and inside message.
5. Type checker fixes (#53759): TypedDict for _SCHEMAS, _FunctionSchema,
_ToolSchema. Removes str() band-aid casts.
All 45 tests pass including new tests for each fix.
Zero core files modified — only plugins/platforms/a2a/ and tests/.
Credits: @davidrobertson (#56437), @knoal (#53736, #53743, #53756,
#53759), @kuangmi-bit (slash command bug report), @gfdsa (k8s URL bug
report), @shivasymbl (#45996 userContext OBO).
Live Tier-3 testing (CLI agent -> a2a tools -> live peer gateway -> model)
surfaced two bugs the kwarg-style unit tests masked:
1. registry.dispatch calls handlers as handler(args, **kwargs) — args is the
whole dict positional. The handlers used keyword params (url=, agent=), so
the dict bound to the first param and .strip() raised
'dict object has no attribute strip'. Rewrote all three handlers to take
args: dict (matching the spotify/google_meet convention). Added a
registry-dispatch regression test that exercises the real call path the
direct-kwarg tests never hit.
2. The model repeatedly reached for agent_name= instead of agent= (6 retries
before success). Accept agent_name/name and message/text/task aliases so a
reasonable guess succeeds first try.
Verified live: client agent discovers the peer's Agent Card, calls it, and
gets the reply back (PONG round-trip confirmed on both client audit log and
peer conversation log). 39 plugin tests pass.
The a2a client tools are registered unconditionally by the plugin, but a
newly-registered plugin toolset defaults to ENABLED for every platform until
the user has seen it in 'hermes tools'. That force-injected 'a2a' into every
agent's enabled_toolsets, leaking 3 tool schemas to all users and breaking
tests that assert exact toolset membership
(test_api_server_toolset::test_create_agent_respects_config_override).
Add 'a2a' to _DEFAULT_OFF_TOOLSETS so it stays opt-in (user enables via
'hermes tools'), matching the spotify precedent. The inbound platform
adapter is already opt-in (only instantiated when the a2a platform is
enabled); this aligns the outbound client tools with the same posture.
Single platform-adapter plugin under plugins/platforms/a2a/ — zero core
edits — that supersedes the entire A2A PR/issue cluster. Built on the
ctx.register_platform + ctx.register_tool surface the codebase now exposes.
Outbound (a2a toolset): a2a_discover / a2a_call / a2a_list let the agent
call any A2A-compliant peer over JSON-RPC message/send. Inbound (platform
adapter): a stdlib http.server serves an Agent Card at
/.well-known/agent.json and routes incoming tasks into the agent's LIVE
gateway session (the #11025 insight) — same agent, full memory — returning
the reply over A2A.
Security on by default: no bearer token => 127.0.0.1-only bind; constant-
time bearer auth; inbound prompt-injection filtering + untrusted-peer
framing; outbound credential redaction; append-only audit log; per-context
conversation persistence outside the compaction pipeline.
Stdlib only (no a2a-sdk). 37 tests incl. a live HTTP round-trip
(card + message/send + reply) and a bearer-auth 401 path.
Production mining (state.db, 400k-msg window): 60.2% of 104k terminal
calls carry a defensive 'cd X && ' prefix (~925k tokens of pure prefix)
and 2,462 failed calls led with cd — the model cannot see cwd state, so
it re-asserts it on every call and runs pwd/ls diagnostics after
directory changes.
The result dict now includes a 'cwd' field whenever the session cwd
after the command differs from the cwd it started in (cd, pushd,
chained cd). Stable-cwd commands are unchanged (no field, no noise).
Per-command workdir overrides stay transient by contract and never
echo. Schema note added so models learn to trust session cwd instead
of prefixing. Pattern borrowed from crush's <cwd> injection.
realpath comparison avoids false echoes through symlinks; the echo is
wrapped defensively so a backend without .cwd can never break the
result path.
The #1 patch failure class in production (state.db mining, 250k-window)
is a re-send of an edit that already landed: 'old_string and new_string
are identical' (299 occurrences) plus a share of hunk-not-found errors
where the new text is already in the file. These errored, sending
models into re-read/re-patch loops.
New tools/fuzzy_match.is_already_applied(content, old, new) — a
conservative check requiring (1) non-trivial new_string (>=8 chars),
(2) EXACT presence of new_string, (3) old_string gone (unless
identical). Wired into three sites:
- patch_replace (replace mode): returns success + no_change: true +
an explicit note instead of the identical-strings / no-match error.
- V4A validation phase: an already-applied hunk validates as a no-op
so multi-hunk patches no longer fail wholesale when one hunk landed
in a prior call.
- V4A apply phase: mirrors the same skip so the two phases agree.
Genuine no-matches (new text absent) and half-applied renames (old
text still present) keep their error behavior — covered by tests.
Production mining (state.db, 28.5k read_file calls in the recent
window) shows 74.3% of reads truncated — nearly all by the 500-line
default, not the char budget (22,443 line-limit vs 13 char-budget
truncations). That churn produced 12,229 redundant re-reads, and after
a truncated result the most common next move was fleeing to terminal
cat/sed (4,608 times) — the pagination contract was not trusted.
Median truncated file is 2,422 total lines, so a 2000-line default
makes 44% of today's truncated first-reads complete in one call while
the unchanged ~100K-char budget still caps worst-case result size
(same ceiling as before: 500 lines x 2000-char line cap = the same
100K). Schema max was already 2000.
Touchpoints: DEFAULT_READ_LIMIT + both read_file signatures
(file_operations.py), read_file_tool + schema text (file_tools.py),
execute_code sandbox stub docs (code_execution_tool.py), 3 tests
pinning the old default.
Builds on the adapter list_channels() hook (cherry-picked from #43545 by
@Guoen0):
- plugins/platforms/simplex: implement list_channels() — enumerates
contacts (/contacts) and groups (/groups) over the live daemon
WebSocket into the channel directory. Returns None when the WS is
down so the directory falls back to session discovery instead of
wiping known targets.
- hermes send --list: merge configured-but-undiscovered platforms into
the listing. Previously a platform configured only via env (e.g. a
fresh SimpleX setup used for outbound sends) was silently omitted,
leaving users guessing at platform names.
- format_directory_for_display(): accept an explicit platforms view and
render empty platforms with a targeting hint instead of hiding them.
- docs: simplex hermes-send section.
Reported by Fedpostoffice on Discord (simplex missing from
hermes send --list; guessed platform names simplex-chat/simplex-relay).
When a command exits non-zero, scan the first 4KB of output for
well-known failure shapes and attach one short, actionable recovery
hint to the tool result ('hint' field):
- gh 'Unknown JSON field' (9.2k occurrences in a 250k-call window)
- git merge conflicts (1.2k) — stop verbatim retries
- command not found (1.0k), incl. python->python3 and pip->pip3
- ModuleNotFoundError (739) — venv activation guidance
- 'already exists' (633), gh rate limits (133), permission denied
- exit-code-only tier: 124 timeout, 126 not-executable, 137 SIGKILL
Hints are suppressed when the existing exit_code_meaning tier already
explains the code (grep=1 etc). Pattern order = production frequency
from state.db mining; first match wins; pure function, no I/O.
The delivery worker is a daemon thread, so a short-lived process
(hermes chat -q, a cron session) could exit right after firing
on_session_end — silently dropping the headline event. Register a
bounded atexit flush (5s) when the worker starts: a dead endpoint can
delay exit slightly, never hang it.
Live-verified: hermes chat -q now delivers on_session_start,
post_tool_call, and on_session_end to a real receiver; regression test
runs a subprocess that exits without flushing (sabotage-verified).
- delivery_id is now generated once per firing and used for both the
X-Hermes-Delivery header and the signed body's delivery_id field —
previously they were two different uuid4s, breaking receiver-side
dedupe as documented.
- 3xx responses are no longer followed: urllib's default redirect
handler converts a redirected POST into a body-less GET, silently
dropping the signed payload. Redirects now log a misconfiguration
warning and count as delivery failure (no retry).
- Docs: receiver-side replay-protection guidance (dedupe on
delivery_id, timestamp freshness window) + redirect semantics.
- Tests: 5xx retry count, redirect-not-followed (sabotage-verified),
header/body delivery_id equality.
The inverse of the inbound webhook platform: hooks.outbound in
config.yaml lists HTTP targets + the plugin-hook events they subscribe
to (on_session_end, subagent_stop, post_tool_call, ...). Each firing
POSTs a JSON payload (same top-level shape as shell hooks' stdin wire)
signed GitHub-style with HMAC-SHA256 (X-Hermes-Signature-256).
Rides the existing hook bus — notify-only callbacks registered on the
plugin manager at the same CLI/gateway/main entry points as shell
hooks. Delivery is fire-and-forget via a bounded queue + single daemon
worker thread, so a dead endpoint can never stall a tool call. Bounded
retries (5xx/conn errors once; 4xx never). secret_env preferred over
inline secret. HERMES_SAFE_MODE skips registration. hermes hooks list
shows outbound targets with signed/UNSIGNED status.
Zero new model tools, zero new subsystems.
Answers and written deliverables that rest on retrieved information now get
inline numbered citations plus a mechanically-rendered Sources list, with a
persistent ledger that makes a hallucinated citation detectable.
- skills/research/grounded-citations/scripts/sources.py: stdlib citation
ledger (add/ingest/list/render/verify) at
$HERMES_HOME/cache/citations/ledger.json, profile-aware, O_EXCL-locked so
parallel subagents sharing a ledger can't collide on ids
- SKILL.md: cite-while-drafting procedure, register-at-retrieval rule,
pitfalls, verification gate
- references/citation-formats.md: per-target placement (markdown, LaTeX/PDF,
docx, pptx, xlsx, wiki, BibTeX handoff to research-paper-writing)
- references/grounding-rationale.md: why numbered ids (ALCE 2305.14627,
WebGPT 2112.09332, Perplexity marker conventions), and how this relates to
the in-process registry in PR #44833
- tests/skills/test_grounded_citations_skill.py: 30 tests
The LRU-cap test in tests/tui_gateway/test_protocol.py stubs
_close_session_by_id with a two-arg lambda; the reaper now passes a
revalidation predicate. Widen the stub signature.