delegate_task gains a control plane: action='list' / 'steer' / 'stop'
let the parent agent see, redirect, and early-stop its own running
subagents mid-flight — the model-facing counterpart of the TUI's
delegation.pause / subagent.interrupt / subagent.steer RPCs.
- action='list': live children of this conversation's spawn tree
(ids, goal, status, running_seconds, accepting_steer, live
transcript path). Ownership is enforced via a _delegate_parent_ref
weakref chain stamped at child build time, so a conversation can
only control its own descendants, never a sibling tree.
- action='steer': queues text into a running child via the existing
steer_subagent() registry path (delivered at the child's next tool
boundary; missed steers surface as missed_steer in the completion).
- action='stop': interrupt_subagent() — child stops at its next
iteration boundary, partial result still re-enters as a completion.
- Spawn dispatch response now includes subagent_ids + control hint.
- Control actions run synchronously (never backgrounded) and bypass
the spawn pause gate and depth limit; they also never consume the
per-turn subagent spawn cap, and remain usable once the cap is hit
(that is when stop matters most).
- Small-model robustness (found live with gpt-5.4-mini on Nous
Portal): tasks=[] alongside goal no longer trips the "Batch mode
requires at least 2 tasks" gate — treated as single-goal.
- CLI display: control calls render as "steer sa-…" / "list" instead
of an empty goal.
Live-tested E2E on Nous Portal (fable-5 + gpt-5.4-mini): full
spawn→list→steer→stop cycle, plus a steer-efficacy run where the
child acked the steer mid-essay and switched topics before finishing.
New desktop_ui tool: the agent proposes an MCP server (install/enable/
authorize + a one-line reason) and blocks on mcp.setup.request until the
renderer's consent card answers mcp.setup.respond with the outcome
(installed/enabled/authorized/declined/unanswered/error). Same lifecycle
as clarify: 10-min timeout, allow_expired late answers, tool lifecycle
events forced on so the card mounts even with tool progress off. Desktop
prompt hint steers the model to the tool instead of hand-editing config;
every other surface keeps the schema out and is pointed at hermes mcp
install.
Desktop-gated (desktop_ui toolset) metadata-only window awareness: the agent
can ask which application window sits directly behind the Hermes window
(app, title, bounds — never pixels). Rides the same blocking bridge as
read_terminal: the gateway emits window.read.request and the renderer
answers window.read.respond.
Post-merge simplify finding on #81613: the wrapper's docstring claimed it
existed for 'existing callers', but every caller was introduced by the same
PR - there was never a pre-existing import path to preserve. All callers
(conversation_loop, tool_executor, run_agent's own flush handler, tests)
now import the canonical hermes_state.classify_persistence_error directly,
matching how is_disk_full_error is consumed. No behavior change; imports
stay lazy inside the exception handlers.
An enterprise deployment hit sustained SQLite write-lock contention on a
shared multi-gigabyte state.db (gateway + CLI processes writing
concurrently). Turns correctly failed closed with
session_persistence_failed, but the only user-facing wording claimed the
disk was full and the gateway rendered a generic failure.
The fast-fail semantics are deliberate and unchanged. This adds a pure
classifier (locked / disk / unknown) applied where the SQLite error is
still visible, threads the cause through the turn-completion explainer,
and stamps a machine-readable failure_reason
(session_persistence_failed:<cause>) plus a guaranteed non-empty error on
the result for downstream surfaces. The cron scheduler's explainer-text
suppression now matches every cause variant so refined wording cannot
leak into scheduled-job deliveries.
- Single source for the approval-derived bound: public human_wait_ceiling()
in tools/approval.py; the gate's lock-timeout helper delegates to it
instead of re-deriving timeout + margin (was duplicated in two modules
and reached for a private _get_approval_timeout).
- Shared _clamped_window_seconds() for the close-time accrual and the
open-window read, so the two clamps are identical by construction.
- Gate __init__ grows session_key kwarg; tests construct via the real
constructor instead of mutating privates post-hoc.
- Gateway test resolves its pending approval via resolve_gateway_approval()
(the production /deny path) instead of hand-rolling queue-entry internals.
- Docstring accuracy: human_wait_seconds monotonicity caveat under cap
eviction; s/pre_tool_block/pre_tool_call/ hook name.
Review-driven follow-up to the #79719 fix:
- Clamp the CLOSE-side accrual too: a wedged window that eventually closed
used to inject its full unclamped overstay into completed_seconds,
retroactively extending a running batch's deadline by hours. Both clamps
now share one ceiling helper (_human_wait_ceiling = approvals.timeout +
HUMAN_WAIT_MARGIN_S), and the gate's lock-timeout uses the same margin
constant so the bounds cannot drift apart.
- Evict idle sessions until the table is under the cap (was: at most one
per insert, so churn could outgrow _HUMAN_WAIT_MAX_SESSIONS). Entries
with an open window are still never evicted.
- Log (debug) instead of silently swallowing a failed session-key snapshot
in the gate constructor.
Tests: close-side clamp regression + table-cap assertion added; suite at
17 passed.
A tool wedged inside _ConcurrentToolAuthorizationGate hung the whole turn
forever (#79719): excluded_seconds() measured residency in gate.run() —
arbitrary code — so an open window grew 1:1 with wall clock and the batch
deadline's remaining was constant (remaining = deadline - window_started;
now cancels out). A hanging pre_tool_call plugin or an approval round-trip
to a dead client defeated the deadline entirely. The serialization lock was
also an unbounded acquire, so every other worker needing authorization
parked behind the wedged holder forever.
Fix, in two halves:
- tools/approval.py grows per-session human-wait accounting
(human_wait_window / human_wait_seconds). The two places that are
verifiably blocked on a HUMAN — the CLI approval prompt and the gateway
approval poll loop — mark their own windows. Both are intrinsically
bounded by approvals.timeout; the open-window read is additionally
clamped to that timeout plus a margin as belt-and-braces.
- _ConcurrentToolAuthorizationGate keeps only serialization, with a bounded
acquire (approvals.timeout + 60s; on expiry the prompt runs unserialized —
the same degradation the start-order gate accepted in #79705).
excluded_seconds() becomes a baseline-delta read of the session's
human-wait total.
A wedged plugin now contributes nothing to the exclusion, so the batch
times out at the normal deadline with correctly labeled results, while a
genuine approval wait — which can legitimately exceed any fixed bound —
still extends the deadline in full. E2E (real AIAgent, worktree imports):
wedged-plugin batch on main never ends (>30s observed, 3s deadline); with
the fix it ends at 3.0s. A 4s simulated approval over a 2s deadline
completes without a timeout label.
Closes#79719
Follow-up to the salvaged start-order gate bound. Two gaps remained, both
reachable through the same knob.
1. The gate bound ignored the batch deadline it sits under. With
HERMES_CONCURRENT_TOOL_TIMEOUT_S below 120s the deadline fired first, so
the parked tools were still reported as "timed out" without ever running --
the exact bug the bound exists to fix. The gate now clamps to
min(120s, batch_timeout / 2), matching the sibling constant's documented
habit of relating the two timeouts.
2. A gate-parked worker released purely by its own timeout could wake up after
the batch was abandoned and dispatch its tool anyway: wasted work whose
result nobody reads, a duplicate post_tool_call for a tool_call_id the turn
already closed as timeout, and agent._current_tool left pointing at a dead
tool for the rest of the session (the main thread's reset already ran).
Abandonment is now a first-class wakeup: both abandon sites set an event and
notify the condition, and a released worker raises _BatchAbandoned instead
of dispatching. Parked threads are reclaimed in milliseconds rather than one
full gate timeout plus a tool runtime.
Also names the tool in the gate-timeout warning. The closure's function_name
binds the last-parsed tool, so logging it directly would have printed the wrong
name; it is threaded through _begin_in_order instead.
Measured, 3-tool batch with the first tool wedged during dispatch:
main PR as-is with this commit
dispatched in batch 0 0 tool_b, tool_c
dispatched after return 0 2 (ghost) 0
_current_tool leaked no "tool_b" no
Adds tests/run_agent/test_start_order_gate.py (3 tests). Mutation-checked
against the parent commit: the starvation guard passes there (it binds the
salvaged fix), while the deadline-clamp and abandonment guards both fail,
reproducing the ghost dispatch as
"tool(s) dispatched after the batch was abandoned: [tool_a, tool_b]".
_begin_in_order parks each concurrent tool worker on a timeout-less
Condition.wait_for until every earlier-ordered tool has advanced through
its dispatch. If one tool wedges during dispatch (observed in production:
a stuck skill_view; also reproducible via any blocking authorization),
three failures compound: every later-ordered worker is starved and never
starts; the batch deadline then falsely reports those never-started tools
as "timed out" (sub-second read_file/search_files calls get blamed while
having done zero work, and the model reasons against that false failure
info); and after the batch is abandoned the parked workers leak forever —
f.cancel() cannot cancel running threads, the per-thread interrupt flag
is never polled inside wait_for, and nothing notifies the condition
again. Confirmed with a faulthandler all-threads dump taken after batch
abandonment showing workers still parked at the gate.
Bound the wait at 120s; on expiry, log a warning and proceed out of
order (worst case: interleaved approval prompts — strictly better than
permanent starvation). The >= predicate lets one worker's timeout-jump
release every skipped worker immediately, and max() keeps the counter
monotonic for out-of-order advancement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(agent): read_preview — the desktop-gated tool that reads the in-app browser
The agent could open the preview pane (open_preview) and read the embedded
terminal (read_terminal), but the browser it had just opened was a black box —
'what does this page say?' had no answer. read_preview mirrors read_terminal
end to end: HERMES_DESKTOP-gated via check_fn (zero schema footprint outside
the GUI), dispatched through the same agent callback pattern, windowed with
start/count so a long page pages instead of flooding context.
* feat(gateway): preview.read blocking bridge
Same lifecycle as terminal.read: the tool blocks on preview.read.request, the
renderer answers preview.read.respond (allow_expired — a slow page extraction
losing the 45s race must not surface a raw 4009), and a timeout emits
preview.read.expire so late answers resolve quietly.
* feat(desktop): the renderer serializes the active preview tab for the agent
preview-reader.ts is the preview analog of the terminal's buffer registry: the
URL pane registers a page reader (webview executeJavaScript → title + visible
innerText) keyed by tab id; readActivePreview resolves the ACTIVE tab, windows
the text (24k cap per read), and answers file/artifact tabs with identity plus
a note pointing at the tool that reads that content directly. The gateway
event handler answers preview.read.request beside terminal.read.request.
Re-applied from #50508 onto current main: the executor moved from
tools/ to agent/ and the JSON-error site was refactored away into
_parse_tool_arguments, but these 8 f-string debug calls survived the
move verbatim. Lazy %s formatting skips string interpolation when
DEBUG is off — the Tool-result line interpolates the FULL tool output
(can be 100KB+) on every tool call otherwise.
The sequential executor's KeyboardInterrupt handlers emitted a cancelled
post-tool-call event for the current tool, called agent.interrupt(), then
re-raised — WITHOUT appending a tool result message for the interrupted call
or any remaining calls in the batch. The assistant tool-call turn was left
with no matching tool results, a message-role alternation violation that
malforms the next provider request (relying on downstream repair passes to
patch it, which don't run on every path).
The cooperative-interrupt block (_interrupt_requested) and the concurrent
executor already emit a result for every call_id; this brings the two hard-
interrupt handlers into line via a shared _append_cancelled_tool_results
helper that appends a cancelled result for the current + remaining calls
before re-raising.
Verified live before/after (0 tool results -> 3 for a 3-call batch
interrupted on the first tool) and with a sabotage-checked regression test.
52 interrupt/executor tests pass.
The 1.0s sleep between sequential tool calls has been present since the
initial commit with no documented rationale. It sleeps between local
tool executions — the next LLM request goes out only after the whole
batch — so it rate-limits nothing, and the parallel read-only path
already runs with no delay. Every multi-tool turn pays (N-1) seconds
of dead time. Remove the sleep, the internal tool_delay plumbing, and
dead test assignments. AIAgent.__init__ keeps tool_delay as a
deprecated no-op keyword for one release so existing programmatic
callers construct cleanly; passing it emits a DeprecationWarning.
Port from nearai/ironclaw#5149 (the describe-first live-hardening fix in
their progressive tool disclosure work): when a model invokes a deferred
tool through the tool_call bridge without the schema-required arguments,
return the tool's parameter schema instead of dispatching blind.
Pre-fix, a blind call produced an opaque downstream failure
("[TOOL_ERROR] Tool execution failed: KeyError: 'document_id'") that
teaches the model nothing about what the tool expects — IronClaw observed
cheap models looping ~30 identical invalid calls until the iteration
budget died. Post-fix, the model repairs the call in one round-trip.
- tools/tool_search.py: new validate_deferred_call_args() — key-absence
check of schema 'required' fields only; no type checking (coerce_tool_args
already repairs types downstream); fails open on any validator error so
it can never block a legitimate dispatch.
- model_tools.py: probe after the scope gate in the bridge dispatch.
- agent/tool_executor.py: probe in both unwrap sites (concurrent +
sequential) before the underlying tool replaces the bridge; sequential
path flattens the payload to match its {"error": str} wrapping.
- tests: TestDeferredCallSchemaProbe — blind call returns schema (not
KeyError), valid/optional calls dispatch, unvalidatable tools fail open,
out-of-scope rejection unchanged.
Follow-ups to the salvaged #23768 commit, which targeted a pre-79559214
codebase:
- agent/tool_executor.py + agent/agent_runtime_helpers.py: pass
multi_select at both current clarify dispatch points (the PR's
run_agent.py edits landed on dead code paths).
- tools/clarify_tool.py: replace the broad TypeError-retry in
_invoke_callback with inspect.signature detection, so a compatible
callback that raises TypeError internally is not invoked twice
(addresses hermes-sweeper review feedback on #23768).
- tests: cover single-invocation on internal TypeError, legacy 2-arg
callbacks, **kwargs callbacks, and registry handler multi_select
pass-through (schema arg → handler → callback).
The last_activity field showed 'tool completed: read_file (120.7s)'
identically whether the tool succeeded or failed, making post-mortem
analysis of silent-hang reports (#69131) unnecessarily hard.
Append ' (error)' to the activity description when the tool result is
classified as a failure, in both the concurrent and sequential
execution paths.
(Salvaged from PR #69467; its conversation_loop.py activity-touch hunk
is the same fix as PR #69577 and lands in the preceding commit.)
_extract_parallel_scope_path used Path.cwd() (process cwd) instead of the
tool's actual execution cwd, and os.path.abspath() instead of os.path.realpath(),
so symlink aliases and relative/absolute path pairs that resolve to the same
physical file were treated as distinct targets and placed in the same parallel
segment. On case-insensitive platforms (Windows) os.path.normcase() was also
absent, allowing Foo.txt and foo.txt to race.
Changes:
- agent/tool_dispatch_helpers.py: introduce _canonical_path(raw_path,
execution_cwd) applying expanduser->abspath->realpath->normcase; thread
execution_cwd through _extract_parallel_scope_path and
_plan_tool_batch_segments
- agent/tool_executor.py: pass get_active_env(effective_task_id).cwd as
execution_cwd to _plan_tool_batch_segments; add pathlib.Path import
- run_agent.py: pass active env cwd to _plan_tool_batch_segments at the
second call site inside _execute_tool_calls
- tests/run_agent/test_tool_batch_segmentation.py: add 5 regression tests
covering relative/absolute same target, symlink alias, execution_cwd vs
process cwd, symlink parent + nonexistent write target, and Windows
case-insensitive alias (skipped on non-Windows)
Fixes a file-corruption / lost-update race introduced by the mixed
tool-batch segmentation feature (perf commit #64460).
A model response containing several parallel-safe reads plus one unsafe
tool used to lose ALL concurrency: _should_parallelize_tool_batch was
all-or-nothing, so a single barrier call (terminal, clarify, unknown
tool, malformed args) forced the entire batch onto the sequential path.
_plan_tool_batch_segments now splits the batch into ordered segments:
maximal contiguous runs of parallel-safe calls execute on the existing
concurrent path, barrier calls on the sequential path, strictly in the
model's emission order. Invariants preserved:
- one tool result per call, appended in emission order (segments are
contiguous, so no result reordering across a barrier)
- side-effect boundaries: no call starts before an earlier barrier ends
- overlapping file targets split into separate ordered parallel runs
- turn-end budget enforcement + /steer injection run exactly once per
batch (segment executors run with finalize=False; the segmented
dispatcher owns the whole-turn finalize)
- interrupt during segment k drains segments k+1..n with cancelled
results, keeping one result per tool_call_id
Homogeneous batches keep their original single-path dispatch (zero
behavior delta); _should_parallelize_tool_batch remains as a thin view
over the planner for existing callers and tests.
Extend the pre_tool_call plugin hook return contract with a new directive:
{"action": "approve", "message": "why this needs human confirmation"}
Previously a pre_tool_call hook could only veto a tool call (action: block)
or allow it silently. It could not escalate to the existing human-approval
flow. This unlocks user-defined runtime approval rules on ANY tool (HTTP
writes, file writes to sensitive paths, email sends), enforced at runtime —
resolving #51221 as a pure plugin, with no core approval.py rule schema.
Mechanism:
- get_pre_tool_call_directive() returns (action, message) for block|approve;
get_pre_tool_call_block_message() kept as a block-only back-compat shim.
- resolve_pre_tool_block() is the single dispatch-site chokepoint: fetches
the directive and, for approve, invokes the human gate; fail-closed to a
block on denial, timeout, or gate exception. ALL FOUR tool-dispatch sites
now call it: tool_executor (concurrent + sequential), agent_runtime_helpers,
and model_tools.handle_function_call.
- request_tool_approval() escalates via the SAME machinery as Tier-2
dangerous commands: session/permanent allowlist, prompt_dangerous_approval
(CLI) / submit_pending (gateway), [o]nce/[s]ession/[a]lways/[d]eny,
timeout fail-closed, approvals.cron_mode for cron contexts.
Architecture: extracted the shared decision core into _run_approval_gate(),
called by BOTH check_dangerous_command() and request_tool_approval() so the
fail-closed / cron / gateway / yolo / persist policy lives in ONE place and
cannot drift. Fixed a latent divergence — the plugin path now honors --yolo.
Approval grain: [a]lways is keyed on tool_name + a hash of the reason (an
explicit plugin rule_key overrides), so distinct reasons on the same tool
persist independently instead of one 'always' blanketing the whole tool.
Non-interactive: cron honors approvals.cron_mode (parity with commands); any
other non-interactive non-gateway context fails CLOSED for the plugin path
(the command path keeps its historical fail-open default, unchanged).
No new config schema, no new env vars, no new hook events.
Three CLI reliability fixes:
1. Interrupt reliability: chat() only re-queued the user's interrupt
message when the turn result carried interrupted=True. When the agent
thread raced past its last interrupt check (or finished) before the
interrupt landed, the message was silently dropped — and the stale
_interrupt_requested flag left on the agent instantly aborted the
NEXT turn. Un-acknowledged interrupt messages are now re-queued as
the next turn and the stale flag is cleared (only when the agent
thread actually exited). The clarify-race path also parks the message
in _pending_input instead of dropping it.
2. Slow exit (5+ min): stdlib ThreadPoolExecutor workers are non-daemon
and joined unconditionally by concurrent.futures' atexit hook — even
after shutdown(wait=False). One wedged tool worker (abandoned after
interrupt/timeout) held the process open forever. Promoted
async_delegation's daemon executor to a shared tools/daemon_pool
module and adopted it in tool_executor (concurrent tool batches),
memory_manager (background sync), delegate_tool (child timeout wrapper
+ batch fan-out), and skills_hub (source fan-out). Added a 30s exit
watchdog (HERMES_EXIT_WATCHDOG_S) armed at _run_cleanup start as a
backstop for wedged cleanup steps.
3. Exit jank: after prompt_toolkit tears down the input/status bars the
terminal sat silent for the whole cleanup window, looking hung. Print
'Shutting down… (finalizing session)' immediately at exit start.
E2E: live PTY interrupt of a foreground 'sleep 120' terminal tool now
aborts in ~1s and the typed message runs as the next turn; wedged-worker
+ wedged-cleanup subprocess exits in 5.8s (watchdog) instead of hanging.
Review follow-up on the concurrent-tool deadline salvage. timed_out_indices is
snapshotted from not_done at the deadline; a worker can still finish and write
results[i] in the window before the post-execution result loop reads it. The
loop unconditionally replaced results[i] with a fabricated 'timed out' message
for any snapshotted index, discarding a genuinely-successful (just-late) result.
Gate the timeout message on 'and r is None' so a real result always wins. Add a
regression test that forces the snapshot-vs-result-loop race deterministically
(mutation-checked: reverting the guard fails it). Also document the intentional
detached-worker leak at the executor abandon site.
A tool with no internal interrupt check (read_file, web_search, or a wedged
terminal backend) that never returns keeps the concurrent-tool poll loop alive
forever: the loop only breaks when all futures finish or an interrupt is
requested, and the 30s heartbeat resets the gateway idle monitor so idle-kill
never fires. The ThreadPoolExecutor was also used as a context manager, so its
__exit__ joined the hung worker with wait=True.
Add a wall-clock batch deadline (HERMES_CONCURRENT_TOOL_TIMEOUT_S, default 420s
— above the 360s web_extract timeout; 0/negative disables). When it fires:
cancel pending futures, signal an interrupt to the worker threads, abandon the
executor (shutdown wait=False, cancel_futures=True) so hung threads aren't
joined, and return a per-tool 'timed out' result for the unfinished calls while
still surfacing the finished ones. Also fixes the latent futures.index(f)
lookup (ambiguous with duplicate futures) by tracking a future->index map.
Salvaged from #54562.
Co-authored-by: Gustavo Mendes <87918773+gustavosmendes@users.noreply.github.com>
Teknium
Brooklyn Nicholson
doncazper
kshitij
Victor Kyriazakos
kshitij
Sylvain Baeriswyl
Jeffrey Quesnelle
EndeavorYen
Alex Fournier
annguyenNous
Xipong
Soju06
Jeffrey Quesnelle
elcocoel
webtecnica
Gille
sprmn24
Gustavo Mendes