on_memory_write spawns a fire-and-forget daemon thread that was never
stored on self, so shutdown() couldn't join it — the exact problem the
PR fixes for the async writer thread. Store as self._memwrite_thread
and include it in the shutdown join loop.
Review follow-up for salvaged PR #83500.
The previous gate compared session.user_peer_id against a fresh
_resolve_user_peer_id() call on the same manager. Both values come from
the same resolver with the same inputs, so a non-owner triggering a new
session in a shared channel passed the check and received the owner's
MEMORY.md/USER.md under their peer.
The owner is now a config fact: _declared_owner_peer_id() returns the
sanitized peerName, and migration runs only when the session's user peer
is that peer. Without a declared peerName, migration runs only when no
runtime gateway identity is present (the single-operator CLI path).
Aliases still work: a platform ID mapped onto peerName resolves to the
owner peer before the comparison.
Tests now derive each session's user peer from the real resolver instead
of hand-picking mismatched ids, so the non-owner test fails against the
old gate.
The owner gate from #82038 compared against config.peer_name directly,
which is None for most single-user setups — sanitizing None would raise
and the gate never accounted for pinned/runtime/aliased identities.
Resolve the owner the same way sessions do, and add the non-owner skip
regression test the original PR shipped without.
Co-authored-by: menhguin <menhguin@users.noreply.github.com>
migrate_memory_files() uploads USER.md/MEMORY.md with peer=user_peer — the
session's runtime user. In shared channels, a non-owner's new thread uploads
the owner's full profile under the NON-OWNER's peer; Honcho's deriver then
attributes the owner's psychometrics/medical/biography to that person. This
was the root contamination vector (55/70 contaminated sessions carried the
payload). Skip migration unless the session user is the configured owner.
SOUL.md unaffected (uploads under assistant peer).
Co-authored-by: Minh Nguyen <menhguin@users.noreply.github.com>
sync_turn called manager._flush_session() directly, which flushes
synchronously every turn no matter what writeFrequency says — the
"async", "session", and every-N-turns modes were dead configuration
on the main turn path. Route through save(), the dispatcher that
actually implements those modes.
Same bug class reported in #19650 (starship-s) and #72708 (Diaspar4u);
this takes the minimal one-line routing fix without their broader
lifecycle refactors.
Co-authored-by: starship-s <45587122+starship-s@users.noreply.github.com>
Provider shutdown() only called manager.flush_all(), which drains the
queue but never joins the async-writer thread — manager.shutdown()
exists and nothing called it. The writer thread could still be blocked
in httpx I/O at interpreter exit (the #37632 crash class). Now
shutdown() calls manager.shutdown() (flush + join) when persistence is
enabled, and a new manager.stop_async_writer() (join only, no flush)
when saveMessages is false, so containment and clean teardown compose.
The containment commit skipped the whole turn when either side was
empty, which would drop a real user message on interrupted or
tool-only turns. Keep the guard for fully-empty turns only and skip
empty sides individually inside the sync loop.
Salvages #67559 — original gated sync_turn/on_memory_write/on_session_end but missed shutdown(), whose flush_all() still persisted on exit. hermes-sweeper review (salvageability=high) flagged this as the one gap.
Guard sits after the worker-thread joins, not at the top: cleanup is independent of persistence, and a top-of-method return would leak _prefetch_thread/_sync_thread. Adds TestShutdown and clarifies the saveMessages=false README row.
Credit @Matroskin86 (original PR author).
The saveMessages knob has been parsed by HonchoClientConfig since its
introduction but was never consumed: sync_turn, on_memory_write and
on_session_end persisted to Honcho regardless. With saveMessages=false the
provider now never writes automatically (raw turns, memory-write conclusion
mirroring, session-end flush) while read/tools paths stay fully functional.
Guard uses getattr with a True default so legacy/injected configs keep the
old behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018giroL5zeMPnPxERxAxXHY
dialectic_query collapsed every backend failure to an empty string, so
the explicit honcho_reasoning tool rendered timeouts, server errors,
and genuinely-empty answers identically as 'No result from Honcho.'
(#36098 issue 4). Operators debugging 'search works but reasoning does
not' were sent down representation/observation rabbit holes when the
real cause was a 30s timeout on a medium-reasoning dialectic call.
Add raise_errors to dialectic_query (default false — automatic
injection keeps its fail-quiet behavior and cadence backoff) and pass
it from the explicit tool call, returning a tool error that names the
failure and points at the timeout knob. Auth errors keep their
dedicated handler.
Two silent-auth-failure paths from #36098 (also #66125):
- the local-URL guard only escaped the 'local' placeholder when the
HOST BLOCK had apiKey. A top-level apiKey in honcho.json — explicit
user intent, and what 'hermes honcho setup' writes for single-host
configs — was dropped on the floor, so AUTH_USE_AUTH self-hosts
401'd on every request. Now any explicit key in honcho.json (host
block or top level) is honored; only env-sourced keys are still
treated as likely-cloud and skipped for local URLs.
- named-profile host blocks do not inherit the default host's apiKey
(credential isolation is by design), but the failure was silent:
the profile ran unauthenticated and every tool said 'no context'.
Affirm isolation and warn loudly at config-resolution time instead,
the outcome #66125 proposed if inheritance is rejected.
_all_profile_host_configs() built per-profile host keys inline as
f"{HOST}.{profile}" ("hermes.work") while profile_host_key() — used by
honcho status/enable/sync and the runtime memory plugin — produces the
underscore form ("hermes_work"). The lookup always missed, so
'hermes honcho peers' showed "(not set)" / leaked the raw malformed key
into the AI-peer column for every non-default profile. Profile names
needing sanitization (dots/spaces) were doubly broken.
Verified live: with hosts["hermes_work"] populated, cmd_peers showed
'work ... hermes.work' before the fix and 'work ... hermes' after.
Tests: host keys match the writer form, sanitized profile names resolve,
peers output shows populated identities with no key leak, and clean
fallback for profiles without a block.
Salvage of #2757 by @teyrebaz33 — rebased onto current Honcho plugin layout.
Stray control characters (e.g. terminal escapes pasted into HONCHO_BASE_URL
or config baseUrl) are dropped with a warning so SDK construction cannot
crash startup on Invalid non-printable ASCII character errors.
_resolve_or_create_client() used a plain dict.get(config.host) that
fails for dot-form profile host keys (e.g. "hermes.profile_a") even
though the _host_block() helper defined nearby handles the legacy
dot-form → underscore-form fallback correctly. The result:
_host_has_key evaluates to False for every authenticating user,
so effective_api_key is set to "local" and every Honcho API call
returns 401 Invalid JWT — cascade failure into silent data loss
for cross-peer queries and message sync.
Fixes by calling the existing _host_block() helper instead of
reimplementing the direct lookup. Local variable renamed from
_host_block → _host_block_local to avoid shadowing the function.
Closes#37436
HonchoClientConfig.from_global_config() only consulted top-level
baseUrl / base_url / HONCHO_BASE_URL in ~/.honcho/config.json. The
Honcho SDK's native config format — and what Claude Desktop writes —
nests the URL at endpoint.baseUrl. Users with that config format had
their self-hosted Honcho container silently ignored: every honcho_*
call routed to https://api.honcho.dev with a workspace_id that does not
exist there, so tools returned empty data with no error anywhere.
Resolution order in from_global_config(), highest first:
1. endpoint.baseUrl (SDK-native, what Claude Desktop writes)
2. baseUrl / base_url (root-level, existing behavior)
3. HONCHO_BASE_URL (existing env var)
4. HONCHO_URL (the SDK's own env var, honcho/client.py:234)
HONCHO_URL is also read in from_env(). from_global_config() delegates to
from_env() whenever the config file is missing or unreadable, so an env
fallback wired into only one of the two would silently do nothing for
users with no config file.
A non-dict endpoint value falls through cleanly rather than raising.
Existing users are unaffected — the new sources are consulted only when
the existing ones resolve to None.
The INFO log for the base_url-unset case now says so explicitly instead
of printing only the host. The SDK resolves that case from its own
ENVIRONMENTS map (honcho/client.py:36-39), which for environment=
production means the public cloud; a self-hosted user whose config was
not picked up otherwise sees a healthy-looking startup line.
Closes#43800.
- _submit_background and _prefetch_provider: replace unreadable
(lambda inner: (lambda: ctx.run(inner)))(fn) with functools.partial(ctx.run, fn)
- from_env(): set config_path=resolve_config_path() so bound_config_path()
doesn't re-resolve from ContextVar on daemon threads (the exact bug
the PR fixes for from_global_config)
Review follow-ups for salvaged PR #83525.
The dict was written on every build and popped/cleared on eviction and
reset, but no read site remained — timeout staleness detection moved
into the cache key itself (a timeout change produces a new identity and
_slot_for evicts the old slot), which the isolation tests already pin.
Flagged in review by @spfcraze.
Profile isolation is a ContextVar; plain threading.Thread targets start
with an empty context, so the plugin's nine daemon threads (session
init, prewarm, first-turn base/prefetch, prefetch, sync, memwrite,
async writer, context prefetch) resolved ambient state — config path,
active host, hermes home, oauth token paths — against the DEFAULT
profile whenever they ran under a routed profile's turn.
Adds spawn_context_thread(), which copies the caller's context at spawn
time so the thread sees the profile scope it was created under, and
routes every plugin thread spawn through it. Defense-in-depth under the
bound-config work: even ambient resolution on these threads now lands
on the right profile.
The copy_context approach follows the gateway's own
_run_in_executor_with_context pattern; #81401 applied it to the init
thread, this extends it to all nine spawns.
Co-authored-by: angel12 <angel12@users.noreply.github.com>
Replaces the process-wide first-config-wins client singleton with a
per-identity slot map. The singleton baked the first profile's
workspace_id and bearer into one shared client, so in multi-profile
processes (gateway multiplexer, dashboard, cron) every profile's
memory landed in whichever workspace initialized first — cross-tenant
bleed with no error (#69123, #74065).
cache key: (host, workspace, base_url, environment, provenance paths,
effective timeout, credential fingerprint). the fingerprint hashes the
OAuth REFRESH token (stable across in-place access-token rotation,
changes on re-auth/account switch) or the static api key — so
re-running 'hermes honcho setup' to switch accounts produces a new
identity instead of silently reusing the old account's client and
writing tenant B's data with tenant A's bearer, a hole per-path keys
alone cannot close.
same-identity slots with a different fingerprint or timeout are
EVICTED on replacement, so credential churn can't accumulate pinned
clients — the replaced client's pools close when its last holder
drops. timeout changes rebuild via the key (the old explicit staleness
check is subsumed). failed in-place OAuth rotation resets only the
client's own slot. reset_honcho_client() clears everything, preserving
test and oauth-flow re-login semantics.
per-config-identity caching was first proposed in #69142; the
provenance-key shape follows #81401. this implementation adds the
credential fingerprint and eviction they lacked.
Co-authored-by: NaMinhyeok <NaMinhyeok@users.noreply.github.com>
Co-authored-by: angel12 <angel12@users.noreply.github.com>
Profile isolation in every multi-profile process (gateway multiplexer,
dashboard, cron) is a ContextVar (set_hermes_home_override) that
threading.Thread targets cannot see. The plugin's daemon threads —
async writer, prefetch, sync, first-turn, init — all funnel through
HonchoSessionManager.honcho, which called get_honcho_client() with NO
config, re-resolving resolve_config_path()/resolve_active_host() from
the ContextVar-blind thread context: every background memory access
landed on the DEFAULT profile. Worse, the OAuth paths did the same, so
a token refresh on a daemon thread could persist the rotated token
into the wrong profile's honcho.json, and a 401 recovery could burn
the wrong profile's single-use refresh token.
- HonchoClientConfig gains provenance (config_path, hermes_home)
captured at resolution time inside the caller's profile scope, with
bound_config_path() for consumers
- manager.honcho passes the bound config instead of re-resolving
- OAuth paths (_apply_fresh_oauth_token, _refresh_cached_oauth,
_reauth_required, _force_reauth) use the bound path
- the honcho.json timeout memo becomes path-keyed instead of
single-slot, so multi-profile processes stop thrashing it and
returning profile A's timeout for profile B
Groundwork for per-identity client caching (#69123, #74065); the
provenance-field shape follows #81401.
Co-authored-by: angel12 <angel12@users.noreply.github.com>
Adds platforms.slack.extra.native_task_cards: when enabled, live tool
calls render as Slack-native plan/task cards via chat.startStream /
chat.appendStream (task_display_mode: plan, task_update chunks) instead
of text/edit progress bubbles. ID-bearing tool_start/tool_complete
callbacks correlate concurrent same-name tool calls correctly; any
native API failure falls back to one continuously edited text update.
The stream is stopped exactly once when the turn finalizes.
Salvaged from PR #29496 onto current main (TurnRunner/TurnContext seam);
closes#29483.
Slack's Agents & AI Apps feature ships a native streaming surface that
renders a live-typing message instead of the edit-based progressive
updates the adapter used until now.
The adapter now implements the existing draft-streaming interface:
- supports_draft_streaming() opts in whenever the app is connected and
native streaming hasn't been detected as unavailable.
- send_draft() starts a stream on the first frame (chat.startStream,
anchored to the resolved thread_ts, with recipient_team_id/user_id
for channel streams) and appends only the delta on subsequent frames
(chat.appendStream is append-only). The consumer's trailing cursor
glyph is stripped before delta computation.
- Unlike Telegram drafts (ephemeral, replaced by a real sendMessage),
a Slack stream IS the final message. send() therefore intercepts the
turn-final delivery for a chat with an active stream whose streamed
text is a prefix of the final content, and seals it via
chat.stopStream with the remaining delta instead of posting a
duplicate. Rich Block Kit (when enabled) is applied to the sealed
message via chat_update, mirroring the finalize path in edit_message.
- Feature-gate errors from chat.startStream (not_allowed,
missing_scope, unknown_method, ...) are cached on the adapter so
subsequent runs skip straight to edit-based streaming with a single
warning naming the fix (enable Agents & AI Apps for the app);
transient errors only disable drafts for the current run via the
consumer's existing send_draft failure handling.
- Segment breaks (new draft_id) and disconnect() seal any open stream
so chats are never left with a dangling live-typing indicator.
No consumer or config changes: streaming.transport auto/draft now
lights up native streaming on Slack through the same interface
Telegram drafts use, and the edit-based path remains the fallback.
_resolve_thread_id() falls back to _last_inbound_thread[chat_id] when no
explicit thread is present. That fallback exists for interactive DMs, where
Google Chat spawns a fresh thread per top-level user message and the adapter
drops thread_id to keep the session key stable. It also fired for cron
deliveries, which carry job_id in their metadata but no thread: the output
landed as a reply inside the last inbound thread instead of starting a new
top-level message.
Bypass the _last_inbound_thread fallback when metadata has a job_id (i.e. the
message is an automated cron delivery), so cron output posts at top level
unless an explicit thread is requested.
1. Use open_credentialed_url() instead of bare urlopen() in
templates.py apply_template() and probe_existing_customization().
Both send Authorization: Bearer headers; bare urlopen forwards
credentials on cross-origin redirects. The codebase has
open_credentialed_url() in hermes_cli/urllib_security.py that
strips credentials on cross-origin redirects — used by 4 other
modules.
2. Guard unavailable_reason() with the dedup set check before
calling it. The gateway builds a fresh AIAgent per message, so
without this guard unavailable_reason() (which calls _load_config()
→ stat + file read + JSON parse, and _check_local_runtime() →
importlib probes) runs on every gateway turn for an unavailable
provider, even though the warning is deduped after the first.
3. Move INDICATOR_GLYPH from Hindsight's eye emoji to a generic
brain (🧠) in core (agent/memory_provider.py). Hindsight overrides
with its own _HINDSIGHT_GLYPH (👁️) in recall_status() and
_emit_saving_indicator(). Other memory providers no longer inherit
Hindsight's brand mark as the default glyph.
Bundles previously-separate Hindsight/memory PRs into a single review surface:
- opt-in synchronous recall (recall_sync) — recall the injected memory in-turn instead of next-turn prefetch (#5820)
- actionable error when local_embedded runtime is missing — tells the user which package to install (#7718)
- default retain_source to 'hermes' so every stored memory self-identifies its provenance
- offer a starter memory template during hermes memory setup, plus warn before overwriting an already-configured bank
- warn when a configured memory provider reports unavailable (#2765)
- deterministic 'recalled N memories' recall indicator — Hermes itself emits a status line when auto-recall injects memory
- 'saving to memory' retain indicator — emitted the moment a turn is dispatched to the writer
Authored by @benfrank241 (ben.bartholomew@vectorize.io).
Salvaged from PR #74379.
Adds an observability/nemo_relay section to the built-in plugins page
(the plugin had no section despite appearing in the shipped table) with
the gateway.telemetry.session_segments keys, defaults-off contract, and
segment metadata; mirrors a summary in the plugin README.
Follow-up fixes from /hermes-pr-review + /simplify-code on PR #83437:
1. Replace _redact_secrets with agent.redact.redact_sensitive_text(force=True)
— the plugin's 11-pattern list was a strict subset of the 50+ patterns in
agent/redact.py. Secrets like Stripe keys, Google API keys, GitLab tokens,
HuggingFace tokens, DB connection strings, and Telegram bot tokens would
all leak through the plugin's list but are caught by the existing redactor.
Added pk-lf- (Langfuse public key) to _PREFIX_PATTERNS in agent/redact.py.
2. Remove dead 'not isinstance(client, object)' check in on_session_finalize —
always False for any Python value.
3. Fix MoAClient.last_reference_metrics() to call the public
self.chat.completions.last_reference_metrics() instead of reaching into
the private _last_reference_metrics attribute via getattr.
4. Deduplicate _coerce_request_messages call in on_pre_llm_request — pass
pre_coerced=input_messages to _messages_for_langfuse_input to avoid
double-coercion + double _capture_content serialization per API request.
5. Add HERMES_LANGFUSE_CAPTURE to OPTIONAL_ENV_VARS in hermes_cli/config.py
for consistency with the other HERMES_LANGFUSE_* env vars.
6. Fix test_sanitized_mode_redacts_secrets test data — the old samples
('sk-abc...1234', 'sk-ant...1234', 'Authorization: Bearer ***') were too
short to match the regex thresholds and never actually tested redaction.
Updated to realistic-length secrets and changed assertions to check that
the output differs from input (redact_sensitive_text masks rather than
inserting the literal string 'REDACTED').
Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).
Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.
Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.
Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.
New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.
Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.
Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.
Closes#29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
Slack Web API calls return `SlackResponse`/`AsyncSlackResponse`, which are
mapping-like but not `dict` subclasses, so every `isinstance(resp, dict)`
gate took its "unexpected shape" branch at runtime: user and channel names
collapsed to raw IDs, every user resolved as a non-bot (defeating the
allow_bots loop guard), ephemeral replies were reported as failures, and
uploads/caption fallbacks lost their message_id.
Normalize responses through a single `_slack_response_payload()` helper
(dict passes through, SDK response yields `.data`, anything else yields
`{}` so callers keep their fallbacks) and use it at every call site.
Existing Slack tests injected plain dicts, which is why the defect was
invisible; the new tests run each behavioral case against a real
`AsyncSlackResponse` as well.
Rebased onto current main, where the OpenRouter tool-use 404 is now
handled natively (the bundled demo's exact reason to exist), so the demo
plugin is removed per the standalone-repo policy and every test now uses
a synthetic unclaimed error (fake provider, neutral message, no status
code) that no present or future built-in rule can claim.
classify_api_error is now explicitly Python-plugin-only: VALID_HOOKS
doubles as the shell-hook allow-list, but the shell response parser has
no channel for the classification directive, so shell registrations are
refused at config parse with a warning instead of being silently
ignored (new SHELL_UNSUPPORTED_HOOKS set + regression test).
The hook is documented in the hooks reference as the third
behavior-changing hook, with the full kwargs contract, return shape,
and the Python-only note.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
Adds a plugin seam at the top of agent/error_classifier.classify_api_error()
(step 0, before the built-in pipeline) so model-provider plugins can classify
their provider's error quirks without patching core:
- New "classify_api_error" entry in VALID_HOOKS. Callbacks receive the parsed
error context (provider, model, status_code, error_type, error_code,
error_message, error_body, error, approx_tokens, context_length,
num_messages), self-scope on `provider`, and return None to pass or a dict
{"reason": "<FailoverReason name>", ...optional recovery-hint overrides}.
- get_plugin_error_classification() helper mirrors
get_pre_tool_call_block_message(): first valid result wins, invalid dicts
and unknown reasons are skipped, callback exceptions are isolated — a
broken plugin can never break classification. Zero behavior change when no
plugin claims the error (all 179 existing classifier tests pass untouched).
- Bundled reference plugin `openrouter-tool-use-404` (opt-in, like all
bundled standalone plugins) re-implements PR #58451: OpenRouter's
"No endpoints found that support tool use" 404 carries no
_MODEL_NOT_FOUND_PATTERNS signal, so it classifies as unknown/retryable
and the retry loop burns 3-5 attempts on a deterministic rejection.
The plugin classifies it as model_not_found (retryable=False,
should_fallback=True) so the fast-fallback path fires immediately —
demonstrating a waiting core PR converted to a publishable plugin.
Motivation: ~10 open PRs are single-provider error-classification patches
(#58451, #58355, #58502, #58474, #58366, ...). This hook turns that whole
class of contribution into plugin territory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FWMcB7RPSYUpsXDfBgwjzM
Implements the remaining observers from RFC #58548 (@thebizfixer),
accepted as the design basis in the #64231 batch disposition:
- on_kanban_worker_spawned: fires in the dispatch loop after spawn_fn
returns and the worker PID is durably persisted (the RFC timing
contract), in both the ready and review lanes.
- on_kanban_worker_exited: tick-derived from detect_crashed_workers;
fires after every reclaim/accounting txn has committed, carrying
exit_kind / exit_code / outcome / retry_status.
- on_kanban_worker_stale_claim: fires when release_stale_claims
reclaims a TTL-expired claim; live-PID claim extensions and deferred
reclaims stay silent.
- on_kanban_task_updated: task-mutation boundary observer carrying
changed_fields (field names only); fired by assign_task,
set_model_override, and set_reasoning_effort, and by the dashboard
plugin API's direct-SQL priority/title/body editors (single and
bulk) through the new kanban_db.notify_task_updated seam.
- on_kanban_dispatch_tick: re-port of PR #56066 (@laboratoiresonore),
renamed per the taxonomy and fired strictly AFTER _dispatch_tick_lock
is released; the sweeper found the original fired inside the lock,
where a slow subscriber could extend the single-writer critical
section and stall a sibling dispatcher.
All five are observer-only (return values ignored), fire after the
relevant write txn commits, and short-circuit on has_hook() so nothing
is built when no consumer registers; every fire site is fully
best-effort so a broken plugin can never break dispatch or a task
mutation. No config surface added. Existing plugins and hook payloads
are untouched.
Mutation-boundary scope: every user-facing task-FIELD editor fires
(assignee, priority, title, body, model/provider override, reasoning
effort). Deliberately not wired: status transitions (they belong to
the lifecycle hook family), dispatcher bookkeeping columns
(worker_pid, workspace_path, claim columns — surfaced through the
worker hooks instead), link/comment/attachment tables (not task-row
writes), and the dispatcher's default-assignee auto-assign (already
surfaced via DispatchResult.auto_assigned_default in the tick
payload). notify_task_updated is the seam for wiring further paths.
Docs: new rows plus a detail section in the shipped plugin-hook catalog.
Tests: 30 new (9 worker lifecycle, 8 dispatch tick, 8 task updated,
5 dashboard mutation boundary), including a lock-probe contract test
that fails if the tick hook ever fires inside the dispatch lock.
Refs: RFC #58548, #64231 batch disposition, folds #56066.
Simplify-code findings:
- _authenticated_json: replace manual header construction with
self._headers(include_tenant=False) — eliminates duplication with
_headers() and includes Content-Type consistently.
- _health_requires_credentials: replace getattr(exc, 'status_code')
with _status_code_from_error(exc) for consistency with the existing
error-classification utility. Drop the fragile string-matching
fallback — _parse_response always sets status_code on
_OpenVikingHTTPError, so 401/403 check is sufficient.
- Relax test header assertions to check presence/absence of specific
headers rather than exact dict equality, so they survive the
header-construction refactor.
Hosted OpenViking (Volcengine) rejects anonymous GET /health with
AuthenticationError, which made the provider look unhealthy and silently
disabled automatic memory mirroring. Keep the anonymous probe first for
identity safety, then retry once with the configured API key only when
the server demands credentials.
Fixes#78410
OpenCode identifies clients by request headers, the same way OpenRouter
does. Our opencode-zen and opencode-go profiles never set any, so every
request went out with the OpenAI SDK default "OpenAI/Python x.y.z"
User-Agent and OpenCode had no way to tell the traffic was Hermes Agent.
Two changes:
- Add HTTP-Referer, X-Title, and a HermesAgent User-Agent to both
OpenCode profiles through profile.default_headers, the same path
Fireworks uses. This covers chat_completions, codex_responses,
auxiliary clients, model switches, and the models catalog fetch.
- Merge the same headers in build_anthropic_client for opencode.ai
base URLs. The Anthropic Messages route (Claude on Zen, MiniMax and
Qwen on Go) builds its client there and never sees profile headers.
Verified against the live Go relay with a real key. Both wire formats
return HTTP 200 and the requests now carry X-Title "Hermes Agent",
HTTP-Referer, and User-Agent HermesAgent/0.20.0.
Follow-up to the salvaged #80489 substring-fallback removal: the
structured 401/403 branch still exited with a bare return, leaving
_running True — dead listener, healthy-looking is_connected(), gateway
never told (the zombie half of the bug, OOF-156 class). It now sets a
non-retryable mattermost_auth_error with token guidance and notifies
the gateway fatal handler.
Also: pytest.importorskip for aiohttp in the verifier probe file
(module-level import crashed collection in envs without the optional
dep), and probe fixtures updated for the escalation attributes.
The WS reconnect loop had a fallback check that looked for "401", "403",
or "unauthorized" as substrings anywhere in an exception's string form.
A transient error whose message happens to contain those digits (a proxy
body, a stack trace, anything) got treated as a permanent auth failure
and stopped reconnection for good.
I removed the substring fallback and kept only the structured check:
aiohttp.WSServerHandshakeError with status in {401, 403}. That's the only
signal that reliably means the server rejected our credentials.
Added two regression tests: one proving a transient error containing
"401" in its text still retries, and one confirming the existing
_closing early-return path is untouched by the removal.
Follow-ups to the salvaged #80032 fatal-error escalation, closing the
gaps its review thread identified plus a sibling of the same class:
1. Partial-batch loss: _check_inbox now dispatches whatever the fetch
returned BEFORE escalating a failure — the early-return dropped
already-fetched messages whose UIDs were marked seen.
2. Seen-after-fetch: UIDs enter _seen_uids only after their fetch
returns a response, so a mid-batch connection failure leaves the
remaining UIDs eligible for the next poll. Per-message processing
moved to _parse_fetched_message behind a poison guard: a message
that fails parsing/auth-verification is marked seen, logged with
its UID, and skipped once — never an eternal crash loop.
3. Reconnect mail loss: connect(is_reconnect=True) restores the
account's seen-UID baseline from a class-level snapshot instead of
re-marking the entire mailbox seen — mail that arrived during an
outage is now processed after the reconnect the escalation triggers.
7 new regression tests.
_fetch_new_messages() wrapped the whole IMAP connect/login/select/search/
fetch sequence in a bare except that logged and returned an empty list —
indistinguishable from a genuinely empty inbox. The adapter never invoked
its fatal-error handler, so the gateway's reconnect/backoff/status
machinery never learned the mailbox was unreachable; outages lasted until
a manual restart.
Track fetch failure on the adapter and, when the poll loop observes it,
set a retryable fatal error (email_imap_fetch_failed) and notify the
gateway handler so the platform enters the reconnect queue just like a
startup connection failure.
The cherry-picked #79448 predated #85049's _classify_connect_exception,
so it added a parallel PrivilegedIntentsRequired branch ahead of the
classifier (plus its own _is_privileged_intents_required detector).
Fold the tailored guidance into the classifier's existing intents arm
instead: one classification path, one error code (discord_intents_required),
and the message now names exactly the intents Hermes requested (Message
Content always; Server Members only when username/role allowlists need it).
Wizard callout, docs corrections, and tests from #79448 kept as-is.
PrivilegedIntentsRequired is a Developer Portal config error; surface which
intents Hermes requested as a non-retryable fatal and teach setup/docs.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Collapse the duplicated discord LoginFailure/PrivilegedIntentsRequired
classification (name-match + isinstance blocks repeated the same code/
message tuples) into a single _is() helper — one message per failure.
- Replace the user-facing HERMES_RECONNECT_ATTENTION_AFTER_SECONDS env var
with agent.reconnect_attention_after in config.yaml (default 7200, 0
disables), bridged internally like gateway_timeout. .env is for secrets.
- Use _float_env for robust parsing instead of bare int(os.getenv(...)).
- Document terminal classification + needs_attention escalation in
website/docs/user-guide/configuration.md.
Fleet triage after the 2026-08-11 storm resolution found agents whose sole
platform had been silently 'retrying' for weeks: revoked Telegram tokens,
Discord privileged-intent rejections, and Photon sidecars that can never
start were all funnelled into the indefinite reconnect queue with no owner
signal (OOF-151/152/153, epic OOF-156).
Two-part fix:
1. Per-adapter classification — by exception TYPE only, never message text:
- telegram: InvalidToken/Forbidden -> telegram_auth_error, retryable=False
(new _looks_like_auth_error, mirrors _looks_like_network_error)
- discord: LoginFailure -> discord_auth_error, PrivilegedIntentsRequired
-> discord_intents_required (both retryable=False); every other path now
sets an explicit code (previously the generic branch set NO fatal info,
which the gateway read as 'probably transient')
- photon: new typed PhotonSidecarStartupError; deps-install failure ->
SIDECAR_DEPS_MISSING and missing node binary -> SIDECAR_NODE_MISSING
(retryable=False); ambiguous startup crashes stay retryable
- email: IMAP/SMTP failures now always set a fatal code;
SMTPAuthenticationError -> email_auth_error, retryable=False (IMAP4.error
is type-ambiguous between bad creds and transient NOs, so IMAP stays
retryable)
2. Gateway escalation — platforms continuously in the reconnect queue past
HERMES_RECONNECT_ATTENTION_AFTER_SECONDS (default 2h, 0 disables) get
needs_attention=true + retrying_since stamped into runtime status, once
per episode, cleared on successful reconnect.
Deliberately NOT a circuit breaker: retries never stop. The auto-pause
mechanism was removed for good reason (transient DNS outages left bots
silently dead); this preserves that and only adds visibility. No new
platform_state enum values — NAS's status schema is strict — only additive
fields.
Unknown exception types always stay retryable: a false terminal recreates
the silently-dead-bot problem, and the escalation path covers
misclassified permanent failures.
Extend the normalized-envelope pipeline shipped in #82063 with new event
types, each with its own versioned, event-local payload contract:
- Telegram: message_edited (edited_message updates; editor-identity auth
extraction, forum topic thread_id, bounded text/caption, ISO edited_at)
- Discord: message_edited, message_deleted, thread_created, thread_renamed
(on_message_edit/delete, on_thread_create/update fire-sites with has_hook
no-subscriber fast-paths, bot-authored events dropped, rename-only
filtering on thread updates)
All events flow through the same gateway-owned post-auth boundary; malformed
or unauthorized events drop, fail closed. Raw SDK payload access is
deliberately NOT shipped (round-2 correction: needs its own
gateway.raw_events capability and design).
The Discord fire-site machinery (no-subscriber fast-path, observer isolation,
connect-time wiring) adapts the observer-hook design from PR #62584
(@paoloantinori) onto the normalized-envelope contract; PR #36875's raw
telegram update hook is superseded by the same correction.
Docs: hooks.md gains per-event payload contract tables.
Co-authored-by: Paolo Antinori <pantinor@redhat.com>
OpenRouter's /v1/models entries advertise reasoning capability
(supported_parameters + reasoning.mandatory/supported_efforts). Use that
metadata as the primary gate in _supports_reasoning_extra_body instead of
the hand-maintained vendor-prefix allowlist, which went stale one vendor at
a time (nvidia/ missing -> #75386). Also clamp the requested effort to the
nearest LOWER catalog-supported level in the OpenRouter profile so ultra/max
against a high-capped route no longer 4xxes.
Cache-only on the hot path: capabilities parse for free out of the existing
fetch_openrouter_models() payload, a background warmer covers cold starts,
and unknown models/offline catalogs fall back to the static prefix list
unchanged.
Scope events export when their OWNING scope closes. Turn scopes close
every turn; session scopes close only at session end. Marks were attached
to the session handle, so a long-lived conversation — a Slack thread open
all day, the normal enterprise case — emitted no approval or turn marks
for hours, and none at all if the process died first. Audit dashboards
showed an empty approval table while approvals were demonstrably firing;
the operator had to end the session to see anything.
Attach marks to the live turn handle when one exists for the mark's
session (active_turn already validates live/same-profile/same-session/
unreleased), falling back to the session handle otherwise — correct for
session-level events like session.end and for marks emitted outside a
turn. Parentage semantics are unchanged: the turn is a child of the
session, so the session tree is identical, only export cadence changes
from per-session to per-turn.
'ab|.*' compiled and carried the accepted 'ab' literal prefix while its
'.*' branch stayed unprefixed, escaping the no-redact-everything
guarantee (_extract_literal_prefix stops at '|'). Registration now
rejects top-level alternation with a regression test for exactly that
shape; grouped alternation after the prefix, escaped pipes, and
character-class pipes remain accepted.
The bundled nvapi-redaction reference plugin is removed per repo policy
(vendor integrations ship as standalone plugin repos); the end-to-end
register() coverage now uses a synthetic plugin written at test time.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnMCvi2vXqfs996AjVeF2F
kshitij
Erosika
carnie[bot]
赵桂雄
eapwrk
Dillon Townsel
spfcraze
Bartok9
mohamedorigami-jpg
LeonSGP43
Rob Sherman
Simon van Laak
Arman
andyst-dev
Ben
Teknium
Nikita Barkov
webdevtodayjason
Christopher
Slobaka
Adolanium
Stephen Chin
kyssta-exe
rainbowgits
Shannon Sands
Victor Kyriazakos