Agent Plugins v1 packages with 'streamable-http' mcp.json entries now load
through Hermes' existing URL-based MCP client instead of being reported and
skipped. The stdio-only limitation was the agreed follow-up slice from
PR #81196.
Boundary rules from the v1 spec (§7.2.1) are enforced:
- URL must be absolute http(s), no user information, no fragment; plain
HTTP only for localhost/loopback hosts.
- Configured package headers are never forwarded across a cross-origin
redirect: translation marks entries strict_redirect_headers, and the
redirect hook in the native runtime strips those headers (plus
Authorization) whenever a redirect leaves the original origin. On mcp <
1.24.0, where the client cannot hook redirects, such servers fail closed
with an actionable upgrade message.
- Legacy 'sse' entries remain reported and skipped.
The redirect hook is extracted into a testable module-level factory
(_make_redirect_header_stripper); default behavior for native config
servers is unchanged (Authorization-only stripping).
Two cache-key correctness follow-ups to #82229 (review feedback):
1. Abbreviated commit pins are immutable too. The installer's
is_valid_commit() accepts 7-40 hex chars, but the Python refresh
exempted only exactly-40-hex names — an abbreviated pin like
install-4ce1994.ps1 could be overwritten with a branch script. The
predicate now mirrors the Rust rule (7-40 hex = immutable, never
rewritten), applied to the sanitized target ref.
2. Refresh only the update-target ref's cache key. The helper rewrote
EVERY mutable-ref entry with the active checkout's script: with
install-main.ps1 and install-bb_gui.ps1 coexisting, updating main
replaced both with main's script — cross-branch cache poisoning in
the other direction. It now computes the single cache key for the
branch being updated, using the installer's own ref sanitization
(sanitize_ref: non [A-Za-z0-9._-] -> '_', so bb/gui ->
install-bb_gui.ps1), and touches nothing else. Entries the
bootstrapper never wrote are not created.
The branch is threaded from the existing `branch =
_resolve_update_branch(args)` in both _cmd_update_impl call sites and
_update_via_zip (main-only by its own guard).
Regression tests lock down both invariants: abbreviated-SHA pin
untouched (including when passed as the branch), coexisting mutable
refs (main refresh leaves install-bb_gui.ps1 byte-identical),
sanitize_ref parity, and uncached-ref no-op.
E2E on the incident machine's real bootstrap-cache: planted a stale
install-main.ps1 + sibling install-bb_gui.ps1 + abbreviated pin
install-4ce1994.ps1; refresh("main") healed main byte-exact and left
both others untouched; refresh("4ce1994") was a no-op. The pre-existing
40-hex pin entry in the real cache was also untouched.
Pre-#67193 hermes-setup binaries (June 2026 and earlier, including the
newest published build) resolve bootstrap-cache/install-<branch>.ps1 by
"exists -> reuse forever": a branch-ref cache entry written at install
time is never re-downloaded, so every GUI update/repair executes a
months-stale install script. The binary has no self-update path, so no
amount of `hermes update` fixes it.
Live incident (2026-08-09, ryanc): install-main.ps1 cached June 4 lacked
the #81327 venv process-tree sweep; the bootstrap venv stage died with
"Cannot remove item venv\Scripts\python.exe: Access denied" on a
straggler backend pair, twice, despite every relevant fix already being
merged on main - the installer simply never ran that code.
Fix: `_refresh_bootstrap_cache_scripts()` runs at the end of every
update path (git, zip, already-up-to-date repair), overwriting mutable
branch-ref cache entries with the freshly pulled scripts/install.ps1 /
install.sh. The stale binary's unconditional reuse becomes a feature: it
"reuses" a file the update keeps permanently current. Post-#67193
installers re-download on every run anyway, so this is a harmless
pre-seed of identical bytes for them.
Scope guards: 40-hex commit-SHA entries are immutable pins and are never
touched; .ps1 gets the UTF-8 BOM to match the installer's cache format
(#67193); best-effort - a failed refresh never fails the update.
E2E on the incident machine: poisoned the real
bootstrap-cache/install-main.ps1 with a stub, ran the real function -
healed byte-exact to the checkout's script (BOM intact, #81327 tree-kill
sweep present).
Follow-up to #82179 addressing helix4u's review comment
(#82179 issuecomment-5229441571). Three parts:
1. Desktop teardown (salvaged from #77436, @4adwentures): the update
hand-off's releaseBackendLock() sent SIGTERM to the primary backend
BEFORE taskkill /T. If the launcher exits first, Windows can no longer
enumerate its descendants and they survive holding the venv — the
Electron path that creates the orphan #82179 then has to repair.
New stopBackendTreesForUpdate() tree-kills the live root first, with
the behavioral vitest from #77436. The scanner half of #77436 is
deliberately NOT taken (superseded by #82158's full-cmdline scan).
2. Tree-aware orphan classification: _orphaned_desktop_backend_pids()
previously refused the whole holder set when any holder had a live
parent. But the scanner legitimately returns an orphaned serve root
AND its descendants (the venv trampoline's uv-managed interpreter
worker — which carries the same backend argv — plus .hermes-runtime
children). Those have a live parent: the orphan root itself. Now
holders inside an accepted orphan root's tree fold into that root
(only roots are returned; taskkill /T reaps descendants), and
live-parent backends defer to the ancestry check instead of refusing
outright. Anything outside an orphan tree still refuses.
3. Tests for the mixed shapes: root+managed-runtime child,
grandchild depth, non-descendant stray alongside an orphan root
(still refuses), descendant exited mid-classify.
E2E on a real Windows box: spawned a detached backend-shaped orphan
that itself spawned children (3 python descendants); the scanner-shaped
mixed holder set classified to [root], taskkill /T reaped root and all
descendants. The live Desktop backend on the box still classified None
(refusal preserved). The first E2E attempt caught exactly the
trampoline/worker case the mocks missed — the live worker re-execs with
the same backend argv and a live parent — which is what part 2 fixes.
Co-Authored-By: 4adwentures <296413879+4adwentures@users.noreply.github.com>
The GUI-updater handoff race: the Desktop fires SIGTERM + app.quit() and
spawns hermes-setup, but its Python backend (`python.exe -m
hermes_cli.main serve`) can survive the teardown. The Desktop is gone --
nothing will respawn that backend -- yet the venv-holder guard refused on
it and the update dead-ended with "Hermes is still running" while the
user had zero windows open (observed twice on 2026-08-09, 01:59 and
02:17, bootstrap-installer.log).
New `_orphaned_desktop_backend_pids()` classifies remaining holders: a
serve/dashboard backend whose supervising parent is provably dead (PID
gone, or recycled -- parent created after the child) is a straggler safe
to reap. Any live-parent backend, non-backend holder, or unprovable case
keeps the refusal exactly as before. Reaping uses the new
`_stop_process_trees()` (taskkill /T /F), mirroring the Desktop's
forceKillProcessTree and install.ps1's venv sweep so the managed
.hermes-runtime interpreter child dies with its launcher (#70026).
Builds on #81327 (salvaged intact underneath): that fixed the same
parent-only-kill gap in install.ps1's venv sweep; this closes the
remaining dead-end in the `hermes update` guard itself.
E2E on a real Windows box: spawned a detached orphan with a
backend-shaped argv -> classifier returned its PID and the tree reap
killed it; a non-backend orphan and the live Desktop backend (parent
alive) both returned None (refusal preserved).
Extends the picker fix to the read that actually uses the key: switch_model's
user-provider credential resolution (the ${VAR} api_key expansion and the
key_env fallback at the resolve-credentials step) still read os.environ raw
and passed the result to resolve_runtime_provider as explicit_api_key — so
under multiplex_profiles the actual switch, not just the picker listing,
could adopt another profile's key. Same _scoped_key_env helper, same
fail-closed semantics; identical behavior when multiplexing is off.
Adds end-to-end switch_model tests pinning that an installed scope wins over
the process environment for both read shapes.
854007d1c routed the remaining main-agent fallback key reads through
agent.secret_scope so the multiplexed gateway's per-profile scope applies.
list_authenticated_providers - which gateway/slash_commands.py calls
directly for /model - still resolved custom-endpoint and fallback-entry
credentials with raw os.environ.get(key_env), so under multiplex_profiles
one profile's picker reads whatever key the process environment happens to
hold, i.e. another profile's.
no multiplexing : profileA-key (unchanged)
scope installed : profileB-key (was profileA-key)
Route both reads through a _scoped_key_env() helper over
secret_scope.get_secret(). get_secret is identical to os.getenv when
multiplexing is off, so single-profile deployments are byte-for-byte
unchanged; a fail-closed UnscopedSecretError is treated as "no credential
visible for this profile", which is how the picker already handles a
missing key.
Scope: only the two key_env credential reads. The other environment reads
in that function are provider-presence probes (AWS creds, LM_BASE_URL),
a separate concern.
_detect_venv_python_processes() returned cmdline_raw[:120]. Gateways
autostarted via the managed-runtime interpreter carry a >120-char exe path
(.hermes-runtime\python\generation-...\cpython-3.11-...), so the truncated
cmdline ended inside the exe path, before '-m hermes_cli.main gateway run'.
The Desktop preflight's pausable-gateway exemption
(_scan_venv_blockers._is_pausable_gateway) therefore never matched, the
gateway was reported as a blocker, and every Desktop update aborted with
'Update didn't finish' even with all windows closed — the updater's own
gateway pause never got a chance to run.
Fix: return the full cmdline from the detector and truncate only at
display time (_format_venv_python_holders_message and the scan's JSON
cmdline field, after redaction).
Reproduced live on Windows 11: scan reported blocked=true for
'...cpython-3.1' (truncated); after the fix the same gateway pair scans
clear with pausable_gateways=2.
An alias that family-matches multiple catalog models (/model opus) used to
silently pick one via _model_sort_key heuristics. The heuristics have
guessed wrong repeatedly — dated snapshots like claude-opus-4-20250514
parsed as version 20,250,514 and outranked claude-opus-4-8; suffix
tiebreaks landed on the cheapest tier — and every wrong guess silently
switches the user to a model they did not ask for.
resolve_alias now raises AmbiguousAliasError whenever more than one model
matches the alias family; switch_model catches it at all three call sites
(explicit-provider path, current-provider path, authenticated-provider
fallback) and returns a failure result listing the candidates
(best-guess-first ordering, capped at 10) with instructions to pick an
exact name. A single match still resolves automatically, and DIRECT_ALIASES
exact mappings are unaffected.
The date-stamp split from #67571 is kept, demoted from selection logic to
display ordering of the candidate list.
Supersedes the auto-pick approach of #67571; credit to @Sahaun and @GottZ
for the date-stamp parser analysis that this builds on.
_model_sort_key treated YYYYMMDD snapshot stamps (e.g.
claude-opus-4-20250514) as version components, so 20250514 > 8
and resolve_alias("opus", "anthropic") returned the wrong model.
Fix: split components ≥ 19_000_101 (smallest plausible date stamp)
out of the version tuple, keeping them as a trailing tiebreaker so
bare IDs sort before their dated snapshots and newer snapshots
before older ones. Shorter numeric components (mistral-large-2411,
gpt-4-0613) keep their current behavior. No models.dev dependency
in the sort path.
get_messages() only deserializes content and tool_calls; the structured
reasoning columns (reasoning_details, codex_reasoning_items,
codex_message_items) come back as the raw TEXT they were stored as.
Feeding those rows straight back into a write, which is exactly what
the POST /api/sessions/{id}/fork handler does by piping get_messages()
into replace_messages(), hit an unguarded json.dumps() and stored the
already-serialized string encoded a second time. On replay of the fork,
json.loads() then yields the inner string instead of a list, and every
consumer's isinstance(..., list) gate silently drops it: preserved
Anthropic thinking blocks, Codex encrypted-reasoning/message-item
replay, and OpenRouter multi-turn reasoning context are all lost after
a fork, with one more encoding layer added per fork.
The /branch copy loop had the same defect from the other side: it
forwarded reasoning but none of the structured columns, and both TUI
branch writers persisted role/content alone, dropping reasoning and
reasoning_content along with them.
Route the six dumps sites in append_message and _insert_message_rows
through a shared guard that keeps already-serialized strings as-is;
structured values from the live runtime are dumped exactly as before.
Forward the reasoning fields in all three branch writers, matching the
set gateway/slash_commands.py already forwards on its own /branch path.
Bundled backends/platforms/providers load without a plugins.enabled
entry ('must just work'), but plugins.manage reported them 'not
enabled' — clients rendered running plugins with an OFF switch.
Surface the truthful default; explicit disable still wins.
plugins.manage list rows now carry the canonical registry key and a
portable flag (Agent Plugins v1 plugin.json packages), and toggles
address the key — bare names collide across category dirs
(image_gen/fal vs video_gen/fal), so name-addressed toggles flipped
both. Portable packages' in-memory MCP servers also fold into
enabled_mcp_server_names(); without that their tools registered with
the MCP runtime but never reached the model's schema.
Titling fired on the first response, so a session sat unnamed for the whole
opening turn - p50 151s, p90 1212s across real sessions, because a turn is
tool calls, not one round-trip. A turn that failed or was interrupted never
got a title at all. Four surfaces each carried their own copy of the call.
Move it into the shared turn prologue and split it in two: a deterministic
title derived from the user's opening message, written inline before the
model runs, then one small-model call that upgrades it. The response is
constrained to a JSON object so there is no preamble to strip, and control
wrappers are stripped rather than refused, so a slash command titles as
what the user asked for instead of the command itself.
* fix(model-picker): serve cached custom-provider catalog on no-probe opens
#58183 stopped GUI picker opens from live-probing saved custom
OpenAI-compatible endpoints so a stopped local server could not stall the
picker. It gated the whole discovery block, not just the network call, so
`cached_fetch_api_models()` was skipped too — and with it the catalog an
earlier probe had already written to `provider_models_cache.json`.
A custom endpoint that is not the current provider therefore renders only
the models named in its config entry. A local server with 8 models loaded
shows the 1 model that was saved when the provider was first added, on
every picker open, while an explicit Refresh shows all 8.
Add `cache_only` to `cached_fetch_api_models()`: answer from disk within
the existing stale-serve window, never fetch, never revalidate off-thread,
return None on a miss. Split the three call sites in
`list_authenticated_providers()` into what the user's config permits
(`discover_models`, an explicit `models:` allowlist) and how we may obtain
it, so suppressing the probe now downgrades to a cached read instead of
skipping discovery outright. `discover_models: false` still pins, and a
cache hit no longer writes back to config since the probe that populated
it already did.
The latency win stands: a cold cache is a miss, so picker opens against
offline endpoints still make zero network calls.
* test(model-picker): pin the cached-catalog contract for no-probe opens
Cover both halves of the invariant, since fixing either one alone
reintroduces a bug the other guards against.
`cache_only` on `cached_fetch_api_models()`: a fresh entry and an entry
past its TTL but inside the stale-serve window both serve; an entry beyond
that window, an empty cache, rotated credentials, `force_refresh`, and a
missing base_url are all misses — and none of them fetch or spawn a
background revalidation.
`list_authenticated_providers()` on the GUI path: a non-current endpoint
with a warm cache reports its full catalog across all three provider
shapes (`custom_providers`, `providers:`, bare `provider: custom`) with no
live fetch attempted. A cold cache keeps the configured list and still
makes no network call, which is the #58183 guarantee. `discover_models:
false` keeps pinning, and a cache hit does not write back to config.
* fix: persist discovered custom-provider models in the hermes model flow
The `hermes model` named-custom-provider flow (_model_flow_named_custom)
probes the endpoint and shows the full catalog, but never persists it to the
entry's `models:` list. No-probe surfaces (dashboard, desktop, ACP) call
build_models_payload(..., probe_custom_providers=False) and only render the
configured `models:` list, so a provider added via `hermes model` collapses
to the single `model:` default everywhere except the CLI. OpenAI-compatible
providers added via a probing picker already benefit from
_save_discovered_models_to_config; the CLI flow did not.
Persist the live catalog after a successful probe, mirroring the picker path
in model_switch.py. A failed save is non-fatal.
* fix(model-picker): stop an auto-saved catalog pinning a keyless endpoint
The cached-catalog read added for no-probe picker opens still sat behind
the no-key discovery gate, so it never reached the shape that motivated
it: a keyless local model server.
`bool(api_key) or not has_explicit_models` is a network-cost gate. It
exists so Hermes does not probe an endpoint it cannot authenticate to
when that endpoint already declares its catalog (5f00f36ba, 1039e90b5).
Reading a catalog an earlier probe already paid for costs nothing, so
the gate belongs on the probe, not on discovery as a whole.
Left on the discovery side it re-pins the endpoint it was meant to
spare. A successful probe calls `_save_discovered_models_to_config()`,
which writes a plain list into `models:` — exactly the shape
`_models_config_is_allowlist()` reads back as an explicit user
allowlist. A keyless server therefore froze on the catalog of its first
probe and could never widen again, which is the "lineup changes after
config was written" case. f66319097 already carved the dict shape out of
this trap for the same reason; the list shape is the other door into it.
Move the clause to `_probe_live` at both custom-endpoint sites. Probe
suppression is unchanged — verified byte-identical to main across the
keyed/keyless x declared/undeclared matrix — and `discover_models: false`
remains the documented way to pin a catalog.
* test(model-picker): cover the keyless auto-save pinning trap
Three tests around the gate move, each failing on the code before it:
- a keyless endpoint carrying an auto-saved `models:` list still reads
its full cached catalog
- the same row, cold cache and probing enabled, still makes zero live
fetches — the network-cost gate the clause exists for
- an end-to-end round trip: persist a probe result via
`_save_discovered_models_to_config()`, reload it, and assert the shape
we wrote does not read back as a user pin
The round-trip test guards the whole chain rather than one branch, so a
future change that makes the saved shape look like an intentional
allowlist fails here even if the gate logic is refactored.
* fix(model-picker): key the custom-endpoint model cache by api_mode
`cached_fetch_api_models()` fingerprints entries with `api_mode`, but no
call site in `list_authenticated_providers()` passed it, so every custom
row resolved to the `api_mode=None` fingerprint. Two rows sharing a
base_url and credential but differing by `api_mode` are deliberately
distinct picker rows — it is part of `group_key` at both sites — yet they
collapsed onto one cache entry.
That was latent while probing was the only way to fill a row: a mismatched
entry was overwritten by the row's own live fetch. Serving that entry
without a probe makes it visible, so an `anthropic_messages` row could
render the catalog an OpenAI-mode row cached against the same URL. The
wire protocols differ (`x-api-key` + `anthropic-version` vs
`Authorization: Bearer`), so those catalogs are not interchangeable.
Persist `api_mode` on the group at both grouping sites — it is already
part of `group_key`, so it is constant across the group — and pass it
into the cache read. Section 3b (bare `provider: custom`) has no
`api_mode` in scope and already reads with the empty-credential
fingerprint, so it is unchanged.
Reported by Copilot review on #81973.
---------
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
Co-authored-by: Navlem <114683850+Navlem@users.noreply.github.com>
Widen of the PowerShell codepage cluster: the autostart registration
subprocess in tools_config.py was the last text=True capture in these
modules still decoding with the locale code page. Standardize on
encoding='utf-8', errors='replace' like the rest of the file (#53428).
A gate runs whatever command the operator configured, so its output is
arbitrary bytes. run_gate captured it with text=True and no encoding, which
decodes with locale.getpreferredencoding() under errors="strict".
One byte the decoder rejects — a test runner's checkmarks or CJK on a
non-UTF-8 Windows console, a stray binary byte anywhere in the stream — kills
subprocess's reader thread. proc.stdout comes back None, the `or ""` fallback
turns that into an empty tail, and an unhandled traceback is dumped to stderr.
The gate's pass/fail verdict still lands on the exit code, but the output tail
is exactly what the retry prompt feeds back so the agent can fix the failure.
With it empty the agent is told a gate failed and given nothing to act on, so
it burns every retry and the goal auto-pauses.
workspace_fingerprint has the same two calls; there a non-ASCII path in
`git status --porcelain` empties the fingerprint, silently disabling the
unchanged-gate skip that exists to stop a stalled agent re-running the same
red suite.
Decode as UTF-8 with errors="replace" — what git and modern toolchains emit,
and what 262 of the repo's 299 text-mode subprocess calls already do.
Complements the cherry-picked contributor fixes and closes out the
remaining sites of the 'missing explicit encoding' bug class, which is
now permanently gated by ruff PLW1514 (enabled repo-wide in
pyproject.toml and enforced by the blocking `ruff check .` step in
.github/workflows/lint.yml):
- tools/memory_tool.py: read MEMORY.md/USER.md via utf-8-sig so a
Notepad BOM never glues U+FEFF onto the first entry (issue #10878,
PR #10888 by @easyvibecoding — strict-decode contract of
_read_raw_checked preserved rather than errors="replace", so
undecodable files still refuse read-modify-write instead of being
lossily rewritten). Regression tests included.
- tools/skills_tool.py: SKILL.md and skill file reads pinned to
utf-8-sig + errors="replace" — deterministic across platforms instead
of the locale fallback proposed in PR #51701 (superseded: falling back
to cp1252/GBK makes the same skill render differently per host); .env
reader aligned with the canonical utf-8-sig dialect in hermes_cli/config.py.
- agent/shell_hooks.py, hermes_cli/main.py, gateway/slash_commands.py:
explicit utf-8 on the remaining fdopen/open text-mode sites flagged by
the AlexFucuson9 sweep series (#56033#56940#65565#66782#66791).
Co-authored-by: easyvibecoding <easyvibecoding@users.noreply.github.com>
Co-authored-by: AlexFucuson9 <AlexFucuson9@users.noreply.github.com>
Co-authored-by: flyingdoubleg <wangzhe00zju@gmail.com>
Co-authored-by: LeonSGP43 <cine.dreamer.one@gmail.com>
Follow-up to the auth.json UTF-8 read fix in this PR. A repo-wide scan for
the same bug class found three more callers that read ~/.hermes/auth.json
via Path.read_text() with no encoding — same Windows cp1252 hazard:
- agent/auxiliary_client.py _read_nous_auth: a non-ASCII byte raised
UnicodeDecodeError, the broad except swallowed it, and Nous silently
stopped being available as the auxiliary (vision/summarization) provider.
- tools/xai_http.py has_xai_credentials: same failure mode — xAI OAuth
silently looked absent on Windows.
- hermes_cli/main.py is_setup_complete: same; has a config.yaml fallback so
the impact is milder, but the read is still wrong.
All three now use read_text(encoding="utf-8-sig"), matching _save_auth_store's
write encoding. A repo-wide grep confirms there are no remaining
json.loads(...read_text()) reads of auth.json without an explicit encoding.
Tests: rewrote the Windows-encoding regression tests to actually exercise the
bug on POSIX too — a new windows_default_encoding fixture forces a no-encoding
read_text() to decode as cp1252 (the Windows default), and _write_utf8 now
emits real non-ASCII UTF-8 bytes (ensure_ascii=False) so the bytes actually
trip cp1252. Verified each test fails when its fix is reverted (including
the two new sibling-reader tests).
The auth store readers (_load_auth_store, _import_codex_cli_tokens, and the
shared Nous store reader) called Path.read_text() with no encoding, so bytes
were decoded with locale.getpreferredencoding() — cp1252 on Windows. The
stores are *written* as UTF-8 (os.fdopen(..., encoding="utf-8")), so any
non-ASCII byte (a CJK or emoji credential label, an accented display name in
OAuth state) raised UnicodeDecodeError on read.
Worst case: _load_auth_store's broad except then copied the file to .corrupt
and returned an empty store, silently wiping every provider credential on the
next launch. The sibling reader at line 2161 already used
read_text(encoding="utf-8"), confirming the omission was unintentional.
Use utf-8-sig (matching the .env handling in config.py) so a BOM from a
Notepad-edited file is tolerated too.
Adds regression tests covering the UTF-8 round-trip with a non-ASCII label,
BOM tolerance, no-corrupt-on-valid-load, and that the readers pass an explicit
encoding (guard against future regressions). Verified the tests fail when the
fix is reverted.
Closes no issue — found via cross-platform code audit (the bug is not in the
issue tracker).
send_cmd._load_hermes_env intentionally reimplements a minimal dotenv
load (no secret-source pulls, no sanitize rewrite, get_hermes_home path
resolution incl. Windows/profile override), so the shared-loader BOM fix
is mirrored in place: utf-8-sig primary read, BOM strip before the
latin-1 stream fallback.
Claude-Session: https://claude.ai/code/session_01JPmJz5u1Bvtw4cCRvRWnYr
utf-8-sig only covers the primary decode. BOM + invalid UTF-8 (e.g.
PowerShell BOM + cp1252 body) forced latin-1, which kept EF BB BF as
part of the first key name and dropped the canonical name. Strip the
BOM before latin-1 decode and load via stream so override= is preserved.
PowerShell 5.1 Set-Content -Encoding UTF8 and Windows Notepad write a
UTF-8 BOM. load_dotenv(encoding="utf-8") kept U+FEFF on the first key
name, so the canonical name was absent from os.environ and Hermes looked
unconfigured with no error. utf-8-sig strips the BOM and is a no-op for
BOM-less UTF-8; latin-1 fallback unchanged.
Prevent UnicodeEncodeError when model text contains U+D800-range
surrogates by sanitizing to U+FFFD before writing to UTF-8 stdout.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ports #68497 forward onto current main per teknium1's review.
input() can raise UnicodeDecodeError when the terminal encoding
cannot decode the byte sequence (e.g. a non-UTF-8 locale, or an
embedded terminal). The prior port targeted hermes_cli/main.py, the
pre-refactor location -- the update pipeline moved to
hermes_cli/update_cmd.py in 927463efcc.
Per review, fixed all three interactive update prompts that call
input() directly, not just the one this originally targeted:
1. Config-migration prompt (update_cmd.py:~3989): extends the existing
except EOFError to also catch UnicodeDecodeError, prints an
actionable 'hermes config migrate' hint, and falls through to the
skip branch (response=n).
2. Stash-restore prompt (_restore_stashed_changes, ~line 971): the raw
input() call here had NO exception guard at all -- not even for
EOFError. Added a try/except covering both EOFError and
UnicodeDecodeError, falling back to the existing skip-restore path
(changes remain safely in git stash, restorable manually).
3. Upstream-remote prompt (_sync_with_upstream_if_needed, ~line 1274):
already caught (EOFError, KeyboardInterrupt) but not
UnicodeDecodeError -- added it to the existing tuple.
Also dropped the incorrect #12884 reference (a TUI sticky-scroll
report, unrelated to this update-encoding issue, per the review).
4 new tests pass covering all three call sites (config-migration prompt
via cmd_update end to end, stash-restore and upstream-remote prompts
via direct unit tests against their own functions), plus an EOFError
sanity test confirming the stash-restore fix doesn't regress that case
either (it had no guard before). 6/6 in the full
tests/hermes_cli/test_update_yes_flag.py file (no regression).
Persist display.personality only; apply rendered text as an in-session
overlay across CLI, TUI config.set, and gateway /personality.
Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: EMT5320 <1908937833@qq.com>
Keep agent.system_prompt user-owned; named personalities resolve as an
ephemeral overlay via display.personality.
Co-authored-by: kyssta-exe <kyssta-exe@users.noreply.github.com>
Co-authored-by: EMT5320 <1908937833@qq.com>
Opt-in via compression.codex_responses_native (default: false). When enabled,
gpt-5.6-family models on the direct OpenAI API (api.openai.com) or a ChatGPT
Codex subscription send context_management=[{type: compaction,
compact_threshold: N}] on Responses requests. OpenAI compacts server-side and
returns an encrypted compaction output item; Hermes captures it into the
existing codex_reasoning_items sidecar and replays it on later turns in place
of the pruned history — inheriting persistence, session replay, the
cross-issuer guard, and the encrypted-replay kill switch with zero new state.
Scope is deliberately hard-gated (agent/native_compaction.py, re-checked per
request): gpt-5.6 family only — gpt-5.1/5.2 fail server-side on the field
(HTTP 500 / stream stall, no structured rejection; live-verified) — and
direct OpenAI/Codex routes only; xAI, GitHub/Copilot, OpenRouter, relays,
and local servers never see the field.
Hermes' local compression stays armed as the fallback owner: the native
threshold is clamped ~8K tokens below the local trigger so the server
compacts first, and a structured provider rejection of context_management
disables native compaction for the session and retries without it
(one-shot guard in TurnRetryState).
Live-verified E2E on api.openai.com/gpt-5.6: server compaction fired at a
4K threshold, checkpoints captured and replayed, recall preserved across
3 turns; gpt-5.1 with the flag enabled stays clean (field never sent).
Direction credit: PR #76950 by @laryhorb explored native Responses
compaction; this is a minimal reimplementation on current main.
Fixes 13 issues found in PR #20774 review:
1. Wiring: engine selection moved from run_agent.py to agent/agent_init.py
(where init_agent lives on current main). Transform hook moved from
run_agent.py to agent/conversation_loop.py (where run_conversation lives).
2. Prompt caching: replace copy.deepcopy with copy-on-write (shallow list
copy + clone only messages that are mutated). Use last_prompt_tokens
from update_from_response instead of re-estimating tokens every call.
System extension injection is idempotent (one-time cache break).
3. Signature mismatch: _message_signature renamed to _content_signature
and now excludes tool_calls/tool_call_id from the hash. This prevents
mismatches when _canonicalize_api_tool_calls re-serializes argument
JSON with sort_keys=True on the API copy.
4. update_model: accepts api_mode parameter (required by agent_init.py).
5. Reconciled with select_context: transform_api_messages is a separate
hook that runs AFTER select_context and sanitization, before
prompt-cache marker placement. Both hooks coexist with clear ordering.
6. Dedup/purge: kept as DCP-specific strategies (different semantics from
ContextCompressor._prune_old_tool_results — DCP deduplicates by
tool+args signature, not by content hash).
7. Removed copy.deepcopy: replaced with shallow list copy + copy-on-write
via _clone_if_needed. Only messages that are actually mutated get
cloned.
8. Removed redundant _ensure_refs call: _match_api_messages_to_refs no
longer calls _ensure_refs (the caller already called it).
9. _message_key still uses index (needed for positional ref assignment),
but _content_signature is cached per id(msg) to avoid re-hashing.
10. _inject_nudge: only injects into user messages, never falls back to
non-user messages (prevents role semantics violations).
11. Memory: _evict_inactive_blocks bounds blocks_by_id to
_MAX_INACTIVE_BLOCKS (50) deactivated blocks.
12. Merged _range_tool_schema and _message_tool_schema into a single
_compress_tool_schema. Merged _handle_range_compress and
_handle_message_compress into _handle_compress.
13. Dropped DCP_CONTEXT_ENGINE_PR_SPEC.md (temporary file, not for tree).
Config defaults kept minimal in hermes_cli/config_defaults.py (only
the keys the engine actually reads, not the full DCP-compatible surface).
Closes#20717
Salvage of PR #17973 by @TKCen (Sebastian Hänisch), re-implemented on
current main to preserve speed/instructions/provider params,
prepare_spoken_text normalization, OPUS_VOICE_PLATFORMS, is_write_denied
path security, microsecond timestamps, and the streaming-TTS gate.
- Split long TTS text into provider-safe chunks instead of truncating
- Pack generated audio against platform upload limits (Discord 10MB,
Telegram 50MB, configurable via tts.delivery_profiles)
- Combine chunks with ffmpeg (OGG/Opus re-encoded, MP3 stream-copied)
- Multi-file delivery when combination fails or would exceed limits
- Remove hard [:4000] truncation from all callers (cli.py, voice.py,
gateway/run.py, gateway/platforms/base.py)
- Gemini TTS raises ValueError instead of silently truncating when
composed prompt exceeds the provider limit
Simplify-code fixes: removed dead all_touched_paths set, added
try/finally for scratch file cleanup on exception, clean error response
on chunk failure instead of leaking stale file_path.
Four fix-forwards from the adversarial post-merge audit of the Aug 7
unreviewed merge batch:
- estop (#81148): is_engaged() now fails SAFE (engaged) on stat errors;
the gateway estop gate lets recognized slash commands and replies owned
by in-flight work (update prompts, clarify, slash-confirm, tool
approvals, running sessions) through instead of consuming them; new
gateway /pause [reason|off] command gives messaging-only operators an
in-band engage/resume path (busy_policy=dispatch so it works mid-run).
- cron monitor mode (#81138): execution-mode invariants (monitor x
no_agent, monitor_script x monitor_url, no_agent-requires-script) now
have ONE owner (_validate_job_mode_invariants) called from BOTH
create_job and update_job, so the create-time invariant can no longer
be silently violated through the update door.
- cron notepad (#81139): remove_job now clears the job's notepad rows
(clear_notepad was dead code -> orphaned KV state forever); clear is
best-effort and no-ops without creating notepad.db.
- delegation batch gate (#81141): template-marker regex narrowed to
multi-word placeholder shapes only (<feature name>, {file_path}) so
generics (Vec<T>), HTML tags, JSON snippets, glob braces and f-string
style no longer reject legitimate batches; duplicate-goal rejection
removed (best-of-N fan-outs are legitimate).
--resume latest resolves the most recent session through the same
workspace-scoped MRU lookup as -c (TUI source first under --tui, with
classic-CLI fallback). --in DIR chdirs before session resolution so the
lookup keys off DIR's workspace, and pins the session there by skipping
the recorded-cwd restore.
Requested by @Jeff9James: hermes --tui --resume latest --in ./dir
- Migrate the missed 6th inline formatter (update_cmd.py backup-size
display) to the shared helper.
- checkpoints._fmt_bytes: plain alias instead of a None-guard wrapper —
every caller feeds ints from checkpoint_manager (all size fields
initialize to 0), so the None path was dead defensive code.
- Drop the fallback= kwarg (zero production callers; '?' default is the
real inherited contract and stays).
- curator_backup + context_references: call format_bytes directly (single
internal call site each, zero external importers — alias was churn
avoidance with nothing to avoid). backup/_format_size and
doctor/_human_bytes keep their aliases (claw.py + tests pin the former;
three call sites use the latter).
- Reshape the loop so the trailing TB return is reachable (no dead line).
- Tests: replace alias-identity assertions (ossified the delegation
mechanism) with behavior-contract equality over a value sweep;
mutation-checked red-green.
- update_cmd parity: byte-identical B-GB vs the old inline loop; gains
the TB tier.
Five modules each carried a private near-identical human-readable byte
formatter (backup._format_size, checkpoints._fmt_bytes,
doctor._human_bytes, context_references._human_bytes,
curator_backup.format_size). Three of them silently topped out at GB and
rendered a 1 TiB value as '1024.0 GB'. All five now alias one shared
format_bytes in hermes_cli/sizefmt.py (sibling of timefmt.py, same
zero-dependency rationale), keeping each module's established local name
so no caller churns.
Deliberately NOT migrated (behavior differs on purpose):
- session_recovery._format_bytes: binary suffixes (KiB/MiB/GiB)
- qqbot chunked_upload.format_size: '100.0 B' one-decimal style, pinned
by its protocol tests
Net -33 production LOC before the new module; parity verified over a
16-value corpus against all five verbatim originals (only divergence:
the TB tier fix). Contract tests mutation-checked red-green.
_format_db_size reimplemented human-readable size formatting two
imports away from backup._format_size, which doctor already leans on
for _QUICK_STATE_FILES. Delegate and keep only the stat-failure wrap.
Sizes now scale units (KB/GB) instead of pinning everything to MB.
hermes doctor already warns when the linked SQLite carries the WAL-reset
bug, but it never said which databases are actually exposed. A database
already in WAL mode on a vulnerable runtime can still corrupt; one on a
rollback journal cannot. Doctor now lists each Hermes-managed database
with its journal mode next to the SQLite version line and marks the WAL
ones as exposed when the runtime is vulnerable.
The probe reads the 20-byte file header and checks byte 18 (2 = WAL,
1 = rollback journal). It deliberately avoids the SQLite engine: even a
read-only open creates -wal/-shm sidecars next to a WAL database, needs
directory write access, and can wait on locks. The header read does none
of that. It cannot tell delete from truncate/persist, so doctor reports
'rollback journal mode' rather than an exact mode name. Missing files
are skipped; empty, unreadable, or corrupt files are reported as
unreadable without failing doctor.
The database list reuses backup.py's _QUICK_STATE_FILES plus per-board
kanban databases. Exposure uses hermes_state.is_sqlite_wal_reset_vulnerable,
so the 3.50.7 and 3.44.6 backports count as fixed.
OpenCode Go serves GPT 5.6 Luna only via the Responses API per its
published endpoint table (https://opencode.ai/docs/go/#endpoints), but
opencode_model_api_mode() had no gpt- case in the Go branch, sending
Luna to /v1/chat/completions. The relay's shim streams full text but
never emits a finish_reason chunk, so every complete answer is
classified as a mid-stream drop and each turn fails with 'Response
remained truncated after 4 continuation attempts'.
Mirror the Zen branch: gpt- on Go -> codex_responses. Base URL needs
no change (normalize_opencode_base_url already keeps /v1 for
codex_responses). Extend test_opencode_go_api_modes_match_docs with
the Luna assertions.
- Move classify_persistence_error into hermes_state beside is_disk_full_error
and delegate the disk bucket to it (fixes 'ENOSPC writing state.db' and
'not enough space' classifying as unknown). run_agent keeps a thin lazy
delegating wrapper so the documented import path and fast import survive.
- Classify CompressionSessionBusyError (and its RPC-wrapped message forms)
as 'locked': the motivating #81227 failure mode stringifies to 'is being
compressed by another writer', which the substring heuristic missed.
- Export PERSISTENCE_ERROR_CAUSES and iterate it in the cron explainer
suppression instead of a hardcoded tuple, so a future cause bucket cannot
silently desynchronize cron delivery.
- Hedge the gateway locked/unknown recovery wording ('should already be
saved' instead of 'was recorded') to match the explainer - the early
turn-start persist may also have failed.
- Drop STATE_DB_WAL_WARN_BYTES (speculative dead constant with no consumer;
the pre-existing 50 MB doctor WAL check covers the warning).
- Tests: compression-busy classification, is_disk_full_error delegation,
causes-tuple coverage; mutation-checked red-green.
Review follow-ups from the pre-push falsification pass:
- collect_state_db_stats now routes through _connect_tracked_db so the
module's byte-probe guard sees the read-only connection (consistency
with the module's own ro-connection precedent; prevents a raw header
probe from cancelling this reader's locks in multi-threaded callers).
- Drop the new >256 MiB WAL warning from the stats renderer: doctor's
pre-existing 50 MB WAL check (with --fix checkpoint) already covers
WAL runaway, and two warnings for one condition is noise. The test now
locks in the dedup decision.
- Locked-cause explainer says the message 'should already be saved'
rather than overclaiming when the early turn-start persist also failed.
Operators had no Hermes surface showing state.db size, WAL health, index
family shape, or how many processes hold the database — all of which were
needed to diagnose a lock-contention incident on a 4.5 GB multi-writer
install.
Adds collect_state_db_stats() (strictly read-only URI connection, no
SessionDB instantiation, per-field best-effort) and a /proc-based
count_db_holders() to hermes_state, and wires a stats block into hermes
doctor's state.db section: logical size, pages/freelist, WAL size,
message/session counts, journal mode, holder count, FTS table presence
and deferred-rebuild status. Advisory warnings at >1 GiB (suggest
sessions.auto_prune and, when the v23 rebuild is pending or the legacy
trigram shape is detected, an offline 'hermes sessions optimize-storage')
and >256 MiB WAL (checkpoint health). Any stats failure degrades to a
single info line.
Two claim-failure diagnostic paths (cronjob_tools.py:629,921) still used
the old inline 'not enabled or state==paused' check. After get_job()
normalizes via effective_job_state, a half-paused record has
state='scheduled' and enabled=True, so the inline check returned False —
mislabeling the job as 'already being fired' instead of 'paused/disabled'.
Also hoists effective_job_state/is_job_runnable to the top-level import in
cronjob_tools.py (was function-local) and updates console_engine.py's
_format_job to use effective_job_state instead of the old inline
state-or-enabled derivation — a fourth display path the original PR missed.
Follow-up to PR #81287.
pause_job already sets enabled=false atomically with state/paused_at, but
get_due_jobs only checked enabled — so a contradictory record
(enabled=true + paused_at/state=paused) still fired. That was the 07-30
outage failure mode: list looked frozen, fleet kept merging.
- is_job_runnable / effective_job_state: pause markers gate fire; display
derives from the scheduler-honoured enabled flag so half-paused never
renders as [paused]
- get_due_jobs self-heals enabled=false + logs error on contradiction
- claim_job_for_fire uses is_job_runnable (paused_at counts too)
- list/format paths use effective_job_state
- behavioural tests: pause blocks due fire; half-pause self-disables
SessionResumeTooLargeError said 'across its lineage' even when the CLI
mid-setup path counted only the tip segment; the exception now takes a
scope phrase.
OFFSET paging made the streaming export O(n^2) on huge transcripts;
after_id keyset paging keeps each page seek O(1). Adds after_id to
SessionDB.get_messages (ascending-only, guarded against latest/offset
combos).
The mid-setup CLI resume path loads only the tip session's rows, so
gate it with a tip-only count instead of the full-lineage count (which
over-rejected heavily-compressed sessions). Transient guard failures
(locked DB, adaptor stores) now log and proceed instead of blocking
resume with a new error.
sessions.max_resume_messages / sessions.max_export_messages (default
20000, 0 disables) replace the hardcoded hard-rejects, and the CLI
'sessions export' guard becomes per-session instead of cumulative so
full-DB backups of many small sessions keep working. Error guidance now
points at the config override instead of the (corruption-only) repair
command.
Teknium
Drexuxux
Sohom Sahaun
cryptoyasenka
brooklyn!
Austin Pickett
Nolan
Paulo Nascimento
rainbowgits
ygd58
kshitij
Erosika
Axmr1
Victor Kyriazakos
rjvandeve