scripts/run_tests.sh runs the suite under `env -i` with an explicit
allowlist. The runner's own documented environment knobs were never on
that list, so all of them were silent no-ops for anyone invoking the
canonical wrapper:
* HERMES_TEST_WORKERS / PATHS / FILE_TIMEOUT / FILE_RETRIES / SLICE
are read by run_tests_parallel.py at argparse-default time — inside
the stripped environment.
* HERMES_TEST_IMAGE is read by tests/docker/conftest.py to skip its
session-scoped docker build.
The HERMES_TEST_IMAGE strip is the expensive one, and it's been biting
CI since docker.yml switched from bare pytest to run_tests.sh
(f0cb04921): the workflow sets HERMES_TEST_IMAGE to the image the build
step just loaded, the wrapper drops it, and every per-file pytest
subprocess falls back to building hermes-agent-harness:latest itself.
The job log timing shows it plainly — the first 8 files dispatched (the
LPT-heaviest) all report 248-297s, which is them waiting out the
concurrent initial `docker build` (~4 min on a cold local builder);
every file dispatched after that rides the layer cache and finishes in
4-38s (e.g. test_dump_build_sha.py, a single `docker run --entrypoint
cat`, reported 256.6s). ~4 min of pure waste per docker job, on both
arches — and the tests exercised a locally-rebuilt image WITHOUT the
HERMES_GIT_SHA build-arg the workflow bakes in, not the artifact being
shipped.
Fix: forward the six knobs the same way the Windows location vars are
forwarded (66c4c9c0b) — an explicit compute-before-drop allowlist, each
var only when set, so POSIX runs without them are byte-for-byte
unchanged and the 'no credential can leak' property stays auditable.
Verified empirically via a probe test through the wrapper:
before: HERMES_TEST_IMAGE=None inside the subprocess
after: HERMES_TEST_IMAGE='sentinel-image', HERMES_TEST_FILE_TIMEOUT
forwarded, HERMES_TEST_WORKERS=3 yields '(3 workers)' in the
summary, and an unrelated SOME_SECRET stays stripped.
bash -n clean; shellcheck: no new findings (SC2046 on the pre-existing
compileall line predates this change).
Ships the hard A/B evaluation used for the August 2026 core-toolset
performance batch (#77056) as a reusable harness: 9 error-inducing trap
tasks derived from measured production waste classes, two-arm
PYTHONPATH-only comparison, ATOF-trace-based scoring, resume-safe
batteries.
Hardened from the original one-off: paths de-hardcoded (ABEVAL_ROOT /
ABEVAL_HOME), encoding= on all file IO, startup crashes retry on resume
instead of polluting cells, post-hoc grading fix for err_inline_script
baked in. Live-smoked end to end (baseline arm, qwen3-coder-30b,
err_multi_dir: exit 0, correct on-disk verification, resume record
written).
The artifact download URL returns a 302 redirect to a signed blob URL.
urllib sent the Authorization header to the blob, and the blob rejected it
with a 401 error. The download now has two hops. The first hop authenticates
to the API. The second hop follows the redirect without the auth header.
The query runs?event=workflow_call returns nothing for this repository.
GitHub flattens reusable-workflow jobs and their artifacts into the caller
run. The fetch now lists the artifacts on the orchestrator run only. The
dead sub-run enumeration is gone. Two API calls per cycle are gone with it.
The 'artifact statuses updated' reason never appeared. The code updated the
count before the comparison. Now the code compares first and updates after.
The code rejects zip members that contain '..' or start with '/'.
tests/ci/test_live_comment.py is deleted. This repository does not keep
tests for CI infrastructure.
The live comment poller got its review statuses from two sources. The first
was the REVIEW_STATUSES environment variable, fixed at the start of the
comment-live job. The second was one ci-timings artifact, downloaded at the
end of the run. Status details (error messages, action_required items)
appeared only after all jobs finished. The job pass/fail results were visible
as each job completed.
Now every status-producing workflow_call uploads a small review-status
artifact when it completes. The poller lists all review-status-* artifacts
from the orchestrator run and its workflow_call runs every cycle. It
downloads each artifact and merges the statuses into the comment. A status
appears as soon as its job finishes.
Changes:
- live_comment.py: _fetch_artifact_statuses became fetch_all_review_statuses.
The new function lists the artifacts via the API, downloads each one, and
parses it. Removed the review_statuses_json parameter, the
--review-statuses-file argument, and the subprocess import.
- ci.yml: removed the REVIEW_STATUSES environment variable, the inline Python
merger, and the --review-statuses-file argument. Renamed the
ci-timings-review-status artifact to review-status-ci-timings.
- Eight workflow_call files: added a step that writes review-status.json and
uploads it as an artifact after each review_status output.
- test_live_comment.py: added tests for _parse_status_file and
_merge_statuses.
The poller logs transitions between polls. It reports newly completed jobs
(with their results), newly appeared jobs, and jobs that left the pending
list. Each comment update shows the reason for the change. For example:
'1 new completion(s); artifact statuses updated'. When nothing changed, the
poller lists the jobs that are still pending. The status line shows the raw
job count from the API and the number of infra jobs that the filter removed.
allow you to simulate the whole official curl | bash installer,
and subsequent hermes updates.
Run development commands in a bubblewrap filesystem and network sandbox
with a local HTTPS MITM fixture server and a fake github
git-upload-pack transport.
Package the sandbox command and expose it from the nix devShell.
Stage the local installer at its canonical fake HTTPS URL and add a
persistent installation/update test path. Route root installs through
sandbox-owned filesystem locations and snapshot dirty source worktrees
into temporary fake commits so update tests can fast-forward without
changing the real checkout.
Includes a --install-ref sandbox installer mode that fetches any commit
(--from-main is a nice shorthand for local development) outside the
sealed sandbox, installs from that snapshot, and then promotes the fake
remote to the current worktree so update flows can be exercised with FF.
Notes on non-root sandboxes:
Giving a non-root sandbox a network is tricky.
slirp4netns joins the target userns and setuids to root before configuring the
netns, so the userns must map a uid 0; bwrap's --unshare-user maps exactly ONE
uid, so --uid 1000 leaves no root to become and slirp diedswith
`setns(CLONE_NEWNET): Operation not permitted`. Stage 1 builds the user+net
namespaces with `unshare` and two one-id ranges:
inner 0 -> a subuid, unused by the payload, present only so slirp can
become root
inner 1000 -> our real host uid
Mapping the payload to the *host* uid (not a second subuid) keeps everything the
sandbox writes owned by us, so `rm -rf` on a persistent sandbox still needs no
privileges. Stage 2 execs bwrap WITHOUT --unshare-user -- it only adds mount/pid
-- sidestepping bwrap's refusal to accept --uid outside a userns it created.
Costs a /etc/subuid range for the invoking user (we error with the exact line to
add) and util-linux `unshare`; `--root` needs neither.
scripts/tests/ has held three PowerShell suites that no workflow ever
invoked -- there is no Windows runner in CI, so they have been inert since
they landed. A regression test nothing executes is worse than none: it
reads as coverage.
Adds a windows-latest job, gated on a new `installer` lane so it only fires
for PRs touching install.ps1 or its tests. The 8.3 suite runs under both
pwsh 7 and Windows PowerShell 5.1, since install.ps1 arrives via `irm | iex`
into whichever shell the user already has and 5.1 is what ships with Windows.
Only the 8.3 suite is wired up. The other two fail on main today for
unrelated reasons; they can join once they are fixed.
The previous suite pulled ConvertTo-LongPath out of install.ps1 via the AST
and dot-sourced the extracted text. AGENTS.md bans source-reading tests, and
this one showed why: it never executed the script-level Add-Type the kernel32
resolver depends on, so the resolver that does the actual work was untestable
by construction.
Each case now spawns install.ps1 as a real subprocess with a crafted
environment. -ProtocolVersion is a side-effect-free early exit below the
normalization block, so the whole block runs exactly as it does mid-install
and the assertions read what it reports back.
Verified RED against the pre-fix install.ps1 (10 of 24 assertions fail) and
GREEN after. The profile-root substitution is pure path arithmetic, so those
cases run on any host including non-Windows CI.
Co-authored-by: xxxigm <tuancanhnguyen706@gmail.com>
Windows aliases a profile folder whose name has a space, a dot, or an
accented character (FIRST~1.LAS, STONE~1.ZEN, RUBN~1). PowerShell's
FileSystem provider then throws "does not exist" the moment such a path
reaches a provider cmdlet, which every Node/Electron stage hits through
Tee-Object and the desktop stage hits again probing the binary it just
built. The install fails on an artifact that is sitting on disk.
install.ps1 already tried to expand these, but only via COM and only for
TEMP/TMP. COM cannot expand an alias on a non-English locale, and it cannot
expand one at all when 8dot3 generation is disabled or the alias is stale
-- both return the short path unchanged. LOCALAPPDATA was never normalized
either, so InstallDir stayed short even when TEMP got fixed.
Three resolvers now run in order, each covering what the last one cannot:
kernel32!GetLongPathNameW (locale-independent), COM (P/Invoke blocked),
and profile-root substitution (nothing to resolve -- rebuild on a root we
can prove is long). All five profile-rooted variables are normalized, and
HermesHome/InstallDir are re-derived from them. An explicitly passed
-HermesHome/-InstallDir is normalized in place, never replaced.
Every resolver degrades to returning its input, so a host where none apply
behaves exactly as before. Rewrites are logged to stderr: this bug class
has only ever been reported as a bare "does not exist" with no hint that a
short alias was involved.
Co-authored-by: Sahil-SS9 <218421507+Sahil-SS9@users.noreply.github.com>
* fix(credential-pool): clear exhaustion state on key rotation
When a user rotates an API key (e.g. via `hermes setup` after hitting a
rate limit), _upsert_entry updates the access_token on the existing pool
entry but preserves the stale last_status=exhausted from the old key.
On the next session the pool finds the entry, sees it exhausted, and
returns no usable credentials — even though the new key is valid.
Fix: when access_token changes on an existing entry, reset last_status,
last_error_code, last_error_reason, last_error_message, and
last_error_reset_at. The exhaustion state belongs to the old key, not
the new one.
* chore: add pasevin@gmail.com to AUTHOR_MAP
* fix: clear last_status_at on key rotation, remove unused pytest import
Address review feedback from teknium1 on PR #22622:
- Add last_status_at=None to the reset block (matches all other
token-sync reset paths in credential_pool.py)
- Assert last_status_at is None in the regression test
- Remove unused pytest import flagged by ruff + ty
startSocket() awaits useMultiFileAuthState() and fetchLatestBaileysVersion()
before it creates a socket or registers event handlers, and the close handler
re-entered it via a bare setTimeout(startSocket, ...). That leaves two
unrecoverable failure modes on a reconnect:
- a rejection is an unhandled promise rejection (fatal on modern Node)
- a hang leaves the bridge permanently disconnected with nothing left to
retry, while its HTTP server keeps answering 503 to the gateway
The second mode was observed in the field: fetchLatestBaileysVersion() is a
plain fetch to raw.githubusercontent.com with no AbortSignal, and after a
stream:error 503 disconnect the bridge logged 'Reconnecting in 3s...' once
and then sat silent and disconnected for 27+ hours until manually restarted.
Fix, as two pure helpers in bridge_helpers.js (keeping bridge.js side-effect
free to test):
- createReconnectScheduler(): every (re)connect entry point now catches a
failed startSocket() and reschedules it instead of dying or going silent
- createVersionResolver(): bounds the version fetch with a 15s timeout and
falls back to the last known-good version (or the Baileys default before
first success) instead of pending forever
The check-attribution CI gate kept bouncing salvage PRs because mapping
contributor emails was a manual, easy-to-forget step (bare
<login>@users.noreply.github.com emails don't auto-resolve like the
<id>+<login> form).
- scripts/audit_pr_attribution.py: mirrors the CI gate's logic exactly
(merge-base scan, same skip rules). Report mode for pre-push checks;
--fix auto-resolves via the bare-noreply local part (verified against
the GitHub users API) or GitHub email search, then writes
contributors/emails/<email> files via add_contributor.py. Prints a
confirm-the-human warning on bare-noreply resolution since the local
part is user-controlled (the bryan->hydraxman case).
- contributor-check.yml: failure output + review_status how_to_fix now
lead with the one-command fix instead of hand-editing instructions
(also drops the stale 'edit AUTHOR_MAP' guidance — AUTHOR_MAP is
frozen).
`hermes desktop` still failed with EBADENGINE demanding Node >=26 after
#76562, on a machine whose `apps/desktop/package.json` already said
`^20.19.0 || >=22.12.0`. #76562 fixed the manifest but not its mirror in
`package-lock.json`, and `npm ci` reads engines from the lockfile:
package.json apps/desktop -> {'node': '^20.19.0 || >=22.12.0'}
package-lock apps/desktop -> {'node': '>=26.0.0'} <- what gated
Chasing that exposed a second, pre-existing problem: the floor #76562
declared was too generous. Running the real `npm ci` against the whole
workspace on Node 22.21.1 fails on a transitive dependency —
npm error notsup Not compatible with your version of node/npm:
react-router@8.3.0
npm error notsup Required: {"node":">=22.22.0"}
react-router 8.3.0 (a direct dependency of both `apps/desktop` and `web`)
declares `>=22.22.0`, which is tighter than Vite's `^20.19 || >=22.12` and
excludes all of Node 20. So `>=20.0.0` promised support the tree cannot
deliver: an install on Node 20 or early 22 passed the installer's gate and
then died inside `npm ci` on someone else's package.
All four engine declarations now state the floor the dependency tree
actually has, `>=22.22.0`: root `package.json`, `apps/desktop/package.json`,
and both of their `package-lock.json` mirrors. The installer gates move with
them (`node_satisfies_build` in install.sh, `Test-NodeVersionOk` in
install.ps1) so a too-old system Node is replaced with the managed one
*before* npm runs, and the failure a user does see names hermes-agent rather
than a transitive package. NODE_VERSION stays 22 — latest-v22.x is 22.23.2,
comfortably above the floor.
The invariant test gains the case that would have caught the mirror drift on
its own: the desktop assertion now pins the tightest floor a dependency
actually declares, and the managed-runtime check compares majors, since
install.sh fetches latest-v{major}.x rather than {major}.0.0.
Verified with real `npm ci --dry-run` over the full workspace:
- node 22.23.2 (what install.sh provisions) -> 1258 packages
- node 26.5.1 -> 1189 packages
- node 22.21.1 (below the floor) -> EBADENGINE naming
hermes-agent, i.e. our own manifest, not react-router
Fresh installs and `hermes update` both fail at the first `npm ci`:
npm error code EBADENGINE
npm error notsup Required: {"node":">=26.0.0","npm":">=12.0.0"}
npm error notsup Actual: {"node":"v24.15.0","npm":"11.12.1"}
`.npmrc` sets engine-strict=true, so `engines` is a hard gate on every
install. The floor was raised to npm >=12 — but no Node release bundles
npm 12: Node 26 ships 11.17.0, 24 ships 11.16.0, 22 ships 10.9.8. The
requirement is unsatisfiable by any stock toolchain, so the installer
provisions a Node and is immediately unable to install with it.
engines.npm becomes `<11.10.0 || >=11.17.0`. That still excludes the band
the strictness was actually for: npm 11.10-11.16 honor `min-release-age`
but ignore `min-release-age-exclude`, both set in .npmrc, so they apply the
14-day gate to packages we exempted. Verified rather than assumed — npm
11.12.1 fails `ETARGET ... vite@8.2.0 with a date before 7/18/2026` while
11.17.0 installs it.
engines.node returns to >=20.0.0 and the toolchain floor to Node 22.
Nothing in the tree needs 26: Vite 8.2.0 declares `^20.19.0 || >=22.12.0`
and Electron 40 declares `>=12.20.55`. Requiring 26 force-migrated every
working install for no dependency reason. apps/desktop drops to Vite's own
floor for the same reason; the desktop bundle builds clean on Node 22.
install.sh gained a second gate: a system Node was accepted on version
alone, so a machine with Node 24 + its bundled npm 11.16.0 (the bad band)
passed the check and then failed `npm ci`. npm_supports_npmrc() now rejects
that band and installs the managed Node instead.
CI, Docker and nix are hermetic and keep pinning Node 26 / npm 12 — they
provision their own toolchain, and both satisfy the relaxed range.
tests/test_engines_satisfiable.py encodes the invariants that would have
caught this: the npm floor must be met by an npm some shipping Node
bundles, the node floor by the runtime install.sh provisions, the desktop
floor by its own build toolchain, and the lockfile mirror must match.
Restoring the broken values fails 5 of them with the reason stated.
Verified end-to-end (real downloads, temp HERMES_HOME):
- fresh install: managed node v22.23.2 / npm 10.9.8 -> npm ci, 209 packages
- existing managed tree (v22.22.3 / npm 10.9.8) -> npm ci, 209 packages
- node 26.5.1 + bundled npm 11.17.0 -> npm ci, 208 packages
- system npm 11.12.1 (the reported case) -> EBADENGINE, recovery provisions
a managed tree and retries green
- apps/desktop `npm run build` on Node 22 -> dist built, assert passes
_nb_ensure_bundled_npm_range ran only at the tail of
_nb_install_bundled_node, so it fired just after a tarball was unpacked.
ensure_node's reuse rung returns before reaching it, leaving an existing
managed tree on whatever npm its Node major bundled.
That strands a real install: the upgrade is best-effort (`|| true`), so
one offline run leaves an at-target Node 26 tree carrying npm 11.17.0 —
below the root package.json's `engines.npm` floor of >=12, fatal under
.npmrc's engine-strict. Heal does not cover it either; the tree is at the
target major and every binary passes --version, so
_nb_managed_node_needs_heal correctly reports it healthy. Re-running the
installer, the documented recovery, never repaired it.
install.ps1 already had this right: Update-ManagedNpm is called from both
branches that yield a managed tree, including the reuse path. This is the
POSIX side of that same call site.
Reproduced on a seeded node-26.5.1/npm-11.17.0 tree: before, ensure_node
left npm at 11.17.0 and `npm ci` died with EBADENGINE; after, it upgrades
to 12.0.2 and `npm ci` installs 208 packages. An already-in-range tree
costs one --version probe (~0.13s), and the system-node path is unchanged.
Co-authored-by: ethernet8023 <arilotter@gmail.com>
Follow-up to 6fdc64efc, which fixed only the POSIX bootstrap. install.ps1
unpacks the same nodejs.org build, so Windows had the same EBADENGINE:
Node 26.5.1 bundles npm 11.17.0, one minor below the root package.json's
`engines.npm` floor of >=12, and .npmrc's engine-strict=true makes that
fatal at the first `npm ci`.
Update-ManagedNpm mirrors _nb_ensure_bundled_npm_range rung for rung —
temp cwd so the checkout's .npmrc cannot gate the upgrade meant to
satisfy it, npm_config_min_release_age=0, and an explicit --prefix at the
managed tree. EAP is relaxed around the npm call for the same reason
Install-Uv does it: npm's stderr would otherwise wrap as ErrorRecords and
short-circuit before $LASTEXITCODE is read. Env vars and location are
restored in a finally.
Called from both branches that yield a managed tree: the fresh portable
unpack, and the reuse-an-existing-tree path, where an older install still
has its original major's npm sitting there. The in-range check makes the
second a one-probe no-op on reruns.
The range comes from Get-NpmRange, which prefers the checkout's
package.json but falls back to a $NpmRange constant — unlike the POSIX
side, Test-Node runs before the repo is cloned, so there is usually no
manifest on disk yet (and none at all when install.ps1 is piped from the
web). The manifest read means a drifted constant self-corrects on any run
against an existing checkout.
Not executed locally: no pwsh on this machine, and the repo runs no
PowerShell in CI.
The bundled-Node bootstrap unpacked the nodejs.org tarball and stopped.
Node 26.5.1 bundles npm 11.17.0, one minor below the root package.json's
own `engines.npm` floor of >=12 — and .npmrc sets `engine-strict=true`,
so that is fatal rather than a warning:
npm error code EBADENGINE
npm error notsup Required: {"node":">=26.0.0","npm":">=12.0.0"}
npm error notsup Actual: {"node":"v26.5.1","npm":"11.17.0"}
A brand-new install died at the first `npm ci` with "Desktop workspace
npm install failed". CI never saw it because the workflows run an
explicit `npm i -g npm@12`; the Python update path recovers through
hermes_cli/npm_engine.py, but the installer path had no such rung.
_nb_ensure_bundled_npm_range() now upgrades the managed tree's npm into
range right after the tarball lands, mirroring upgrade_managed_npm():
- temp cwd, so the checkout's own .npmrc (engine-strict,
min-release-age) does not gate the upgrade meant to satisfy it;
- npm_config_min_release_age=0, which also neutralises a user ~/.npmrc;
- explicit --prefix at the managed tree, because
_nb_configure_npm_prefix writes prefix=~/.local into its etc/npmrc
and a bare `npm i -g` would install a second npm elsewhere while the
managed tree stayed stale.
The range is read out of package.json rather than duplicated, so the two
cannot drift, with HERMES_NPM_TARGET_RANGE as an override and a >=12.0.0
fallback for a stripped install tree. An already-in-range npm skips the
network round-trip. Best-effort: a failed upgrade warns with the manual
command and keeps the working Node, since npm_engine.py still covers the
EBADENGINE that follows.
Verified against a real tree provisioned by this bootstrap: node v26.5.1
/ npm 12.0.2, bin/npm and bin/npx still relative-symlinked into the
upgraded lib/node_modules/npm, the ~/.local/bin links resolving to 12.0.2
through the tree, and no stray second npm under ~/.local/lib.
Existing users who only ever launch Hermes (never re-run an installer)
kept their managed Node 22 tree forever: the heal path only fired for
*broken* trees, and a healthy 22 passes the --version probe. Now
"outdated" heals the same way "broken" does, on both sides of the mirror:
- hermes_constants.py: find_hermes_node_executable() checks
_managed_node_tree_outdated() (managed node major <
_HERMES_NODE_TARGET_MAJOR) and routes through the existing
once-per-process heal_hermes_managed_node(), which redownloads
latest-v26.x. When the heal fails (offline, download error) the
outdated-but-runnable tree is still returned — old Node beats no Node.
- scripts/lib/node-bootstrap.sh: _nb_managed_node_needs_heal() gains the
matching _nb_managed_node_outdated() rung, so heal_managed_node agrees
with the Python side.
This is the same shape as the managed-uv flow: resolve the managed
runtime, notice it can't satisfy the requirement, provision the right one
in place, fall back gracefully.
Tests (tests/test_hermes_constants.py): outdated tree triggers heal and
returns the upgraded binary; failed heal still serves the old tree; an
at-target tree never heals (heal stub raises).
Hermes now pins its toolchain to Node 26 everywhere. Every path that
installs, accepts, heals, or upgrades a Node runtime moves from the old
22-default / `^20.19 || >=22.12` floor to a single rule: Node >=26.
Installers:
- scripts/install.sh — NODE_VERSION=26; node_satisfies_build() collapses
the two-branch Vite floor to `major >= 26`; user-facing messages updated.
- scripts/install.ps1 — $NodeVersion=26; Test-NodeVersionOk likewise;
winget fallback switches OpenJS.NodeJS.LTS -> OpenJS.NodeJS (26 is
Current, not LTS — the LTS manifest would reinstall a too-old Node).
- Dockerfile — node_source stage node:22-bookworm-slim -> node:26 (digest
pinned, amd64 sha256:9e6f...bf73).
- nix/ was already on nodejs_26 (lib.nix, npm-12-0-2.nix); the checks.nix
wrapper check ratchets from `>= 20` to `>= 26`.
Heal/upgrade paths:
- scripts/lib/node-bootstrap.sh — HERMES_NODE_TARGET_MAJOR default 22->26
and HERMES_NODE_MIN_VERSION default 20->26, so heal_managed_node,
_nb_install_bundled_node, and the fnm/proto/nvm/brew rungs all target 26
and stop accepting an on-PATH Node below it. Both remain env-overridable.
- hermes_constants.py — _HERMES_NODE_TARGET_MAJOR fallback 22->26, which
drives the Windows heal path's latest-v26.x download.
Version gates:
- package.json engines.node >=20 -> >=26; apps/desktop engines
`^20.19.0 || >=22.12.0` -> `>=26.0.0`.
- CI setup-node: all five workflows 22 -> 26.
- Docs describing Hermes's own toolchain updated (windows-native, docker,
acp, nix-setup, contributing). Skill docs describing third-party tools'
own requirements are untouched.
Termux still installs via `pkg install nodejs` best-effort (nodejs.org
ships no Android tarballs); that path was never version-gated.
Verified: bash -n on both shell scripts, PowerShell AST parse of
install.ps1, latest-v26.x index resolves (node-v26.5.1), and the install
test suite — 18 tests across the 5 install/runtime test files — passes.