Commit Graph

551 Commits

Author SHA1 Message Date
Ben Barclay bb597e1c02
fix(cron): managed-cron fires execute in the gateway process (live adapters + dashboard forwarder) (#84339)
* fix(gateway): pass live adapters to cron fire webhook's fire_due

The Chronos fire webhook (/api/cron/fire) called
provider.fire_due(job_id, adapters=None, loop=loop), so every
externally-triggered fire delivered through the standalone path even
with a live gateway in-process. E2EE platforms and relay-fronted
logical platforms (whose ONLY send path is the live relay adapter — no
native credential exists on the box) failed every external fire with
"platform 'X' not configured/enabled", while the same job delivered
fine under the built-in ticker (gateway/run.py passes runner.adapters).

Resolve the runner (self.gateway_runner → app['gateway_runner'] →
_gateway_runner_ref(), the same chain the drain check uses) and forward
its adapters. No runner → adapters=None, preserving the historical
standalone path byte-identically.

Note: does not by itself fix Fly-hosted scale-to-zero deployments where
NAS's callback lands on the DASHBOARD process (internal_port 9119) —
_fire_cron_job_for_profile there has no gateway runner in-process. That
topology needs a separate fire handoff (design pending).

* fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable)

The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD
process via _fire_cron_job_for_profile with adapters=None. On hosted
deployments (Fly proxy exposes only the dashboard's port) that made
every managed-cron fire deliver through the standalone send path, which
cannot serve relay-fronted logical platforms (their only sender is the
live relay adapter in the gateway process — no native credential exists
on the box) or E2EE rooms. It also ran the whole agent turn inside the
dashboard: wrong process for memory/session ownership and fire-claim
attribution.

Restore the invariant that the GATEWAY owns cron execution:

- Dashboard route: after verifying the NAS JWT and resolving the job's
  profile, FORWARD the fire to the gateway api_server's own
  /api/cron/fire on loopback, NAS bearer preserved (the gateway
  re-verifies the JWT — defense in depth, no new trust link), and pass
  the gateway's response through. Gateway unreachable → 503 so NAS
  retries per the Chronos contract (non-2xx = retryable; the store CAS
  de-dupes the eventual double fire). Deliberately NO local-execution
  fallback.
- Endpoint resolution mirrors gateway/config.py's api_server load order
  per target profile (config.yaml extra.port → API_SERVER_PORT from
  process env or the profile's .env → 8642), with /p/<profile>/ prefix
  routing under multiplex.
- docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on
  first boot when absent (never overwrites an operator value), so the
  loopback api_server passes its startup guard on hosted images. The
  fire route itself is NAS-JWT-authed; the key gates the rest of the
  api_server surface. The listener binds 127.0.0.1 by default and the
  Fly service exposes only the dashboard port.
- _fire_cron_job_for_profile kept but deprecated (late-binding seam
  compatibility); no route calls it.
- docs/chronos-managed-cron-contract.md: document the two-hop inbound
  topology and the 503-retry semantics.

Depends on the previous commit (fire webhook passes live adapters to
fire_due) — together they make NAS→dashboard→gateway fires deliver over
relay end to end.

* fix(cron): read the profile api_server port via the canonical config loader

CI guard test_config_read_guard flagged the new _gateway_fire_endpoint
for a raw yaml.safe_load of the profile's config.yaml — the exact drift
class the guard exists to kill (raw reads miss the managed-scope
overlay, ${ENV_VAR} expansion, and root-model normalization).

Read through load_config() under a HERMES_HOME override scoped to the
target profile instead (the same pattern the deprecated
_fire_cron_job_for_profile uses for its store scope), and pull the port
with cfg_get. Test updated to stub load_config rather than write a raw
config.yaml.

* fix(gateway): only messaging platforms count for the scale-to-zero arm gate

The stage2 hook now generates API_SERVER_KEY for every Docker container,
and key presence force-enables the api_server platform. The scale-to-zero
arm gate counted every enabled platform, so the loopback api_server
listener made messaging_is_relay_only_or_absent False on every hosted
instance — silently disarming the feature (the not-armed log would show
enabled platforms=['relay','api_server']).

The arm gate and the not-armed logger now share one helper that filters
to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK —
the same non-messaging exclusion set _connect_platforms already uses.
A genuinely enabled direct-socket platform (Discord/Telegram) still
disarms. Two of the three new tests fail without this fix.
2026-08-12 17:04:44 +10:00
Brooklyn Nicholson 1e6a7b3315 fix(desktop): scope custom provider settings to the active profile
The custom-endpoint REST handlers ran bare load_config/save_config, so
every add/activate/delete landed in the process-level default profile
regardless of which profile the desktop settings UI was targeting. A
provider added under a non-default profile silently went to default:
visible only in default-bound sessions, absent everywhere else, and
un-addable to another profile without hand-editing its config.yaml.

Scope all four handlers (list/upsert/activate/delete) to the requested
profile via _config_profile_scope, matching /api/config, and spread the
active profile into the four hermes.ts wrappers alongside their existing
validateCustomEndpoint sibling.
2026-08-10 22:26:54 -05:00
Math 1edfdeee81 fix(desktop): keep serve backend alive through Windows launcher 2026-08-10 19:16:59 -07:00
Teknium 33f8e96a72 fix: guard has_env profile probe with _safe like its sibling fields
An unreadable profile dir made (entry / '.env').exists() raise
PermissionError out of the sidebar fallback, 500ing /api/profiles.
Found by hostile fixture during live E2E of the scandir conversion.
2026-08-10 18:04:59 -07:00
Michael Gannotti 373631bea1 fix(dashboard): raise fd soft limit + replace iterdir with scandir to stop fd leak (#81547)
Two-part fix for the dashboard fd exhaustion reported in #81547:

1. Raise RLIMIT_NOFILE soft limit on startup (before uvicorn binds).
   macOS defaults to 256 for LaunchAgent processes — too tight for the
   dashboard which opens 3 fds (db+wal+shm) per SessionDB per request
   across all profiles. After days of polling the soft limit exhausts
   and every os.listdir/open raises OSError [Errno 24]. The helper raises
   to the hard limit (or minimum 4096), matching the reporter's ulimit
   workaround. No-op on Windows (no resource module).

2. Replace bare Path.iterdir() with context-managed os.scandir() in four
   dashboard hot paths: _fallback_profile_dicts, file manager list,
   checkpoint listing, and plugin discovery. iterdir() returns a
   generator that holds an open directory fd until fully consumed; if
   an exception interrupts iteration the fd leaks. os.scandir() is an
   explicit context manager that guarantees close on exit, following
   the same idiom already used in /api/fs/list.

Tests: 6 passed, 3 skipped (resource-module tests skip on Windows).
2026-08-10 18:04:59 -07:00
Quark Assistant 0b15eb5f05 fix(desktop): terminate app-managed gateway on shutdown 2026-08-10 18:04:59 -07:00
RelaxJonh 07298df805 fix(gateway): reap orphaned gateways before spawning restart (#77276)
_spawn_gateway_restart() now calls _reap_unsupervised_gateway_orphans()
before spawning a new `hermes gateway restart` child.  On desktop-app
restart the old serve exits but its gateway child gets reparented to
launchd (PPID=1) and keeps its platform connection alive.  The new
serve then spawns a fresh gateway, resulting in two live gateways
racing the same connection.

The reap was already implemented for the CLI restart path (#75936) but
the dashboard's _spawn_gateway_restart path was not covered.

Fixes #77276
2026-08-10 18:04:59 -07:00
cadezhou bc1223840d fix(desktop): reap orphan gateways at startup
On Desktop serve startup, reap orphan gateway processes (PPID=1) left
behind by a previous serve session that exited abnormally. This prevents
the old and new gateways from racing for the same QQ WebSocket
credential, which splits messages across parallel session trees (#77276).
2026-08-10 17:02:56 -07:00
Leon Phull 888624ae61 fix(cli): never reap serve processes owned by a valid backend.lock.json
Production incident: the orphan reap killed a legitimate SSH remote backend
started by another client machine. Its process sat at ppid 1 with the same
cmdline shape as a genuine orphan, and the exclusion list only covered THIS
app instance's children — ownership by OTHER clients was invisible.

The reap now treats every backend.lock.json under ~/.hermes/desktop-ssh/*/
as an ownership claim: lock payloads are schema-validated (mirroring
remote-lifecycle.ts) and their PIDs are excluded both before the scan and
re-checked after it (defense in depth against a lock written mid-scan).

Regression tests cover the exact incident shape: a lock-owned PID and a
genuine orphan with identical process shapes — only the orphan is reaped.

Also: fold the new single-field `runtime` config category into `agent`
(_CATEGORY_MERGE) and fix an env leak in the serve-startup test
(HERMES_SERVE_HEADLESS restored via monkeypatch) so the combined suites
run green in any order.
2026-08-10 17:02:56 -07:00
Leon Phull 6386c75306 fix(desktop): reap orphaned local serve backends on desktop boot
When Desktop exits uncleanly, leftover `hermes serve --host 127.0.0.1 --port 0`
processes can be reparented to pid 1 and keep full MCP trees alive. The next
boot then stacks another backend on top of the corpses until EMFILE kills
sidebar/session APIs and tabs disappear.

- Detect Desktop-local serve shape (loopback + ephemeral port 0)
- Only reap processes whose ppid is 0/1 (true orphans)
- Spare fixed-port remote serves (e.g. --port 9119) and HERMES_DESKTOP_CHILD_PID
- Run at Desktop backend start (HERMES_DESKTOP=1) before parent-death watchdog

Complements parent-death watchdog (prevents future orphans) and configurable
nofile soft limit (capacity floor). Together these stop the multi-backend
pile-up cascade observed on macOS Desktop SSH/local installs.
2026-08-10 17:02:56 -07:00
XiaoZAZA a9a0648f49 fix(desktop): reap orphaned serve backends via parent-death watchdog + group-kill
An unclean desktop exit (crash / SIGKILL / update handoff) stranded every
`hermes serve` profile backend as an orphan (ppid=1) still serving, each
holding its MCP child subtree — 31 orphans / ~1.3 GiB RSS on one install.

Root causes + fixes:
- serve had no parent-death watchdog: add _start_parent_death_watchdog() in
  web_server.py (mirrors slash_worker.py), gated on HERMES_PARENT_PID; os._exit
  cascades to MCP watchdogs. No-op for standalone `hermes serve`.
- desktop passes HERMES_PARENT_PID in both serve spawn env blocks (main.ts).
- POSIX teardown now group-kills (process.kill(-pid, ...)) so MCP grandchildren
  die too (backend-child.ts + waitForBackendExit SIGKILL fallback).

Windows path unchanged (forceKillProcessTree). Tests updated + passing.
2026-08-10 17:02:56 -07:00
kshitij 4ecdee38a6 fix(dashboard): close config-RMW gaps left by the off-loop sweep 2026-08-08 13:28:28 +05:30
Royalaid 965a548788 fix(gateway): serialize config mutations and finish the router off-loop sweep
Two follow-ups to the off-loop move, from external review (both verified,
the second larger than reported):

- Config read-modify-write handlers moved to worker threads could now
  interleave — _CONFIG_LOCK covers each load/save individually, never the
  span between them; the event loop used to serialize these accidentally.
  New _CONFIG_MUTATION_LOCK (worker-threads only, so it can never block
  the loop) held across the whole load→mutate→save span in all seven RMW
  handlers. update_config_raw skipped: it's a full-document replace with
  no server-side read, so a lock cannot close its client-side window.

- The review flagged two skills routes still taking _SKILLS_PROFILE_LOCK
  on the event loop; a systematic audit of hermes_cli/web_routers/ found
  24 on-loop routes (skills 5, mcp 9, tools 10, cron 1). All moved to the
  same inner-_run + asyncio.to_thread pattern, mutating ones under the
  mutation lock, uniform lock order (_SKILLS_PROFILE_LOCK →
  _CONFIG_MUTATION_LOCK). Await-safe _config_profile_scope routes, plain
  def routes, and already-threaded routes unchanged.

Regression tests: concurrent theme+font updates both survive (fails with
the lock nulled: "theme write lost to a concurrent font write"); event
loop stays responsive while the profile lock is held during GET
/api/skills. 214 tests passing across the touched suites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 13:28:28 +05:30
Royalaid 52c9aee3bf fix(gateway): move _profile_scope and config I/O off the event loop in async handlers
The diagnostics loop watchdog caught GET /api/config freezing the gateway
event loop for >1s, stack-sampled blocking on _SKILLS_PROFILE_LOCK inside
_profile_scope. Any async handler that entered _profile_scope (process-wide
threading lock) or called load_config()/save_config() on-loop could stall
every chat and WebSocket at once while a slow lock-holder ran.

Move 28 such handlers to the existing inner-_run + asyncio.to_thread
pattern (contextvar-safe: the whole scope enter/body/exit stays inside one
worker thread). Handlers using the await-safe _config_profile_scope, plain
def endpoints (FastAPI threadpool), and tui_gateway's contextvar-only
decorator are unaffected and unchanged.

Regression test holds _SKILLS_PROFILE_LOCK in a thread while calling
GET /api/config and asserts an event-loop heartbeat keeps ticking; it fails
against the pre-fix code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 13:28:28 +05:30
Teknium c228d1c559 fix(dashboard): fold one-field doctor category into general tab
doctor.live_probe_timeout is the only schema-surfaced doctor.* field;
merge it into the general tab per the no-orphan-category invariant.
2026-08-07 09:07:48 -07:00
kshitij afb46fdab4 refactor(cron): polish registration partial-failure surfaces
Follow-up to the salvaged registration contract:
- share one _raise_if_cron_registration_error() helper for the two
  byte-identical dashboard 424 except-blocks (web_server + cron router,
  via the existing late() seam)
- add endpoint-level 424 coverage for /api/cron/blueprints/instantiate
  (previously only the sync worker was tested)
- give chat/CLI surfaces a human-facing user_message() (job name, no
  exception class name) and add a recovery hint (pause/resume or update
  re-registers via provider reconcile) to the model/REST message
- consolidate five inline provider test doubles into one ABC-subclassing
  make_cron_provider conftest factory; the web_server test double now
  subclasses CronScheduler so an ABC rename fails loudly
- narrow the wrapper facade to keyword-only (**kwargs) and route the
  tool's partial-failure return through tool_error()
2026-08-07 17:45:06 +05:30
Gille f346458f29 fix(cron): surface initial scheduler registration failures 2026-08-07 17:45:06 +05:30
emozilla bdee48928f fix(dashboard): derive the stale-schema read probe from SCHEMA_SQL
After `hermes update`, the desktop sidebar showed "No sessions yet" until
the user's first message. #72424 added sessions.last_activity_at, which
list_sessions_rich now selects — but column adds only land through
_reconcile_columns() in the writable _init_schema, and read-only opens
skip that by design. Every sidebar read path opens state.db read-only, so
each poll raised "no such column: s.last_activity_at" until the first
prompt's lazy session-row persist forced a writable open and reconciled.

A heal for exactly this class already existed (_open_session_db_for_profile
probes the read-only handle and does a one-time writable reopen on
staleness), but its probe was a hand-written four-column list that never
learned last_activity_at — it went stale three days after shipping. And the
batched sidebar route (/api/profiles/sessions/sidebar) bypassed the helper
entirely, swallowing per-profile failures into an errors array the desktop
never surfaces, so the incident produced an empty sidebar with clean logs.

The fix removes the maintenance burden instead of paying it once more:

- hermes_state_schema.schema_read_probe_statements() derives one
  `SELECT <every declared column> FROM <table> LIMIT 0` per table from
  SCHEMA_SQL via the existing _parse_schema_columns() — the same source of
  truth the writable reconciler diffs against, so any future ADD COLUMN is
  probed with no list to update. Column references are table-qualified:
  an unqualified double-quoted identifier that fails to resolve silently
  degrades to a string literal (SQLite's double-quoted-string misfeature)
  and would make the probe pass on exactly the store it exists to catch.

- web_server splits the heal into a path-level _open_session_db_at_path
  (semantics unchanged) so the cross-profile session routes can share it;
  both profiles.py loops and _count_status_active_sessions (the remaining
  raw read-only sibling) now open through it. The heal stays a helper
  rather than a SessionDB classmethod on purpose: escalation-to-writable
  must remain an explicit caller decision — update_cmd.py opens read-only
  mid-update and must never write.

- Exhaustion guard: if the writable heal SUCCEEDS and the re-probe still
  fails (a schema problem ADD COLUMN cannot express), the store is marked
  exhausted — warn once, skip the probe, serve reads probe-less — instead
  of re-running the full writable init on every poll against a possibly
  live DB. A FAILED writable open (transient lock) is deliberately not
  recorded, so the next poll retries the heal.

- The per-profile swallow sites in profiles.py now also log a deduplicated
  warning, so a persistent read failure is loud in errors.log even though
  the response errors array stays invisible to the sidebar.

Tests: probe/SCHEMA_SQL coverage invariants (tests/test_schema_read_probe.py),
last_activity_at added to the /api/sessions heal parametrize, a sidebar-route
heal test reproducing the shipped symptom (errors == [] and the session
returned against a store missing the column), and an exhaustion test pinning
exactly one writable open. The sidebar and last_activity_at tests fail on
main.
2026-08-07 00:41:58 -04:00
Brooklyn Nicholson 950b55d4d7 feat(update): emit an action-scoped terminal receipt from hermes update
The dashboard now mints an action_id per backend update, hands it to the
spawned `hermes update` via HERMES_ACTION_ID, and reuses an in-flight
update action instead of spawning a duplicate. The updater prints a
bounded `=== hermes-update completed <id> ===` receipt on every success
path — normal, zip, dependency-repair, and the no-op "Already up to
date!" path that previously ended with no terminal marker at all
(#58764) — so the Desktop can prove completion across the dashboard
restart boundary instead of guessing from stale log text.

Co-authored-by: Vitor Cepeda Lopes <vitor@vitorcepedalopes.com>
Co-authored-by: doncazper <caztronics@yahoo.com>
2026-08-05 10:34:18 -06:00
jinglun010 25a9c2c245 perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)
Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.
2026-08-03 21:16:09 +05:30
ehz0ah e43bc0b7aa fix(openviking): integrate reliability and configuration hardening 2026-08-03 20:35:47 +05:30
kshitij decf12eda0 perf(dashboard): serve hashed /assets bundles with immutable cache headers
Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.

Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
  on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
  /fonts/, /fonts-terminal/, /ds-assets/, /assets/.

index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.

The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.

Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-03 18:51:43 +05:30
youzi 9645ea8d52 fix(web): avoid blocking provider validation 2026-08-03 18:47:30 +05:30
aydnOktay 105aba6705 fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)
Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).
2026-08-03 18:46:58 +05:30
Rod Boev e3ce092f8d perf(dashboard): keep tools in focused analytics usage (#18511) 2026-08-03 18:45:43 +05:30
Rod Boev c1639322c2 perf(dashboard): skip full InsightsEngine on /api/analytics/usage (#18511) 2026-08-03 18:45:43 +05:30
kshitij 1f1acc0e4d fix(dashboard): warm cold check_fn verdicts with a background probe
On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.
2026-08-03 18:44:38 +05:30
Hafiz Ahmad Ashfaq 0de8c32f52 fix(dashboard): cache plugins hub payload and avoid auth probes 2026-08-03 18:44:38 +05:30
wayne1992127 579672d87d fix(desktop): keep cold gateway config off event loop 2026-08-02 23:20:24 +05:30
lost9999 47fa4385df fix(serve): cache /api/status profile-gateway topology scan
/api/status is the desktop's boot liveness probe (polled ~1/s) but since
#60537 every call ran a full topology scan — per-profile yaml.safe_load
(pure-Python loader), psutil process probes, realpath walks — in the
default executor. On multi-profile installs concurrent polls pile up and
hold the GIL 14-16s, starving the event loop: the WS sidecar cannot
flush gateway.ready, the desktop times out into the next stall, and boot
escalates to the 'Hermes couldn't start' overlay (#60800).

Memoize the scan behind a 10s TTL with a collapse lock so concurrent
polls share one scan. Topology only changes on gateway start/stop, so a
<=10s stale badge is an acceptable trade for not starving the loop. The
cache also keys on the collector's identity: tests monkeypatch
_collect_profile_gateway_topology per case, and the identity check keeps
them hermetic (a swapped collector is a miss) without a reset hook.

py-spy captures during a failing boot land in _profile_platform_ports ->
yaml.safe_load on executor threads (7 profiles, Windows). After: one
cold-start scan, zero recurring stalls, desktop boots.
2026-08-02 23:10:28 +05:30
Teknium 5c6cc38010 fix(secrets): scope-aware credential reads in core tool/gateway/web-server paths
TOOL_GATEWAY_USER_TOKEN (managed_tool_gateway), OPENROUTER_API_KEY presence
(openrouter_client), SUDO_PASSWORD (terminal_tool), GATEWAY_PROXY_KEY
(gateway/run), SLACK_BOT_TOKEN presence (gateway/session), and the
ELEVENLABS_API_KEY env fallback (web_server voices endpoint) now honor the
installed profile secret scope; unscoped callers keep legacy env reads via
the UnscopedSecretError fallback (Slack pattern).
2026-08-02 10:02:33 -07:00
joelbrilliant 57197cd48d fix(dashboard): preserve maintenance writes on read polling
Signed-off-by: joelbrilliant <joelbrilliant1@gmail.com>
2026-08-02 21:30:54 +05:30
joaomarcos 5aa7594993 Merge remote-tracking branch 'origin/main' into fix-73914-atomic-cancel
# Conflicts:
#	tests/hermes_cli/test_web_oauth_dispatch.py
2026-07-31 03:35:08 -03:00
Teknium ad12df6ba4 Revert "remove Vercel AI Gateway and Vercel Sandbox (#33067)"
This reverts commit febc4cfec0.
2026-07-29 19:48:37 -07:00
Rod Boev e461e86502 test(cli): remove importlib.reload seam from web_server session-token tests (#38034)
Extract _resolve_session_token() in hermes_cli/web_server.py so tests can
exercise token resolution directly instead of importlib.reload(ws), which
re-executed the whole module mid-suite (fresh FastAPI app + token) and
split module identity between test and app state.

Salvaged from PR #39038 by @rodboev (maintainer-endorsed direction);
rebased onto the rewritten test_web_server.py — dropped the PR's hunks for
test_falls_back_to_random_token's old body (test deleted in the prune,
re-added here in the PR's new form).

Co-authored-by: Rod Boev <rod.boev@gmail.com>
2026-07-29 18:55:10 -07:00
Brooklyn Nicholson f174c0b6bb fix(pairing): scope the approve/revoke endpoints to a profile
The gateway keeps one PairingStore per served profile, but every
`/api/pairing` endpoint built the global one. An operator managing a named
profile saw the wrong pending list, and approving wrote a grant into a
whitelist their running gateway never consults — the user stays locked out
while the UI shows them as approved.

`_pairing_store(profile)` now resolves per profile and validates the name
(400/404 on an unknown one). No `_profile_scope` needed: PairingStore
resolves the profile's home itself, so nothing process-global is swapped
across an await.

Both GUIs had to change to match. The listing rides the query param — for
the dashboard that meant deleting `pairing` from the "machine-global, must
NOT be rewritten" exclusion list, a comment this change makes false. The
mutating endpoints read the profile off the BODY, which no query-param
rewrite reaches, so approve/revoke send it explicitly on both surfaces.
2026-07-29 18:43:50 -05:00
teknium1 011ec4513e refactor(web): extract sessions/mcp/skills/tools routes to APIRouter modules (wave 2; route-table equality verified)
- hermes_cli/web_routers/sessions.py: 14 routes across 3 routers
  (list_router, search_router, manage_router) mounted at the three original
  registration points so global route order is preserved exactly.
- hermes_cli/web_routers/mcp.py: 11 routes; OAuth flow registry
  (_mcp_oauth_flows/lock/cap) stays in web_server, reached via new
  web_deps.LateState live proxies so tests mutating web_server._mcp_oauth_flows
  keep working.
- hermes_cli/web_routers/skills.py: 12 routes across hub_router + router
  (two original registration points straddle the profiles router include).
- hermes_cli/web_routers/tools.py: 12 routes; toolset/terminal catalogs stay
  in web_server (some are defined after the mount point), reached via LateState.
- web_deps.py: add LateState — operation-time proxy for web_server-owned
  module state (getattr/item/iter/len/contains/context-manager/comparisons).
- Handler bodies byte-identical; legacy re-exports keep
  web_server.<handler> importable for tests.
- Verified: ordered route table (method, path) identical to pre-refactor app
  (291 routes); import smoke; ruff; windows-footguns clean.
- test_web_server_sessiondb_eventloop.py: structural AST scan now reads both
  web_server.py and web_routers/sessions.py (handlers moved; helpers stayed).
2026-07-29 16:21:54 -07:00
Brooklyn Nicholson 37d0766ba5 fix(pairing): keep GUI approvals off the code brute-force lockout
Follow-up hardening on the request-id grant path.

approve_request took the same lockout treatment as approve_code: gated by
it, and recording a miss toward it. But the two paths defend different
things. The lockout exists to stop guessing at the 8-char code space over a
messaging channel; a request id is only ever obtained by an admin already
authenticated to the store, so a miss means the row they clicked went stale.
Counting those let a handful of clicks on a stale list lock the operator out
of `hermes pairing approve` for an hour — the GUI DoSing the CLI.

Also drops the `code`/`code_hash_prefix` compat fields from list_pending.
The hash prefix is what admin surfaces mistook for an approvable code in the
first place, and re-exporting the request id under the old `code` key just
preserves the ambiguity; both consumers in the tree read `request_id` now.
The 16-hex sniffing that had been copy-pasted into the CLI and the endpoint
(where a chained conditional consulted it against the wrong field) moves to
one owner, PairingStore.looks_like_request_id.

The endpoint no longer reports a 429 on the request-id path, where lockout
can't apply — a stale id surfaced as a bogus "locked out" while the platform
sat locked for something else entirely.
2026-07-29 17:51:40 -05:00
Flownium 774d92fbfa fix: approve listed pairing requests 2026-07-29 17:49:33 -05:00
teknium1 27b1377b4c refactor(web): extract git/profiles/cron routes to APIRouter modules (web_deps seam; route-table equality verified) 2026-07-29 12:23:28 -07:00
teknium1 bd93ccb890 refactor(gateway): shared fence-aware markdown chunker core (yuanbao-derived) + canonical table-row splitter 2026-07-29 11:53:59 -07:00
teknium1 5b751dc0ad chore: remove unused imports and dead locals (ruff F401/F841 sweep)
Cleans F401 unused imports and F841 dead local assignments across
root *.py, agent/, hermes_cli/, tools/, gateway/, cron/, tui_gateway/
(tests/, plugins/, skills/ excluded).

Intentionally KEPT (false positives / test-patch surfaces):
- agent/transports/__init__.py package re-exports
- cli.py browser_connect re-exports (DEFAULT_BROWSER_CDP_URL area,
  used by tests/cli/test_cli_browser_connect.py)
- hermes_cli/main.py _prompt_auth_credentials_choice /
  _model_flow_bedrock_api_key (accessed via main_mod attr in tests)
- gateway/run.py aliased replay_cleanup + whatsapp_identity re-exports
  and _PORT_BINDING_PLATFORM_VALUES (test-referenced)
- hermes_cli/web_server.py get_running_pid (tests monkeypatch it) and
  _OAUTH_TOKEN_URL availability probe
- hermes_cli/config.py get_process_hermes_home re-export (noqa'd F811
  chain) and yaml availability-probe import
- hermes_cli/nous_subscription.py managed_nous_tools_enabled
  (tests patch hermes_cli.nous_subscription.managed_nous_tools_enabled)
- try/except ImportError availability probes (env_loader, tts_tool,
  mcp_tool, web_server anthropic OAuth block)
- tools/web_tools.py noqa F401 re-exports
- hermes_cli/setup_whatsapp_cloud.py:263 'proceed' skipped: possible
  missing-guard bug, flagged for separate review
- unused function parameters (signature changes out of scope)

Side-effect RHS calls preserved where only the binding was dead
(e.g. web_server proc = _spawn_hermes_action -> bare call).
2026-07-29 11:53:39 -07:00
teknium1 ed33ebca1d refactor: canonical config loaders for behavioral reads + guarded raw-read primitive (kills the managed-scope/env-expansion drift class)
The disease: ~15 scattered raw yaml.safe_load(config.yaml) reads that
silently miss managed-scope overlay, ${ENV_VAR} expansion, profile-aware
pathing, and root-model normalization. Every new config feature needed an
N-site sweep (incident chain 9cbcc0c9c8732293cf87b0e47a98f91928aa0443). This commit assigns every raw read to an owner and adds a
lint-guard test so the class cannot regrow.

New primitive (additive-only change to hermes_cli/config.py):
  read_user_config_raw(path=None) — reads the user file EXACTLY as
  written; docstring states it is ONLY legal for write-back round-trips
  and raw-file diagnostics. Behavioral reads must use
  load_config()/load_config_readonly().

BEHAVIOR FIXES (class-a sites migrated to a canonical loader — these
previously read values that could DIFFER from the effective config):

  gateway/run.py _try_resolve_fallback_provider → _load_gateway_runtime_config
    keys: fallback_providers/fallback_model (provider, model, base_url,
    api_key). Drift fixed: a managed-pinned fallback chain was ignored;
    an api_key of "${OPENROUTER_API_KEY}" reached the resolver unexpanded.
  gateway/run.py GatewayRunner._load_provider_routing → same loader
    key: provider_routing. Drift fixed: managed-pinned routing prefs and
    ${VAR} templates were ignored.
  gateway/run.py GatewayRunner._load_fallback_model → same loader
    keys: fallback chain. Same drift as above.
  gateway/run.py GatewayRunner._refresh_fallback_model
    keeps the raw primitive (its last-known-good-on-parse-failure contract
    forbids the fail-open loader, which returns {} on a torn write) but now
    applies managed overlay + env expansion inline. Drift fixed: chain
    edits under managed scope / env templates were previously frozen out.
  tui_gateway/server.py _load_cfg (72 behavioral call sites)
    now = raw read + managed overlay (pre-existing) + NEW ${VAR} expansion,
    split from a new _load_cfg_raw() write-back primitive. Drift fixed:
    e.g. custom_prompt: "hello ${VAR}", agent.system_prompt, model,
    api_key/base_url templates reached sessions unexpanded. DEFAULT_CONFIG
    is deliberately NOT merged (callers treat missing keys as unset;
    `_load_cfg() == {}` sentinels and _save_cfg round-trips depend on it).
  tui_gateway/server.py _profile_configured_cwd
    keys: terminal.cwd of a NON-launch profile. Drift fixed: managed
    overlay + ${VAR} expansion now apply (load_config() would resolve the
    wrong profile's home, so the raw primitive + inline pipeline is used).
  plugins/platforms/telegram/adapter.py _reload_dm_topics_from_config
    → load_config_readonly(). keys: platforms.telegram.extra.dm_topics.
    Drift fixed: managed overlay + profile-aware pathing + expansion.
  plugins/memory/holographic _load_plugin_config → load_config_readonly().
    keys: plugins.hermes-memory-store.*. Same drift class.

WRITE-BACK ROUND-TRIPS (class-b: stay raw BY DESIGN via read_user_config_raw;
merging defaults/overlay would pollute the saved user file):
  gateway/slash_commands.py: model persist x2, _save_gateway_config_key,
    memory/skills write_approval toggles
  gateway/platforms/yuanbao.py auto-sethome
  tui_gateway/server.py _write_config_key + all cfg→_save_cfg blocks
    (reasoning show/hide/full/clamp, details_mode[.section], prompt)
    → new _load_cfg_raw()
  plugins/memory/holographic save_config

RAW-FILE DIAGNOSTICS + presence-sensitive bridges (class-c: stay raw,
now via the shared primitive with an explanatory comment):
  hermes_cli/doctor.py x5 (model validation, stale-root-keys, .env drift,
    deprecation sweep, memory-provider probe — the latter two keep their
    inline managed overlay where they had one)
  gateway/run.py _bridge_max_turns_from_config and the module-level
    TERMINAL_*/HERMES_* env bridge (bridging merged defaults would export
    all of DEFAULT_CONFIG into the environment; both keep their inline
    overlay + expansion)
  hermes_cli/send_cmd.py env bridge (same presence-sensitivity)
  hermes_cli/gateway.py multiplex-conflict probe (reads the DEFAULT root's
    config, not the active profile's — load_config is the wrong owner)
  hermes_cli/profiles.py / hermes_cli/web_server.py / tools/wake_word.py
    multi-profile reads (load_config targets only the ACTIVE profile home)
  cron/jobs.py _resolve_default_model_snapshot and cron/scheduler.py
    run_job config read keep their existing inline overlay+expansion but
    now share the primitive (their fail-open + last-value semantics and
    the deliberate no-defaults merge are preserved exactly).

Failure-semantics audit: every migrated site preserves its exact previous
behavior on missing file ({} / early return) and parse failure (raise into
the caller's existing except, warn, last-known-good, or fail-open) —
read_user_config_raw intentionally mirrors bare open()+safe_load semantics
(raises on parse errors, {} only on FileNotFoundError/non-dict root).

Guard: tests/hermes_cli/test_config_read_guard.py scans the tree for
yaml.safe_load within 6 lines of a 'config.yaml' reference outside an
explicit ALLOWLIST (hermes_cli/config.py, gateway/config.py, gateway/run.py
fallback path, hermes_cli/managed_scope.py which reads the MANAGED file,
gateway/readiness.py parse-health probe) and fails on new offenders.

E2E: tests/hermes_cli/test_config_loader_e2e.py runs a subprocess with a
temp HERMES_HOME (config.yaml containing ${E2E_PROMPT_SUFFIX}) plus a
HERMES_MANAGED_DIR overlay pinning agent.reasoning_effort, asserting
tui _load_cfg resolves "hello world"/"high" while _load_cfg_raw +
_save_cfg round-trip the template and user value verbatim with no
managed/default leakage.
2026-07-29 10:53:29 -07:00
brooklyn! c8f911112c
Merge pull request #74033 from NousResearch/jb/session-detail-profile-stamp
fix(sessions): always stamp owning profile so cross-profile open works toward default
2026-07-29 12:39:09 -05:00
brooklyn! abd9edbea3
Merge pull request #74234 from NousResearch/bb/session-pins-server-owned
Pins are server-owned, so they survive paging and follow you between apps
2026-07-29 12:31:38 -05:00
teknium1 9179fb72ea refactor: extract web_server Pydantic models to web_models.py (pure schema move) 2026-07-29 10:15:07 -07:00
teknium1 3d48f893da refactor: single build_subprocess_env() factory for all child-process spawns (profile + secret-scrub single owner) 2026-07-29 10:14:11 -07:00
Brooklyn Nicholson 3290807e60 feat(api): session lists carry the pin flag and never drop a pinned row
The three list endpoints (`/api/sessions`, `/api/profiles/sessions`, and
the batched sidebar route) now request the pinned back-fill and expose
`pinned` as a real JSON boolean alongside `archived`.

Both merge paths re-window rows after sorting, which would have thrown
away exactly what the back-fill fetched, so pinned rows survive the cap.
The sidebar's "load more" signal discounts them — they arrive past the
LIMIT by design and would otherwise fake a full page on a short list.

Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com>
Co-authored-by: Kyzcreig <9063726+Kyzcreig@users.noreply.github.com>
Co-authored-by: konsisumer <11262660+konsisumer@users.noreply.github.com>
Co-authored-by: eason2026 <209090628+eason2026@users.noreply.github.com>
2026-07-29 12:00:31 -05:00
yoniebans a8ec50f681 fix(sessions): always stamp owning profile so cross-profile open works toward default
Session rows served without ?profile= carried no profile field, so in
multi-profile desktops the default profile's sessions circulated unowned:
resolveStoredSession cached them profile-less, resolveSessionProfile returned
undefined, and session.resume targeted whichever gateway was active -- opening
a default-profile session from a non-default window failed with
"session can't be found" while the reverse direction worked (#67603 family).

Server: GET /api/sessions/{id} and GET /api/sessions now stamp profile/
is_default_profile unconditionally -- the serving profile is always known
(_cron_default_profile() when the request is unscoped).

Renderer: resolveStoredSession treats a profile-less $sessions cache hit as
unresolved when >1 profile exists (falls through to the stamped by-id ladder)
and back-fills the active profile on bare by-id hits from older backends, so
unowned rows are never re-cached.

Verified via CDP against a live 4-profile renderer: bare by-id GET returned
hasProfileField:false and the stale cache rows matched; with the fix both
lookups return the owning profile and the resume routes to the right backend.
2026-07-29 13:54:12 +02:00
JonthanaHanh a9af49df09 fix(web-server): absorb _warm_gateway_module import before lifespan yield (#73083)
On Windows + Python 3.11 the gateway import triggers heavy .pyc
compilation and Defender real-time scans that do not release the GIL.
Running in run_in_executor still froze the event loop for 15-22 s,
causing the Desktop's 10-second WebSocket ready-probe to time out.

Move the call from the executor to a synchronous invocation before the
lifespan yield, so the GIL block is absorbed during backend
initialisation — before the server socket accepts probes.

Fixes #73083
2026-07-29 13:59:11 +05:30