* fix(desktop): evict settled session states nothing on screen references
Closing a tile never removed its runtime's entry from $sessionStates, so
every tile ever closed parked its full transcript in the map for the life
of the process. Each leftover entry taxes every subsequent stream flush —
the map is spread-copied per delta and the busy/attention/draft projections
walk every entry per publish — so the app got slower the longer it ran,
which users read as "I need to clean my sessions/dbs".
Publish now evicts a settling state when no tile and not the primary view
holds its runtime (transition side effects still fire, so the settle keeps
its unread dot), and closing a tile drops an already-settled state on the
spot. Busy and needs-input states stay: background turns feed the sidebar
dots, and a first publish always lands because a resume can publish a beat
before the surface binds the runtime.
16 tiles streaming in a 2x2 grid with a day's worth of closed-tile residue:
worst-second 34 -> 58 fps, p99 frame 90 -> 28 ms, longtasks 37 -> 0.
* perf(desktop): index lineage aliases per sessions-list reference
lineageAliases scanned the whole recents list per call, and it is called
per cached session state per status projection per message delta — with a
populated sessions DB and a few busy sessions that multiplied out to
millions of row checks a second during streaming. Build the alias index
once per list reference (the list is replaced wholesale, never mutated)
and look aliases up in O(1).
* perf(desktop): journal each in-flight turn under its own storage key
The v1 journal kept every session's tail in one localStorage key, so each
throttled write re-parsed and re-stringified EVERY busy session's snapshot
— a grid of concurrent streams turned that into a whole-store JSON round
trip dozens of times a second, all on the main thread. Per-session keys
make a write O(own tail) no matter how many other sessions are streaming.
A v1 store migrates on first touch; expired/overflow crash residue is
pruned once per renderer.
* perf(desktop): stress the multitab scenario across grid/streaming/DB axes
The one-stack multitab run hid every cost this round of fixes removed: it
drove hook.publish (store only — no journal, no wiring cache), with an
empty recents list and no closed-tile residue. Streaming now routes through
hook.update (the real gateway write path), and the scenario grows axes for
the workloads users actually hit: --zones splits tiles across visible grid
zones, --streaming caps how many sessions are mid-turn (zone leaders
first), --sessions seeds a lived-in recents list, --dead models settled
sessions no surface references. launch.mjs pins HERMES_DESKTOP_CDP_PORT so
a non-default --port survives the app's own dev-CDP flag.
Profiling the boot answered "is there a real cold-start win?": no wasteful
hotspot — the renderer does only ~tens of ms of work at mount, no heavy library
(shiki/mermaid/katex/d3/motion) initializes at startup; the rest is Electron
runtime + waiting, near the Electron floor.
It also exposed that the cold-start number was pessimistic: a fresh
--user-data-dir per run means a COLD V8 code cache and worst-case bundle
recompile every launch. Real users reuse their profile. Measured delta:
fresh (cold cache): spawn→interactive ~1.48s
reused (warm cache): ~1.0s
So representative launch is ~1.0s; only first-launch-after-install pays ~+400ms.
- coldStartSamples() reuses one profile (run 0 warms the cache, discarded;
runs 1..N are warm samples), stepping ports + pausing so the single-instance
lock releases. `--cold-fresh` measures the first-launch worst case.
- Re-baselined cold-start with the representative warm numbers.
Net: nothing high-ROI left to optimize. The only lever is shipping a pre-warmed
V8 code cache to make first launch match warm (~400ms, once per update) — real
packaging complexity for a marginal win, deliberately not pursued.
* bench(desktop): measure the full picture — prod build, cold-start, first-token
Stop drip-feeding scenarios: extend the harness to cover the latencies that
actually dominate perceived speed, and measure them on a REAL production build.
- --prod: build a production renderer with the probe included (VITE_PERF_PROBE=1,
off in normal builds) and launch it from dist/. Measures minified React, so
numbers are representative shipped figures instead of ~3x-inflated dev ones.
- cold-start scenario (tier "cold"): launch → CDP → driver → first paint, via a
fresh isolated spawn per run. Captures spawn_to_cdp_ms, spawn_to_driver_ms, fcp_ms.
- first-token scenario (backend tier): Enter → first assistant token painted —
the TTFT latency an agent app is uniquely judged on.
- run.mjs gained --prod (build once), cold-start fresh-spawn loop, and gates
ci+cold tiers against the baseline.
Baseline re-captured on a PRODUCTION build (median of 5), darwin-arm64 — all
green. Representative numbers:
cold-start spawn→interactive ~1.6s, FCP ~0.5s
stream frame p95 22ms, 1 longtask
keystroke p50 2ms, p95 8.7ms
transcript mount 145ms, 82ms longtask (400-msg open)
The prod build also settled the open question from the dev numbers: the
transcript-mount "lead" (221ms longtask in dev) is only ~72-82ms in prod — not
actionable. Measurement did its job.
* bench(desktop): trustworthy cold-start measurement (code-splitting is NOT the lever)
Investigated code-splitting the ~22MB renderer bundle to cut cold start. It is
the wrong fix on both counts:
1. Intentional design: vite.config disables codeSplitting because Shiki emits
thousands of dynamic chunks and electron-builder OOMs scanning them — a
packaging/installer constraint, not an oversight.
2. The data says it wouldn't help. Fixing the cold-start measurement to be
trustworthy and reading the boot composition (prod build):
spawn → interactive ~1.5s
renderer nav → DOMInteractive ~0.8s, → DOMContentLoaded ~1.06s
so the whole 22MB bundle EVAL is only ~0.27s (DCL − DOMInteractive) of the
~1.5s. The dominant costs are Electron/window startup and React app mount —
neither touched by splitting.
The measurement fixes (the real content of this PR — no app change, since the
optimization was rejected):
- Drop HERMES_DESKTOP_BOOT_FAKE from spawned instances — it injected artificial
per-phase boot-overlay sleeps that inflated cold-start (and slowed every run).
- Unique debug/dev port per cold-start run — a just-killed instance can hold
:9222 briefly, so reusing it made CDP attach to the DYING instance and report
garbage (spawn_to_cdp of ~4ms). Stepping the port per run fixes the race.
- Richer boot marks (dom_interactive, dom_content_loaded, main-script size) so
cold-start composition is visible, not just a single number.
- Forward all numeric boot marks from the cold-start loop.
- Re-baseline cold-start with the clean numbers.
A real cold-start win would target Electron startup / app-mount (e.g. V8 code
cache, deferred non-critical mount) — a future pass, now that it's measurable.
Stop drip-feeding scenarios: extend the harness to cover the latencies that
actually dominate perceived speed, and measure them on a REAL production build.
- --prod: build a production renderer with the probe included (VITE_PERF_PROBE=1,
off in normal builds) and launch it from dist/. Measures minified React, so
numbers are representative shipped figures instead of ~3x-inflated dev ones.
- cold-start scenario (tier "cold"): launch → CDP → driver → first paint, via a
fresh isolated spawn per run. Captures spawn_to_cdp_ms, spawn_to_driver_ms, fcp_ms.
- first-token scenario (backend tier): Enter → first assistant token painted —
the TTFT latency an agent app is uniquely judged on.
- run.mjs gained --prod (build once), cold-start fresh-spawn loop, and gates
ci+cold tiers against the baseline.
Baseline re-captured on a PRODUCTION build (median of 5), darwin-arm64 — all
green. Representative numbers:
cold-start spawn→interactive ~1.6s, FCP ~0.5s
stream frame p95 22ms, 1 longtask
keystroke p50 2ms, p95 8.7ms
transcript mount 145ms, 82ms longtask (400-msg open)
The prod build also settled the open question from the dev numbers: the
transcript-mount "lead" (221ms longtask in dev) is only ~72-82ms in prod — not
actionable. Measurement did its job.
Chased the "stream frame p95 = 60ms with ZERO longtasks" mystery to its actual
cause: the default stream chunk had no paragraph breaks, so it grew into one
giant ~22KB block that re-rendered fully every flush — defeating the block
memoization real streaming relies on. Plain text = 21ms; realistic chunk with
`\n\n` breaks (blocks settle, only the tail re-renders) = 23ms. Fixed the
default chunk to model real LLM output; a break-less `--chunk` remains available
as a single-block worst-case stress.
Also hardened the isolated instance so measurements reflect real cost:
- Wait for the gateway socket to actually connect before measuring (a booting/
absent backend's reconnect backoff churns the main thread). Exposed via a new
__PERF_DRIVE__.connected() probe reading $gateway.connectionState.
- Focus emulation + anti-throttle/occlusion flags so a backgrounded perf window
isn't frame-throttled (no OS focus stealing).
- Generation-guarded the rAF frame recorder so repeated runs don't leave
overlapping recorders polluting frame intervals.
Baseline re-captured as the median of 5 --spawn runs (darwin-arm64); all three
CI scenarios now green and stable. Absolute values are dev-build (noted in
_meta) — regression guards, not shipped numbers.
- Resolve the vite CLI via vite/package.json `bin` (Vite 8's exports block
importing vite/bin/vite.js directly — --spawn failed with ERR_PACKAGE_PATH_NOT_EXPORTED).
- Add a post-launch settle so cold-start contention (vite dep pre-bundling,
first backend-connect attempts) doesn't contaminate the first scenario.
- Drop the raw autolink from the default stream chunk (resolvable URLs trigger
link-embed DNS lookups unrelated to render cost).
- Replace seed baseline with real numbers from a darwin-arm64 --spawn run.
keystroke + transcript are clean; stream is a clean single-run capture (the
isolated backend may not connect, and its reconnect churn inflates frame
pacing — re-capture on a connected instance for tighter tolerances).
Replaces the dozen ad-hoc measure-*/profile-* scripts (each reinventing the
CDP client — 4 different copies — plus its own arg parsing, stats, output
path, and none with a baseline) with one framework under scripts/perf/:
- lib/cdp.mjs one CDP client + target discovery + typing + CPU-profile wrapper + DOM selectors
- lib/stats.mjs percentiles, histograms, CPU-profile self-time ranking
- lib/baseline.mjs load/compare/update baseline + regression gate (new capability)
- lib/launch.mjs attach, OR spawn a fully ISOLATED instance
- scenarios/* one module per measurement, registered in scenarios/index.mjs
- run.mjs / serve.mjs, baseline.json, README.md
Isolation solves the long-standing measurement blocker: a running `hgui` held
the Electron single-instance lock, so a second instance quit. `--spawn` /
`perf:serve` launch with their own --user-data-dir (separate lock scope), their
own HERMES_HOME (separate backend/sessions, config seeded from ~/.hermes so it
reaches a chat view without onboarding), and their own --remote-debugging-port.
Synthetic scenarios drive $messages via window.__PERF_DRIVE__, so no LLM credits.
Scenario -> sunset script mapping:
stream <- measure-synthetic-stream, profile-synth-stream, profile-long-stream
stream --real <- measure-real-stream, profile-real-stream
keystroke <- measure-latency, profile-typing, leak-typing
transcript <- (new: long-transcript mount cost)
submit <- measure-submit, measure-jump
session-switch <- profile-session-switch
profile-switch <- measure-profile-switch
CPU profiling is now a cross-cutting --cpuprofile flag, not 5 separate scripts.
CI-tier scenarios (stream, keystroke, transcript) need no backend/credits and
are gated against baseline.json (seed values; re-capture with --update-baseline
on a reference device). Backend-tier scenarios are report-only.
perf-probe.tsx gains loadTranscript() for the transcript scenario. No core
files touched; isolation is via CLI args, not env-gated app changes.
Verified: node --check all modules, tsc, eslint, and a unit smoke of the
stats + regression-gate logic. The end-to-end GUI run (which opens a window)
is left to run interactively via `npm run perf -- --spawn`.
brooklyn!