RomanNum3ral
/
gstack
mirror of https://github.com/garrytan/gstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541) 

* test(helpers): shared skill-census helper with three explicit counts

physicalSkillFiles (symlinked dirs included, root router included),
authoredSkills (realpath-deduped, router excluded), registryEntries
(what ./setup registers: unique frontmatter names + _gstack-command).

One counting authority for the hermetic seeder, context-bill ground
truth, and the catalog-budget test — connect-chrome's dir symlink and
the root router otherwise produce three subtly different hand-rolled
censuses. Ported-wave foundation (C11).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(evals): stop the harness grading itself

findPreviousRun excluded only the file being written, by name, so every
suite compared against _partial-e2e.json — the current run's own
accumulator, relabelled with the current tier just before each flush.
That is why every block read '+$0.00, +0s, Stable run, no regressions.'
This harness has never been able to detect a regression, and reassuring
output that cannot fail is worse than none. In-progress runs are now
excluded by role, and a run with nothing to compare against says NO
BASELINE instead of claiming stability.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit f3140b5245221fff7fb9411c7ec07c2ca11587b5)

* refactor(evals): shared partial-run predicate + finalized-run lookup

isPartialEval(data, filename) is the one place that decides what counts
as an in-progress accumulator (the _partial flag OR a _partial-prefixed
filename), and findLatestFinalizedRun(evalDir, tier) is the one place
that finds the newest real run — scanning the eval dir plus one level of
shards/<slug>/ subdirs, where the sharded paid runner points each
shard's collector. skill-budget-regression.test.ts's hand-rolled
findLatestRun (flag-blind: a flagged-but-renamed accumulator passed its
name check) is replaced by the shared helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit b55fcf6966366fd21a8cdc46de61aab6e1b1d100)

* feat(evals): register shipped skills for hermetic PTY children

Hermetic children get a config dir that deliberately seeds no skills —
right for children that install their own, fatal for the PTY family that
TYPES /office-hours or /plan-ceo-review: claude rejects the command as
Unknown before any model turn, so the plan-family gate smokes measure
nothing. hermeticSkillsConfigDir() is a second, opt-in config dir under
the same runRoot that mirrors ./setup's registration exactly (real dir
per registry name, SKILL.md + sections/ symlinks, frontmatter-name
resolution, _gstack-command root alias), driven by the shared
skill-census so connect-chrome's dir symlink collapses the same way
setup's idempotent overwrite does.

Ported from fork commit 03c4eca2, tree walk rewritten for the upstream
layout (top-level <skill>/SKILL.md dirs, no skills/ tree). Unit tests
are new: seed shape, census parity, symlink resolution, connect-chrome
collapse, idempotence, no-API-key seed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 93dae6107b30ce453a07c2d342b60262bba6ce0b)

* feat(evals): seedSkills opt-in for PTY slash-command tests + tripwire

Wire ClaudePtyOptions.seedSkills through launchClaudePty: when set (and
hermetic, and no per-test CLAUDE_CONFIG_DIR override), the child gets
hermeticSkillsConfigDir() so typed /skill slash commands resolve instead
of dying as Unknown command before any model turn. Opted in at the three
runPlanSkill* helpers and the four direct-launch slash-command tests
(plan-design-with-ui, plan-ceo-mode-routing, autoplan-chain,
ship-idempotency).

New static tripwire (test/pty-skill-seeding-wiring.test.ts): any test
file that sends a slash command over the PTY must route through a
runPlanSkill* helper or pass seedSkills: true — an unseeded slash-command
test spends money and measures nothing. hermetic-wiring.test.ts now
blesses the repo-tree seeding path explicitly (config dir under runRoot,
symlinks into the repo checkout, never operator ~/.claude).

The CI "Register gstack skills for PTY smoke" step keeps a keep-me note:
container cross-mount symlinks defeat the TUI scanner and HOME is not
hermeticized, so the real-file copies there must survive this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 63c52269daaffb833b3105ea9b4b99be6df8fec7)

* refactor(evals): single shared paid-test-set module

test/helpers/paid-test-set.ts is now the one definition of which test
files are paid (the exact globs package.json's test:gate expands).
scripts/test-free-shards.ts derives its free/paid exclusion from it
instead of a private regex list, dropping the dead
browse/test/security-review-fullstack.test.ts pattern (file no longer
exists). The sharded paid runner derives its enumeration from the same
module, so a file added to one list can no longer silently miss the
other.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit a7f36479a6a1f3656452370f5883371f3cb65623)

* feat(evals): env-driven lazy eval dir + shard-aware store and tooling

Importing eval-store no longer spawns the gstack-slug subprocess: the
module-level DEFAULT_EVAL_DIR constant is now a memoized defaultEvalDir()
resolved at collector construction. Resolution order: explicit
constructor arg, then GSTACK_EVAL_DIR, then slug detection — so the
sharded paid runner can point each shard child at its own
<evalDir>/shards/<slug>/ dir with plain env, no --preload.

Runs collected under a shards/ subdir record their slug in the eval
JSON (EvalResult.shard). findPreviousRun scans one shards/<slug>/ level
and prefers same-slug priors, so each shard baselines against its own
history instead of whichever shard flushed last. eval:list,
eval:summary, and eval:compare enumerate the same one level of shard
subdirs; eval:compare's no-arg mode also stops picking an in-progress
accumulator as the after-run.

eval-watch stays flat (documented follow-up): it tails a single dir for
live progress and gains nothing from per-shard baselines until the
runner emits a merged stream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit e1f53f7d9c7fe6b65877d843f2e25bd2e2d12ffd)

* feat(evals): sharded paid tier runner

scripts/test-paid-shards.ts runs the gate/periodic tier one Bun process
per test file, with an EXTERNAL wall-clock timeout that SIGKILLs the
shard's detached process group and an aggregate that distinguishes
passed / failed / timed-out / never-started — partial execution can no
longer read as a pass. Bun's native --shard/--isolate covers none of
this: no process-group kill (hung claude/codex PTY grandchildren
survive in-process isolation), no never-started taxonomy, no per-shard
env. Each shard child gets GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/
(slug = test filename sans extension, stable across runs) so shard
baselines compare against their own prior runs.

Output classification lives in scripts/test-strict-output.ts (strict
exit-code derivation, incremental fail-line classifier, child signal
forwarding) so the runner and any future strict bun-test wrapper share
one implementation. Enumeration derives from the shared paid-test-set
module; tier exclusion fires only on an explicit whole-file
EVALS_TIER === '<other>' guard.

package.json gains test:gate:sharded / test:periodic:sharded, and
eval:bg:gate / eval:bg:periodic now run the sharded scripts with detach
timeouts sized to the worst case (gate: 49 shards x 30min / 4 jobs ~
6.2h -> 25200s; periodic: 59 -> 28800s).

test/paid-shards.test.ts pins enumeration, tier classification, and the
kill-and-continue property with a real busy-loop shard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 5e76bd5931836257f896cedfe4e93912cb759c70)

* feat(security): hash-chained egress receipt ledger (core)

Port lib/egress-receipt from the v2 fork as TypeScript: writeReceipt
(sync, fail-closed via typed EGRESS_RECEIPT_FAILED), best-effort
writeOutcome, readLedger/listReceipts/verifyLedger, GSTACK_HOME ->
GSTACK_STATE_DIR -> ~/.gstack resolution, 0600 ledger under a 0700
security dir, and an mkdir spin lock (2.5s budget) with documented
>10s-mtime stale-lock reclaim.

Changes vs the fork:
- lastRawLine tail-reads the final 4KB instead of loading the whole
  ledger, so appends stay O(1) as the file grows.
- WARN-at-size: past 25MB writeReceipt emits one self-explanatory
  stderr warning per process (what the ledger is, how to inspect it,
  rotation TODO); verifyLedger gains a sizeWarning field. Rotation
  TODO carries the chain-genesis sketch (new generation's first record
  embeds the prior file's tail hash).

bin/gstack-egress-receipt is a bun script bridging shell callers:
write|outcome subcommands, exit 3 + EGRESS_RECEIPT_FAILED on stderr on
failure; --no-payload records sha256:null for git-class ops.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 619726a3d77d987a2e50151a5727b3faaaf5fc6a)

* chore(bin): delete dead brain-consumer/reader scripts

bin/gstack-brain-consumer and bin/gstack-brain-reader are byte-identical
dead scripts that POST the repo URL + a Bearer token to a /ingest-repo
endpoint gbrain removed (docs/gbrain-sync.md already documents the
removal in past tense). No live references remain; CHANGELOG mentions
are historical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 254ddc69fc5a0270fcc973e36b6a81766d835d2d)

* feat(security): shared shell receipt helpers

bin/gstack-egress-lib.sh (sourced library, gstack-gbrain-lib.sh
precedent) provides _receipted_curl and _receipted_git: write the
egress receipt BEFORE the send via gstack-egress-receipt, hand curl the
SAME payload file via --data-binary @file so the receipt hash matches
the wire bytes exactly, then append a best-effort outcome. Per-call
fail policy: 'closed' refuses the send (return 3, problem/cause/fix
message on stderr) and 'open' warns and proceeds. Payload temp files
are consumed immediately per call — no EXIT traps, since callers like
gstack-telemetry-sync own their own EXIT trap and a sourced trap would
clobber it.

Tested end-to-end against a local Bun.serve listener: receipt sha256
equals the sha256 of the bytes the listener received, fail-closed
refusal never touches the network and carries the problem/cause/fix
stderr shape, fail-open warns and proceeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 6d067dce2d4c8815dec98be551763c85a3671357)

* feat(security): receipt core shell sinks

Wire the three core bash egress sinks through gstack-egress-lib.sh:

- gstack-telemetry-sync: the batch POST now writes the payload to a
  temp file, receipts those exact bytes fail-closed, and hands curl the
  SAME file. On refusal nothing is sent and the cursor does not
  advance, so the batch stays buffered for the next run. The HTTP
  status is recorded as the receipt outcome.
- gstack-update-check: fail-open receipts (warn + proceed) on the
  Supabase ping POST, both VERSION curls (via a local
  _receipted_version_fetch helper that skips non-network schemes), and
  git ls-remote. The ping receipt is written inside the backgrounded
  subshell, so it can never block the script's exit.
- gstack-brain-sync: fail-closed git-class receipts. The push receipt
  is written BEFORE the commit consumes the queue, so a refused receipt
  leaves the queue intact and the next run retries the whole drain
  (pinned by a new queue-intact-on-refusal test, including the
  problem/cause/fix refusal message shape). The retry-path fetch and
  retry push carry their own fail-closed receipts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 3c60f699acceaf1c92a218874711e05fc17dca5d)

* feat(security): receipt TS module sinks + tunnel

writeReceipt (fail-closed, sha256:null — a subprocess or SDK owns the
wire bytes) before every TS-module network-bearing operation:

- bin/gstack-gbrain-sync.ts: before the gbrain code walk that ships
  repo content to the user's gbrain DB (may be remote Postgres). A
  refused receipt fails the stage with status refused-egress-receipt.
- bin/gstack-memory-ingest.ts: before the gbrain batch import of
  transcript pages. A refused receipt returns a system_error verdict
  without spawning the import.
- browse/src/server.ts: before both ngrok.forward call sites (start-up
  BROWSE_TUNNEL=1 path and the /tunnel/start endpoint). A receipt
  failure lands in the existing catch that tears the tunnel listener
  back down and refuses the start.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 5677d618a48fcd0ae2b068bf868781d90f809cb5)

* feat(design): receipted fetch for OpenAI calls

design/src/receipted-fetch.ts wraps every api.openai.com call: a
content-free egress receipt (sink design-openai, sha256 of the JSON
body — hash only, never the body) is written BEFORE the send. Polarity
is FAIL-OPEN: user-facing generation must not die because an audit log
hiccuped, so a receipt failure warns on stderr and the call proceeds.
Streams pass through untouched (response bodies returned as-is;
non-string request bodies receipted as sha256:null rather than drained
to hash).

All ten call sites converted with per-command payload classes:
generate, variants (injected fetchFn passes through), iterate (both
threaded and fresh paths), evolve (image + screenshot analysis), check,
diff, design-to-code, memory.

Unit-tested with injected fetch: receipt-before-send ordering, stream
passthrough, and fail-open on an unwritable ledger.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit c0e5ff6639414ac2fd98e8ac3affb51401746b55)

* feat(security): receipt admin scripts + user git-ops (zero exceptions)

Wire the remaining shell egress through gstack-egress-lib.sh:

- gstack-gbrain-mcp-verify: both JSON-RPC probe POSTs (initialize +
  tools/list) receipted fail-closed via payload files (hash == wire
  bytes). A refused receipt lands in the NETWORK class — no send.
- gstack-security-dashboard / gstack-community-dashboard: the
  community-pulse GETs receipted fail-open (read-only stats must not
  break over an audit hiccup).
- gstack-gbrain-supabase-provision: api_call receipted fail-closed.
  Each retry attempt hands the helper a fresh copy of the body file
  (the helper consumes its payload). The receipt hashes the request
  body only — the PAT never reaches the ledger or any log. Refusal
  exits 8 without retrying.
- git-class sha256:null receipts, fail-open: gstack-artifacts-init
  (ls-remote, initial push, fetch/pull recovery, retry push),
  gstack-brain-restore (staging clone, existing-repo fetch),
  gstack-session-update (self-update pull).

gstack-team-init needs no wiring: every git clone in it is inside an
echoed instruction string, not an executed command.

The lib now self-locates with shell builtins only (no dirname), so
sourcing works under the whitelist-PATH test harnesses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit b8c5e2055b21ab72878b3e46f8047782ee65a11c)

* test(security): egress wiring tripwire + polarity contract

Static-grep tripwire pinning the egress-receipt wiring (threat model in
the header: the ledger is forensic observability of ATTEMPTED egress,
not an exfiltration control):

- Per-sink assertions: every wired TS module imports egress-receipt and
  calls writeReceipt; every wired shell sink sources
  gstack-egress-lib.sh with each network op under a receipt;
  ngrok-proximity check for server.ts; every design api.openai.com call
  routes through receiptedFetch.
- Absence assertions: the dead brain-consumer/reader scripts stay
  deleted (lstat, so a dangling symlink also fails).
- Polarity table pinned as data (fail-closed: brain-sync,
  memory-ingest, gbrain-sync, telemetry-sync, ngrok, mcp-verify,
  supabase-provision; fail-open: design-openai, update-check,
  dashboards, git-class user ops, context-bill --exact) plus per-file
  polarity spot-checks.
- NEW-SINK SCANNER with zero KNOWN_UNWIRED: sweeps bin/, lib/,
  scripts/, design/src, browse/src for curl, absolute-URL fetch(, and
  git remote ops (never local rev-parse/get-url; heredoc bodies and
  message strings excluded) and requires every hit to be receipted or
  in a REASONED exemption list where each entry carries its why.
  Preamble-generated skill prose documented out-of-scope in the header.
- Shebang tripwire: no bin/gstack-* file may carry a node shebang.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit ff69ceeafaf9c017d539b6ad77ff8f95b680b979)

* feat(cli): gstack-egress reader

bin/gstack-egress (bun) — the auditor's view of the receipts ledger:

- list: one row per receipt (what gstack ATTEMPTED to send), with
  --since/--host/--sink filters and --json.
- verify: recompute the hash chain; exit 3 on tamper naming the first
  broken line; prints the sizeWarning when the ledger passes 25MB.
- grants: what CAN leave, built on the upstream config keys only
  (telemetry, artifacts_sync_mode, redact_repo_visibility,
  redact_prepush_hook via gstack-config get) — each grant names its
  file, key, and the exact revoke command.

CLI smoke tests spawn the real bin against a temp GSTACK_HOME,
including a broken-chain fixture asserting exit 3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9e24eca0f1069fea2ea69e7df4e9b256e93d59a3)

* feat(cli): context-bill — token bill-of-materials (stripped port)

lib/context-bill.ts, ported from the v2 fork and STRIPPED to the tiers
this repo's skills can exercise: ALWAYS-ON (per-skill frontmatter bytes
with dead-key and foreign-host-file flags), EAGER (SKILL.md + any
forced 'for every invocation' references), on-disk totals, --diff,
--budget, and --exact with the calibration table. The fork's
CONDITIONAL/TRANSITIVE/LAZY/FAST-PATH parsers understand only its
dispatcher layout and were dropped; the tier fields stay in the report
shape (empty/zero/null) so re-adding a parser is additive.
TOKEN_DIVISORS and their provenance docblock kept; --help notes
recalibration via --exact's calibration block.

Three upstream fixes over the fork:
(a) findSkillDirs treats the walk ROOT as a container — the repo root's
    router SKILL.md is billed AND its children are walked (the fork
    short-circuited and billed one skill); walkMd skips node_modules
    and dot-directories.
(b) installed-tree layout: subdirs that are their own repo checkout
    (a gstack/ clone inside ~/.claude/skills, detected by .git) are
    skipped, and directory symlinks (connect-chrome) are followed with
    a container-recursion cycle guard.
(c) ROUTER_KEYS widened to the upstream frontmatter contract {name,
    description, version, allowed-tools, triggers, preamble-tier}.

--exact writes an egress receipt (sink context-bill-exact, host
api.anthropic.com) BEFORE any count_tokens POST; if the receipt cannot
be written the run degrades to the offline estimate with a warning —
nothing is sent unrecorded. bin/gstack-context-bill is the bun shim.

Tests: fixture-tree ledgers, the three fixes, --diff/--budget exit
codes, --exact with injected fetch (envelope subtraction, receipt
ordering, fail-open degradation), CLI smoke test, and ground truth
against THIS repo via test/helpers/skill-census.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 675c19876b87ec927b555f5f64c7f93130b3de90)

* test(catalog): aggregate discovery-surface budget with ratchet protocol

Every host loads every skill's frontmatter name + description at
discovery, every session. applyCatalogTrim in scripts/gen-skill-docs.ts
shapes each description and the 160KB per-file warn covers body size,
but nothing capped the aggregate frontmatter — the catalog could grow
one reasonable-looking description at a time. This test is that
enforcement layer.

Measures the catalog via test/helpers/skill-census.ts authoredSkills
(symlink-deduped, root router counted separately as the _gstack-command
alias line item): 53 skills + router = 4,420 bytes = 1,105
token-equivalents today, asserted <= 1,150 (~4% headroom). Per-skill
sub-cap of 260 bytes (largest today: design-consultation at 229), plus
a non-empty-description check.

Failure messages are self-service ratchets: they print the new total,
the delta, and the update protocol (bump the constant AND the
derivation comment in the same commit; trim instead of grow for
existing descriptions). Parser handles folded block scalars
(description: >-) for fork parity; import-free by design so it
survives generator refactors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit c106fb36f768181b80c257e5cff1cde4f435f9c0)

* fix(browse): extension token bootstrap moves to pinned-origin POST; /health carries no token

GET /health is now liveness/status only in every mode — both token
carve-outs (headed-mode disjunct AND chrome-extension:// Origin
disjunct) are removed. Token bootstrap is POST /extension-token on the
local listener: the Origin header must be exactly
chrome-extension://<GSTACK_EXTENSION_ID> and the Host header's hostname
must parse to 127.0.0.1 or localhost (parsed via new URL, never literal
equality — Host arrives as '127.0.0.1:34567'). Wrong origin/host → 403
with no detail. The tunnel surface 404s the endpoint (not in
TUNNEL_PATHS, verified by test).

The extension ID is pinned by a new "key" field (RSA public key) in
extension/manifest.json; browse/scripts/extension-id.ts reproduces the
ID derivation (first 16 bytes of SHA-256 of the DER public key, hex
mapped 0-9a-f → a-p). The private key is not committed anywhere —
unpacked/baked-in loads only need the public key.

Extension side: background.js bootstraps and refreshes the token via
POST /extension-token (403 → disconnected state); sidepanel.js direct
connect path does the same; sidepanel-terminal.js's dead /health token
fallback (read AUTH_TOKEN/authToken keys the server never sent,
hardcoded port) is replaced with the window.gstackAuthToken path.

MIGRATION NOTE: the manifest key pins the extension ID, so existing
installs' side-panel local state (saved port, snoozes) resets once —
explained in-product via a one-time notice (flag
gstack_id_migrated_v162). After upgrading the server, restart the
browser so the old service worker stops polling for a token GET /health
no longer serves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit e9a0b6847a2d17fe6656a4686b4efd0c8380eb09)

* docs: correct stale compiled-binaries claim; file three egress/eval follow-ups

CLAUDE.md's compiled-binaries section claimed browse/dist binaries are
tracked by git and appear as modified in git status — false since
64d5a3e4 (v0.11.16.0) untracked them, and actively harmful: it trained
agents to ignore dist binaries in git status. The section now states
the truth (untracked + gitignored; a dist binary in git status means
someone force-added it) and covers make-pdf/dist too.

TODOS.md gains the three follow-ups filed by the v1.62 port-wave
reviews: ledger rotation with chain-genesis records, launch-nonce
token bootstrap, and eval-watch shard-awareness.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: pre-landing review fixes for the v2 port wave

Review army (checklist + 5 specialists) + coverage/plan audits on the
assembled branch. Genuine correctness/security/hygiene fixes:

- test-paid-shards: strictTestExitCode now receives expectedFiles on the
  real bun path, so a shard that runs fewer files than planned (harness
  crash, nothing loaded) with exit 0 is no longer recorded 'passed' — the
  invisible-non-execution class the runner exists to kill. Pinned by the
  new test/strict-output.test.ts (also covers the chunk-boundary classifier).
- test-paid-shards: EVALS_TIER env is validated (gate|periodic) like the
  --tier flag, so a typo can't self-skip every test and exit 0 green.
- package.json: test:periodic:sharded sets EVALS_ALL=1, restoring the
  full-tier semantics the pre-shard script had (CI already set it; local
  eval:bg:periodic silently under-measured without it).
- brain-sync.test: run() pins HOME to the temp home so gstack-artifacts-init
  stops writing/clobbering the operator's real ~/.gstack-artifacts-remote.txt
  every free-suite run; afterEach now also scrubs the current filename.
- egress-receipt: cap each receipt field at 512B so a serialized line always
  fits the 4KB tail-read window — a longer line would make the next append
  hash a truncated prior line and verifyLedger report a permanent false
  TAMPER. warnLedgerSize short-circuits before statSync once fired (append
  hot path).
- gstack-egress: import.meta.dir (Windows-safe) instead of new URL().pathname
  so grants doesn't silently report defaults on Windows; strip control chars
  from ledger-derived fields on render so a crafted receipt can't spoof the
  auditor's view.
- extension/background.js + CLAUDE.md: renumber the identity-pin migration
  refs v1.62 -> v1.63 (main claimed 1.62.0.0; this wave queue-advances).
- egress-receipt-wiring: pin lib/context-bill.ts unconditionally (both land
  together now); drop the dead RunShardsOptions.tier field.

All fix-affected test files green; gate failures triaged as external-env
(codex/gemini CLI drift) or pre-existing (hermetic-canary fails identically
on base). Deferred polish tracked in the PR body + decision store.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: bump version and changelog (v1.63.0.0)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: file TODO to harden plan-design-with-ui PTY detection

The v1.63 seedSkills change made this gate test execute for the first
time; it reliably times out because its terminal scraper can't parse the
(correctly-rendered) scope-gate AskUserQuestion out of a spinner-mangled
PTY buffer. Shipped skill behavior is correct — test-harness limitation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: re-slot release as v1.62.1.0 (PATCH per user)

Main claimed 1.62.0.0 while the wave was in flight; the user chose the
PATCH slot over queue-advancing MINOR. Renumbers the identity-pin
migration notice (now version-free flag name so a re-slot never orphans
an already-set flag), the CLAUDE.md /health note, the CHANGELOG heading,
and the TODOS section titles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): stop hard-requiring the literal ok) case label in gbrain-refresh guards

The extractor grepped for 'ok)' but the case label grew to
ok|timeout|thin-client) (#1964, #2051), so the whole file errored on
import — the free suite's only red for months. The extractor now matches
any label starting with ok and its alternations; all 7 guard assertions
run again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): hermetic-canary probes with ${VAR:-} so nounset shells can't fail success

The probe echoed bare $CONDUCTOR_WORKSPACE_PATH — when scrubbing WORKS
the var is unset, and under a nounset shell the echo errors, failing the
canary exactly when isolation succeeds. Defaulted expansions assert
identically under any shell. Fails identically on base; fixed here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(evals): absorb codex/gemini CLI drift; external-service tests go periodic-tier

- codex exec gains --skip-git-repo-check: newer CLIs refuse exec in an
  untrusted non-git dir (our temp skill dirs) — empirically verified.
- gemini: --skip-trust was removed in gemini-cli 0.34 (argv parse error);
  dropped from the session runner and the benchmark adapter. A present-
  but-unusable CLI (deprecated individual code-assist auth path) now
  classifies as SKIP, not a false adapter failure; the benchmark live
  smoke skips on auth/rate_limit error codes (environmental) while still
  failing on timeout/unknown (the drift classes it exists to catch).
- codex-e2e, gemini-e2e, and benchmark-providers gain the canonical
  whole-file EVALS_TIER === 'periodic' guard per CLAUDE.md tiering rule 3
  (external service -> periodic) — the sharded gate runner now excludes
  all three (gate: 45 -> 42 shards).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(evals): parse single-logical-line AskUserQuestions in the PTY runner

When the PTY reflows a boxed AUQ, ALL options land on ONE logical line
after stripAnsi — parseNumberedOptions parsed one option per line, found
only '1.', and the >=2 check failed forever while the correct question
sat on screen (plan-design-with-ui timed out this way twice, with the
rendered scope-gate AUQ visible in both failure buffers). The cursor
line is now parsed as a stream of ascending N. tokens; DEC cursor-
visibility residue is stripped before matching; plan-design-with-ui's
budgets grow to fit observed ~6min preamble+thinking latency. Pinned by
test/pty-auq-single-line.test.ts using the real failure buffers; all 142
existing parser-consumer unit tests still green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: restore v1.63.0.0 (MINOR — user-confirmed final slot)

The wave ships new capability (egress receipts + two CLIs, sharded paid
runner, hermetic skill seeding) at ~8K lines — MINOR scale per the
scale-aware bump rules. Supersedes the brief v1.62.1.0 re-slot; the
version-free migration flag means no state churn from the renumber.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: spell out AskUserQuestion in the PTY single-line fixture

Rename test/pty-auq-single-line.test.ts to
test/pty-askuserquestion-single-line.test.ts and expand the AUQ
abbreviation in identifiers and comments. House style writes
AskUserQuestion in full in filenames, identifiers, and comments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: sync every doc surface with the v1.63 release

/document-release audit (4-lane, all claims verified against branch code):

- README: gstack-egress + gstack-context-bill rows in the standalone-binaries
  table; Privacy & Telemetry gains the receipted-egress bullet (attempted-
  egress framing per the shipped threat model).
- ARCHITECTURE: /health is liveness-only, POST /extension-token endpoint row
  + bootstrap mechanics paragraph; new Egress receipt ledger subsection under
  Security model; eval persistence covers the sharded runner, GSTACK_EVAL_DIR,
  and the finalized-run baseline rule.
- CLAUDE.md: sharded test scripts in Commands; sharded semantics in the
  detached-evals section; PTY skill seeding in the hermetic section; egress
  invariant block beside the other server-egress invariants; catalog-budget
  ceiling beside the 160KB token ceiling; project-tree entries for
  lib/egress-receipt.ts, lib/context-bill.ts, scripts/test-paid-shards.ts.
- CONTRIBUTING: seedSkills + live-tree seeding in the hermetic paragraph;
  sharded runner in detached runs; catalog-budget in the Tier 1 list.
- BROWSER: extension token bootstrap section, tunnel egress receipts section,
  identity-pin migration note in manual install.
- REMOTE_BROWSER_ACCESS: tunnel-start receipt bullet in the security model.
- gbrain docs: /sync-gbrain + brain-sync egress-receipt behavior documented;
  dead consumer-token instructions removed (consumer machinery deleted this
  release); new fail-closed refusal added to the error catalog.
- CHANGELOG: measured-vs-ceiling catalog numbers, contributor notes for the
  external-service tier move and the PTY single-line AskUserQuestion parser,
  release date.
- TODOS: /health token-distribution TODO resolved by this release, removed;
  port-wave follow-up sections re-labeled to the shipped version.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: sweep drift that predates this release

Surfaced by the /document-release audit; every fix verified against the
current binaries:

- gstack-brain-init was replaced by gstack-artifacts-init in v1.27.0.0
  (hard-delete, no compat shim), but README, USING_GBRAIN_WITH_GSTACK,
  docs/gbrain-sync.md, and docs/gbrain-sync-errors.md still instructed
  users to run it — command-not-found on every follow. Same sweep updates
  ~/.gstack-brain-remote.txt to the canonical ~/.gstack-artifacts-remote.txt
  (legacy name still honored on restore, noted where users copy the file).
- gbrain-sync-errors.md headings re-matched to the literal messages the
  binaries print today (the doc's whole value is grep-by-exact-message):
  'gstack-artifacts-init: ~/.gstack/ is already a git repo pointing at:',
  'Remote not reachable via SSH:', 'Failed to create or find ...'. The
  already-a-repo fix now leads with the command's own set-url suggestion.
- docs/gbrain-sync.md 'Under the hood' linked a plan file that does not
  exist in the repo; replaced with the decisions themselves.
- SIDEBAR_MESSAGE_FLOW startup timeline: /pty-session responds with
  {terminalPort, sessionId, attachToken, leaseExpiresAt} (v1.44 shape,
  verified at browse/src/server.ts:1860), not the retired
  {terminalPort, ptySessionToken} pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: fold the Codex accuracy review of the release docs

Six findings, all verified against source before fixing:

1. 'Every send writes a receipt' overclaimed — fail-open sinks proceed with
   a stderr warning when the receipt write fails, so a fail-open send can go
   unrecorded (lib/egress-receipt.ts:8-14). Descriptive prose now says so;
   the receipted framing keeps 'attempted'.
2. 'Receipts hash the request body' is wrong for subprocess-owned sends —
   git pushes record sha256: null (lib/egress-receipt.ts:71).
3. 'grants shows every consent in force' overclaimed — it reports the four
   standing config settings (bin/gstack-egress:139-181). Reworded in
   README, ARCHITECTURE, and the CHANGELOG entry.
4. 'Zero-exception scanner' vs reality: the new-sink scanner carries a
   reasoned SCANNER_EXEMPT list (user-directed fetches, probes, instruction
   strings, skill prose). CLAUDE.md now names it.
5. Error-catalog cause/fix for the receipt refusal: the writer mkdirs the
   ledger dir itself, so 'missing' isn't a cause and bare chmod fails when
   it is absent — cause reworded, fix is mkdir -p && chmod.
6. gbrain-sync first-run steps described the retired binary's behavior:
   default repo is gstack-artifacts-$USER, and init PRINTS the gbrain
   hookup command (never auto-executes; bin/gstack-artifacts-init:384-419).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
This commit is contained in:
Garry Tan 2026-08-14 09:28:56 -07:00 committed by GitHub
parent d078622b73
commit 1d41ee3ab3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
103 changed files with 7174 additions and 618 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
.github/workflows/evals.yml vendored
View File

@ -183,6 +183,14 @@ jobs:
# bin/ + sections/ are committed). $HOME is /github/home here; the spawned
# claude inherits it (this runner adds no HOME/CLAUDE_CONFIG_DIR override,
# no hermetic mode) and the Seed step already proved claude reads $HOME.
#
# KEEP THIS STEP even though seedSkills/hermeticSkillsConfigDir() now
# registers skills for hermetic PTY children: that registry is SYMLINKS
# into the repo checkout, and this container's cross-mount symlinks
# defeat the TUI skill scanner (see the note inside the step below) —
# the real-file copies here are what the TUI actually reads. HOME is
# also not hermeticized, so the absolute ~/.claude/skills/gstack/...
# preamble paths resolve through the gstack root symlink this step makes.
- name: Register gstack skills for PTY smoke
if: matrix.suite.name == 'e2e-pty-plan-smoke'
run: |

17
ARCHITECTURE.md
View File

@ -91,14 +91,15 @@ When a user runs `pair-agent --client`, the daemon starts an ngrok tunnel so a r
The fix is **two HTTP listeners**, not one:
- **Local listener** (`127.0.0.1:LOCAL_PORT`) — always bound. Serves bootstrap (`/health` with token delivery), `/cookie-picker`, `/inspector/*`, `/welcome`, `/refs`, the sidebar-agent API, and the full command surface. Never forwarded.
- **Local listener** (`127.0.0.1:LOCAL_PORT`) — always bound. Serves token bootstrap (`POST /extension-token`, released only to the pinned extension identity), `/health` (liveness/status only — never a token), `/cookie-picker`, `/inspector/*`, `/welcome`, `/refs`, the sidebar-agent API, and the full command surface. Never forwarded.
- **Tunnel listener** (`127.0.0.1:TUNNEL_PORT`) — bound lazily on `/tunnel/start`, torn down on `/tunnel/stop`. Serves a locked allowlist: `/connect` (pairing ceremony, unauth + rate-limited), `/command` (scoped tokens only, further restricted to a browser-driving command allowlist), and `/sidebar-chat`. Everything else 404s.
ngrok forwards only the tunnel port. The security property comes from **physical port separation**: a tunnel caller cannot reach `/health` or `/cookie-picker` because those paths don't exist on that TCP socket. Header inference (check `x-forwarded-for`, check origin) is unreliable (ngrok header behavior changes; local proxies can add these headers); socket separation isn't.
| Endpoint | Local listener | Tunnel listener | Notes |
|---|---|---|---|
| `GET /health` | public (no token unless headed/extension) | 404 | Token bootstrap for extension happens locally only |
| `GET /health` | public (liveness/status only — never a token) | 404 | Token bootstrap moved to `POST /extension-token` (v1.63) |
| `POST /extension-token` | pinned Origin (`chrome-extension://<GSTACK_EXTENSION_ID>`) + loopback Host | 404 | The only endpoint that hands out the root token |
| `GET /connect` | public (`{alive:true}`) | public (`{alive:true}`) | Probe path for tunnel liveness |
| `POST /connect` | public (rate-limited 300/min) | public (rate-limited) | Setup-key exchange for pair-agent |
| `POST /command` | auth (Bearer root OR scoped) | auth (scoped only, allowlisted commands) | Root token on tunnel = 403 |
@ -114,6 +115,8 @@ ngrok forwards only the tunnel port. The security property comes from **physical
| `GET /inspector/events` | Bearer OR HttpOnly `gstack_sse` cookie | 404 | SSE. Same cookie as /activity/stream |
| `POST /sse-session` | auth (Bearer) | 404 | Mints the view-only 30-min SSE session cookie |
**Extension token bootstrap (v1.63.0.0).** `GET /health` never carries a token in any mode — it is liveness/status only. The sidebar extension obtains the root token via `POST /extension-token`, which releases it only when the caller's Origin is exactly `chrome-extension://<GSTACK_EXTENSION_ID>` (pinned by the `key` field in `extension/manifest.json`; reproduce the derivation with `bun browse/scripts/extension-id.ts`) and the Host header parses to a loopback hostname — parsed with `new URL()`, never compared raw, because Host carries the port. Web pages cannot forge a `chrome-extension://` Origin, and the endpoint is never added to the tunnel allowlist, so the tunnel surface 404s it by default-deny.
**Tunnel surface denial logs.** Every rejection on the tunnel listener (`path_not_on_tunnel`, `root_token_on_tunnel`, `missing_scoped_token`, `disallowed_command:*`) is recorded asynchronously to `~/.gstack/security/attempts.jsonl` with timestamp, source IP (from `x-forwarded-for`), path, and method. Rate-capped at 60 writes/min globally to prevent log-flood DoS. Shares the attempt log with the prompt-injection scanner.
**SSE session cookies.** EventSource can't send Authorization headers, so the extension POSTs `/sse-session` once at bootstrap with the root Bearer and receives a 30-minute view-only cookie (`gstack_sse`, HttpOnly, SameSite=Strict). The cookie is valid ONLY for `/activity/stream` and `/inspector/events` — it is NOT a scoped token and cannot be used on `/command`. Scope isolation is enforced by the module boundary: `sse-session-cookie.ts` has no imports from `token-registry.ts`.
@ -144,6 +147,14 @@ Cookies are the most sensitive data gstack handles. The design:
The browser registry (Comet, Chrome, Arc, Brave, Edge) is hardcoded. Database paths are constructed from known constants, never from user input. Keychain access uses `Bun.spawn()` with explicit argument arrays, not shell string interpolation.
### Egress receipt ledger (v1.63.0.0)
Every enumerated gstack-initiated off-machine sink writes a hash-chained, tamper-evident receipt to `~/.gstack/security/egress.jsonl` BEFORE the send — `writeReceipt` in `lib/egress-receipt.ts` for TypeScript callers, `_receipted_curl` / `_receipted_git` from `bin/gstack-egress-lib.sh` for shell scripts. Receipts record a sha256 of the exact bytes sent when the caller owns them (subprocess-owned sends like git pushes record `sha256: null`); they never store the body.
Failure polarity is per-class and pinned by tests. Sensitive sinks are fail-closed: brain-sync pushes, memory-ingest, gbrain-sync, telemetry, ngrok tunnel starts, mcp-verify, and supabase-provision refuse to send if the receipt can't be written (each refusal prints problem + cause + fix). User-facing sinks fail open with a stderr warning — the design binary's OpenAI calls, update-check, the read-only dashboards, and git-class receipts proceed even when the receipt write failed, so a fail-open send can go unrecorded (warned, by design). The new-sink scanner in `test/egress-receipt-wiring.test.ts` fails CI when an off-machine sink ships unwired; its only exemptions are enumerated with reasons (user-directed page fetches, reachability probes, install-doc strings, skill prose).
Inspect the ledger with `bin/gstack-egress`: `list` (what gstack attempted to send), `verify` (recompute the chain, exit 3 on tamper), `grants` (the standing consent settings and how to revoke each). Threat model: the ledger is forensic observability of ATTEMPTED egress — it records what gstack tried to send so accidents are auditable; it is not an exfiltration control.
### Unicode sanitization at server egress (v1.38.0.0)
Page content harvested by CDP can contain lone UTF-16 surrogate halves (orphaned high or low surrogates from broken JavaScript string handling on the page). When those reach `JSON.stringify`, Bun emits them as `\uD800`-style escape sequences that the downstream consumer's `JSON.parse` accepts, but the Anthropic API rejects with a 400 — turning a single weird page into a session-killing error. Defense is single-point, applied at every server egress that ships page-derived strings.
@ -414,7 +425,7 @@ The `EvalCollector` accumulates test results and writes them in two ways:
1. **Incremental:** `savePartial()` writes `_partial-e2e.json` after each test (atomic: write `.tmp`, `fs.renameSync`). Survives kills.
2. **Final:** `finalize()` writes a timestamped eval file (e.g. `e2e-20260314-143022.json`). The partial file is never cleaned up — it persists alongside the final file for observability.
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack-dev/evals/`.
`eval:compare` diffs two eval runs. `eval:summary` aggregates stats across all runs in `~/.gstack-dev/evals/`. Both are shard-aware (v1.63.0.0): the sharded paid runner (`scripts/test-paid-shards.ts`, run via `test:gate:sharded` / `test:periodic:sharded` — the `eval:bg:gate` / `eval:bg:periodic` scripts now point at these) gives each shard's collector its own directory at `<evalDir>/shards/<slug>/` through the `GSTACK_EVAL_DIR` env var (honored by the `EvalCollector` constructor), and `eval:list` / `eval:compare` / `eval:summary` scan one level of `shards/<slug>/` subdirectories. Baseline lookups exclude `_partial` accumulators (`isPartialEval` / `findLatestFinalizedRun` in `eval-store.ts`), so auto-comparison never uses the current run's own partial file as its baseline.
### Test tiers

26
BROWSER.md
View File

@ -705,6 +705,10 @@ Or do it manually: `chrome://extensions` → toggle Developer mode → Load
unpacked → navigate to `~/.claude/skills/gstack/extension` → pin the
extension → enter the port from `$B status`.
v1.63 pinned the extension identity via the manifest `key` field, so existing
unpacked installs get a new extension ID and panel-local state (saved port)
resets once — a one-time in-product notice explains this.
---
## Pair-agent
@ -758,6 +762,15 @@ remote agent that tries them gets a 403 plus a fresh entry in the denial log.
+ domain only (no raw IP, no full request body), rotates at 10MB with 5
generations. Per-device salt at `~/.gstack/security/device-salt` (mode 0600).
### Tunnel egress receipts (v1.63+)
Every tunnel session open writes a hash-chained egress receipt (sink
`browse-tunnel`) to `~/.gstack/security/egress.jsonl` BEFORE ngrok forwards
anything. Fail-closed: if the receipt can't be written, the tunnel listener
is torn down and the start is refused. Inspect the ledger with
`bin/gstack-egress list` and verify chain integrity with
`bin/gstack-egress verify` (exit 3 on tamper).
See [`docs/REMOTE_BROWSER_ACCESS.md`](docs/REMOTE_BROWSER_ACCESS.md) for the
full operator guide.
@ -800,6 +813,19 @@ The Terminal pane uses a separate session cookie, `gstack_pty`, minted via
PTY, can't dispatch arbitrary `/command` calls. `/health` endpoint MUST NOT
surface this token.
### Extension token bootstrap (v1.63+)
`GET /health` is liveness/status only — it never carries a token, in any
mode. The Side Panel extension bootstraps the root token via
`POST /extension-token` on the local listener. The server releases the
token only when the caller's Origin is exactly
`chrome-extension://<GSTACK_EXTENSION_ID>` — the `key` field in
`extension/manifest.json` pins the extension ID (`GSTACK_EXTENSION_ID` in
`browse/src/server.ts`; derivation reproducible via
`bun browse/scripts/extension-id.ts`) — AND the parsed Host hostname is
loopback. Anything else gets a detail-free 403. The endpoint is never
added to `TUNNEL_PATHS`, so the tunnel surface 404s it by default-deny.
### Token registry
`browse/src/token-registry.ts` handles mint/validate/revoke for all three

108
CHANGELOG.md
View File

@ -1,5 +1,113 @@
# Changelog
## [1.63.0.0] - 2026-08-13
**Everything gstack sends off your machine now leaves a receipt you can read.**
**And the eval harness stopped grading itself a passing grade.**
This release ports the parts of the GStack 2 fork that earned their way back into
main. The headline is a hash-chained egress ledger: every place gstack itself
sends data off your machine now writes a local, tamper-evident receipt first, and
`gstack-egress list` / `verify` show you exactly what left and prove the chain is
intact. Two new command-line tools ship with it: `gstack-egress` (the auditor's
view) and `gstack-context-bill` (a token bill-of-materials for any skills tree, so
you can see what a gstack install costs your context window before you invoke
anything). The test harness got three real fixes, one of them a bug that had been
quietly lying to every contributor for months.
### The numbers that matter
Source: the assembled branch (`git log 1.62.0.0..HEAD`), the free suite
(`bun test`), and the discovery-surface gate (`test/catalog-budget.test.ts`).
| Metric | Before | After | Δ |
|---|---|---|---|
| gstack-owned off-machine sinks with a receipt | 0 | every enumerated sink | tripwire-enforced, zero exceptions |
| Eval "no regressions" lines that were self-comparisons | every one | 0 | the harness compared runs against their own in-progress accumulator |
| Paid gate runner isolation | one process, one hung file kills the tier | one process per file, group-SIGKILL on stall | + never-started accounting |
| Discovery catalog budget | unenforced | 1,105 token-equivalents measured, 1,150 ceiling | ratchet-protocol on every skill add |
| Browser `/health` endpoint | served the root auth token to any localhost caller in headed mode | serves no token in any mode | token bootstrap moved to a pinned-origin POST |
The eval-store line is the one that matters most for anyone hacking on gstack:
`findPreviousRun` picked the newest same-tier file as the baseline, and the
in-progress `_partial` accumulator always won that sort, so the auto-comparison
compared a run against itself and printed "no regressions" no matter what. That is
fixed, with regression tests, and the fix was confirmed against the bug on the
prior release before landing.
### What this means for you
If you care what gstack does with your data, you can now audit it: run
`gstack-egress list` after any session and see every off-machine send, or
`gstack-egress verify` to confirm nothing was rewritten. If you contribute to
gstack, your eval comparisons mean something again, the paid gate can't be taken
down by one wedged test, and `gstack-context-bill` tells you what your skill
changes cost before you ship them. Nothing new phones home; the ledger is local
and the receipts record what gstack *attempts* to send, so accidents are auditable.
Ported from the GStack 2 fork by Sina Matian (time-attack/gstack); the eval-store
bug fix and the port shortlist were selected and hardened for upstream.
### Itemized changes
#### Added
- `gstack-egress` — read the hash-chained egress receipt ledger: `list` (what
gstack attempted to send off-machine), `verify` (recompute the chain, exit 3 on
tamper), `grants` (the standing consent settings and how to revoke each).
- `gstack-context-bill` — offline token bill-of-materials for a skills tree:
always-on discovery cost vs per-invocation cost, `--diff` between two trees,
`--budget`, and `--exact` (opt-in, measures against the real tokenizer).
- Hash-chained egress receipts (`lib/egress-receipt.ts`): fail-closed
receipt-before-send for sensitive sinks (brain-sync, memory-ingest, gbrain-sync,
telemetry, tunnels), fail-open with a warning for user-facing sinks (the design
binary's model calls, update-check, dashboards). A tripwire test enforces that
every off-machine sink in the tree is wired, with zero silent exceptions.
- Sharded paid-gate runner (`test:gate:sharded` / `test:periodic:sharded`): one
process per test file, an external wall-clock timeout that group-SIGKILLs a
wedged file's whole process tree, and four-way per-shard status so a crash can't
masquerade as a pass.
- `gstack-context-bill` and the egress tools install through the standard `./setup`
path like every other gstack binary.
#### Changed
- The browser `/health` endpoint no longer carries the root auth token in any
mode. The sidebar extension bootstraps its token through a new
`POST /extension-token` that requires the pinned extension origin and a loopback
Host; the tunnel listener never exposes it. Upgrading resets the sidebar's
panel-local state once, explained in-product.
- Hermetic PTY test children can register the repo's shipped skills, so
slash-command gate tests actually exercise the skill under test instead of
silently measuring nothing.
- Discovery-surface cost is now gated: `test/catalog-budget.test.ts` pins the
aggregate skill name+description budget with a self-service ratchet protocol.
#### Fixed
- The eval harness auto-comparison compared every run against its own in-progress
accumulator and reported "no regressions" unconditionally. Fixed with regression
tests; comparisons now find the latest *completed* same-tier run.
- The browser `/health` token leak (a headed-mode carve-out that handed the root
token to any localhost caller).
#### For contributors
- Shared modules replace duplicated logic: one paid-test-set definition consumed by
both the free-suite filter and the paid runner, one skill-census helper with three
explicit counts (physical files, authored skills, registry entries) consumed by
the seeder, context-bill, and the catalog gate.
- `CLAUDE.md`'s compiled-binaries note corrected: the `browse/dist` binaries have
been untracked since v0.11.16.0, so they no longer appear in `git status`.
- External-service E2E tests (Codex, Gemini, benchmark providers) are declared
periodic-tier with the canonical whole-file guard, so the merge-blocking gate
never waits on a third-party CLI. The Codex runner passes
`--skip-git-repo-check` (now required in non-git working dirs) and the Gemini
runner classifies an unusable CLI (removed flags, retired auth paths) as a
skip instead of a false failure.
- The PTY test runner parses AskUserQuestion prompts that reflow onto a single
logical line and strips DEC cursor-visibility residue, pinned by
`test/pty-askuserquestion-single-line.test.ts` — the failure class that
previously ate a gate test's whole time budget.
- New follow-ups filed in `TODOS.md`: egress ledger rotation (chain-genesis
records), a launch-nonce token bootstrap, and eval-watch shard-awareness.
## [1.62.0.0] - 2026-08-12
## **Plan reviews stop asking what to review when you're in plan mode.**

96
CLAUDE.md
View File

@ -9,6 +9,8 @@ bun run test:evals # run paid evals: LLM judge + E2E (diff-based, ~$4/run max)
bun run test:evals:all # run ALL paid evals regardless of diff
bun run test:gate # run gate-tier tests only (CI default, blocks merge)
bun run test:periodic # run periodic-tier tests only (weekly cron / manual)
bun run test:gate:sharded # gate tier via the sharded paid runner (one Bun process per test file)
bun run test:periodic:sharded # periodic tier via the sharded paid runner (implies EVALS_ALL=1)
bun run test:e2e # run E2E tests only (diff-based, ~$3.85/run max)
bun run test:e2e:all # run ALL E2E tests regardless of diff
bun run eval:select # show which tests would run based on current diff
@ -48,13 +50,21 @@ MCP servers / skills), a temp `GSTACK_HOME`, and `--strict-mcp-config`.
Local eval signal matches CI. Debug against real operator state with
`EVALS_HERMETIC=0` (restores the legacy env AND drops the strict-MCP flag).
Per-test `env:` overrides merge last, so deliberate contamination
(`CONDUCTOR_WORKSPACE_PATH`, per-test `GSTACK_HOME`) keeps working. Wiring
is pinned by `test/hermetic-wiring.test.ts` (static tripwire) and two
gate-tier canaries in `test/skill-e2e-hermetic-canary.test.ts`.
(`CONDUCTOR_WORKSPACE_PATH`, per-test `GSTACK_HOME`) keeps working. The
hermetic config dir seeds NO skills by default; a PTY test that types a
`/skill` slash command must pass `seedSkills: true` to the PTY runner, which
points the child's `CLAUDE_CONFIG_DIR` at `hermeticSkillsConfigDir()` — a
seeded registry that symlinks the LIVE working tree's SKILL.md files (by
design: the skills ARE the subject under test; a snapshot would measure stale
copies). Wiring is pinned by `test/hermetic-wiring.test.ts` (static tripwire),
two gate-tier canaries in `test/skill-e2e-hermetic-canary.test.ts`, and the
seeding tripwires in `test/hermetic-skills-seeding.test.ts` /
`test/pty-skill-seeding-wiring.test.ts`.
E2E tests stream progress in real-time (tool-by-tool via `--output-format stream-json
--verbose`). Results are persisted to `~/.gstack-dev/evals/` with auto-comparison
against the previous run.
against the previous finalized run (in-flight `_partial` files are never used as
a baseline, so a run can't compare against itself).
**Diff-based test selection:** `test:evals` and `test:e2e` auto-select tests based
on `git diff` against the base branch. Each test declares its file dependencies in
@ -110,6 +120,7 @@ gstack/
│ ├── host-adapters/ # Host-specific adapters (OpenClaw tool mapping)
│ ├── resolvers/ # Template resolver modules (preamble, design, review, gbrain, etc.)
│ ├── skill-check.ts # Health dashboard
│ ├── test-paid-shards.ts # Sharded paid-tier runner (one Bun process per shard)
│ └── dev-skill.ts # Watch mode
├── test/ # Skill validation + eval tests
│ ├── helpers/ # skill-parser.ts, session-runner.ts, llm-judge.ts, eval-store.ts
@ -147,7 +158,7 @@ gstack/
│ ├── test/ # Integration tests
│ └── dist/ # Compiled binary
├── extension/ # Chrome extension (side panel + activity feed + CSS inspector)
├── lib/ # Shared libraries (worktree.ts)
├── lib/ # Shared libraries (worktree.ts, egress-receipt.ts, context-bill.ts, redact-engine.ts)
├── docs/designs/ # Design documents
├── setup-deploy/ # /setup-deploy skill (one-time deploy config)
├── .github/ # CI workflows + Docker image
@ -184,6 +195,16 @@ behavior). If you blow past 40K, the right fix is usually: (1) look at WHAT grew
or as a reference doc, (3) only compress carefully-tuned prose as a last resort —
cuts to the coverage audit, review army, or voice directive have real quality cost.
A second, harder ceiling guards the DISCOVERY surface: `test/catalog-budget.test.ts`
caps the aggregate frontmatter `name` + `description` across all skills at 1,150
token-equivalents (260-byte per-skill sub-cap), counted through the shared census
in `test/helpers/skill-census.ts`. This one is enforced, not a warning — every
host loads the full catalog every session, so growth here taxes every
conversation. The failure message carries the re-measure + ratchet protocol.
`bin/gstack-context-bill` shows the full token bill-of-materials for a skills
tree (always-on vs per-invocation, `--diff`, `--budget`; `--exact` opts into the
real tokenizer and POSTs file text to api.anthropic.com with an egress receipt).
**Merge conflicts on SKILL.md files:** NEVER resolve conflicts on generated SKILL.md
files by accepting either side. Instead: (1) resolve conflicts on the `.tmpl` templates
and `scripts/gen-skill-docs.ts` (the sources of truth), (2) run `bun run gen:skill-docs`
@ -282,10 +303,14 @@ PTY via `window.gstackInjectToTerminal(text)`, exposed by
`sidepanel-terminal.js`. No `/sidebar-command` POST — the live REPL is
the only execution surface in the sidebar now.
**`/health` MUST NOT surface any shell-grant token.** It already leaks
`AUTH_TOKEN` to localhost callers in headed mode (a v1.1+ TODO). Don't
make that worse by adding the PTY session token there. PTY auth flows
through `POST /pty-session` only.
**`/health` MUST NOT surface any token — and it no longer does** (v1.63+).
The historical headed-mode leak of `AUTH_TOKEN` is fixed: `GET /health` is
liveness/status only in every mode. Token bootstrap is `POST /extension-token`,
which validates the caller's Origin against the pinned extension identity
(the `key` field in `extension/manifest.json` pins the extension ID —
`GSTACK_EXTENSION_ID` in `browse/src/server.ts`, derivation reproducible via
`bun browse/scripts/extension-id.ts`) plus a loopback Host. PTY auth still
flows through `POST /pty-session` only. Don't add any token to `/health`.
**Transport-layer security** (v1.6.0.0+). When `pair-agent` starts an ngrok tunnel,
the daemon binds two HTTP listeners: a local listener (127.0.0.1, full command
@ -315,6 +340,23 @@ response in `server.ts`, read
`browse/test/server-sanitize-surrogates.test.ts` pins the wiring with invariant
tests, so bypasses fail CI.
**Egress receipts at every off-machine sink** (v1.63.0.0+). Every gstack-initiated
send off the machine MUST write a hash-chained receipt to
`~/.gstack/security/egress.jsonl` BEFORE the send: TypeScript callers use
`writeReceipt` from `lib/egress-receipt.ts`; shell scripts source
`bin/gstack-egress-lib.sh` and use `_receipted_curl` / `_receipted_git`. Failure
polarity is per-class: fail-closed for sensitive sinks (brain-sync, memory-ingest,
gbrain-sync, telemetry, ngrok tunnels, mcp-verify, supabase-provision), fail-open
+ stderr warning for user-facing ones (design OpenAI calls, update-check,
dashboards, git-class ops). The new-sink scanner in
`test/egress-receipt-wiring.test.ts` fails CI on an unreceipted `curl` /
`git push` / `fetch` to a non-loopback host unless the file carries a reasoned
entry in its `SCANNER_EXEMPT` list (user-directed page fetches, reachability
probes, instruction strings, skill prose) — if you add a new off-machine sink,
wire it through the helpers and add it to the enumerated sink list. Inspect with
`bin/gstack-egress` (`list` | `verify`, exit 3 on tamper | `grants`). Threat
model: forensic observability of ATTEMPTED egress, not an exfiltration control.
**SSE endpoint helper** (v1.51.0.0+). New SSE endpoints in `server.ts` MUST route
through `createSseEndpoint(req, config)` from `browse/src/sse-helpers.ts`. The
helper owns the cleanup contract (abort + enqueue-throw + heartbeat-throw, all
@ -425,19 +467,21 @@ migration script to `gstack-upgrade/migrations/`. Read CONTRIBUTING.md's "Upgrad
migrations" section for the format and testing requirements. The upgrade skill runs
these automatically after `./setup` during `/gstack-upgrade`.
## Compiled binaries — NEVER commit browse/dist/ or design/dist/
## Compiled binaries — never commit browse/dist/, design/dist/, or make-pdf/dist/
The `browse/dist/` and `design/dist/` directories contain compiled Bun binaries
(`browse`, `find-browse`, `design`, ~58MB each). These are Mach-O arm64 only — they
do NOT work on Linux, Windows, or Intel Macs. The `./setup` script already builds
from source for every platform, so the checked-in binaries are redundant. They are
tracked by git due to a historical mistake and should eventually be removed with
`git rm --cached`.
The `browse/dist/`, `design/dist/`, and `make-pdf/dist/` directories contain
compiled Bun binaries (`browse`, `find-browse`, `design`, ~62MB each). These are
Mach-O arm64 only — they do NOT work on Linux, Windows, or Intel Macs. The
`./setup` script builds from source for every platform.
**NEVER stage or commit these files.** They show up as modified in `git status`
because they're tracked despite `.gitignore` — ignore them. When staging files,
always use specific filenames (`git add file1 file2`) — never `git add .` or
`git add -A`, which will accidentally include the binaries.
These directories are **untracked and gitignored** (`.gitignore:3-6`; the
`browse/dist/` binaries were untracked in `64d5a3e4`, v0.11.16.0; the others were
never tracked). They will NOT appear in `git status`. If a dist binary ever does
show up in `git status`, something force-added it (`git add -f`) — do not commit
it; unstage it and find out how it got there.
When staging files, always use specific filenames (`git add file1 file2`) — never
`git add .` or `git add -A`, which can sweep in build outputs and junk.
## Redaction guard (PII / secrets / legal content)
@ -864,7 +908,17 @@ the run can also die to idle-sleep. `gstack-detach` fixes both: a fresh session
machine-wide `gstack-evals` lock (concurrent worktrees serialize instead of
saturating the shared model API), a per-tier watchdog, and a **run-scoped** log
under `~/.gstack-dev/eval-runs/` (no shared-`/tmp` collision). Each prints its
log path. Or call `gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
log path. `eval:bg:gate` / `eval:bg:periodic` run their tier through the
sharded paid runner (`scripts/test-paid-shards.ts`, also exposed as
`test:gate:sharded` / `test:periodic:sharded`): one Bun process per test
file, an external wall-clock timeout that kills the shard's process GROUP
(stray `claude`/`codex` grandchildren included), a per-shard
`GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/` honored by the `EvalCollector`
constructor, and an aggregate that separates failed vs timed-out vs
never-started shards — the detach timeouts (25200s gate / 28800s periodic)
are sized against worst-case shard wall clock. `eval:list` / `eval:compare` /
`eval:summary` read the shard dirs too. Or call
`gstack-detach [--lock NAME] [--timeout SECS] [--label LBL] --
<cmd>` directly for any long agent job. Export `ANTHROPIC_API_KEY` first (never
pass keys in argv).
- Then **poll the printed logfile** with a death-aware watcher: break on the

25
CONTRIBUTING.md
View File

@ -160,6 +160,7 @@ Runs automatically with `bun test`. No API keys needed.
- **Skill validation tests** (`test/skill-validation.test.ts`) — Validates that SKILL.md files reference only real commands and flags, and that command descriptions meet quality thresholds.
- **Generator tests** (`test/gen-skill-docs.test.ts`) — Tests the template system: verifies placeholders resolve correctly, output includes value hints for flags (e.g. `-d <N>` not just `-d`), enriched descriptions for key commands (e.g. `is` lists valid states, `press` lists key examples).
- **Tier-alignment invariant** (`test/e2e-tier-alignment.test.ts`) — For every self-gated `test/skill-e2e-*.test.ts` named in a touchfiles dep list, the file's `EVALS_TIER` self-gate must match its declared tier in `E2E_TIERS`. Kills the "inert demotion" class where a test is re-tiered in `touchfiles.ts` but the file still gates on the old tier and keeps running in the wrong lane. Unmapped or mixed-tier files are reported, never silently skipped.
- **Catalog budget** (`test/catalog-budget.test.ts`) — Caps the aggregate discovery surface: the sum of every skill's frontmatter `name` + `description` (what every host loads at discovery, every session) must stay under 1,150 token-equivalents, with a 260-byte per-skill cap. Counting goes through the shared census in `test/helpers/skill-census.ts` (physical files vs authored skills vs registry entries — three deliberately different counts). Adding a skill? The failure message carries the re-measure + ratchet protocol.
### Tier 2: E2E via `claude -p` (~$3.85/run)
@ -184,10 +185,16 @@ seeded `CLAUDE_CONFIG_DIR`, a temp `GSTACK_HOME`, and `--strict-mcp-config`. You
operator `~/.claude` config, MCP servers (gbrain, Conductor), skills, `~/.gstack`
decision logs, and `CONDUCTOR_*` env never leak into the child, so local eval
signal matches CI instead of disagreeing for reasons unrelated to the code under
test. Set `EVALS_HERMETIC=0` to debug against your real operator state (this also
test. The hermetic `CLAUDE_CONFIG_DIR` seeds no skills by default; a PTY test
that types a `/skill` slash command passes `seedSkills: true` to the PTY runner,
which swaps in `hermeticSkillsConfigDir()` — a seeded skill registry that
symlinks the LIVE working tree's SKILL.md files (by design: the skills are the
subject under test, so a snapshot would measure stale copies). Set
`EVALS_HERMETIC=0` to debug against your real operator state (this also
drops `--strict-mcp-config`). The wiring is pinned by `test/hermetic-wiring.test.ts`
(a free static tripwire) and two gate-tier isolation canaries in
`test/skill-e2e-hermetic-canary.test.ts`.
(a free static tripwire), two gate-tier isolation canaries in
`test/skill-e2e-hermetic-canary.test.ts`, and the skill-seeding tripwires in
`test/hermetic-skills-seeding.test.ts` / `test/pty-skill-seeding-wiring.test.ts`.
### E2E observability
@ -227,8 +234,16 @@ bun run eval:bg:gate # detached gate-tier suite
bun run eval:bg:periodic # detached periodic-tier suite
```
Each prints its log path. Humans running `bun run test:evals` foreground in their
own terminal don't need this — Ctrl-C is intended there.
Each prints its log path. The gate and periodic variants run their tier through
the sharded paid runner (`scripts/test-paid-shards.ts`, also available directly
as `bun run test:gate:sharded` / `bun run test:periodic:sharded`): one Bun
process per test file, an external wall-clock timeout that kills the shard's
whole process group (stray `claude`/`codex` grandchildren included), a per-shard
eval dir (`GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/`), and an aggregate that
distinguishes failed vs timed-out vs never-started shards. `eval:list`,
`eval:compare`, and `eval:summary` are shard-aware. Humans running
`bun run test:evals` foreground in their own terminal don't need this — Ctrl-C
is intended there.
**Eval comparison commentary:** `eval:compare` generates natural-language Takeaway sections interpreting what changed between runs — flagging regressions, noting improvements, calling out efficiency gains (fewer turns, faster, cheaper), and producing an overall summary. This is driven by `generateCommentary()` in `eval-store.ts`.

7
README.md
View File

@ -235,7 +235,7 @@ Each skill feeds into the next. `/office-hours` writes a design doc that `/plan-
| `/ios-qa` | **iOS Live-Device QA (v1.43.0.0+)** — drive a real iPhone over USB CoreDevice via an embedded `StateServer` in the app. Read Swift source, codegen typed `@Observable` accessors, run the agent loop. Optional `--tailnet` flag exposes the device to OpenClaw or any HTTP-capable agent on your Tailscale tailnet so remote agents can run iOS QA without ever touching the hardware. Capability-tier allowlist (observe/interact/mutate/restore), per-device session lock, audit log. |
| `/ios-fix`, `/ios-design-review`, `/ios-clean`, `/ios-sync` | iOS bug-fix loop, designer's-eye HIG audit, debug-bridge cleanup, and accessor resync. See `docs/skills.md`. End-to-end walkthrough: [docs/howto-ios-testing-with-gstack.md](docs/howto-ios-testing-with-gstack.md). |
### New binaries (v0.19)
### Standalone binaries
Beyond the slash-command skills, gstack ships standalone CLIs for workflows that don't belong inside a session:
@ -243,6 +243,8 @@ Beyond the slash-command skills, gstack ships standalone CLIs for workflows that
|---------|-------------|
| `gstack-model-benchmark` | **Cross-model benchmark** — run the same prompt through Claude, GPT (via Codex CLI), and Gemini; compare latency, tokens, cost, and (optionally) LLM-judge quality score. Auth detected per provider, unavailable providers skip cleanly. Output as table, JSON, or markdown. `--dry-run` validates flags + auth without spending API calls. |
| `gstack-taste-update` | **Design taste learning** — writes approvals and rejections from `/design-shotgun` into a persistent per-project taste profile. Decays 5%/week. Feeds back into future variant generation so the system learns what you actually pick. |
| `gstack-egress` | **Egress receipt auditor** — every gstack-initiated off-machine send writes a tamper-evident, hash-chained receipt to `~/.gstack/security/egress.jsonl` before the send. `list` shows what gstack attempted to send and to which host, `grants` shows the standing consent settings plus the exact command that revokes each, `verify` recomputes the hash chain and exits 3 on tamper. |
| `gstack-context-bill` | **Token bill-of-materials** — read-only, offline audit of what an installed skills tree costs in tokens: always-on frontmatter every session pays vs per-invocation SKILL.md + forced references. `--diff` compares two trees, `--budget` enforces a ceiling, `--exact` opts into Anthropic `count_tokens` (sends file text off-machine; writes an egress receipt first, degrades to the offline estimate if the receipt can't be written). |
| `gstack-ios-qa-daemon` | **iOS QA daemon** — Mac-side broker between an agent and a connected iPhone over USB CoreDevice. Loopback by default; `--tailnet` opens a Tailscale-facing listener with identity-gated capability tiers. Single-instance via flock on `~/.gstack/ios-qa-daemon.pid`. See [docs/howto-ios-testing-with-gstack.md](docs/howto-ios-testing-with-gstack.md). |
| `gstack-ios-qa-mint` | **iOS allowlist manager** — owner-grant CLI for the tailnet allowlist. `grant`/`revoke`/`list` against `~/.gstack/ios-qa-allowlist.json` (mode 0600). Remote agents never auto-allowlist; this is the explicit-intent path. |
| `gstack-ios-qa-regen` | **iOS bridge regenerator** — deterministically installs the canonical DebugBridge package, generates typed state accessors, and records the installed gstack version. Safe to rerun after source changes or upgrades. |
@ -418,7 +420,7 @@ The skill asks once per repo. The decision is sticky across worktrees and branch
**GStack memory sync (different feature, same private-repo infra).** Optionally pushes your gstack state (learnings, CEO plans, design docs, retros, developer profile) to a private git repo so your memory follows you across machines, with a one-time privacy prompt (everything allowlisted / artifacts only / off) and a defense-in-depth secret scanner that blocks AWS keys, tokens, PEM blocks, and JWTs before they leave your machine.
```bash
gstack-brain-init
gstack-artifacts-init
```
**Running gstack in Conductor?** Conductor explicitly strips `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` from every workspace's process env, so paid evals and gbrain embeddings won't work out of the box. Set `GSTACK_ANTHROPIC_API_KEY` and `GSTACK_OPENAI_API_KEY` in Conductor's workspace env config instead — gstack's TS entry points promote them to canonical names at runtime. Full details and the contributor checklist for adding the import to new entry points: [Conductor + GSTACK_* env vars](USING_GBRAIN_WITH_GSTACK.md#conductor--gstack_-env-vars).
@ -450,6 +452,7 @@ gstack includes **opt-in** usage telemetry to help improve the project. Here's e
- **What's sent (if you opt in):** skill name, duration, success/fail, gstack version, OS. That's it.
- **What's never sent:** code, file paths, repo names, branch names, prompts, or any user-generated content.
- **Change anytime:** `gstack-config set telemetry off` disables everything instantly.
- **Every off-machine send is receipted.** Any gstack-initiated network send — telemetry included — writes a hash-chained, tamper-evident receipt to `~/.gstack/security/egress.jsonl` before the send; sensitive sinks refuse to send at all if the receipt can't be written. Audit with `gstack-egress list`, verify the chain with `gstack-egress verify` (exit 3 on tamper), see the standing consent settings with `gstack-egress grants`. The ledger records attempted sends so accidents are auditable — it's an audit trail, not a network firewall.
Data is stored in [Supabase](https://supabase.com) (open source Firebase alternative). The schema is in [`supabase/migrations/`](supabase/migrations/) — you can verify exactly what's collected. The Supabase publishable key in the repo is a public key (like a Firebase API key) — row-level security policies deny all direct access. Telemetry flows through validated edge functions that enforce schema checks, event type allowlists, and field length limits.

148
TODOS.md
View File

@ -875,32 +875,6 @@ plus a TTL so abandoned PTYs eventually exit.
---
### v1.1+: Audit `/health` token distribution
**What:** Codex's outside-voice review on cc-pty-import flagged that
`/health` already surfaces `AUTH_TOKEN` to any localhost caller in headed
mode (`server.ts:1657`). That's a pre-existing soft leak — anything
running on localhost gets the root token by hitting `/health`.
**Why:** cc-pty-import sidesteps it by NOT putting the PTY token there
(uses an HttpOnly cookie path instead). But the underlying leak is still
shippable surface. A second extension or a localhost web app could
currently scrape `AUTH_TOKEN` and hit any browse-server endpoint.
**Pros:** Closes a real privilege-escalation path on multi-extension
machines. **Cons:** Either we tighten the gate (Origin must be OUR
extension id, not just any chrome-extension://) or we move bootstrap
discovery off `/health` entirely. Either has migration cost for tests
and the existing extension.
**Context:** codex finding #2 on cc-pty-import plan-eng review. Not in
scope of that PR; deliberately deferred to keep PTY-import small.
**Priority:** P2.
**Effort:** M.
---
## Testing
## P2: Per-finding AskUserQuestion count assertion for /plan-ceo-review
@ -2654,3 +2628,125 @@ CI-hard-fail contract has to land five times.
five green files at the tail of a release. Zero user-facing value; pure DRY.
**Effort:** S (human ~3h, CC ~20min). **Depends on:** None.
## Egress-receipt follow-ups (filed via /plan-eng-review + /codex on the v1.63 port wave)
### P2: egress ledger rotation with chain-genesis records
**What:** Rotate `~/.gstack/security/egress.jsonl` at a size threshold (match
`attempts.jsonl`'s 10MB/5-generation pattern in `browse/src/security.ts`), where
each new generation's FIRST record embeds the prior file's tail hash so
`gstack-egress verify` can walk across generations.
**Why:** v1.63 ships WARN-at-25MB (visible growth) but nothing bounds the file.
Rotation was deliberately deferred: it changes the verify contract, and a wrong
implementation makes healthy ledgers verify as "broken".
**Pros:** Bounded disk forever; verify stays meaningful across generations.
**Cons:** Chain-genesis semantics are subtle; needs its own focused tests
(cross-generation verify, mid-rotation crash).
**Context:** `lib/egress-receipt.ts` (`appendChained`/`verifyLedger`) carries the
design sketch in its rotation TODO comment. Start from the `attempts.jsonl`
rotation precedent.
**Effort:** S (human ~4h, CC ~25min). **Depends on:** v1.63 port wave landed.
### P3: launch-nonce token bootstrap (local-process impersonation)
**What:** Add a launch-time nonce to the `/extension-token` bootstrap: `browse`
mints a nonce at headed launch, seeds it into the extension (CDP
`chrome.storage` injection or a launcher-written sidecar), and the endpoint
requires it alongside the pinned origin.
**Why:** v1.63's pinned-origin check authenticates browser contexts; any local
PROCESS can still forge an Origin header with curl. That threat is explicitly
outside the current model (any local process can hit the port anyway) — this
TODO documents the deliberate boundary and the designed path across it.
**Pros:** Closes the local-process impersonation path (strongest of the three
options evaluated in the v1.63 plan review).
**Cons:** Largest bootstrap change; CDP seeding is fiddly across the three
launch paths (`--load-extension`, baked-in Browser.app, real-Chrome fallback);
low present-day value.
**Context:** `browse/src/server.ts` `/extension-token` handler +
`GSTACK_EXTENSION_ID`; launch paths in `browse/src/browser-manager.ts` (~358,
~455, ~1562); `extension/background.js` bootstrap.
**Effort:** M (human ~2 days, CC ~1h). **Depends on:** none.
### P3: eval-watch shard-awareness
**What:** Teach `scripts/eval-watch.ts` (hardcoded `_partial-e2e.json` path at
~line 17) about the sharded layout: watch `<evalDir>/shards/*/_partial-e2e.json`
and aggregate live progress across shard subdirs.
**Why:** v1.63's sharded runner gives each shard its own eval subdir (so shards
baseline against their own priors); `findPreviousRun`, `eval-compare`,
`eval-list`, and `eval-summary` were all made shard-aware, but the live watcher
intentionally stayed flat — it shows nothing during sharded runs.
**Pros:** Live progress during `eval:bg:gate` sharded runs again.
**Cons:** Multi-file watch + aggregation UI; low stakes (the run-scoped detach
log already streams per-shard results).
**Context:** `scripts/eval-watch.ts`; shard layout defined in
`scripts/test-paid-shards.ts` (slug = test filename); `listEvalJsonFiles` in
`test/helpers/eval-store.ts` already enumerates the layout — reuse it.
**Effort:** S (human ~2h, CC ~15min). **Depends on:** v1.63 port wave landed.
## v1.63 port-wave review follow-ups (deferred from /ship review army — non-blocking polish)
Genuine review findings deferred from the v1.63 ship because they are
informational/polish, not correctness-blocking, and several want their own
tests. Filed so they are tracked, not dropped.
- **P2 — telemetry-sync HTTP-status outcome is dead code.** `_GSTACK_EGRESS_LAST_RECEIPT`
is set inside a command-substitution subshell in `bin/gstack-telemetry-sync`, so the
parent-shell guard that would append the HTTP status to the receipt never fires. The
generic `exit:N` outcome is still recorded, so the ledger is correct, just less
precise. Fix: have `_receipted_curl` persist the receipt id to a caller-readable temp
file, or restructure the call out of the subshell. (Confirmed by 3 review specialists.)
- **P2 — context-bill "TOTAL on disk" double-counts child skills** in a root-as-container
tree (this repo's own layout): `buildBill` sums the root skill's whole-tree walk plus
each child's subtree again (~2x the TOTAL line). ALWAYS-ON / EAGER / --diff / --budget
are all unaffected — only the informational TOTAL is wrong. Fix: compute the tree total
from a single deduplicated `walkMd(root)` pass, or exclude child dirs from the root
skill's `totalMd`. Needs a fixture test. (`lib/context-bill.ts`.)
- **P3 — DRY/robustness polish:** one shared `_gstack_egress_host_of` helper for the
~11 hand-rolled URL-to-host extractions across the egress shell sinks; extract the
duplicated tunnel-open `writeReceipt` block in `browse/src/server.ts` (two sites);
hoist the per-iteration `SharedArrayBuffer` alloc out of the egress-receipt lock spin;
replace context-bill's exact-mode `errorPct === 0` sentinel with an explicit flag;
reuse `frontmatterName()` from `skill-census.ts` in `catalog-budget.test.ts`.
- **P3 — test-coverage gaps the audit named:** `PAID_TEST_GLOBS` ↔ `package.json`
`test:gate` parity test; `GSTACK_EXTENSION_ID` ↔ `manifest.json` key derivation parity
test (`browse/scripts/extension-id.ts`); a runner test asserting each shard child gets
its own `GSTACK_EVAL_DIR` under `shards/<slug>`; receipt-refusal branch tests for
supabase-provision / gbrain-sync / memory-ingest.
## P2: harden or re-tier skill-e2e-plan-design-with-ui PTY detection
**What:** The gate-tier `test/skill-e2e-plan-design-with-ui.test.ts` began executing
for the first time once v1.63's `seedSkills` registered skills in hermetic PTY
children (the fork had deleted this file; it measured nothing before). It now
reliably TIMES OUT even though the skill runs correctly: the transcript shows
`/plan-design-review` reaching its scope-gate AskUserQuestion (5 options, the
`<gstack-qid:plan-design-review-scope-gate>` marker present), but the test's
`isNumberedOptionListVisible`/`parseNumberedOptions` scraping can't classify it out
of the PTY buffer because spinner frames (`[?25l✻Sprouting… still thinking`) are
interleaved character-by-character with the option text.
**Why:** Shipped behavior is correct — this is a test-harness detection limitation,
not a product bug. But a gate test that always times out is worse than no test.
**Fix options:** (a) harden the tail-scraping (drop DEC private-mode + spinner
residue before matching; widen/clean the window); (b) add an LLM-judge fallback
classifier (the file's own comments note the regex detectors are "brittle to PTY
rendering quirks"); or (c) move this test to periodic until (a)/(b) lands.
**Context:** `test/skill-e2e-plan-design-with-ui.test.ts`,
`test/helpers/claude-pty-runner.ts:308` (`isNumberedOptionListVisible`). Evidence:
`~/.gstack-dev/eval-runs/pdwu-verify-*.log`. **Effort:** M (human ~half day / CC ~30min).

10
USING_GBRAIN_WITH_GSTACK.md
View File

@ -132,6 +132,8 @@ Storage: `~/.gstack/gbrain-repo-policy.json`, mode 0600, schema-versioned so fut
The skill runs three stages — code, memory, brain-sync — independently. A failure in one doesn't block the others. State persists to `~/.gstack/.gbrain-sync-state.json` so re-running picks up cleanly.
Stages that can send data off-machine (code sync into a possibly-remote gbrain DB, memory ingest, the brain-sync push) each write a tamper-evident receipt to the egress ledger (`~/.gstack/security/egress.jsonl`) before sending, fail-closed: if the receipt can't be written, the stage refuses with `EGRESS_RECEIPT_FAILED` instead of syncing unrecorded. Fix is usually `mkdir -p ~/.gstack/security && chmod -R u+w ~/.gstack/security`, then re-run. Inspect receipts with `gstack-egress list`.
**What it does on a fresh worktree:**
1. **Pre-flight.** Checks `gbrain_local_status` (the local engine's health). If the engine is `broken-db` or `broken-config`, the skill STOPs with a remediation menu — it refuses to silently degrade. If the local engine is missing and you're in remote-MCP mode (Path 4), the code stage SKIPs cleanly and only brain-sync runs.
@ -167,14 +169,16 @@ This is different from gbrain itself. Your gstack state (`~/.gstack/` — learni
Turn it on with:
```bash
gstack-brain-init
gstack-artifacts-init
```
You'll get a one-time privacy prompt: **everything allowlisted** / **artifacts only** (plans, designs, retros, learnings — skip behavioral data like timelines) / **off**. Every skill run syncs the queue at start and end — no daemon, no background process.
Secret-shaped content (AWS keys, GitHub tokens, PEM blocks, JWTs, bearer tokens) is blocked from sync before it leaves your machine.
**On a new machine:** Copy `~/.gstack-brain-remote.txt` over, run `gstack-brain-restore`, and yesterday's learnings surface on today's laptop.
**On a new machine:** Copy `~/.gstack-artifacts-remote.txt` over (the legacy
`~/.gstack-brain-remote.txt` name still works), run `gstack-brain-restore`, and
yesterday's learnings surface on today's laptop.
Full guide: [docs/gbrain-sync.md](docs/gbrain-sync.md). Error index: [docs/gbrain-sync-errors.md](docs/gbrain-sync-errors.md).
@ -239,7 +243,7 @@ Gbrain itself ships with these that gstack wraps:
| `~/.gstack/.setup-gbrain.lock.d` | Concurrent-run lock (atomic mkdir). Released on normal exit + SIGINT. |
| `~/.gstack/.brain-queue.jsonl` | Pending sync entries for gstack memory sync |
| `~/.gstack/.brain-last-push` | Timestamp of last sync push (for `/health` scoring) |
| `~/.gstack-brain-remote.txt` | URL of your gstack memory sync remote (safe to copy between machines) |
| `~/.gstack-artifacts-remote.txt` | URL of your gstack memory sync remote (safe to copy between machines; legacy name `~/.gstack-brain-remote.txt` still read) |
| `~/.gstack/.setup-gbrain-inflight.json` | Reserved for future `--resume-provision` persisted state |
### Environment variables

2
VERSION
View File

@ -1 +1 @@
1.62.0.0
1.63.0.0

24
bin/gstack-artifacts-init
View File

@ -40,6 +40,16 @@ SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
URL_BIN="$SCRIPT_DIR/gstack-artifacts-url"
REMOTE_FILE="$HOME/.gstack-artifacts-remote.txt"
# Egress receipt helpers (_receipted_git): fail-open for user-directed
# git ops against the user's own artifacts remote.
. "$SCRIPT_DIR/gstack-egress-lib.sh"
# remote host for receipt records (github.com etc). Set once PUSH_URL exists.
_artifacts_host() {
local h="${PUSH_URL#*://}"; h="${h#*@}"; h="${h%%[/:]*}"
echo "${h:-unknown}"
}
REMOTE_URL=""
HOST_PREF=""
URL_FORM_SUPPORTED="false"
@ -185,7 +195,8 @@ PUSH_URL=$("$URL_BIN" --to ssh "$CANONICAL_HTTPS" 2>/dev/null || echo "$CANONICA
# ---- verify push URL is reachable ----
echo "Verifying remote connectivity: $PUSH_URL"
if ! git ls-remote "$PUSH_URL" >/dev/null 2>&1; then
if ! _receipted_git open artifacts-init "$(_artifacts_host)" artifacts-remote-ls-remote "user ran gstack-artifacts-init" \
bash -c 'git ls-remote "$1" >/dev/null 2>&1' _ "$PUSH_URL"; then
cat >&2 <<EOF
Remote not reachable via SSH: $PUSH_URL
This could mean:
@ -346,10 +357,15 @@ else
fi
# ---- initial push ----
if ! git push -q -u origin main 2>/dev/null; then
if ! _receipted_git open artifacts-init "$(_artifacts_host)" artifacts-initial-push "user ran gstack-artifacts-init" \
bash -c 'git push -q -u origin main 2>/dev/null'; then
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
if git fetch origin 2>/dev/null && git pull --ff-only origin "$CURRENT_BRANCH" 2>/dev/null; then
git push -q -u origin "$CURRENT_BRANCH" || {
if _receipted_git open artifacts-init "$(_artifacts_host)" artifacts-fetch "user ran gstack-artifacts-init" \
bash -c 'git fetch origin 2>/dev/null' \
&& _receipted_git open artifacts-init "$(_artifacts_host)" artifacts-pull "user ran gstack-artifacts-init" \
bash -c 'git pull --ff-only origin "$1" 2>/dev/null' _ "$CURRENT_BRANCH"; then
_receipted_git open artifacts-init "$(_artifacts_host)" artifacts-initial-push "user ran gstack-artifacts-init" \
git push -q -u origin "$CURRENT_BRANCH" || {
echo "Push to $PUSH_URL failed. The remote may have divergent content." >&2
echo "Try: cd ~/.gstack && git pull --rebase origin $CURRENT_BRANCH && git push origin $CURRENT_BRANCH" >&2
exit 1

201
bin/gstack-brain-consumer
View File

@ -1,201 +0,0 @@
#!/usr/bin/env bash
# gstack-brain-consumer — manage the consumer (reader) registry.
#
# DEPRECATED in v1.17.0.0. This binary targets a gbrain HTTP /ingest-repo
# endpoint that never shipped on the gbrain side. Live federation now uses
# `gbrain sources` directly via bin/gstack-gbrain-source-wireup. This file
# stays for one cycle to avoid breaking external scripts; removal in v1.18.0.0.
#
# Consumer = a reader that ingests the gstack-brain git repo as a source of
# session memory. v1 primary consumer is GBrain; later versions can register
# Codex, OpenClaw, or third-party readers.
#
# NOTE ON NAMING: internally this helper uses "consumer" (correct data-model
# term). User-facing copy and the alias `gstack-brain-reader` use "reader"
# (matches user mental model: "what's reading my brain?").
#
# Usage:
# gstack-brain-consumer add <name> --ingest-url <url> --token <token>
# gstack-brain-consumer list
# gstack-brain-consumer remove <name>
# gstack-brain-consumer test <name>
#
# Env:
# GSTACK_HOME — override ~/.gstack
set -euo pipefail
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
CONSUMERS_FILE="$GSTACK_HOME/consumers.json"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
ensure_file() {
mkdir -p "$GSTACK_HOME"
if [ ! -f "$CONSUMERS_FILE" ]; then
echo '{"consumers": []}' > "$CONSUMERS_FILE"
fi
}
get_remote_url() {
git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo ""
}
sub_add() {
local name="" url="" token=""
local positional=""
while [ $# -gt 0 ]; do
case "$1" in
--ingest-url) url="$2"; shift 2 ;;
--token) token="$2"; shift 2 ;;
--) shift; break ;;
-*) echo "Unknown flag: $1" >&2; exit 1 ;;
*) positional="$1"; shift ;;
esac
done
name="$positional"
if [ -z "$name" ] || [ -z "$url" ]; then
echo "Usage: gstack-brain-consumer add <name> --ingest-url <url> [--token <token>]" >&2
exit 1
fi
ensure_file
# Upsert in consumers.json, store token in gstack-config under `<name>_token`.
python3 - "$CONSUMERS_FILE" "$name" "$url" <<'PYEOF'
import sys, json
path, name, url = sys.argv[1:4]
try:
with open(path) as f:
data = json.load(f)
except Exception:
data = {"consumers": []}
entry = {"name": name, "ingest_url": url, "status": "unknown", "token_ref": f"{name}_token"}
cs = data.setdefault("consumers", [])
for i, c in enumerate(cs):
if c.get("name") == name:
cs[i] = entry
break
else:
cs.append(entry)
with open(path, "w") as f:
json.dump(data, f, indent=2)
f.write("\n")
print(f"registered consumer: {name}")
PYEOF
if [ -n "$token" ]; then
"$CONFIG_BIN" set "${name}_token" "$token"
echo "token stored: gstack-config get ${name}_token to retrieve"
fi
# Attempt registration with remote (HTTP POST).
sub_test "$name"
}
sub_list() {
if [ ! -f "$CONSUMERS_FILE" ]; then
echo '{"consumers": []}'
return 0
fi
cat "$CONSUMERS_FILE"
}
sub_remove() {
local name="${1:-}"
if [ -z "$name" ]; then
echo "Usage: gstack-brain-consumer remove <name>" >&2
exit 1
fi
ensure_file
python3 - "$CONSUMERS_FILE" "$name" <<'PYEOF'
import sys, json
path, name = sys.argv[1:3]
try:
with open(path) as f:
data = json.load(f)
except Exception:
data = {"consumers": []}
before = len(data.get("consumers", []))
data["consumers"] = [c for c in data.get("consumers", []) if c.get("name") != name]
after = len(data["consumers"])
with open(path, "w") as f:
json.dump(data, f, indent=2)
f.write("\n")
print(f"removed: {before - after} entry(ies)")
PYEOF
}
sub_test() {
local name="${1:-}"
if [ -z "$name" ]; then
echo "Usage: gstack-brain-consumer test <name>" >&2
exit 1
fi
ensure_file
# Look up the consumer by name.
local info
info=$(python3 - "$CONSUMERS_FILE" "$name" <<'PYEOF'
import sys, json
path, name = sys.argv[1:3]
try:
with open(path) as f:
data = json.load(f)
except Exception:
data = {"consumers": []}
for c in data.get("consumers", []):
if c.get("name") == name:
print(c.get("ingest_url", ""))
sys.exit(0)
sys.exit(1)
PYEOF
) || { echo "No such consumer: $name" >&2; exit 1; }
local url="$info"
local token
token=$("$CONFIG_BIN" get "${name}_token" 2>/dev/null || echo "")
if [ -z "$url" ] || [ -z "$token" ]; then
echo "consumer '$name': url or token missing; cannot test"
return 0
fi
local repo_url
repo_url=$(get_remote_url)
echo "Testing $name at ${url%/}/ingest-repo ..."
local resp
resp=$(curl -sS -X POST "${url%/}/ingest-repo" \
-H "Authorization: Bearer $token" \
-H "Content-Type: application/json" \
--data "{\"repo_url\":\"$repo_url\"}" \
-w "\n%{http_code}" 2>&1 || echo -e "\ncurl-error")
local code
code=$(echo "$resp" | tail -1)
if [ "$code" = "200" ] || [ "$code" = "201" ] || [ "$code" = "204" ]; then
echo "ok (HTTP $code)"
# Update status in consumers.json.
python3 - "$CONSUMERS_FILE" "$name" "ok" <<'PYEOF'
import sys, json
path, name, status = sys.argv[1:4]
with open(path) as f: data = json.load(f)
for c in data.get("consumers", []):
if c.get("name") == name:
c["status"] = status
with open(path, "w") as f: json.dump(data, f, indent=2); f.write("\n")
PYEOF
else
echo "failed (HTTP $code)"
python3 - "$CONSUMERS_FILE" "$name" "error" <<'PYEOF'
import sys, json
path, name, status = sys.argv[1:4]
with open(path) as f: data = json.load(f)
for c in data.get("consumers", []):
if c.get("name") == name:
c["status"] = status
with open(path, "w") as f: json.dump(data, f, indent=2); f.write("\n")
PYEOF
fi
}
case "${1:-}" in
add) shift; sub_add "$@" ;;
list) sub_list ;;
remove) shift; sub_remove "$@" ;;
test) shift; sub_test "$@" ;;
--help|-h|"") sed -n '2,20p' "$0" | sed 's/^# \{0,1\}//' ;;
*) echo "Unknown subcommand: $1" >&2; exit 1 ;;
esac

1
bin/gstack-brain-reader
View File

@ -1 +0,0 @@
gstack-brain-consumer

11
bin/gstack-brain-restore
View File

@ -30,6 +30,10 @@ set -euo pipefail
GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
# Egress receipt helpers (_receipted_git): fail-open for user-directed
# git ops against the user's own artifacts remote.
. "$SCRIPT_DIR/gstack-egress-lib.sh"
# v1.27.0.0+ canonical name; brain-remote is the legacy fallback during the
# migration window. The migration script renames the file in place.
if [ -f "$HOME/.gstack-artifacts-remote.txt" ]; then
@ -78,7 +82,9 @@ STAGING=$(mktemp -d "${TMPDIR:-/tmp}/gstack-brain-restore.XXXXXX")
trap 'rm -rf "$STAGING" 2>/dev/null' EXIT
echo "Cloning $REMOTE_URL to staging..."
if ! git clone --quiet "$REMOTE_URL" "$STAGING/repo" 2>/dev/null; then
RESTORE_HOST="${REMOTE_URL#*://}"; RESTORE_HOST="${RESTORE_HOST#*@}"; RESTORE_HOST="${RESTORE_HOST%%[/:]*}"
if ! _receipted_git open brain-restore "${RESTORE_HOST:-unknown}" brain-restore-clone "user ran gstack-brain-restore" \
bash -c 'git clone --quiet "$1" "$2" 2>/dev/null' _ "$REMOTE_URL" "$STAGING/repo"; then
echo "Clone failed. Check:" >&2
echo " - URL is correct: $REMOTE_URL" >&2
echo " - Auth: gh auth status (github) / glab auth status (gitlab)" >&2
@ -160,7 +166,8 @@ done
# ---- move .git into place ----
if [ -d "$GSTACK_HOME/.git" ]; then
# Existing .git with matching remote — just fetch + fast-forward.
git -C "$GSTACK_HOME" fetch origin >/dev/null 2>&1 || true
_receipted_git open brain-restore "${RESTORE_HOST:-unknown}" brain-restore-fetch "user ran gstack-brain-restore" \
bash -c 'git -C "$1" fetch origin >/dev/null 2>&1' _ "$GSTACK_HOME" || true
else
mv "$STAGING/repo/.git" "$GSTACK_HOME/.git"
fi

36
bin/gstack-brain-sync
View File

@ -32,6 +32,17 @@ DISCOVER_CURSOR="$GSTACK_HOME/.brain-discover-cursor"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
CONFIG_BIN="$SCRIPT_DIR/gstack-config"
# Egress receipt helpers (_receipted_git): receipt-before-send, fail-closed.
. "$SCRIPT_DIR/gstack-egress-lib.sh"
# origin host for receipt records (github.com etc).
remote_host() {
local url host
url=$(git -C "$GSTACK_HOME" remote get-url origin 2>/dev/null || echo "")
host="${url#*://}"; host="${host#*@}"; host="${host%%[/:]*}"
echo "${host:-unknown}"
}
# Remote-specific hint for auth errors (branch on origin URL).
remote_auth_hint() {
local url
@ -276,6 +287,21 @@ subcmd_once() {
exit 0
fi
# Egress receipt for the push, written BEFORE the commit consumes the
# queue (amendment C7 ordering): a refused receipt exits HERE, before any
# queue mutation or local commit, so the queue stays intact and the next
# run retries the whole drain. Content-free: git owns the bytes
# (sha256:null). Fail-closed.
local push_host receipt_err
push_host=$(remote_host)
if ! receipt_err=$(GSTACK_HOME="$GSTACK_HOME" "$SCRIPT_DIR/gstack-egress-receipt" write \
--sink brain-sync --host "$push_host" --class curated-memory-git-push \
--no-payload --consent "artifacts_sync_mode!=off" 2>&1 >/dev/null); then
write_status "push_failed" "EGRESS_RECEIPT_FAILED: receipt not writable; push refused (queue preserved)"
_gstack_egress_refusal "brain-sync push" "$(printf '%s' "$receipt_err" | head -c 300)"
exit 1
fi
# Commit with template message.
local n ts
n=$(wc -l < "$paths_file" | tr -d ' ')
@ -303,12 +329,16 @@ subcmd_once() {
exit 0
fi
# Try a fetch-and-merge + retry.
if git -C "$GSTACK_HOME" fetch origin 2>/dev/null; then
# Try a fetch-and-merge + retry. The fetch and the retry push are their
# own attempted-egress ops, each receipted fail-closed (a refusal falls
# through to the push_failed path below).
if GSTACK_HOME="$GSTACK_HOME" _receipted_git closed brain-sync "$push_host" curated-memory-git-fetch "artifacts_sync_mode!=off" \
bash -c 'git -C "$1" fetch origin 2>/dev/null' _ "$GSTACK_HOME"; then
local branch
branch=$(git -C "$GSTACK_HOME" rev-parse --abbrev-ref HEAD 2>/dev/null || echo main)
if git -C "$GSTACK_HOME" merge --no-edit "origin/$branch" >/dev/null 2>&1; then
if git -C "$GSTACK_HOME" push origin HEAD 2>/dev/null; then
if GSTACK_HOME="$GSTACK_HOME" _receipted_git closed brain-sync "$push_host" curated-memory-git-push "artifacts_sync_mode!=off" \
bash -c 'git -C "$1" push origin HEAD 2>/dev/null' _ "$GSTACK_HOME"; then
: > "$QUEUE"
date -u +%Y-%m-%dT%H:%M:%SZ > "$LAST_PUSH_FILE"
write_status "ok" "pushed $n file(s) after rebase"

10
bin/gstack-community-dashboard
View File

@ -12,6 +12,9 @@ set -uo pipefail
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
# Egress receipt helpers (_receipted_curl): fail-open for read-only stats.
. "$GSTACK_DIR/bin/gstack-egress-lib.sh"
# Source Supabase config if not overridden by env
if [ -z "${GSTACK_SUPABASE_URL:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
. "$GSTACK_DIR/supabase/config.sh"
@ -35,10 +38,11 @@ fi
# never as a healthy "Weekly active installs: 0".
TMPBODY="$(mktemp)"
trap 'rm -f "$TMPBODY"' EXIT
HTTP_CODE="$(curl -s --max-time 15 -w '%{http_code}' -o "$TMPBODY" \
SUPA_HOST="${SUPABASE_URL#*://}"; SUPA_HOST="${SUPA_HOST%%/*}"
HTTP_CODE="$(_receipted_curl open community-dashboard "$SUPA_HOST" community-pulse-fetch "user-invoked dashboard" --no-payload \
curl -s --max-time 15 -w '%{http_code}' -o "$TMPBODY" \
"${SUPABASE_URL}/functions/v1/community-pulse" \
-H "apikey: ${ANON_KEY}" \
2>/dev/null || true)"
-H "apikey: ${ANON_KEY}" || true)"
# curl prints its own 000 before a non-zero exit — a `|| echo` here would
# double it to "000000" in user-facing output. Normalize to the last 3 chars.
HTTP_CODE="$(printf '%s' "$HTTP_CODE" | tr -d '[:space:]' | tail -c 3)"

7
bin/gstack-context-bill Executable file
View File

@ -0,0 +1,7 @@
#!/usr/bin/env bun
// gstack-context-bill — token bill-of-materials for an installed skills tree.
// All behavior lives in lib/context-bill.ts; this is the CLI shim.
import { contextBillMain } from '../lib/context-bill';
process.exit(await contextBillMain(process.argv.slice(2)));

207
bin/gstack-egress Executable file
View File

@ -0,0 +1,207 @@
#!/usr/bin/env bun
/**
* gstack-egress — the auditor's view of the receipts ledger.
*
* list what gstack ATTEMPTED to send off this machine (one row per receipt)
* grants what CAN leave: every consent grant in force, where it lives,
* and the exact command that revokes it (pure config reads)
* verify recompute the hash chain; exit 3 on tamper
*
* THREAT MODEL: the ledger is forensic observability — it records ATTEMPTED
* egress so accidents are auditable; it is not an exfiltration control.
*
* The ledger is written by lib/egress-receipt.ts at every enumerated sink
* (see test/egress-receipt-wiring.test.ts for the pinned list).
*
* Home: GSTACK_HOME, legacy GSTACK_STATE_DIR, else ~/.gstack.
*/
import path from 'node:path';
import { spawnSync } from 'node:child_process';
import {
egressLedgerPath,
listReceipts,
resolveEgressHome,
verifyLedger,
} from '../lib/egress-receipt';
// import.meta.dir is Windows-safe; new URL(import.meta.url).pathname yields
// '/C:/...' with percent-encoded spaces there, which would make the
// gstack-config spawn silently fail and grants report every default. Matches
// the sibling bin/*.ts convention.
const BIN_DIR = import.meta.dir;
/**
* Strip control characters (incl. ANSI escapes) from ledger-derived strings
* before printing. A receipt's host/payloadClass can derive from semi-trusted
* input (a URL argument, a git remote); JSON.parse restores \u001b escapes to
* live ESC bytes, so an attacker-shaped receipt could hide or spoof rows in
* the very output an auditor reads. The hash chain is unaffected; this only
* sanitizes the human render.
*/
function sanitizeForDisplay(value: unknown): string {
return String(value).replace(/[\u0000-\u001F\u007F]/g, '');
}
function usage(message: string): never {
process.stderr.write(`gstack-egress: ${message}\n`);
process.stderr.write(
'Usage: gstack-egress list [--since <ISO>] [--host <host>] [--sink <sink>] [--json]\n' +
' gstack-egress verify [--json]\n' +
' gstack-egress grants [--json]\n',
);
process.exit(2);
}
function configGet(key: string): string {
const result = spawnSync(path.join(BIN_DIR, 'gstack-config'), ['get', key], {
encoding: 'utf-8',
});
return (result.stdout || '').trim();
}
function egressList(args: string[], home: string): number {
const values = new Map<string, string>();
let json = false;
for (let index = 0; index < args.length; index += 1) {
const arg = args[index];
if (['--since', '--host', '--sink'].includes(arg)) {
const value = args[++index];
if (value == null || value.startsWith('--')) usage(`${arg} requires a value`);
values.set(arg, value);
} else if (arg === '--json') {
json = true;
} else {
usage(`unknown option: ${arg}`);
}
}
const since = values.get('--since');
if (since != null && !Number.isFinite(Date.parse(since))) usage('--since must be an ISO timestamp');
let receipts = listReceipts(home);
if (since) receipts = receipts.filter((r) => Date.parse(r.ts) >= Date.parse(since));
if (values.has('--host')) receipts = receipts.filter((r) => r.host === values.get('--host'));
if (values.has('--sink')) receipts = receipts.filter((r) => r.sink === values.get('--sink'));
if (json) {
process.stdout.write(`${JSON.stringify(receipts, null, 2)}\n`);
return 0;
}
if (!receipts.length) {
process.stdout.write(`no receipts\nledger: ${egressLedgerPath(home)}\n`);
return 0;
}
for (const r of receipts) {
// sink/host/payload_class/consent can carry semi-trusted content — strip
// control chars so a crafted receipt can't spoof the auditor's view. ts,
// bytes, and sha256 are format-constrained at write time.
process.stdout.write(
`${r.ts} ${sanitizeForDisplay(r.sink)} -> ${sanitizeForDisplay(r.host)} ` +
`${sanitizeForDisplay(r.payload_class)} ${r.bytes}B ` +
`sha256=${r.sha256 ?? '(subprocess-owned)'} consent=${sanitizeForDisplay(r.consent)} ` +
`status=${r.status ? sanitizeForDisplay(r.status) : '-'}\n`,
);
}
process.stdout.write(`${receipts.length} receipt(s) ledger: ${egressLedgerPath(home)}\n`);
return 0;
}
function egressVerify(args: string[], home: string): number {
for (const arg of args) if (arg !== '--json') usage(`unknown option: ${arg}`);
const result = verifyLedger(home);
if (args.includes('--json')) {
process.stdout.write(`${JSON.stringify({ ...result, ledger: egressLedgerPath(home) }, null, 2)}\n`);
} else {
if (result.ok) {
process.stdout.write(`chain intact: ${result.count} line(s) verified\n`);
} else {
process.stdout.write(`TAMPER: chain broken at line ${result.brokenLine} (${result.reason})\n`);
}
if (result.sizeWarning) process.stdout.write(`${result.sizeWarning}\n`);
}
return result.ok ? 0 : 3;
}
interface Grant {
grant: string;
value: string;
granted: boolean;
detail: string;
file: string;
key: string;
revoke: string;
}
function egressGrants(args: string[], home: string): number {
for (const arg of args) if (arg !== '--json') usage(`unknown option: ${arg}`);
const configFile = path.join(home, 'config.yaml');
const telemetry = configGet('telemetry') || 'off';
const syncMode = configGet('artifacts_sync_mode') || 'off';
const repoVisibility = configGet('redact_repo_visibility') || 'unknown';
const prepushHook = configGet('redact_prepush_hook') || 'false';
const grants: Grant[] = [
{
grant: 'telemetry',
value: telemetry,
granted: telemetry !== 'off',
detail: 'anonymous and community tiers upload usage events to Supabase; off stays local-only',
file: configFile,
key: 'telemetry',
revoke: 'gstack-config set telemetry off',
},
{
grant: 'brain-sync',
value: syncMode,
granted: syncMode !== 'off' && syncMode !== '',
detail: 'git push of curated allowlisted memory to the user-configured artifacts remote',
file: configFile,
key: 'artifacts_sync_mode',
revoke: 'gstack-config set artifacts_sync_mode off',
},
{
grant: 'redact_repo_visibility',
value: repoVisibility,
granted: repoVisibility === 'public',
detail: 'redaction strictness assumption for external sinks; public gets per-finding confirmation',
file: configFile,
key: 'redact_repo_visibility',
revoke: 'gstack-config set redact_repo_visibility unknown (unknown = public-strict)',
},
{
grant: 'redact_prepush_hook',
value: prepushHook,
granted: prepushHook === 'true',
detail: 'opt-in git pre-push redaction scan; granted here means the guard is ON',
file: configFile,
key: 'redact_prepush_hook',
revoke: 'gstack-config set redact_prepush_hook false (disables the guard)',
},
];
if (args.includes('--json')) {
process.stdout.write(`${JSON.stringify(grants, null, 2)}\n`);
return 0;
}
for (const grant of grants) {
process.stdout.write(
`${grant.granted ? '[GRANTED]' : '[off] '} ${grant.grant}: ${grant.value}\n` +
` ${grant.detail}\n` +
` lives in: ${grant.file} (${grant.key})\n` +
` revoke: ${grant.revoke}\n`,
);
}
process.stdout.write("What was ATTEMPTED: 'gstack-egress list'. Chain check: 'gstack-egress verify'.\n");
return 0;
}
const [action, ...rest] = process.argv.slice(2);
const home = resolveEgressHome();
let code: number;
if (action === 'list') code = egressList(rest, home);
else if (action === 'verify') code = egressVerify(rest, home);
else if (action === 'grants') code = egressGrants(rest, home);
else usage(`unknown subcommand: ${action ?? '(none)'}`);
process.exit(code);

118
bin/gstack-egress-lib.sh Normal file
View File

@ -0,0 +1,118 @@
# gstack-egress-lib.sh — shared egress-receipt helpers for bash sinks.
#
# This file is NOT executable; source it:
#
# . "$(dirname "$0")/gstack-egress-lib.sh"
#
# THREAT MODEL: the egress ledger is forensic observability — it records
# ATTEMPTED egress so accidents are auditable; it is not an exfiltration
# control. Receipts are written before send, outcomes are best-effort, and
# fail-open callers can send unrecorded with a warning.
#
# Provides:
# _receipted_curl <closed|open> <sink> <host> <class> <consent> \
# (<payload-file>|--no-payload) <curl-cmd> [args...]
# — Writes an egress receipt BEFORE running the wrapped curl command.
# When a payload file is given, the receipt hashes that file and curl
# receives THE SAME FILE via an appended `--data-binary @file`, so the
# recorded sha256 matches the exact wire bytes (scan-at-sink precedent).
# The payload file is CONSUMED: the helper deletes it after the send
# (and on refusal) — callers need no cleanup trap for it.
# stdout is the wrapped command's stdout; returns its exit code.
# A best-effort outcome record (`exit:N`) is appended after the send;
# the receipt id is exported as _GSTACK_EGRESS_LAST_RECEIPT so callers
# can append a more specific outcome (e.g. the HTTP status).
#
# Fail policy (first argument, per call):
# closed — receipt failure REFUSES the send: nothing hits the network,
# return 3, problem/cause/fix message on stderr.
# open — receipt failure warns on stderr and proceeds unrecorded.
#
# _receipted_git <closed|open> <sink> <host> <class> <consent> <git-cmd> [args...]
# — Same contract for git-class ops: sha256:null receipt (a subprocess
# owns the bytes), no payload file, command runs unmodified.
#
# NO EXIT traps in this file, ever: callers (gstack-telemetry-sync) own
# their own EXIT traps and a trap set by a sourced library would clobber
# the caller's. All temp handling is immediate, per call.
# Self-locate without dirname (builtins only), so the lib works even under
# a stripped test PATH.
case "${BASH_SOURCE[0]}" in
*/*) _gstack_egress_lib_dir="$(cd "${BASH_SOURCE[0]%/*}" && pwd)" ;;
*) _gstack_egress_lib_dir="$(pwd)" ;;
esac
_gstack_egress_home() {
if [ -n "${GSTACK_HOME:-}" ]; then
printf '%s' "$GSTACK_HOME"
elif [ -n "${GSTACK_STATE_DIR:-}" ]; then
printf '%s' "$GSTACK_STATE_DIR"
else
printf '%s' "$HOME/.gstack"
fi
}
# Problem + cause + fix, in plain language (DX contract for every
# fail-closed refusal). $1 = sink, $2 = cause text from the receipt bridge.
_gstack_egress_refusal() {
local home
home="$(_gstack_egress_home)"
echo "gstack: $1 NOT sent — the egress receipt could not be written (${2:-unknown cause}). Fix: chmod -R u+w $home/security (or check GSTACK_HOME). What this is: gstack records everything it ATTEMPTS to send off-machine; see gstack-egress." >&2
}
# Shared core. $6 is a payload file path or --no-payload; the rest is the
# command to run. Payload files are deleted here (immediate, no traps).
_gstack_egress_run() {
local policy="$1" sink="$2" host="$3" class="$4" consent="$5" payload="$6"
shift 6
local bin="$_gstack_egress_lib_dir/gstack-egress-receipt"
local payload_flag=(--no-payload)
[ "$payload" != "--no-payload" ] && payload_flag=(--payload-file "$payload")
local receipt_id="" receipt_err="" err_file=""
err_file="$(mktemp "${TMPDIR:-/tmp}/gstack-egress-err-XXXXXX" 2>/dev/null)" || err_file=""
if [ -n "$err_file" ]; then
receipt_id="$("$bin" write --sink "$sink" --host "$host" --class "$class" \
"${payload_flag[@]}" --consent "$consent" 2>"$err_file")" || receipt_id=""
receipt_err="$(head -c 500 "$err_file" 2>/dev/null | tr '\n' ' ')"
rm -f "$err_file"
else
receipt_id="$("$bin" write --sink "$sink" --host "$host" --class "$class" \
"${payload_flag[@]}" --consent "$consent" 2>/dev/null)" || receipt_id=""
fi
_GSTACK_EGRESS_LAST_RECEIPT="$receipt_id"
if [ -z "$receipt_id" ]; then
if [ "$policy" = "closed" ]; then
_gstack_egress_refusal "$sink" "$receipt_err"
[ "$payload" != "--no-payload" ] && rm -f "$payload"
return 3
fi
echo "gstack: egress receipt could not be written for $sink (${receipt_err:-unknown cause}) — sending anyway (fail-open). gstack normally records everything it ATTEMPTS to send off-machine; see gstack-egress." >&2
fi
local status=0
if [ "$payload" = "--no-payload" ]; then
"$@" || status=$?
else
"$@" --data-binary @"$payload" || status=$?
rm -f "$payload"
fi
if [ -n "$receipt_id" ]; then
"$bin" outcome "$receipt_id" "exit:$status" 2>/dev/null || true
fi
return $status
}
_receipted_curl() {
_gstack_egress_run "$@"
}
_receipted_git() {
local policy="$1" sink="$2" host="$3" class="$4" consent="$5"
shift 5
_gstack_egress_run "$policy" "$sink" "$host" "$class" "$consent" --no-payload "$@"
}

84
bin/gstack-egress-receipt Executable file
View File

@ -0,0 +1,84 @@
#!/usr/bin/env bun
// gstack-egress-receipt — bun script that BRIDGES shell callers (the bash
// egress sinks: gstack-telemetry-sync, gstack-update-check, gstack-brain-sync,
// and the sourced helpers in gstack-egress-lib.sh) into lib/egress-receipt.ts.
//
// Usage:
// gstack-egress-receipt write --sink S --host H --class C \
// (--payload-file F | --no-payload) [--consent "key=value"]
// → prints the receipt id on stdout, exit 0.
// → exit 3 + "EGRESS_RECEIPT_FAILED: ..." on stderr when the receipt
// cannot be written. Fail-closed callers MUST then refuse the send.
//
// gstack-egress-receipt outcome <receipt-id> <status>
// → best-effort response-status record; never blocks anything.
//
// Home: GSTACK_HOME, legacy GSTACK_STATE_DIR, else ~/.gstack.
// The payload file is hashed as-is: pass the SAME file to curl (`-d @file`)
// so the receipt hash matches the exact bytes sent (scan-at-sink precedent).
import fs from 'node:fs';
import { EGRESS_RECEIPT_FAILED, sha256Hex, writeOutcome, writeReceipt } from '../lib/egress-receipt';
function parseArgs(args: string[], valueFlags: string[], boolFlags: string[]) {
const values = new Map<string, string>();
const flags = new Set<string>();
for (let index = 0; index < args.length; index += 1) {
const arg = args[index];
if (valueFlags.includes(arg)) {
const value = args[++index];
if (value == null) usage(`${arg} requires a value`);
values.set(arg, value);
} else if (boolFlags.includes(arg)) {
flags.add(arg);
} else {
usage(`unknown option: ${arg}`);
}
}
return { values, flags };
}
function usage(message: string): never {
process.stderr.write(`gstack-egress-receipt: ${message}\n`);
process.exit(2);
}
const [command, ...rest] = process.argv.slice(2);
if (command === 'write') {
const { values, flags } = parseArgs(rest,
['--sink', '--host', '--class', '--payload-file', '--consent'], ['--no-payload']);
const payloadFile = values.get('--payload-file');
if (!payloadFile && !flags.has('--no-payload')) usage('write requires --payload-file or --no-payload');
try {
let bytes = 0;
let sha256: string | null = null; // --no-payload = git-class op, subprocess owns the bytes
if (payloadFile) {
const payload = fs.readFileSync(payloadFile);
bytes = payload.byteLength;
sha256 = sha256Hex(payload);
}
const { id } = writeReceipt({
sink: values.get('--sink') as string,
host: values.get('--host') as string,
payloadClass: values.get('--class') as string,
bytes,
sha256,
consent: values.get('--consent') ?? 'unspecified',
});
process.stdout.write(`${id}\n`);
} catch (error) {
process.stderr.write(`${EGRESS_RECEIPT_FAILED}: ${(error as Error)?.message ?? error}\n`);
process.exit(3);
}
} else if (command === 'outcome') {
const [receipt, status, ...extra] = rest;
if (!receipt || !status || extra.length) usage('Usage: gstack-egress-receipt outcome <receipt-id> <status>');
try {
writeOutcome({ receipt, status });
} catch {
// Best-effort: the pre-send receipt is the invariant, the outcome is bookkeeping.
}
} else {
usage('Usage: gstack-egress-receipt write|outcome ...');
}

23
bin/gstack-gbrain-mcp-verify
View File

@ -43,6 +43,10 @@ URL="$1"
command -v curl >/dev/null 2>&1 || { echo "gstack-gbrain-mcp-verify: curl is required" >&2; exit 2; }
command -v jq >/dev/null 2>&1 || { echo "gstack-gbrain-mcp-verify: jq is required (brew install jq)" >&2; exit 2; }
# Egress receipt helpers (_receipted_curl): receipt-before-send, fail-closed.
. "$(cd "$(dirname "$0")" && pwd)/gstack-egress-lib.sh"
MCP_HOST=$(echo "$URL" | sed -E 's|^[a-z]+://([^/]+).*|\1|')
emit() {
# emit <status> <server_name> <server_version> <error_class> <error_text> <url_supported> <raw_body>
jq -n \
@ -73,15 +77,20 @@ INIT_BODY='{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVers
TMPBODY=$(mktemp -t gstack-mcp-verify.XXXXXX)
trap 'rm -f "$TMPBODY"' EXIT
# Receipted fail-closed: the payload file is hashed and handed to curl as
# the exact wire bytes. A refused receipt never hits the network — it lands
# in the NETWORK class below (curl never ran, no HTTP code).
INIT_PAYLOAD=$(mktemp -t gstack-mcp-init.XXXXXX)
printf '%s' "$INIT_BODY" > "$INIT_PAYLOAD"
set +e
HTTP_CODE=$(curl -s -o "$TMPBODY" -w '%{http_code}' \
HTTP_CODE=$(_receipted_curl closed gbrain-mcp-verify "$MCP_HOST" mcp-initialize-probe "user-invoked mcp verify" "$INIT_PAYLOAD" \
curl -s -o "$TMPBODY" -w '%{http_code}' \
--max-time 10 \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-H "Authorization: Bearer $GBRAIN_MCP_TOKEN" \
-d "$INIT_BODY" \
"$URL" 2>/dev/null)
"$URL")
CURL_EXIT=$?
set -e
@ -150,14 +159,18 @@ URL_SUPPORTED=false
TOOLS_BODY_FILE=$(mktemp -t gstack-mcp-tools.XXXXXX)
TOOLS_REQ='{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}'
# Receipted fail-closed like the initialize probe. A refused receipt skips
# the probe (nonzero TOOLS_EXIT) and the field stays false — best-effort.
TOOLS_PAYLOAD=$(mktemp -t gstack-mcp-tools-req.XXXXXX)
printf '%s' "$TOOLS_REQ" > "$TOOLS_PAYLOAD"
set +e
curl -s -o "$TOOLS_BODY_FILE" \
_receipted_curl closed gbrain-mcp-verify "$MCP_HOST" mcp-tools-list-probe "user-invoked mcp verify" "$TOOLS_PAYLOAD" \
curl -s -o "$TOOLS_BODY_FILE" \
--max-time 10 \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-H "Authorization: Bearer $GBRAIN_MCP_TOKEN" \
-d "$TOOLS_REQ" \
"$URL" >/dev/null 2>&1
TOOLS_EXIT=$?
set -e

25
bin/gstack-gbrain-supabase-provision
View File

@ -69,6 +69,12 @@ set -euo pipefail
SUPABASE_API_BASE="${SUPABASE_API_BASE:-https://api.supabase.com}"
API_VERSION="v1"
# Egress receipt helpers (_receipted_curl): receipt-before-send, fail-closed.
# The receipt hashes the request body only — the PAT (Authorization header)
# is never receipted or logged.
. "$(cd "$(dirname "$0")" && pwd)/gstack-egress-lib.sh"
SUPABASE_API_HOST="${SUPABASE_API_BASE#*://}"; SUPABASE_API_HOST="${SUPABASE_API_HOST%%/*}"
DEFAULT_WAIT_TIMEOUT=180
POLL_INTERVAL=5
CURL_TIMEOUT=30
@ -135,11 +141,24 @@ api_call() {
-H "Content-Type: application/json"
-H "User-Agent: gstack-gbrain-supabase-provision"
)
# Receipted fail-closed. The retry loop reuses $body_file across
# attempts, but the helper consumes its payload file — so each attempt
# hands it a fresh copy (hash still equals the exact wire bytes; the
# helper appends --data-binary @copy). Bodyless calls use --no-payload.
local payload_arg="--no-payload"
if [ -n "$body_file" ]; then
curl_args+=(--data-binary "@$body_file")
payload_arg=$(mktemp)
cp "$body_file" "$payload_arg"
fi
local status
if ! status=$(curl "${curl_args[@]}" "$url" 2>/dev/null); then
local status rc=0
status=$(_receipted_curl closed supabase-provision "$SUPABASE_API_HOST" "provision-api-call ($method $apipath)" "user ran gstack-gbrain-supabase-provision" "$payload_arg" \
curl "${curl_args[@]}" "$url") || rc=$?
if [ "$rc" -eq 3 ] && [ -z "$status" ]; then
# Egress receipt refused — the send never happened (the helper's
# problem/cause/fix message is already on stderr). Don't retry.
exit 8
fi
if [ "$rc" -ne 0 ]; then
# curl itself failed (network, timeout, etc.). Retry.
if [ "$attempt" -ge "$max_attempts" ]; then
die_net "network failure calling $method $apipath after $attempt attempts"

30
bin/gstack-gbrain-sync.ts
View File

@ -39,6 +39,7 @@ import "../lib/conductor-env-shim";
import { detectEngineTier, withErrorContext, canonicalizeRemote } from "../lib/gstack-memory-helpers";
import { ensureSourceRegistered, sourcePageCount, parseSourcesList, cycleCompleted, type CycleStatus } from "../lib/gbrain-sources";
import { detectAutopilot, decideSourceRemove, decideCodeSync } from "../lib/gbrain-guards";
import { writeReceipt } from "../lib/egress-receipt";
import { localEngineStatus, type LocalEngineStatus } from "../lib/gbrain-local-status";
import { buildGbrainEnv, spawnGbrain, execGbrainJson, NEEDS_SHELL_ON_WINDOWS } from "../lib/gbrain-exec";
import { checkOwnedStagingDir } from "../lib/staging-guard";
@ -67,7 +68,13 @@ interface CodeStageDetail {
source_path?: string;
page_count?: number | null;
last_imported?: string;
status?: "ok" | "skipped" | "failed" | "refused-autopilot" | "refused-reclone";
status?:
| "ok"
| "skipped"
| "failed"
| "refused-autopilot"
| "refused-reclone"
| "refused-egress-receipt";
}
interface StageResult {
@ -902,6 +909,27 @@ async function runCodeImport(args: CliArgs): Promise<StageResult> {
};
}
// Egress receipt BEFORE the code walk (fail-closed): the walk ships repo
// content to the user's gbrain DB, which may be a remote Postgres. The
// gbrain subprocess owns the wire bytes, so the receipt is content-free
// (destination + payload class only; sha256 null).
try {
writeReceipt({
sink: "gbrain-sync",
host: "gbrain-db (user-configured DATABASE_URL)",
payloadClass: `repo-code-index source=${sourceId} (sent by gbrain subprocess)`,
bytes: 0,
sha256: null,
consent: "gbrain setup consent + per-repo policy chokepoint (repoPolicyTier)",
});
} catch (err) {
return {
name: "code", ran: true, ok: false, duration_ms: Date.now() - t0,
summary: `EGRESS_RECEIPT_FAILED: ${(err as Error).message} — code sync refused`,
detail: { source_id: sourceId, source_path: root, status: "refused-egress-receipt" },
};
}
const walkResult = spawnGbrain(["sync", "--strategy", "code", "--source", sourceId], {
stdio: args.quiet ? ["ignore", "ignore", "ignore"] : ["ignore", "inherit", "inherit"],
timeout: codeTimeoutMs,

31
bin/gstack-memory-ingest.ts
View File

@ -65,6 +65,7 @@ import {
withErrorContext,
} from "../lib/gstack-memory-helpers";
import { execGbrainText, spawnGbrainAsync } from "../lib/gbrain-exec";
import { writeReceipt } from "../lib/egress-receipt";
import { checkOwnedStagingDir, STAGING_MARKER } from "../lib/staging-guard";
// ── Types ──────────────────────────────────────────────────────────────────
@ -1690,6 +1691,36 @@ async function ingestPass(args: CliArgs): Promise<BulkResult> {
// spawn, parent termination orphans the gbrain process (observed
// during 2026-05-10 cold-run testing — gbrain kept running 15 min
// after the orchestrator timed out).
//
// Egress receipt BEFORE the import (fail-closed): the gbrain DB may be a
// remote Postgres, so the ingest is a potential off-machine send. The
// gbrain subprocess owns the wire bytes (content-free receipt, sha256
// null). The remote-http branch above stages locally only — its egress
// happens in gstack-brain-sync, which writes its own receipt at the push.
try {
writeReceipt({
sink: "memory-ingest",
host: "gbrain-db (user-configured DATABASE_URL)",
payloadClass: `transcript-pages count=${staging.written} (sent by gbrain subprocess)`,
bytes: 0,
sha256: null,
consent: "gbrain setup consent (/setup-gbrain)",
});
} catch (err) {
const msg = `EGRESS_RECEIPT_FAILED: ${(err as Error).message} — ingest refused`;
console.error(`[memory-ingest] ERR: ${msg}`);
failed += prep.prepared.length;
return {
written: 0,
skipped_secret: prep.skippedSecret,
skipped_dedup: prep.skippedDedup,
skipped_unattributed: prep.skippedUnattributed,
failed,
duration_ms: Date.now() - t0,
partial_pages: prep.partialPages,
system_error: msg,
};
}
const importResult = await runGbrainImport(stagingDir, resolveImportTimeoutMs());
const stdout = importResult.stdout || "";

10
bin/gstack-security-dashboard
View File

@ -17,6 +17,9 @@ set -uo pipefail
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
# Egress receipt helpers (_receipted_curl): fail-open for read-only stats.
. "$GSTACK_DIR/bin/gstack-egress-lib.sh"
# Source Supabase config
if [ -z "${GSTACK_SUPABASE_URL:-}" ] && [ -f "$GSTACK_DIR/supabase/config.sh" ]; then
. "$GSTACK_DIR/supabase/config.sh"
@ -46,10 +49,11 @@ fi
# surface are indistinguishable from good news.
TMPBODY="$(mktemp)"
trap 'rm -f "$TMPBODY"' EXIT
HTTP_CODE="$(curl -s --max-time 15 -w '%{http_code}' -o "$TMPBODY" \
SUPA_HOST="${SUPABASE_URL#*://}"; SUPA_HOST="${SUPA_HOST%%/*}"
HTTP_CODE="$(_receipted_curl open security-dashboard "$SUPA_HOST" community-pulse-fetch "user-invoked dashboard" --no-payload \
curl -s --max-time 15 -w '%{http_code}' -o "$TMPBODY" \
"${SUPABASE_URL}/functions/v1/community-pulse" \
-H "apikey: ${ANON_KEY}" \
2>/dev/null || true)"
-H "apikey: ${ANON_KEY}" || true)"
# curl prints its own 000 before a non-zero exit — a `|| echo` here would
# double it to "000000" in user-facing output. Normalize to the last 3 chars.
HTTP_CODE="$(printf '%s' "$HTTP_CODE" | tr -d '[:space:]' | tail -c 3)"

9
bin/gstack-session-update
View File

@ -11,6 +11,10 @@ set +e
GSTACK_DIR="${GSTACK_DIR:-$HOME/.claude/skills/gstack}"
STATE_DIR="${GSTACK_STATE_DIR:-$HOME/.gstack}"
# Egress receipt helpers (_receipted_git): fail-open — an update pull must
# never block a session over a receipt hiccup.
. "$(cd "$(dirname "$0")" && pwd)/gstack-egress-lib.sh"
THROTTLE_FILE="$STATE_DIR/.last-session-update"
LOCK_DIR="$STATE_DIR/.setup-lock"
LOG_FILE="$STATE_DIR/analytics/session-update.log"
@ -76,7 +80,10 @@ fi
# ── Pull latest ──
OLD_HEAD=$(git -C "$GSTACK_DIR" rev-parse HEAD 2>/dev/null)
git -C "$GSTACK_DIR" pull --ff-only -q 2>/dev/null
UPDATE_URL=$(git -C "$GSTACK_DIR" remote get-url origin 2>/dev/null || echo "")
UPDATE_HOST="${UPDATE_URL#*://}"; UPDATE_HOST="${UPDATE_HOST#*@}"; UPDATE_HOST="${UPDATE_HOST%%[/:]*}"
GSTACK_HOME="$STATE_DIR" _receipted_git open session-update "${UPDATE_HOST:-unknown}" gstack-self-update-pull "auto_upgrade=true" \
bash -c 'git -C "$1" pull --ff-only -q 2>/dev/null' _ "$GSTACK_DIR"
PULL_EXIT=$?
NEW_HEAD=$(git -C "$GSTACK_DIR" rev-parse HEAD 2>/dev/null)

28
bin/gstack-telemetry-sync
View File

@ -13,6 +13,9 @@ set -uo pipefail
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
STATE_DIR="${GSTACK_STATE_DIR:-$HOME/.gstack}"
# Egress receipt helpers (_receipted_curl): receipt-before-send, fail-closed.
. "$GSTACK_DIR/bin/gstack-egress-lib.sh"
ANALYTICS_DIR="$STATE_DIR/analytics"
JSONL_FILE="$ANALYTICS_DIR/skill-usage.jsonl"
CURSOR_FILE="$ANALYTICS_DIR/.last-sync-line"
@ -114,12 +117,31 @@ RESP_FILE="$(mktemp "${TMPDIR:-/tmp}/gstack-sync-XXXXXX")" || {
exit 0
}
trap 'rm -f "$RESP_FILE"' EXIT
HTTP_CODE="$(curl -s -w '%{http_code}' --max-time 10 \
# Egress receipt BEFORE the send (fail-closed): the batch is written to a
# temp file, the receipt hashes those exact bytes, and curl sends the SAME
# file. On refusal nothing is sent and the cursor does not advance, so the
# batch stays buffered and the next run retries. The helper consumes the
# payload file.
PAYLOAD_FILE="$(mktemp "${TMPDIR:-/tmp}/gstack-sync-payload-XXXXXX")" || {
echo "gstack-telemetry-sync: mktemp failed — skipping this run" >&2
exit 0
}
printf '%s' "$BATCH" > "$PAYLOAD_FILE"
DEST_HOST="${SUPABASE_URL#*://}"
DEST_HOST="${DEST_HOST%%/*}"
HTTP_CODE="$(GSTACK_HOME="$STATE_DIR" _receipted_curl closed telemetry-sync "$DEST_HOST" telemetry-events "telemetry=$TIER" "$PAYLOAD_FILE" \
curl -s -w '%{http_code}' --max-time 10 \
-X POST "${SUPABASE_URL}/functions/v1/telemetry-ingest" \
-H "Content-Type: application/json" \
-H "apikey: ${ANON_KEY}" \
-o "$RESP_FILE" \
-d "$BATCH" 2>/dev/null || echo "000")"
-o "$RESP_FILE" || echo "000")"
# Best-effort response-status record for the receipt above (overrides the
# helper's generic exit-code outcome with the HTTP status).
if [ -n "${_GSTACK_EGRESS_LAST_RECEIPT:-}" ]; then
GSTACK_HOME="$STATE_DIR" "$GSTACK_DIR/bin/gstack-egress-receipt" outcome "$_GSTACK_EGRESS_LAST_RECEIPT" "$HTTP_CODE" 2>/dev/null || true
fi
# ─── Update cursor on success (2xx) ─────────────────────────
case "$HTTP_CODE" in

52
bin/gstack-update-check
View File

@ -15,6 +15,29 @@ set -euo pipefail
GSTACK_DIR="${GSTACK_DIR:-$(cd "$(dirname "$0")/.." && pwd)}"
STATE_DIR="${GSTACK_STATE_DIR:-$HOME/.gstack}"
# Egress receipt helpers (_receipted_curl / _receipted_git). Update checks
# are fail-OPEN: a receipt hiccup warns but never blocks the version check.
. "$GSTACK_DIR/bin/gstack-egress-lib.sh"
# _receipted_version_fetch <url>
# Receipted bodyless GET for a VERSION file (fail-open). Local reads
# (file://, or a URL with no network host) never leave the machine — they
# are not egress, so they get no receipt.
_receipted_version_fetch() {
local url="$1" host scheme
scheme="${url%%://*}"
host="${url#*://}"; host="${host%%/*}"
case "$scheme" in
http|https|git|ssh|ftp|ftps)
GSTACK_HOME="$STATE_DIR" _receipted_curl open update-check "$host" version-fetch "update_check!=false" --no-payload \
curl -sf --max-time 5 "$url" || true
;;
*)
curl -sf --max-time 5 "$url" 2>/dev/null || true
;;
esac
}
CACHE_FILE="$STATE_DIR/last-update-check"
MARKER_FILE="$STATE_DIR/just-upgraded-from"
SNOOZE_FILE="$STATE_DIR/update-snoozed"
@ -172,12 +195,19 @@ _SUPA_KEY="${GSTACK_SUPABASE_ANON_KEY:-}"
_TEL_TIER="$("$GSTACK_DIR/bin/gstack-config" get telemetry 2>/dev/null || true)"
if [ -n "$_SUPA_URL" ] && [ -n "$_SUPA_KEY" ] && [ "${_TEL_TIER:-off}" != "off" ]; then
_OS="$(uname -s | tr '[:upper:]' '[:lower:]')"
curl -sf --max-time 5 \
-X POST "${_SUPA_URL}/functions/v1/update-check" \
-H "Content-Type: application/json" \
-H "apikey: ${_SUPA_KEY}" \
-d "{\"version\":\"$LOCAL\",\"os\":\"$_OS\"}" \
>/dev/null 2>&1 &
# Receipted (fail-open) and fully backgrounded: the receipt write happens
# inside the background subshell, so it can never block this script's exit.
_PING_FILE="$(mktemp "${TMPDIR:-/tmp}/gstack-update-ping-XXXXXX" 2>/dev/null)" || _PING_FILE=""
if [ -n "$_PING_FILE" ]; then
printf '{"version":"%s","os":"%s"}' "$LOCAL" "$_OS" > "$_PING_FILE"
_SUPA_HOST="${_SUPA_URL#*://}"; _SUPA_HOST="${_SUPA_HOST%%/*}"
GSTACK_HOME="$STATE_DIR" _receipted_curl open update-check "$_SUPA_HOST" update-check-ping "telemetry=$_TEL_TIER" "$_PING_FILE" \
curl -sf --max-time 5 \
-X POST "${_SUPA_URL}/functions/v1/update-check" \
-H "Content-Type: application/json" \
-H "apikey: ${_SUPA_KEY}" \
>/dev/null 2>&1 &
fi
fi
# Resolve VERSION via a SHA-pinned raw URL. GitHub's branch-raw CDN
@ -193,12 +223,14 @@ REMOTE=""
if [ -z "${GSTACK_REMOTE_URL:-}" ]; then
# Disable credential prompts and apply a 5-second low-speed timeout so a
# flaky network or captive portal can't hang every skill preamble.
_LSR_LINE="$(GIT_TERMINAL_PROMPT=0 GIT_HTTP_LOW_SPEED_LIMIT=1000 GIT_HTTP_LOW_SPEED_TIME=5 \
git ls-remote "$REMOTE_REPO" refs/heads/main 2>/dev/null || true)"
_REPO_HOST="${REMOTE_REPO#*://}"; _REPO_HOST="${_REPO_HOST#*@}"; _REPO_HOST="${_REPO_HOST%%[/:]*}"
_LSR_LINE="$(GSTACK_HOME="$STATE_DIR" _receipted_git open update-check "${_REPO_HOST:-unknown}" version-ls-remote "update_check!=false" \
bash -c 'GIT_TERMINAL_PROMPT=0 GIT_HTTP_LOW_SPEED_LIMIT=1000 GIT_HTTP_LOW_SPEED_TIME=5 \
git ls-remote "$1" refs/heads/main 2>/dev/null' _ "$REMOTE_REPO" || true)"
_REMOTE_SHA="$(echo "$_LSR_LINE" | awk '{print $1}')"
if echo "$_REMOTE_SHA" | grep -qE '^[0-9a-f]{40}$'; then
_SHA_URL="https://raw.githubusercontent.com/garrytan/gstack/${_REMOTE_SHA}/VERSION"
REMOTE="$(curl -sf --max-time 5 "$_SHA_URL" 2>/dev/null || true)"
REMOTE="$(_receipted_version_fetch "$_SHA_URL")"
fi
fi
@ -206,7 +238,7 @@ fi
# network, mirror without refs/heads/main) or when GSTACK_REMOTE_URL was
# explicitly overridden.
if [ -z "$REMOTE" ]; then
REMOTE="$(curl -sf --max-time 5 "$REMOTE_URL" 2>/dev/null || true)"
REMOTE="$(_receipted_version_fetch "$REMOTE_URL")"
fi
REMOTE="$(echo "$REMOTE" | tr -d '[:space:]')"

45
browse/scripts/extension-id.ts Executable file
View File

@ -0,0 +1,45 @@
#!/usr/bin/env bun
/**
* Derive the Chrome extension ID from the "key" field in
* extension/manifest.json.
*
* Chrome computes an extension's ID as the first 16 bytes of the SHA-256
* hash of the DER-encoded public key, with each hex nibble mapped from
* 0-9a-f to a-p (the "mpdecimal" alphabet). Pinning the public key in the
* manifest pins the ID, which lets the browse server verify the Origin
* header on POST /extension-token against a single known extension
* identity (GSTACK_EXTENSION_ID in browse/src/server.ts).
*
* The private half of the keypair is intentionally NOT in the repo — the
* extension is loaded unpacked (or baked into Browser.app), so only the
* public key is needed to pin the ID. Regenerating the keypair changes
* the ID and requires updating both the manifest "key" and the
* GSTACK_EXTENSION_ID constant.
*
* Usage: bun browse/scripts/extension-id.ts [path/to/manifest.json]
*/
import { createHash } from 'node:crypto';
import * as fs from 'node:fs';
import * as path from 'node:path';
const manifestPath = process.argv[2]
?? path.join(import.meta.dir, '../../extension/manifest.json');
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf-8'));
if (typeof manifest.key !== 'string' || manifest.key.length === 0) {
console.error(`No "key" field in ${manifestPath}`);
process.exit(1);
}
export function extensionIdFromPublicKey(publicKeyBase64: string): string {
const der = Buffer.from(publicKeyBase64, 'base64');
const hex = createHash('sha256').update(der).digest('hex').slice(0, 32);
let id = '';
for (const c of hex) {
id += String.fromCharCode('a'.charCodeAt(0) + parseInt(c, 16));
}
return id;
}
console.log(extensionIdFromPublicKey(manifest.key));

4
browse/src/browser-manager.ts
View File

@ -1561,8 +1561,8 @@ export class BrowserManager {
if (extensionPath) {
launchArgs.push(`--disable-extensions-except=${extensionPath}`);
launchArgs.push(`--load-extension=${extensionPath}`);
// Auth token is served via /health endpoint now (no file write needed).
// Extension reads token from /health on connect.
// Auth token is served via POST /extension-token (pinned-origin
// bootstrap, no file write needed). /health is liveness-only.
console.log(`[browse] Handoff: loading extension from ${extensionPath}`);
} else {
console.log('[browse] Handoff: extension not found — headed mode without side panel');

102
browse/src/server.ts
View File

@ -49,6 +49,7 @@ import { isProcessAlive } from './error-handling';
import { sanitizeBody, stripLoneSurrogateEscapes } from './sanitize';
import { startSocksBridge, testUpstream, type BridgeHandle } from './socks-bridge';
import { parseProxyConfig, toUpstreamConfig, ProxyConfigError } from './proxy-config';
import { writeReceipt } from '../../lib/egress-receipt';
import { redactProxyUrl } from './proxy-redact';
import { shouldSpawnXvfb, pickFreeDisplay, spawnXvfb, xvfbInstallHint, type XvfbHandle } from './xvfb';
import { logTunnelDenial } from './tunnel-denial-log';
@ -305,6 +306,17 @@ const TUNNEL_PATHS = new Set<string>([
'/sidebar-chat',
]);
/**
* The gstack sidebar extension's pinned Chrome extension ID. Derived from
* the "key" field in extension/manifest.json (first 16 bytes of SHA-256 of
* the DER public key, hex nibbles mapped 0-9a-f → a-p). Reproduce with:
* bun browse/scripts/extension-id.ts
* POST /extension-token releases AUTH_TOKEN only to an Origin of exactly
* `chrome-extension://<this id>`. If the manifest keypair is ever rotated,
* this constant must be updated in the same commit.
*/
export const GSTACK_EXTENSION_ID = 'dgbkdbjebeiblbajiilljmhjdpmiglep';
/**
* Commands reachable via POST /command over the tunnel surface. A paired
* remote agent can drive the browser (goto, click, text, etc.) but cannot
@ -1769,7 +1781,51 @@ export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
);
}
// Health check — no auth required, does NOT reset idle timer
// ─── POST /extension-token — pinned-origin token bootstrap ──────
//
// The ONLY endpoint that hands out AUTH_TOKEN. GET /health used to
// carry the token (headed mode + any chrome-extension:// Origin),
// which meant ANY extension — or any localhost caller in headed
// mode — could read the root token. Now the token is released only
// to the one extension identity we ship: the Origin header must be
// exactly `chrome-extension://<GSTACK_EXTENSION_ID>`, where the ID
// is pinned by the "key" field in extension/manifest.json (derive
// it with `bun browse/scripts/extension-id.ts`). Chrome sets Origin
// on cross-origin POSTs from extension contexts and web pages
// cannot forge a chrome-extension:// Origin.
//
// Local listener only: NEVER added to TUNNEL_PATHS, so the tunnel
// surface 404s it by default-deny.
if (url.pathname === '/extension-token' && req.method === 'POST') {
// Defense-in-depth alongside the 127.0.0.1 bind: a DNS-rebinding
// page can't present a localhost Host header. Host arrives as
// '127.0.0.1:34567', so parse out the hostname — never compare
// the raw header (which carries the port) against a literal.
let hostname: string | null = null;
try {
hostname = new URL(`http://${req.headers.get('host') ?? ''}`).hostname;
} catch (err) {
if (!(err instanceof TypeError)) throw err; // TypeError = malformed Host
}
const originOk =
req.headers.get('origin') === `chrome-extension://${GSTACK_EXTENSION_ID}`;
const hostOk = hostname === '127.0.0.1' || hostname === 'localhost';
if (!originOk || !hostOk) {
// No detail in the body — don't teach a probing caller which
// check failed.
return new Response(JSON.stringify({ error: 'Forbidden' }), {
status: 403, headers: { 'Content-Type': 'application/json' },
});
}
return new Response(JSON.stringify({ token: authToken }), {
status: 200, headers: { 'Content-Type': 'application/json' },
});
}
// Health check — no auth required, does NOT reset idle timer.
// NEVER carries a token in any mode: token bootstrap is
// POST /extension-token (pinned extension Origin) and shell auth
// is POST /pty-session. Liveness/status only.
if (url.pathname === '/health') {
const healthy = await browserManager.isHealthy();
return new Response(JSON.stringify({
@ -1777,14 +1833,6 @@ export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
mode: browserManager.getConnectionMode(),
uptime: Math.floor((Date.now() - startTime) / 1000),
tabs: browserManager.getTabCount(),
// Auth token for extension bootstrap. Safe: /health is localhost-only.
// Previously served unconditionally, but that leaks the token if the
// server is tunneled to the internet (ngrok, SSH tunnel).
// In headed mode the server is always local, so return token unconditionally
// (fixes Playwright Chromium extensions that don't send Origin header).
...(browserManager.getConnectionMode() === 'headed' ||
req.headers.get('origin')?.startsWith('chrome-extension://')
? { token: authToken } : {}),
// The chat queue is gone — Terminal pane is the sole sidebar
// surface. Keep `chatEnabled: false` so any older extension
// build still treats the chat input as disabled.
@ -2309,7 +2357,7 @@ export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
// Dual-listener model: binds a SECOND Bun.serve listener on an
// ephemeral 127.0.0.1 port dedicated to tunnel traffic, then points
// ngrok.forward() at THAT port. The existing local listener (which
// serves /health+token, /cookie-picker, /inspector/*, welcome, etc.)
// serves /extension-token, /cookie-picker, /inspector/*, welcome, etc.)
// is never exposed to ngrok.
//
// Hard fail if the tunnel listener bind fails — NEVER fall back to
@ -2375,6 +2423,19 @@ export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
const forwardOpts: any = { addr: tunnelPort, authtoken };
if (domain) forwardOpts.domain = domain;
// Egress receipt BEFORE the tunnel session opens, fail-closed: a
// writeReceipt failure lands in this catch, which tears the tunnel
// listener back down and refuses the start. One receipt per session
// open; browse command behavior over the tunnel is unchanged.
writeReceipt({
sink: 'browse-tunnel',
host: domain || 'connect.ngrok-agent.com',
payloadClass: 'tunnel-session-open (scoped-token browser-command surface)',
bytes: 0,
sha256: null,
consent: 'pair_agent=on',
});
tunnelListener = await ngrok.forward(forwardOpts);
tunnelUrl = tunnelListener.url();
tunnelServer = boundTunnel;
@ -2794,11 +2855,10 @@ export function buildFetchHandler(cfg: ServerConfig): ServerHandle {
// GET /memory — diagnostic snapshot (auth required, does NOT reset idle).
// Same auth model as /activity/stream and /inspector/events: Bearer header
// OR view-only SSE-session cookie. Does NOT extend /health (which already
// leaks AUTH_TOKEN to any localhost caller in headed mode — see TODOS.md
// "Audit /health token distribution"); a separate endpoint with the
// standard SSE auth keeps the future /health fix from cascading into the
// sidebar footer poll.
// OR view-only SSE-session cookie. Does NOT extend /health (which is
// unauthenticated liveness-only — token bootstrap moved to the pinned
// POST /extension-token); a separate endpoint with the standard SSE auth
// keeps /health free of anything worth stealing.
if (url.pathname === '/memory' && req.method === 'GET') {
const cookieToken = extractSseCookie(req);
if (!validateAuth(req) && !validateSseSessionToken(cookieToken)) {
@ -3084,6 +3144,18 @@ export async function start() {
const forwardOpts: any = { addr: tunnelPort, authtoken };
if (domain) forwardOpts.domain = domain;
// Egress receipt BEFORE the tunnel session opens, fail-closed: a
// writeReceipt failure lands in this catch, which cleans up the
// listener and skips the tunnel (same as any other startup failure).
writeReceipt({
sink: 'browse-tunnel',
host: domain || 'connect.ngrok-agent.com',
payloadClass: 'tunnel-session-open (scoped-token browser-command surface)',
bytes: 0,
sha256: null,
consent: 'pair_agent=on (BROWSE_TUNNEL=1)',
});
tunnelListener = await ngrok.forward(forwardOpts);
tunnelUrl = tunnelListener.url();
tunnelServer = boundTunnel;

2
browse/test/dual-listener.test.ts
View File

@ -57,7 +57,7 @@ describe('Tunnel path allowlist', () => {
const paths = extractSetContents(SERVER_SRC, 'TUNNEL_PATHS');
// These must never be on the tunnel surface
const forbidden = [
'/health', '/welcome', '/cookie-picker',
'/health', '/extension-token', '/welcome', '/cookie-picker',
'/inspector', '/inspector/pick', '/inspector/events', '/inspector/style',
'/tunnel/start', '/tunnel/stop',
'/pair', '/token', '/refs',

178
browse/test/extension-token.test.ts Normal file
View File

@ -0,0 +1,178 @@
/**
* Live behavioral tests for the v1.62 token-bootstrap contract:
*
* - GET /health NEVER carries a token — not in headed mode, not for a
* chrome-extension:// Origin (the two pre-v1.62 carve-outs). IRON-RULE
* regression tests.
* - POST /extension-token releases the token ONLY to the pinned extension
* Origin (chrome-extension://GSTACK_EXTENSION_ID) with a loopback Host.
* - Host arrives with a port ('127.0.0.1:34567') and must be parsed to a
* hostname, not compared literally (amendment C9). 'localhost:34567'
* is accepted too.
* - The tunnel surface 404s /extension-token (not in TUNNEL_PATHS).
*
* Uses the buildFetchHandler factory (same pattern as server-factory.test.ts)
* so no listener/browser is needed. Real-HTTP coverage (Host header set by
* the network stack) lives in pair-agent-e2e.test.ts.
*/
import { describe, test, expect, beforeEach } from 'bun:test';
import * as crypto from 'crypto';
import {
buildFetchHandler,
GSTACK_EXTENSION_ID,
type ServerConfig,
} from '../src/server';
import { __resetRegistry } from '../src/token-registry';
import { BrowserManager } from '../src/browser-manager';
import { resolveConfig } from '../src/config';
const PINNED_ORIGIN = `chrome-extension://${GSTACK_EXTENSION_ID}`;
function makeConfig(overrides: Partial<ServerConfig> = {}): ServerConfig {
const token = 'ext-token-test-' + crypto.randomBytes(16).toString('hex');
return {
authToken: token,
browsePort: 34567,
idleTimeoutMs: 1_800_000,
config: resolveConfig(),
browserManager: new BrowserManager(),
startTime: Date.now(),
...overrides,
};
}
function headedBrowserManager(): BrowserManager {
const bm = new BrowserManager();
// connectionMode is private; force the headed value the old /health
// carve-out keyed on.
(bm as any).connectionMode = 'headed';
return bm;
}
function tokenRequest(headers: Record<string, string>): Request {
// Direct handler invocation — no network stack to synthesize Host, so
// every test sets it explicitly (Bun.serve always delivers one).
return new Request('http://127.0.0.1:34567/extension-token', {
method: 'POST',
headers,
});
}
describe('GET /health never carries a token (IRON RULE)', () => {
beforeEach(() => __resetRegistry());
test('headed mode: no token field in the body', async () => {
const handle = buildFetchHandler(makeConfig({ browserManager: headedBrowserManager() }));
const resp = await handle.fetchLocal(new Request('http://127.0.0.1:34567/health'), null);
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.token).toBeUndefined();
expect(body.mode).toBe('headed');
});
test('chrome-extension Origin (even the pinned one): no token field', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(new Request('http://127.0.0.1:34567/health', {
headers: { Origin: PINNED_ORIGIN },
}), null);
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.token).toBeUndefined();
});
test('headed mode AND pinned chrome-extension Origin together: still no token', async () => {
const handle = buildFetchHandler(makeConfig({ browserManager: headedBrowserManager() }));
const resp = await handle.fetchLocal(new Request('http://127.0.0.1:34567/health', {
headers: { Origin: PINNED_ORIGIN },
}), null);
const body = await resp.json() as any;
expect(body.token).toBeUndefined();
});
});
describe('POST /extension-token pinned-origin bootstrap', () => {
beforeEach(() => __resetRegistry());
test('pinned Origin + Host with port → 200 with the token', async () => {
const cfg = makeConfig();
const handle = buildFetchHandler(cfg);
const resp = await handle.fetchLocal(tokenRequest({
Origin: PINNED_ORIGIN,
Host: '127.0.0.1:34567',
}), null);
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.token).toBe(cfg.authToken);
});
test("Host 'localhost:34567' is accepted too (C9 hostname parse)", async () => {
const cfg = makeConfig();
const handle = buildFetchHandler(cfg);
const resp = await handle.fetchLocal(tokenRequest({
Origin: PINNED_ORIGIN,
Host: 'localhost:34567',
}), null);
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.token).toBe(cfg.authToken);
});
test('wrong extension Origin → 403, no token, no detail', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(tokenRequest({
Origin: 'chrome-extension://aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
Host: '127.0.0.1:34567',
}), null);
expect(resp.status).toBe(403);
const body = await resp.json() as any;
expect(body.token).toBeUndefined();
// No detail about WHICH check failed
expect(JSON.stringify(body)).not.toContain('origin');
expect(JSON.stringify(body)).not.toContain('host');
});
test('missing Origin → 403', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(tokenRequest({
Host: '127.0.0.1:34567',
}), null);
expect(resp.status).toBe(403);
});
test('web-page Origin → 403 (DNS-rebinding page cannot mint a token)', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(tokenRequest({
Origin: 'http://evil.example.com',
Host: '127.0.0.1:34567',
}), null);
expect(resp.status).toBe(403);
});
test('non-loopback Host → 403 even with the pinned Origin', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(tokenRequest({
Origin: PINNED_ORIGIN,
Host: 'evil.example.com:34567',
}), null);
expect(resp.status).toBe(403);
});
test('malformed Host → 403, not a crash', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchLocal(tokenRequest({
Origin: PINNED_ORIGIN,
Host: ':::not a host:::',
}), null);
expect(resp.status).toBe(403);
});
test('tunnel surface 404s /extension-token (not in TUNNEL_PATHS)', async () => {
const handle = buildFetchHandler(makeConfig());
const resp = await handle.fetchTunnel(tokenRequest({
Origin: PINNED_ORIGIN,
Host: '127.0.0.1:34567',
}), null);
expect(resp.status).toBe(404);
});
});

35
browse/test/pair-agent-e2e.test.ts
View File

@ -22,6 +22,7 @@ import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { GSTACK_EXTENSION_ID } from '../src/server';
const ROOT = path.resolve(import.meta.dir, '../..');
const SERVER_ENTRY = path.join(ROOT, 'browse/src/server.ts');
@ -94,22 +95,44 @@ describe('pair-agent flow end-to-end (HTTP only, no ngrok)', () => {
if (daemon) killDaemon(daemon);
});
test('GET /health returns daemon status and includes token for chrome-extension origin', async () => {
test('GET /health returns daemon status and NEVER includes a token (even for chrome-extension origins)', async () => {
const resp = await fetch(`${daemon.baseUrl}/health`, {
headers: { Origin: 'chrome-extension://test-extension-id' },
headers: { Origin: `chrome-extension://${GSTACK_EXTENSION_ID}` },
});
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.status).toBeDefined();
// Extension bootstrap — local listener delivers the token
expect(body.token).toBe(daemon.token);
// v1.62: token bootstrap moved to POST /extension-token. /health is
// liveness-only in every mode.
expect(body.token).toBeUndefined();
});
test('GET /health without chrome-extension origin does NOT include token', async () => {
test('GET /health without origin does NOT include token', async () => {
const resp = await fetch(`${daemon.baseUrl}/health`);
expect(resp.status).toBe(200);
const body = await resp.json() as any;
// Headless mode + no chrome-extension origin → token withheld
expect(body.token).toBeUndefined();
});
test('POST /extension-token with pinned Origin over real HTTP (Host carries port) returns the token', async () => {
// Real fetch → Host arrives as '127.0.0.1:<port>'; the server must parse
// the hostname out rather than compare the raw header (amendment C9).
const resp = await fetch(`${daemon.baseUrl}/extension-token`, {
method: 'POST',
headers: { Origin: `chrome-extension://${GSTACK_EXTENSION_ID}` },
});
expect(resp.status).toBe(200);
const body = await resp.json() as any;
expect(body.token).toBe(daemon.token);
});
test('POST /extension-token with a non-pinned extension Origin returns 403 without the token', async () => {
const resp = await fetch(`${daemon.baseUrl}/extension-token`, {
method: 'POST',
headers: { Origin: 'chrome-extension://aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' },
});
expect(resp.status).toBe(403);
const body = await resp.json() as any;
expect(body.token).toBeUndefined();
});

29
browse/test/server-auth.test.ts
View File

@ -22,14 +22,29 @@ function sliceBetween(source: string, startMarker: string, endMarker: string): s
}
describe('Server auth security', () => {
// Test 1: /health serves token conditionally (headed mode or chrome extension only)
test('/health serves token only in headed mode or to chrome extensions', () => {
// Test 1 (IRON RULE, inverted in v1.62): /health NEVER serves a token in
// ANY mode. Both carve-outs (headed-mode disjunct + chrome-extension://
// Origin disjunct) are gone. Token bootstrap moved to POST /extension-token
// with a pinned extension Origin.
test('/health never serves a token — no headed-mode or chrome-extension carve-out', () => {
const healthBlock = sliceBetween(SERVER_SRC, "url.pathname === '/health'", "url.pathname === '/connect'");
// v1.35.0.0: AUTH_TOKEN const was deleted; factory uses cfg-derived authToken.
// Token must be conditional, not unconditional
expect(healthBlock).toContain('token: authToken');
expect(healthBlock).toContain('headed');
expect(healthBlock).toContain('chrome-extension://');
expect(healthBlock).not.toContain('token: authToken');
expect(healthBlock).not.toContain("getConnectionMode() === 'headed'");
expect(healthBlock).not.toContain("startsWith('chrome-extension://')");
});
// Test 1a: the pinned-origin bootstrap endpoint exists and gates on both
// the exact extension Origin and a loopback Host.
test('POST /extension-token gates on pinned Origin and loopback Host', () => {
const tokenBlock = sliceBetween(SERVER_SRC, "url.pathname === '/extension-token'", "url.pathname === '/health'");
expect(tokenBlock).toContain('GSTACK_EXTENSION_ID');
expect(tokenBlock).toContain('token: authToken');
// Host is parsed to a hostname (arrives as '127.0.0.1:34567'), never
// compared literally against the raw header.
expect(tokenBlock).toContain('.hostname');
expect(tokenBlock).toContain("'127.0.0.1'");
expect(tokenBlock).toContain("'localhost'");
expect(tokenBlock).toContain('403');
});
// Test 1b: /health does not expose sensitive browsing state

3
design/src/check.ts
View File

@ -5,6 +5,7 @@
import fs from "fs";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
export interface CheckResult {
pass: boolean;
@ -22,7 +23,7 @@ export async function checkMockup(imagePath: string, brief: string): Promise<Che
const timeout = setTimeout(() => controller.abort(), 60_000);
try {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
const response = await receiptedFetch("check-screenshot-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

3
design/src/design-to-code.ts
View File

@ -6,6 +6,7 @@
import fs from "fs";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
import { readDesignConstraints } from "./memory";
export interface DesignToCodeResult {
@ -37,7 +38,7 @@ export async function generateDesignToCodePrompt(
? `\n\nExisting DESIGN.md (use these as constraints):\n${designConstraints}`
: "";
const response = await fetch("https://api.openai.com/v1/chat/completions", {
const response = await receiptedFetch("design-to-code-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

3
design/src/diff.ts
View File

@ -6,6 +6,7 @@
import fs from "fs";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
export interface DiffResult {
differences: { area: string; description: string; severity: string }[];
@ -28,7 +29,7 @@ export async function diffMockups(
const timeout = setTimeout(() => controller.abort(), 60_000);
try {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
const response = await receiptedFetch("diff-screenshots-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

5
design/src/evolve.ts
View File

@ -8,6 +8,7 @@
import fs from "fs";
import path from "path";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
export interface EvolveOptions {
screenshot: string; // Path to current site screenshot
@ -55,7 +56,7 @@ export async function evolve(options: EvolveOptions): Promise<void> {
const timeout = setTimeout(() => controller.abort(), 240_000);
try {
const response = await fetch("https://api.openai.com/v1/responses", {
const response = await receiptedFetch("evolve-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
@ -113,7 +114,7 @@ async function analyzeScreenshot(apiKey: string, imageBase64: string): Promise<s
const timeout = setTimeout(() => controller.abort(), 30_000);
try {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
const response = await receiptedFetch("evolve-screenshot-analysis-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

3
design/src/generate.ts
View File

@ -5,6 +5,7 @@
import fs from "fs";
import path from "path";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
import { parseBrief } from "./brief";
import { createSession, sessionPath } from "./session";
import { checkMockup } from "./check";
@ -40,7 +41,7 @@ async function callImageGeneration(
const timeout = setTimeout(() => controller.abort(), 240_000);
try {
const response = await fetch("https://api.openai.com/v1/responses", {
const response = await receiptedFetch("generate-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

5
design/src/iterate.ts
View File

@ -9,6 +9,7 @@
import fs from "fs";
import path from "path";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
import { readSession, updateSession } from "./session";
export interface IterateOptions {
@ -85,7 +86,7 @@ async function callWithThreading(
const timeout = setTimeout(() => controller.abort(), 240_000);
try {
const response = await fetch("https://api.openai.com/v1/responses", {
const response = await receiptedFetch("iterate-threaded-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
@ -133,7 +134,7 @@ async function callFresh(
const timeout = setTimeout(() => controller.abort(), 240_000);
try {
const response = await fetch("https://api.openai.com/v1/responses", {
const response = await receiptedFetch("iterate-fresh-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

3
design/src/memory.ts
View File

@ -14,6 +14,7 @@
import fs from "fs";
import path from "path";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
export interface ExtractedDesign {
colors: { name: string; hex: string; usage: string }[];
@ -34,7 +35,7 @@ export async function extractDesignLanguage(imagePath: string): Promise<Extracte
const timeout = setTimeout(() => controller.abort(), 60_000);
try {
const response = await fetch("https://api.openai.com/v1/chat/completions", {
const response = await receiptedFetch("memory-distill-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,

57
design/src/receipted-fetch.ts Normal file
View File

@ -0,0 +1,57 @@
/**
* receipted-fetch — egress-receipted wrapper for the design binary's OpenAI
* calls (sink 'design-openai').
*
* Writes a content-free receipt BEFORE the send: sha256 of the JSON body
* plus a byte count — the hash only, never the body itself. FAIL-OPEN: a
* receipt hiccup warns on stderr and the call proceeds. User-facing image
* generation must not die because an audit log could not be written (the
* ledger records ATTEMPTED egress for auditing; it is not a send gate here).
*
* Streams pass through untouched: the response is returned as-is, and a
* non-string request body (e.g. a ReadableStream) is receipted as
* sha256:null rather than being consumed to hash it.
*/
import { sha256Hex, writeReceipt } from "../../lib/egress-receipt";
export type FetchLike = typeof globalThis.fetch;
/**
* Drop-in fetch replacement for api.openai.com calls.
*
* @param payloadClass content-free description of what is being sent
* (e.g. 'generate-image-request', 'check-screenshot-request')
* @param fetchImpl injectable fetch for tests / callers with their own
* fetch (variants.ts passes its stubbed fetchFn through)
*/
export async function receiptedFetch(
payloadClass: string,
url: string,
init?: RequestInit,
fetchImpl: FetchLike = globalThis.fetch,
): Promise<Response> {
try {
const body = init?.body;
let bytes = 0;
let sha256: string | null = null;
if (typeof body === "string") {
bytes = Buffer.byteLength(body);
sha256 = sha256Hex(body);
}
writeReceipt({
sink: "design-openai",
host: new URL(url).host,
payloadClass,
bytes,
sha256,
consent: "user ran design command (OPENAI_API_KEY configured)",
});
} catch (err) {
process.stderr.write(
`[design] egress receipt could not be written (${(err as Error).message}) — proceeding (fail-open). ` +
`gstack records what it ATTEMPTS to send off-machine; see gstack-egress.\n`,
);
}
return fetchImpl(url, init);
}

5
design/src/variants.ts
View File

@ -7,6 +7,7 @@
import fs from "fs";
import path from "path";
import { requireApiKey } from "./auth";
import { receiptedFetch } from "./receipted-fetch";
import { parseBrief } from "./brief";
import { normalizeIntFlag } from "./flag-utils";
@ -67,7 +68,7 @@ export async function generateVariant(
const timeout = setTimeout(() => controller.abort(), 240_000);
try {
const response = await fetchFn("https://api.openai.com/v1/responses", {
const response = await receiptedFetch("variants-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Authorization": `Bearer ${apiKey}`,
@ -79,7 +80,7 @@ export async function generateVariant(
tools: [{ type: "image_generation", model: "gpt-image-2", size, quality }],
}),
signal: controller.signal,
});
}, fetchFn);
clearTimeout(timeout);

136
design/test/receipted-fetch.test.ts Normal file
View File

@ -0,0 +1,136 @@
/**
* receipted-fetch — egress receipts for the design binary's OpenAI calls.
*
* Pins the sink contract (fail-OPEN polarity, amendment T3/C8):
* - receipt is written BEFORE the send (ordering observable via the
* ledger's existence at fetch time)
* - the sha256 recorded is the hash of the JSON body; the body itself is
* never stored
* - streaming response bodies pass through the wrapper intact
* - an unwritable ledger warns on stderr and the call still proceeds
*/
import { describe, test, expect, beforeEach, afterEach } from "bun:test";
import fs from "fs";
import os from "os";
import path from "path";
import { receiptedFetch } from "../src/receipted-fetch";
import { egressLedgerPath, listReceipts, sha256Hex } from "../../lib/egress-receipt";
let home: string;
let savedHome: string | undefined;
beforeEach(() => {
home = fs.mkdtempSync(path.join(os.tmpdir(), "design-receipt-"));
savedHome = process.env.GSTACK_HOME;
process.env.GSTACK_HOME = home;
});
afterEach(() => {
if (savedHome === undefined) delete process.env.GSTACK_HOME;
else process.env.GSTACK_HOME = savedHome;
try { fs.chmodSync(path.join(home, "security"), 0o700); } catch {}
fs.rmSync(home, { recursive: true, force: true });
});
describe("receiptedFetch", () => {
test("writes the receipt BEFORE the send; sha256 is the hash of the JSON body", async () => {
const body = JSON.stringify({ model: "gpt-4o", input: "a prompt" });
let receiptsAtFetchTime = -1;
const stub = (async (_url: any, init?: any) => {
// Receipt-before-send: by the time fetch runs, the receipt exists.
receiptsAtFetchTime = listReceipts(home).length;
expect(init.body).toBe(body); // body passes through untouched
return new Response("{}", { status: 200 });
}) as typeof globalThis.fetch;
const response = await receiptedFetch("generate-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
body,
}, stub);
expect(response.status).toBe(200);
expect(receiptsAtFetchTime).toBe(1);
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].sink).toBe("design-openai");
expect(receipts[0].host).toBe("api.openai.com");
expect(receipts[0].payload_class).toBe("generate-image-request");
expect(receipts[0].sha256).toBe(sha256Hex(body));
expect(receipts[0].bytes).toBe(Buffer.byteLength(body));
// Hash only — the ledger never contains the body text.
const raw = fs.readFileSync(egressLedgerPath(home), "utf-8");
expect(raw).not.toContain("a prompt");
});
test("streaming response body arrives intact through the wrapper", async () => {
const chunks = ["data: one\n", "data: two\n", "data: [DONE]\n"];
const stream = new ReadableStream<Uint8Array>({
start(controller) {
for (const c of chunks) controller.enqueue(new TextEncoder().encode(c));
controller.close();
},
});
const stub = (async () => new Response(stream, { status: 200 })) as typeof globalThis.fetch;
const response = await receiptedFetch("evolve-image-request", "https://api.openai.com/v1/responses", {
method: "POST",
body: JSON.stringify({ stream: true }),
}, stub);
expect(response.body).toBeInstanceOf(ReadableStream);
expect(await response.text()).toBe(chunks.join(""));
});
test("non-string request body (ReadableStream) is receipted as sha256:null, not consumed", async () => {
const requestStream = new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(new TextEncoder().encode("streamed-bytes"));
controller.close();
},
});
let receivedBody: any = null;
const stub = (async (_url: any, init?: any) => {
receivedBody = init.body;
return new Response("{}", { status: 200 });
}) as typeof globalThis.fetch;
await receiptedFetch("stream-upload", "https://api.openai.com/v1/responses", {
method: "POST",
body: requestStream,
}, stub);
expect(receivedBody).toBe(requestStream); // same stream object, untouched
const receipts = listReceipts(home);
expect(receipts[0].sha256).toBeNull();
// The stream is still readable by the consumer (was not drained to hash).
expect(await new Response(receivedBody).text()).toBe("streamed-bytes");
});
test("fail-open: unwritable ledger warns on stderr and the call proceeds", async () => {
if (process.platform === "win32" || process.getuid?.() === 0) return;
fs.mkdirSync(path.join(home, "security"), { recursive: true, mode: 0o500 });
let fetched = false;
const stub = (async () => { fetched = true; return new Response("{}", { status: 200 }); }) as typeof globalThis.fetch;
const captured: string[] = [];
const originalWrite = process.stderr.write.bind(process.stderr);
(process.stderr as any).write = (chunk: string) => { captured.push(String(chunk)); return true; };
let response: Response;
try {
response = await receiptedFetch("check-screenshot-request", "https://api.openai.com/v1/chat/completions", {
method: "POST",
body: "{}",
}, stub);
} finally {
(process.stderr as any).write = originalWrite;
}
expect(fetched).toBe(true); // the call proceeded
expect(response.status).toBe(200);
const warning = captured.join("");
expect(warning).toContain("egress receipt could not be written");
expect(warning).toContain("fail-open");
expect(warning).toContain("gstack-egress");
});
});

6
design/test/variants-retry-after.test.ts
View File

@ -44,13 +44,19 @@ function makeStubFetch(
describe("generateVariant Retry-After handling", () => {
let tmpDir: string;
let outputPath: string;
let savedHome: string | undefined;
beforeEach(() => {
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "variants-retry-after-"));
outputPath = path.join(tmpDir, "variant.png");
// The fetch path now writes egress receipts — keep them in the temp home.
savedHome = process.env.GSTACK_HOME;
process.env.GSTACK_HOME = tmpDir;
});
afterEach(() => {
if (savedHome === undefined) delete process.env.GSTACK_HOME;
else process.env.GSTACK_HOME = savedHome;
fs.rmSync(tmpDir, { recursive: true, force: true });
});

6
docs/REMOTE_BROWSER_ACCESS.md
View File

@ -56,7 +56,7 @@ All command endpoints require a Bearer token:
Authorization: Bearer gsk_sess_...
```
`/connect` is unauthenticated (rate-limited) — it's how a remote agent exchanges a setup key for a scoped session token. `/health` is unauthenticated on the local listener (bootstrap) but does NOT exist on the tunnel listener (404).
`/connect` is unauthenticated (rate-limited) — it's how a remote agent exchanges a setup key for a scoped session token. `/health` is unauthenticated on the local listener (liveness/status only — never a token) but does NOT exist on the tunnel listener (404). Extension token bootstrap is `POST /extension-token` on the local listener, gated by the pinned `chrome-extension://` Origin; it is not on the tunnel surface either.
SSE endpoints (`/activity/stream`, `/inspector/events`) accept either a Bearer token or the HttpOnly `gstack_sse` cookie (minted via `POST /sse-session`, 30-minute TTL, stream-scope only — cannot be used against `/command`). As of v1.6.0.0 the `?token=<ROOT>` query-string auth is no longer accepted.
@ -80,6 +80,9 @@ Response: (plain text result of the command)
#### GET /health
Server status. No auth required. Returns status, tabs, mode, uptime.
Never carries a token — extension token bootstrap is `POST /extension-token`
(local listener only, validates the pinned `chrome-extension://` Origin and a
loopback Host; 403 otherwise). Not reachable over the tunnel (404).
### Commands
@ -176,6 +179,7 @@ Each agent owns the tabs it creates. Rules:
- **Path traversal guarded** on `/welcome` — `GSTACK_SLUG` must match `^[a-z0-9_-]+$` or falls back to the built-in template.
- **SSRF guards** on `goto`, `download`, and scrape paths — validates URL target against a localhost/private-range blocklist.
- **Tunnel surface denial logging.** Every rejection on the tunnel listener (`path_not_on_tunnel`, `root_token_on_tunnel`, `missing_scoped_token`, `disallowed_command:*`) is appended to `~/.gstack/security/attempts.jsonl` with timestamp, source IP, path, method. Rate-capped at 60 writes/min.
- **Egress receipt on tunnel start (v1.63+).** Every tunnel session open writes a hash-chained receipt (sink `browse-tunnel`) to `~/.gstack/security/egress.jsonl` BEFORE ngrok forwards anything. Fail-closed: if the receipt can't be written, the tunnel refuses to start. Audit with `bin/gstack-egress list` / `bin/gstack-egress verify`.
- All agent activity is logged with attribution (clientId).
**Known non-goal (tracked as #1136):** on Windows, the cookie-import-browser path launches Chrome with `--remote-debugging-port=<random>`. With App-Bound Encryption v20, a same-user local process can connect to that port and exfiltrate decrypted v20 cookies — an elevation path relative to reading the SQLite DB directly. Fix direction is `--remote-debugging-pipe` instead of TCP.

11
docs/designs/SIDEBAR_MESSAGE_FLOW.md
View File

@ -62,13 +62,18 @@ T+500ms terminal-agent.ts boots
└── Probes claude → writes claude-available.json
T+1-3s Extension loads, sidebar opens
├── background.js: GET /health (liveness only — no token) then
│ POST /extension-token → AUTH_TOKEN. The server releases the
│ token only to Origin chrome-extension://<pinned id>; the
│ manifest "key" pins the ID (browse/scripts/extension-id.ts)
├── sidepanel-terminal.js: setState(IDLE), shows "Starting Claude Code..."
└── tryAutoConnect() polls until window.gstackServerPort + token are set
T+ready tryAutoConnect calls connect()
├── POST /pty-session (Authorization: Bearer AUTH_TOKEN)
│ └── server mints session token, posts /internal/grant to agent
│ └── responds with {terminalPort, ptySessionToken}
│ └── server mints attach token, posts /internal/grant to agent
│ └── responds with {terminalPort, sessionId, attachToken,
│ leaseExpiresAt}
├── GET /claude-available (preflight)
├── new WebSocket(`ws://127.0.0.1:<terminalPort>/ws`,
│ [`gstack-pty.<token>`])
@ -105,7 +110,7 @@ The protocol-token path is what the browser actually uses.
| Token | Lives in | Used for | Lifetime |
|-------|----------|----------|----------|
| `AUTH_TOKEN` | `<stateDir>/browse.json`; in-memory in server.ts | `/pty-session` POST (mint cookie + token) | server lifetime |
| `AUTH_TOKEN` | `<stateDir>/browse.json`; in-memory in server.ts; extension memory via pinned-origin `POST /extension-token` (never `GET /health`) | `/pty-session` POST (mint cookie + token) | server lifetime |
| `gstack-pty.<...>` (Sec-WebSocket-Protocol) | Browser memory only; agent `validTokens` Set | `/ws` upgrade auth | 30 min, auto-revoked on WS close |
| `INTERNAL_TOKEN` | `<stateDir>/terminal-internal-token`; in agent memory | server → agent loopback `/internal/grant` | agent lifetime |

60
docs/gbrain-sync-errors.md
View File

@ -9,8 +9,9 @@ the command output.
## `BRAIN_SYNC: brain repo detected: <url>`
**Problem.** You're on a machine that has `~/.gstack-brain-remote.txt` (copied
from another machine) but no local git repo at `~/.gstack/.git`.
**Problem.** You're on a machine that has `~/.gstack-artifacts-remote.txt`
(or the legacy `~/.gstack-brain-remote.txt`, copied from another machine) but
no local git repo at `~/.gstack/.git`.
**Cause.** You've set up GBrain sync elsewhere and your gstack hasn't been
restored on this machine yet.
@ -92,23 +93,48 @@ your local commit still exists — the next skill run will retry the push.
---
## `gstack-brain-init: ~/.gstack/.git is already a git repo pointing at <url>`
## `gstack: brain-sync push NOT sent — the egress receipt could not be written`
**Problem.** You tried to init with a remote URL that doesn't match the
existing one.
**Problem.** The push was refused before anything left your machine. Every
brain-sync push writes a tamper-evident receipt to the egress ledger
(`~/.gstack/security/egress.jsonl`) before sending, fail-closed. The
receipt could not be written, so nothing was sent, no local commit was
made, and the queue is preserved — the next run retries the whole drain.
`gstack-brain-sync --status` shows `EGRESS_RECEIPT_FAILED` as the failure
detail.
**Cause.** You already ran `gstack-brain-init` with a different remote.
**Cause.** `~/.gstack/security/` is not writable (the receipt writer creates
it when missing, so absence alone is not the cause), the disk is full, or
`GSTACK_HOME` points at a read-only location.
**Fix.** Either:
- Use the existing remote: run `gstack-brain-init` without `--remote`, or
with the matching URL.
- Switch remotes: `gstack-brain-uninstall` first, then re-init with the new
URL. This does not delete your data.
**Fix.**
```bash
mkdir -p ~/.gstack/security && chmod -R u+w ~/.gstack/security
```
Then run any skill (or `gstack-brain-sync --once`) to retry. Inspect the
ledger with `gstack-egress list`; verify its hash chain with
`gstack-egress verify`.
---
## `Remote not reachable: <url>`
## `gstack-artifacts-init: ~/.gstack/ is already a git repo pointing at: <url>`
**Problem.** You tried to init with a remote URL that doesn't match the
existing one. The command refuses to overwrite.
**Cause.** You already ran `gstack-artifacts-init` with a different remote.
**Fix.** Either:
- Use the existing remote: run `gstack-artifacts-init` without `--remote`, or
with the matching URL.
- Switch remotes: `git -C ~/.gstack remote set-url origin <url>` (the
command's own suggestion), or `gstack-brain-uninstall` first, then re-init
with the new URL. Neither deletes your data.
---
## `Remote not reachable via SSH: <url>`
**Problem.** Init couldn't reach the git remote to verify connectivity.
@ -126,7 +152,7 @@ If that fails, check:
---
## `gstack-brain-init: failed to create or find '<name>'`
## `Failed to create or find '<name>'. Try --remote <url>.`
**Problem.** Auto-repo-creation via `gh repo create` failed and the repo
isn't discoverable via `gh repo view` either.
@ -141,7 +167,7 @@ gh auth status
If unauth'd, run `gh auth login`. If the repo name collides, pass a different
name:
```bash
gstack-brain-init --remote git@github.com:YOURUSER/custom-name.git
gstack-artifacts-init --remote git@github.com:YOURUSER/custom-name.git
```
---
@ -168,7 +194,7 @@ gstack session, or (b) a previous failed restore left partial state.
**Fix (three options).**
1. **If this machine's state should become the new truth**: run
`gstack-brain-init` instead of restore — this creates a brand-new brain
`gstack-artifacts-init` instead of restore — this creates a brand-new brain
repo from this machine's state.
2. **If you want to adopt the remote and discard this machine's state**:
@ -189,7 +215,7 @@ and `.gitattributes`.
**Cause.** You pointed restore at a random git repo, or someone deleted the
canonical config files from the brain repo.
**Fix.** Verify the URL. If it's correct, run `gstack-brain-init --remote
**Fix.** Verify the URL. If it's correct, run `gstack-artifacts-init --remote
<url>` to re-seed the canonical config.
---

51
docs/gbrain-sync.md
View File

@ -18,7 +18,7 @@ GBrain.
By design, these stay local even when sync is on:
- Credentials: `.auth.json`, `auth-token.json`, `sidebar-sessions/`,
`security/device-salt`, consumer tokens in `config.yaml`
`security/device-salt`
- Machine-specific state: Chromium profiles, ONNX model weights,
caches, eval-cache, CDP-profile, one-time prompt markers
(`.welcome-seen`, `.telemetry-prompted`, `.vendoring-warned-*`, etc.)
@ -31,25 +31,25 @@ it; you can append your own entries below the marker line.
## First-run setup (30–90 seconds)
```bash
gstack-brain-init
gstack-artifacts-init
```
The command:
1. Turns `~/.gstack/` into a git repo.
2. Asks for a remote URL (default: `gh repo create --private
gstack-brain-$USER`). Any git remote works — GitHub, GitLab, Gitea,
gstack-artifacts-$USER`). Any git remote works — GitHub, GitLab, Gitea,
self-hosted.
3. Pushes an initial commit with just the config.
4. Writes `~/.gstack-brain-remote.txt` (URL-only, no secrets —
4. Writes `~/.gstack-artifacts-remote.txt` (URL-only, no secrets —
safe to copy to another machine).
5. Wires the gstack-brain repo into your local gbrain as a federated
source (via `gbrain sources add` + `git worktree`) so `gbrain search`
can index your synced learnings, plans, and designs. Implementation
lives in `bin/gstack-gbrain-source-wireup`. The old
`gstack-brain-reader add --ingest-url ...` HTTP path was removed in
v1.15.1.0 — it depended on a `/ingest-repo` endpoint gbrain never
shipped.
5. Prints the `gbrain sources add` hookup command for the brain host
(never auto-executed — run it yourself, or on your own machine
`bin/gstack-gbrain-source-wireup` does the same wiring) so
`gbrain search` can index your synced learnings, plans, and designs.
The old `gstack-brain-reader add --ingest-url ...` HTTP path was
removed in v1.15.1.0 — it depended on a `/ingest-repo` endpoint gbrain
never shipped.
After init, the **next skill you run** will ask you ONE question about
privacy mode:
@ -65,14 +65,15 @@ Your answer is persisted. You won't be asked again.
## Cross-machine workflow
On machine A: run `gstack-brain-init` once. That's it — every skill
On machine A: run `gstack-artifacts-init` once. That's it — every skill
invocation now drains the sync queue at its start and end boundaries
(~200–800 ms network pause per skill).
On machine B:
1. Copy `~/.gstack-brain-remote.txt` from machine A to machine B
(password manager, dotfile repo, USB stick — your call).
1. Copy `~/.gstack-artifacts-remote.txt` from machine A to machine B
(password manager, dotfile repo, USB stick — your call; the legacy
`~/.gstack-brain-remote.txt` name is still recognized).
2. Run any gstack skill. The preamble sees the URL file and prints:
```
BRAIN_SYNC: brain repo detected: <url>
@ -80,9 +81,7 @@ On machine B:
```
3. Run `gstack-brain-restore`. That clones the repo, rehydrates your
learnings/plans/retros, and re-registers the git merge drivers.
4. Re-enter consumer tokens (they're machine-local and NOT synced —
`gstack-config set gbrain_token <your-token>`).
5. Next skill: your yesterday-on-machine-A learning surfaces. That's the
4. Next skill: your yesterday-on-machine-A learning surfaces. That's the
magical moment.
## Status, health, and queue depth
@ -141,6 +140,12 @@ To remediate:
There's a defense-in-depth hook at `~/.gstack/.git/hooks/pre-commit` that
runs the same scan if you manually `git commit` against the repo.
Separately (v1.63.0.0+), every push writes a tamper-evident receipt to the
egress ledger (`~/.gstack/security/egress.jsonl`) *before* anything is
sent, fail-closed: if the receipt can't be written, the push is refused
and the queue is preserved. Inspect the ledger with `gstack-egress list`
and verify its hash chain with `gstack-egress verify`.
## Two-machine conflicts
If you write on machine A and machine B the same day, both will push
@ -176,7 +181,7 @@ This:
Add `--delete-remote` to also delete the private GitHub repo (GitHub only,
uses `gh repo delete`).
Re-init anytime with `gstack-brain-init`.
Re-init anytime with `gstack-artifacts-init`.
## Troubleshooting
@ -185,8 +190,8 @@ error message gstack-brain may print, with problem / cause / fix for each.
## Under the hood
For the architectural decisions behind this feature (allowlist vs
denylist, daemon vs preamble-boundary sync, JSONL merge driver, privacy
stop-gate), see the
[approved plan](../system-instruction-you-are-working-jaunty-kahn.md) in
the gstack plans directory.
The architectural decisions behind this feature: allowlist over denylist
(unknown files stay local by default), preamble-boundary sync over a daemon
(no background process to babysit), a JSONL merge driver so concurrent
machines union their queues instead of conflicting, and a privacy stop-gate
that asks once before anything syncs.

94
extension/background.js
View File

@ -32,22 +32,34 @@ function getBaseUrl() {
// ─── Auth Token Bootstrap ─────────────────────────────────────
// Token bootstrap: POST /extension-token. The server validates our Origin
// (chrome-extension://<pinned id> — the manifest "key" pins the ID) before
// releasing the token. GET /health is liveness/status only and never
// carries a token. Returns true on success, false on failure; a 403 means
// the server doesn't trust this extension identity — treat as disconnected
// rather than retrying forever with a stale token.
async function loadAuthToken() {
if (authToken) return;
// Get token from browse server /health endpoint (localhost-only, safe).
// Previously read from .auth.json in extension dir, but that breaks
// read-only .app bundles and codesigning.
const base = getBaseUrl();
if (!base) return;
if (!base) return false;
try {
const resp = await fetch(`${base}/health`, { signal: AbortSignal.timeout(3000) });
const resp = await fetch(`${base}/extension-token`, {
method: 'POST',
signal: AbortSignal.timeout(3000),
});
if (resp.status === 403) {
console.error('[gstack bg] /extension-token 403 — extension identity not trusted by server');
authToken = null;
setDisconnected();
return false;
}
if (resp.ok) {
const data = await resp.json();
if (data.token) authToken = data.token;
if (data.token) { authToken = data.token; return true; }
}
} catch (err) {
console.error('[gstack bg] Failed to load auth token:', err.message);
}
return false;
}
// ─── Health Polling ────────────────────────────────────────────
@ -59,17 +71,16 @@ async function checkHealth() {
return;
}
// Retry loading auth token if we don't have one yet
if (!authToken) await loadAuthToken();
try {
const resp = await fetch(`${base}/health`, { signal: AbortSignal.timeout(3000) });
if (!resp.ok) { setDisconnected(); return; }
const data = await resp.json();
if (data.status === 'healthy') {
// Always refresh auth token from /health — the server generates a new
// token on each restart, so the old one becomes stale.
if (data.token) authToken = data.token;
// Always refresh the auth token — the server generates a new token
// on each restart, so the old one becomes stale. loadAuthToken()
// already flips to disconnected on a 403.
const gotToken = await loadAuthToken();
if (!gotToken && !authToken) return;
// Forward chatEnabled so sidepanel can show/hide chat tab
setConnected({ ...data, chatEnabled: !!data.chatEnabled });
} else {
@ -577,27 +588,50 @@ chrome.tabs.onUpdated.addListener((_id, changeInfo) => {
}
});
// ─── identity-pin migration notice (v1.63) ────────────────────
//
// The manifest "key" added in v1.63 pins the extension ID, which changes
// the ID for existing installs — chrome.storage.local is keyed by
// extension ID, so panel-local state (saved port, snoozes) resets once.
// Explain that in-product, one time. The flag name is version-free so a
// release re-slot never orphans an already-set flag.
async function announceIdentityPinOnce() {
try {
const data = await chrome.storage.local.get('gstack_id_pin_migrated');
if (data.gstack_id_pin_migrated) return;
console.log('[gstack] gstack sidebar: extension identity pinned in v1.63 — panel state reset once.');
chrome.runtime.sendMessage({
type: 'gstack-migration-notice',
message: 'gstack sidebar: extension identity pinned in v1.63 — panel state reset once.',
}).catch(() => {
// Expected: panel not open. The console line above still lands.
});
await chrome.storage.local.set({ gstack_id_pin_migrated: true });
} catch (err) {
console.debug('[gstack] identity-pin notice failed (non-fatal):', err.message);
}
}
// ─── Startup ────────────────────────────────────────────────────
// Fast-retry health check on startup. The server may not be listening yet
// (Chromium launches before Bun.serve starts). Retry every 1s for the
// first 15 seconds, then switch to 10s polling.
loadAuthToken().then(() => {
loadPort().then(() => {
let startupAttempts = 0;
const startupCheck = setInterval(async () => {
startupAttempts++;
await checkHealth();
if (isConnected || startupAttempts >= 15) {
clearInterval(startupCheck);
// Switch to slow polling now that we're connected (or gave up)
if (!healthInterval) {
healthInterval = setInterval(checkHealth, 10000);
}
if (!isConnected) {
console.log('[gstack] Startup health checks failed after 15 attempts, falling back to 10s polling');
}
announceIdentityPinOnce();
loadPort().then(() => {
let startupAttempts = 0;
const startupCheck = setInterval(async () => {
startupAttempts++;
await checkHealth();
if (isConnected || startupAttempts >= 15) {
clearInterval(startupCheck);
// Switch to slow polling now that we're connected (or gave up)
if (!healthInterval) {
healthInterval = setInterval(checkHealth, 10000);
}
}, 1000);
});
if (!isConnected) {
console.log('[gstack] Startup health checks failed after 15 attempts, falling back to 10s polling');
}
}
}, 1000);
});

1
extension/manifest.json
View File

@ -3,6 +3,7 @@
"name": "gstack browse",
"version": "0.1.0",
"description": "Live activity feed and @ref overlays for gstack browse",
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApp4uyDmQADJ/MPoKybEvBPpuGWxsXiNMo5jJFFaEaC3yPJnB4y8E0UuvE56n2KlQzaqlnBOt4T8w0ApTbNABZpEnSGQJVmkbT8a62WXefYZm79bMgzW/bNIZ4QYWNEAtZb0wvncMNSOyU9mga1s3eGWtukHs2Zf5spXRLQGV/on9l8iN9QPRM/VB0AxtUc2DTYjwkTGCAOMFiaq02miP0/hW6AeltBW9R0aHgbnJw2H2YVrgQXRvGxD1DMQe6NzGVVqKGhpUdYGPw4ONWOKijdignz+j+90HSrCK06HUy80jiKAYmdZePnn++N5meIJY+bWk7RqxS6er8Ow2U65TywIDAQAB",
"permissions": ["sidePanel", "storage", "activeTab", "scripting", "tabs"],
"host_permissions": ["http://127.0.0.1:*/", "ws://127.0.0.1:*/"],
"action": {

23
extension/sidepanel-terminal.js
View File

@ -504,7 +504,8 @@
window.gstackScanForPTYInject = async function (text, origin) {
if (!text) return { allow: false, verdict: 'BLOCK', reasons: ['empty-text'] };
try {
const resp = await fetch('http://127.0.0.1:34567/pty-inject-scan', {
const serverPort = getServerPort() || 34567;
const resp = await fetch(`http://127.0.0.1:${serverPort}/pty-inject-scan`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
@ -529,21 +530,13 @@
};
// The auth token for /pty-inject-scan comes from the same source the
// sidepanel uses for /pty-session — a runtime fetch from /health (which
// already returns AUTH_TOKEN in headed mode per CLAUDE.md's v1.1 TODO).
// We don't echo the token here; this helper is a thin proxy around the
// existing pattern.
// sidepanel uses for /pty-session — window.gstackAuthToken, set by
// sidepanel.js after the pinned-origin POST /extension-token bootstrap.
// The old fallback here fetched /health and read token keys the server
// never sent (AUTH_TOKEN/authToken) — dead code since /health stopped
// carrying any token.
async function getAuthTokenForScan() {
if (window.__gstackPtyScanToken) return window.__gstackPtyScanToken;
try {
const resp = await fetch('http://127.0.0.1:34567/health');
const body = await resp.json();
const token = body.AUTH_TOKEN || body.authToken || '';
if (token) window.__gstackPtyScanToken = token;
return token;
} catch {
return '';
}
return getAuthToken() || '';
}
async function connect() {

56
extension/sidepanel.js
View File

@ -1304,21 +1304,36 @@ async function tryConnect() {
});
if (healthResp.ok) {
const data = await healthResp.json();
if (data.status === 'healthy' && data.token) {
if (data.status === 'healthy') {
// /health is liveness-only — the token comes from the pinned-origin
// POST /extension-token bootstrap (our chrome-extension:// Origin
// is validated server-side against the manifest-pinned ID).
const tokenResp = await fetch(`http://127.0.0.1:${port}/extension-token`, {
method: 'POST',
signal: AbortSignal.timeout(2000),
});
const tokenData = tokenResp.ok ? await tokenResp.json() : null;
if (tokenData?.token) {
setLoadingStatus(
`Server healthy on port ${port}, connecting...`,
`token: yes (from /extension-token)\nStarting SSE + activity feed...`
);
updateConnection(`http://127.0.0.1:${port}`, tokenData.token);
// The SEC shield used to drive off /health.security via the chat
// path's classifier; with the chat path ripped, the indicator is
// not driven yet. Leaving the shield element hidden by default.
return;
}
setLoadingStatus(
`Server healthy on port ${port}, connecting...`,
`token: yes (from /health)\nStarting SSE + activity feed...`
`Server healthy but token bootstrap failed (attempt ${connectAttempts})`,
`POST /extension-token → ${tokenResp.status}${tokenResp.status === 403 ? ' (extension identity not trusted)' : ''}`
);
} else {
setLoadingStatus(
`Server responded but not healthy (attempt ${connectAttempts})`,
`status: ${data.status}`
);
updateConnection(`http://127.0.0.1:${port}`, data.token);
// The SEC shield used to drive off /health.security via the chat
// path's classifier; with the chat path ripped, the indicator is
// not driven yet. Leaving the shield element hidden by default.
return;
}
setLoadingStatus(
`Server responded but not healthy (attempt ${connectAttempts})`,
`status: ${data.status}\ntoken: ${data.token ? 'yes' : 'no'}`
);
} else {
setLoadingStatus(
`Server returned ${healthResp.status} (attempt ${connectAttempts})`,
@ -1355,6 +1370,23 @@ chrome.runtime.onMessage.addListener((msg) => {
fetchRefs();
}
}
// One-time v1.62 identity-pin notice from background.js. Transient banner —
// no dedicated element in sidepanel.html since this fires once per install.
if (msg.type === 'gstack-migration-notice' && msg.message) {
console.log('[gstack sidebar]', msg.message);
try {
const banner = document.createElement('div');
banner.textContent = msg.message;
banner.style.cssText =
'position:fixed;left:8px;right:8px;bottom:40px;z-index:9999;' +
'background:#1f2937;color:#f5a623;border:1px solid #f5a623;' +
'border-radius:6px;padding:8px 10px;font-size:12px;text-align:left;';
document.body.appendChild(banner);
setTimeout(() => banner.remove(), 8000);
} catch (err) {
console.debug('[gstack sidebar] migration banner failed:', err && err.message);
}
}
if (msg.type === 'inspectResult') {
inspectorPickerActive = false;
inspectorPickBtn.classList.remove('active');

1003
lib/context-bill.ts Normal file
View File

File diff suppressed because it is too large Load Diff

392
lib/egress-receipt.ts Normal file
View File

@ -0,0 +1,392 @@
/**
* egress-receipt — hash-chained, content-free receipts for every
* gstack-initiated off-machine send (`~/.gstack/security/egress.jsonl`, 0600).
*
* THREAT MODEL: the egress ledger is forensic observability — it records
* ATTEMPTED egress so accidents are auditable; it is not an exfiltration
* control. Receipts are written before send, outcomes are best-effort, and
* fail-open sinks can send unrecorded with a warning.
*
* Semantics:
* - Receipt-before-send: the receipt line is appended BEFORE the network
* call. Fail-closed sinks MUST refuse the send with the typed code
* EGRESS_RECEIPT_FAILED when it cannot be written; fail-open sinks warn
* on stderr and proceed.
* - Content-free: never payload text, never credentials — a sha256 of the
* exact bytes sent plus a byte count only (semantic-reviews.jsonl
* precedent). Sinks where a subprocess/SDK owns the bytes record
* sha256: null.
* - Tamper-evident: each line carries `prev` = sha256 of the previous raw
* line ("" for line 1). `verifyLedger` recomputes the chain.
*
* Node builtins only, so bun TS binaries and the compiled browse binary can
* both import it (same constraint as browse/src/security.ts: no native
* modules).
*/
import { createHash } from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
export const EGRESS_RECEIPT_FAILED = 'EGRESS_RECEIPT_FAILED';
const SHA256_HEX = /^[0-9a-f]{64}$/;
/**
* WARN-at-size threshold. Above this the ledger still appends (never blocks
* on size), but writeReceipt emits one stderr warning per process so the
* user learns the file exists and how to inspect it before it gets silly.
*/
export const LEDGER_WARN_BYTES = 25 * 1024 * 1024;
/**
* Tail window for reading the last raw line. Receipt lines are ~300 bytes;
* 4KB covers any legal line with an order of magnitude to spare.
*/
const TAIL_READ_BYTES = 4096;
// TODO(rotation): ledger rotation. Chain-genesis sketch: when the ledger
// exceeds the size threshold, rename it to egress.jsonl.1 and start a new
// generation whose FIRST record embeds `genesis: sha256(<tail raw line of
// the prior file>)`, so verifyLedger can walk generations end-to-end
// (verify each file's internal chain, then check each genesis hash against
// the previous generation's last line). Until then we warn at 25MB.
type Env = Record<string, string | undefined>;
export interface WriteReceiptOptions {
/** gstack home; resolved from env when omitted */
home?: string;
/** env for home resolution (tests) */
env?: Env;
/** which gstack component is sending */
sink: string;
/** destination host[:port] */
host: string;
/** content-free payload description */
payloadClass: string;
/** exact byte count sent (0 for bodyless requests) */
bytes?: number;
/** sha256 hex of the exact bytes sent; null when a subprocess/SDK owns the bytes */
sha256?: string | null;
/** the consent key+value that authorizes this send */
consent: string;
}
export interface WriteOutcomeOptions {
home?: string;
env?: Env;
/** receipt id returned by writeReceipt */
receipt: string;
status?: string | number;
}
export interface LedgerLine {
lineNo: number;
raw: string;
record: Record<string, unknown> | null;
}
export interface VerifyResult {
ok: boolean;
count: number;
brokenLine: number | null;
reason: string | null;
/** present when the ledger file exceeds LEDGER_WARN_BYTES */
sizeWarning: string | null;
}
/**
* Same resolution order as the rest of gstack (shell sinks, selection code):
* GSTACK_HOME, legacy GSTACK_STATE_DIR, then $HOME/.gstack.
*/
export function resolveEgressHome(env: Env = process.env): string {
const configured = env.GSTACK_HOME || env.GSTACK_STATE_DIR;
if (configured) return path.resolve(configured);
return path.join(env.HOME || os.homedir(), '.gstack');
}
export function egressLedgerPath(home: string): string {
return path.join(home, 'security', 'egress.jsonl');
}
export function sha256Hex(data: string | Uint8Array): string {
return createHash('sha256').update(data).digest('hex');
}
function receiptError(message: string, cause?: unknown): Error & { code: string } {
const error = (cause === undefined ? new Error(message) : new Error(message, { cause })) as Error & { code: string };
error.code = EGRESS_RECEIPT_FAILED;
return error;
}
// A serialized receipt line must stay under TAIL_READ_BYTES so the O(1)
// tail-read always captures the FULL previous line before hashing it into the
// chain. A caller-controlled field (sink/host/payloadClass/consent — e.g.
// context-bill builds payloadClass dynamically) long enough to push the line
// past the tail window would make the next append hash a truncated prior line,
// and verifyLedger would then report a permanent false TAMPER. Cap each field
// well under the window so the invariant holds by construction.
const MAX_FIELD_BYTES = 512;
function requireString(value: unknown, name: string): string {
if (typeof value !== 'string' || !value) throw receiptError(`Egress receipt requires a non-empty ${name}`);
if (Buffer.byteLength(value) > MAX_FIELD_BYTES) {
throw receiptError(`Egress receipt ${name} exceeds ${MAX_FIELD_BYTES} bytes (${Buffer.byteLength(value)})`);
}
return value;
}
/**
* mkdir spin lock, ~2.5s budget. Egress events are rare (minutes apart); the
* lock only protects the read-last-line → append window.
*
* Stale-lock reclaim: a crashed writer strands the lock dir. Once the spin
* budget is exhausted, a lock dir whose mtime is >10s old is stale by
* definition (appends take milliseconds), so the waiter removes it and
* retries instead of failing. The rmdir/stat races with a concurrent
* reclaimer or the owner's own cleanup are harmless — losers just loop.
*/
function withLedgerLock<T>(ledger: string, callback: () => T): T {
const lock = `${ledger}.lock`;
const deadline = Date.now() + 2500;
for (;;) {
try {
fs.mkdirSync(lock);
break;
} catch (error) {
if ((error as NodeJS.ErrnoException)?.code !== 'EEXIST') throw error;
if (Date.now() > deadline) {
try {
const age = Date.now() - fs.statSync(lock).mtimeMs;
if (age > 10_000) { fs.rmdirSync(lock); continue; }
} catch { /* raced with the owner's cleanup — retry */ }
throw receiptError(`Egress ledger is locked: ${lock}`);
}
// Sync sleep (node + bun): the API is sync on purpose so shell, bun,
// and node callers all share one implementation.
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 10);
}
}
try {
return callback();
} finally {
try { fs.rmdirSync(lock); } catch { /* best-effort unlock */ }
}
}
/**
* Last raw line via a tail read: open the file, read the final
* TAIL_READ_BYTES, take the last newline-terminated chunk. Never loads the
* whole ledger, so appends stay O(1) as the file grows.
*/
function lastRawLine(ledger: string): string | null {
let fd: number;
try {
fd = fs.openSync(ledger, 'r');
} catch (error) {
if ((error as NodeJS.ErrnoException)?.code === 'ENOENT') return null;
throw error;
}
try {
const size = fs.fstatSync(fd).size;
if (size === 0) return null;
const length = Math.min(size, TAIL_READ_BYTES);
const buffer = Buffer.alloc(length);
fs.readSync(fd, buffer, 0, length, size - length);
const tail = buffer.toString('utf8');
const lines = tail.split('\n').filter((line) => line.length > 0);
return lines.length ? lines[lines.length - 1] : null;
} finally {
fs.closeSync(fd);
}
}
let warnedLedgerSize = false;
/** Test-only: re-arm the once-per-process size warning. */
export function resetLedgerSizeWarningForTests(): void {
warnedLedgerSize = false;
}
function warnLedgerSizeOnce(ledger: string): void {
// Short-circuit BEFORE the stat: the warning fires at most once per process,
// so after it has fired there is no reason to stat the ledger on every
// subsequent writeReceipt (this runs on the append hot path).
if (warnedLedgerSize) return;
let size: number;
try {
size = fs.statSync(ledger).size;
} catch {
return; // no file yet — nothing to warn about
}
if (size <= LEDGER_WARN_BYTES) return;
warnedLedgerSize = true;
process.stderr.write(ledgerSizeWarning(ledger, size) + '\n');
}
/**
* Self-explanatory size warning: says what the ledger is (records what
* gstack ATTEMPTS to send off-machine), how to inspect it, and that
* trimming arrives with rotation.
*/
export function ledgerSizeWarning(ledger: string, size: number): string {
const mb = (size / (1024 * 1024)).toFixed(1);
return (
`gstack: egress ledger is large (${mb}MB): ${ledger}. ` +
`This file records what gstack ATTEMPTS to send off-machine (content-free receipts, for auditing). ` +
`Inspect it with 'gstack-egress list'. Trimming arrives with ledger rotation (TODO); until then it only grows.`
);
}
function appendChained(
homeOrNull: string | null,
record: Record<string, unknown>,
env?: Env,
): { id: string; path: string } {
const home = homeOrNull ?? resolveEgressHome(env);
const ledger = egressLedgerPath(home);
try {
fs.mkdirSync(path.dirname(ledger), { recursive: true, mode: 0o700 });
return withLedgerLock(ledger, () => {
const previous = lastRawLine(ledger);
const line = JSON.stringify({ ...record, prev: previous == null ? '' : sha256Hex(previous) });
const existed = fs.existsSync(ledger);
fs.appendFileSync(ledger, `${line}\n`, { mode: 0o600 });
if (!existed) fs.chmodSync(ledger, 0o600); // umask must not weaken the ledger
return { id: sha256Hex(line), path: ledger };
});
} catch (error) {
if ((error as NodeJS.ErrnoException)?.code === EGRESS_RECEIPT_FAILED) throw error;
throw receiptError(
`Egress receipt could not be written to ${ledger}: ${(error as Error)?.message ?? error}`,
error,
);
}
}
/**
* Append one content-free receipt BEFORE a network send.
*
* @returns id = sha256 of the written line (for writeOutcome)
* @throws Error with code EGRESS_RECEIPT_FAILED — fail-closed callers must
* refuse the send; fail-open callers warn and proceed
*/
export function writeReceipt(opts: WriteReceiptOptions): { id: string; path: string } {
const sink = requireString(opts.sink, 'sink');
const host = requireString(opts.host, 'host');
const payloadClass = requireString(opts.payloadClass, 'payloadClass');
const consent = requireString(opts.consent, 'consent');
const bytes = opts.bytes ?? 0;
if (!Number.isSafeInteger(bytes) || bytes < 0) throw receiptError('Egress receipt bytes must be a non-negative integer');
const sha256 = opts.sha256 ?? null;
if (sha256 !== null && !SHA256_HEX.test(String(sha256))) throw receiptError('Egress receipt sha256 must be 64 lowercase hex chars or null');
const home = opts.home ?? resolveEgressHome(opts.env);
warnLedgerSizeOnce(egressLedgerPath(home));
return appendChained(home, {
ts: new Date().toISOString(),
type: 'egress',
sink,
host,
payload_class: payloadClass,
bytes,
sha256,
consent,
}, opts.env);
}
/**
* Append the response status for an earlier receipt (best-effort companion
* record — the pre-send receipt is the invariant, the outcome is
* bookkeeping). Chained like every other line.
*/
export function writeOutcome(opts: WriteOutcomeOptions): { id: string; path: string } {
const receipt = requireString(opts.receipt, 'receipt id');
return appendChained(opts.home ?? null, {
ts: new Date().toISOString(),
type: 'outcome',
receipt,
status: String(opts.status ?? 'unknown'),
}, opts.env);
}
/** Raw parsed lines: [{lineNo, raw, record|null}]. Missing ledger → []. */
export function readLedger(home: string): LedgerLine[] {
const ledger = egressLedgerPath(home);
let content: string;
try {
content = fs.readFileSync(ledger, 'utf8');
} catch (error) {
if ((error as NodeJS.ErrnoException)?.code === 'ENOENT') return [];
throw error;
}
return content.split('\n').filter((line) => line.length > 0).map((raw, index) => {
let record: Record<string, unknown> | null = null;
try {
const parsed = JSON.parse(raw);
if (parsed && typeof parsed === 'object') record = parsed;
} catch { /* malformed line — verifyLedger reports it */ }
return { lineNo: index + 1, raw, record };
});
}
export interface Receipt {
ts: string;
type: 'egress';
sink: string;
host: string;
payload_class: string;
bytes: number;
sha256: string | null;
consent: string;
prev: string;
id: string;
status: string | null;
}
/** Receipts with their joined outcome status (`status: null` = none recorded). */
export function listReceipts(home: string): Receipt[] {
const lines = readLedger(home);
const receipts: Receipt[] = [];
const byId = new Map<string, Receipt>();
for (const { raw, record } of lines) {
if (!record) continue;
if (record.type === 'egress') {
const entry = { ...(record as unknown as Omit<Receipt, 'id' | 'status'>), id: sha256Hex(raw), status: null };
receipts.push(entry);
byId.set(entry.id, entry);
} else if (record.type === 'outcome' && byId.has(record.receipt as string)) {
byId.get(record.receipt as string)!.status = String(record.status);
}
}
return receipts;
}
/**
* Recompute the hash chain. `brokenLine` is the 1-indexed first line whose
* `prev` no longer matches the sha256 of the previous raw line (or that
* fails to parse). `sizeWarning` is set when the ledger exceeds
* LEDGER_WARN_BYTES.
*/
export function verifyLedger(home: string): VerifyResult {
const ledger = egressLedgerPath(home);
let sizeWarning: string | null = null;
try {
const size = fs.statSync(ledger).size;
if (size > LEDGER_WARN_BYTES) sizeWarning = ledgerSizeWarning(ledger, size);
} catch { /* missing ledger — verify of an empty chain below */ }
const lines = readLedger(home);
let previousRaw: string | null = null;
for (const { lineNo, raw, record } of lines) {
if (!record || typeof record.prev !== 'string') {
return { ok: false, count: lines.length, brokenLine: lineNo, reason: 'unparseable or missing prev', sizeWarning };
}
const expected = previousRaw == null ? '' : sha256Hex(previousRaw);
if (record.prev !== expected) {
return { ok: false, count: lines.length, brokenLine: lineNo, reason: 'prev hash does not match previous line', sizeWarning };
}
previousRaw = raw;
}
return { ok: true, count: lines.length, brokenLine: null, reason: null, sizeWarning };
}

8
package.json
View File

@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.62.0.0",
"version": "1.63.0.0",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
@ -27,6 +27,8 @@
"test:e2e:all": "EVALS=1 EVALS_ALL=1 bun test --retry 2 --concurrent --max-concurrency ${EVALS_CONCURRENCY:-15} test/skill-e2e-*.test.ts test/skill-routing-e2e.test.ts test/codex-e2e.test.ts test/gemini-e2e.test.ts",
"test:gate": "EVALS=1 EVALS_TIER=gate bun test --retry 2 --concurrent --max-concurrency ${EVALS_CONCURRENCY:-15} test/skill-llm-eval.test.ts test/skill-e2e-*.test.ts test/skill-routing-e2e.test.ts test/codex-e2e.test.ts test/gemini-e2e.test.ts",
"test:periodic": "EVALS=1 EVALS_TIER=periodic EVALS_ALL=1 bun test --retry 2 --concurrent --max-concurrency ${EVALS_CONCURRENCY:-15} test/skill-e2e-*.test.ts test/skill-routing-e2e.test.ts test/codex-e2e.test.ts test/gemini-e2e.test.ts",
"test:gate:sharded": "bun run scripts/test-paid-shards.ts --tier gate",
"test:periodic:sharded": "EVALS_ALL=1 bun run scripts/test-paid-shards.ts --tier periodic",
"test:codex": "EVALS=1 bun test test/codex-e2e.test.ts",
"test:codex:all": "EVALS=1 EVALS_ALL=1 bun test test/codex-e2e.test.ts",
"test:gemini": "EVALS=1 bun test test/gemini-e2e.test.ts",
@ -36,8 +38,8 @@
"start": "bun run browse/src/server.ts",
"eval:bg": "bin/gstack-detach --label evals --lock gstack-evals --timeout 5400 -- bun run test:evals",
"eval:bg:all": "bin/gstack-detach --label evals-all --lock gstack-evals --timeout 7200 -- bun run test:evals:all",
"eval:bg:gate": "bin/gstack-detach --label evals-gate --lock gstack-evals --timeout 3600 -- bun run test:gate",
"eval:bg:periodic": "bin/gstack-detach --label evals-periodic --lock gstack-evals --timeout 5400 -- bun run test:periodic",
"eval:bg:gate": "bin/gstack-detach --label evals-gate --lock gstack-evals --timeout 25200 -- bun run test:gate:sharded",
"eval:bg:periodic": "bin/gstack-detach --label evals-periodic --lock gstack-evals --timeout 28800 -- bun run test:periodic:sharded",
"eval:list": "bun run scripts/eval-list.ts",
"eval:compare": "bun run scripts/eval-compare.ts",
"eval:summary": "bun run scripts/eval-summary.ts",

34
scripts/eval-compare.ts
View File

@ -10,12 +10,13 @@
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import {
findPreviousRun,
compareEvalResults,
formatComparison,
getProjectEvalDir,
isPartialEval,
listEvalJsonFiles,
} from '../test/helpers/eval-store';
import type { EvalResult } from '../test/helpers/eval-store';
@ -52,25 +53,34 @@ if (args.length === 2) {
}
beforeFile = prev;
} else {
// No args — find two most recent of the same tier
let files: string[];
try {
files = fs.readdirSync(EVAL_DIR)
.filter(f => f.endsWith('.json'))
.sort()
.reverse();
} catch {
// No args — find two most recent of the same tier. Scans the flat dir plus
// one level of shards/<slug>/; in-progress accumulators are never the
// "after" run (comparing against a half-finished run says nothing).
const files = listEvalJsonFiles(EVAL_DIR)
.sort((a, b) => path.basename(b).localeCompare(path.basename(a)));
if (files.length === 0) {
console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
process.exit(0);
}
if (files.length < 2) {
console.log('Need at least 2 eval runs to compare. Run evals again.');
process.exit(0);
}
// Most recent file
afterFile = path.join(EVAL_DIR, files[0]);
// Most recent finalized file
const latest = files.find(f => {
try {
return !isPartialEval(JSON.parse(fs.readFileSync(f, 'utf-8')), f);
} catch {
return false;
}
});
if (!latest) {
console.log('No completed eval runs yet. Run: EVALS=1 bun run test:evals');
process.exit(0);
}
afterFile = latest;
const afterResult = loadResult(afterFile);
const prev = findPreviousRun(EVAL_DIR, afterResult.tier, afterResult.branch, afterFile);
if (!prev) {

16
scripts/eval-list.ts
View File

@ -6,9 +6,7 @@
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { getProjectEvalDir } from '../test/helpers/eval-store';
import { getProjectEvalDir, listEvalJsonFiles } from '../test/helpers/eval-store';
const EVAL_DIR = getProjectEvalDir();
@ -37,14 +35,8 @@ for (let i = 0; i < args.length; i++) {
else if (args[i] === '--limit') { limit = parseLimit(args[++i]); }
}
// Read eval files
let files: string[];
try {
files = fs.readdirSync(EVAL_DIR).filter(f => f.endsWith('.json'));
} catch {
console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
process.exit(0);
}
// Read eval files (flat dir plus one level of shards/<slug>/)
const files = listEvalJsonFiles(EVAL_DIR);
if (files.length === 0) {
console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
@ -68,7 +60,7 @@ interface RunSummary {
const runs: RunSummary[] = [];
for (const file of files) {
try {
const data = JSON.parse(fs.readFileSync(path.join(EVAL_DIR, file), 'utf-8'));
const data = JSON.parse(fs.readFileSync(file, 'utf-8'));
if (filterBranch && data.branch !== filterBranch) continue;
if (filterTier && data.tier !== filterTier) continue;
const totalTurns = (data.tests || []).reduce((s: number, t: any) => s + (t.turns_used || 0), 0);

15
scripts/eval-summary.ts
View File

@ -6,20 +6,13 @@
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import type { EvalResult } from '../test/helpers/eval-store';
import { getProjectEvalDir } from '../test/helpers/eval-store';
import { getProjectEvalDir, listEvalJsonFiles } from '../test/helpers/eval-store';
const EVAL_DIR = getProjectEvalDir();
let files: string[];
try {
files = fs.readdirSync(EVAL_DIR).filter(f => f.endsWith('.json'));
} catch {
console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
process.exit(0);
}
// Flat dir plus one level of shards/<slug>/
const files = listEvalJsonFiles(EVAL_DIR);
if (files.length === 0) {
console.log('No eval runs yet. Run: EVALS=1 bun run test:evals');
@ -30,7 +23,7 @@ if (files.length === 0) {
const results: EvalResult[] = [];
for (const file of files) {
try {
results.push(JSON.parse(fs.readFileSync(path.join(EVAL_DIR, file), 'utf-8')));
results.push(JSON.parse(fs.readFileSync(file, 'utf-8')));
} catch { continue; }
}

14
scripts/test-free-shards.ts
View File

@ -27,22 +27,12 @@
import * as fs from 'fs';
import * as path from 'path';
import { spawnSync } from 'child_process';
import { isPaidTestFile } from '../test/helpers/paid-test-set';
const ROOT = path.resolve(import.meta.dir, '..');
const TEST_ROOTS = ['browse/test', 'test', 'make-pdf/test'] as const;
const TEST_FILE_REGEX = /\.test\.(?:[cm]?[jt]s|tsx|jsx)$/;
// Tests that require API spend, external services, or e2e harnesses.
// These are filtered out before any sharding or curation.
const PAID_EVAL_TESTS = [
/^browse\/test\/security-review-fullstack\.test\.ts$/,
/^test\/skill-e2e-.*\.test\.ts$/,
/^test\/skill-llm-eval\.test\.ts$/,
/^test\/skill-routing-e2e\.test\.ts$/,
/^test\/codex-e2e\.test\.ts$/,
/^test\/gemini-e2e\.test\.ts$/,
] as const;
// POSIX-only patterns that indicate a test will fail on windows-latest no
// matter how the runner shards. Codex's v1.18.0.0 review flagged the first
// three as concrete examples in the existing free suite (test/ship-version-sync.test.ts:72,
@ -118,7 +108,7 @@ export function normalizeRelativePath(filePath: string): string {
export function isFreeTestFile(relativePath: string): boolean {
const normalized = normalizeRelativePath(relativePath);
if (!TEST_FILE_REGEX.test(normalized)) return false;
return !PAID_EVAL_TESTS.some(pattern => pattern.test(normalized));
return !isPaidTestFile(normalized);
}
/**

479
scripts/test-paid-shards.ts Normal file
View File

@ -0,0 +1,479 @@
#!/usr/bin/env bun
/**
* test-paid-shards — enumerate, shard, and run the paid (gate/periodic) tier.
*
* The single-process `test:gate` fan-out has never completed a run: one wedged
* or spinning file takes the whole tier down, and an in-process `--timeout`
* cannot save it because a spinning main thread never fires a timer. This
* runner applies the free tier's proven fix — one Bun process per shard — plus
* the two things the paid tier additionally needs:
*
* - an EXTERNAL wall-clock timeout that kills the shard's process GROUP, and
* - an aggregate that distinguishes failed from timed-out from never-started,
* so 26% execution can never again look like a pass.
*
* Why not Bun 1.3.13's native `--shard` / isolated runs? Three gaps, each one
* fatal for this tier:
* 1. No detached-process-group SIGKILL. Paid tests spawn `claude` / `codex`
* PTY grandchildren; when a shard hangs, in-process isolation kills the
* Bun worker but the grandchildren survive and burn cores for hours.
* 2. No never-started taxonomy. A run that aborts partway reports only what
* executed — the shards that never ran are invisible, which is exactly
* the 26%-execution-looks-like-a-pass bug.
* 3. No per-shard env / eval dir. Each shard needs its own GSTACK_EVAL_DIR
* so eval baselines are per-test-file instead of last-flush-wins.
*
* Worst-case wall clock (all shards hit the 30min timeout, 4 parallel jobs):
* gate tier is 49 shards × 30min / 4 jobs ≈ 6.2h; periodic is 59 shards ≈ 7.4h.
* The eval:bg:* detach timeouts (25200s / 28800s) are sized against these.
*
* Enumeration matches package.json's `test:gate` globs (via the shared
* test/helpers/paid-test-set.ts) and honors EVALS_TIER against the E2E_TIERS
* map in test/helpers/touchfiles.ts. Output classification reuses
* scripts/test-strict-output.ts rather than reimplementing it.
*
* Parallelism now lives ACROSS shards (--jobs), not inside one Bun process, so
* each shard runs its own file sequentially and can be killed independently.
*
* Usage:
* bun run scripts/test-paid-shards.ts --list # shard plan only
* bun run scripts/test-paid-shards.ts --tier gate # run gate tier
* bun run scripts/test-paid-shards.ts --timeout 600 --jobs 2
*/
import { spawn, type ChildProcess } from 'node:child_process';
import * as fs from 'node:fs';
import * as path from 'node:path';
import { normalizeRelativePath } from './test-free-shards';
import {
BunTestOutputClassifier,
exactTestFileSelectors,
forwardAndClassify,
installChildSignalForwarding,
strictTestExitCode,
} from './test-strict-output';
import { PAID_TEST_GLOBS, isPaidTestFile } from '../test/helpers/paid-test-set';
import { getProjectEvalDir } from '../test/helpers/eval-store';
export { PAID_TEST_GLOBS, isPaidTestFile };
const ROOT = path.resolve(import.meta.dir, '..');
export type PaidTier = 'gate' | 'periodic';
export const DEFAULT_TIER: PaidTier = 'gate';
export const DEFAULT_SHARD_TIMEOUT_MS = 30 * 60_000;
export const DEFAULT_MAX_FILES_PER_SHARD = 1;
export const DEFAULT_JOBS = 4;
export function collectPaidTestFiles(rootDir = ROOT): string[] {
const testDir = path.join(rootDir, 'test');
if (!fs.existsSync(testDir)) return [];
return fs.readdirSync(testDir)
.map((name) => `test/${name}`)
.filter(isPaidTestFile)
.sort();
}
export interface TierClassification {
included: boolean;
reason: string;
}
/**
* Decide whether a paid test file has anything to run in `tier`.
*
* Per-TEST tier filtering already happens at runtime: test/helpers/e2e-helpers.ts
* intersects the selected tests with E2E_TIERS whenever EVALS_TIER is set, and
* this runner passes EVALS_TIER down to every shard. So this file-level pass is
* only an optimization — skipping a file merely saves one near-instant shard.
*
* Exclusion is the dangerous direction (a wrongly-skipped gate test is exactly
* the invisible-non-execution bug this runner exists to kill), so the only
* exclusion evidence accepted is an explicit whole-file `EVALS_TIER === '<other>'`
* guard. Inferring a file's tier from which E2E_TIERS names appear in its source
* is guesswork that silently drops real work: short keys like 'retro' match
* unrelated strings, and LLM-judge tests are keyed off LLM_JUDGE_TOUCHFILES and
* carry no E2E_TIERS name at all. Everything without an explicit other-tier
* guard runs and self-skips.
*/
export function classifyPaidTestFile(source: string, tier: PaidTier): TierClassification {
const other: PaidTier = tier === 'gate' ? 'periodic' : 'gate';
const declares = (candidate: PaidTier) =>
new RegExp(`EVALS_TIER\\s*===\\s*['"\`]${candidate}['"\`]`).test(source);
if (declares(tier)) return { included: true, reason: `declares EVALS_TIER === '${tier}'` };
if (declares(other)) return { included: false, reason: `declares EVALS_TIER === '${other}' only` };
return { included: true, reason: 'no whole-file tier guard — runtime E2E_TIERS filter decides' };
}
export interface TierSelection {
selected: string[];
excluded: Array<{ file: string; reason: string }>;
}
export function selectPaidTestFiles(files: string[], tier: PaidTier, rootDir = ROOT): TierSelection {
const selected: string[] = [];
const excluded: Array<{ file: string; reason: string }> = [];
for (const file of files) {
const source = fs.readFileSync(path.join(rootDir, file), 'utf8');
const classification = classifyPaidTestFile(source, tier);
if (classification.included) selected.push(file);
else excluded.push({ file, reason: classification.reason });
}
return { selected, excluded };
}
export function planPaidShards(
files: string[],
options: { maxFilesPerShard?: number } = {},
): string[][] {
const size = Math.max(1, options.maxFilesPerShard ?? DEFAULT_MAX_FILES_PER_SHARD);
const unique = [...new Set(files.map(normalizeRelativePath))].sort();
const shards: string[][] = [];
for (let index = 0; index < unique.length; index += size) shards.push(unique.slice(index, index + size));
return shards;
}
export function buildPaidShardArgs(files: string[], timeoutMs: number): string[] {
return ['test', ...files, '--retry', '2', `--timeout=${timeoutMs}`];
}
/**
* Stable per-shard eval-dir slug: test filename sans extension, sanitized.
* Stable across runs so each shard baselines against its own prior run.
*/
export function shardSlug(files: string[]): string {
return files
.map((file) => path.basename(normalizeRelativePath(file)).replace(/\.test\.(?:[cm]?[jt]s|tsx|jsx)$/, ''))
.join('+')
.replace(/[^a-zA-Z0-9._+-]/g, '-');
}
export type ShardStatus = 'passed' | 'failed' | 'timed-out' | 'never-started';
export interface ShardOutcome {
shard: number;
files: string[];
status: ShardStatus;
exitCode: number | null;
elapsedMs: number;
groupPid: number | null;
}
export interface ShardCommand {
command: string;
args: string[];
}
export interface RunShardsOptions {
timeoutMs?: number;
jobs?: number;
rootDir?: string;
env?: NodeJS.ProcessEnv;
/** When set, each shard child gets GSTACK_EVAL_DIR=<evalDirBase>/shards/<slug>/. */
evalDirBase?: string;
/** Override the spawned command. Tests inject fake slow/spinning commands. */
commandFor?: (files: string[]) => ShardCommand;
log?: (line: string) => void;
}
/**
* SIGKILL the shard's whole process group. Orphaned grandchildren (browsers,
* claude sessions) are how a stalled run once burned a core for 15.7 hours.
*/
function killProcessGroup(child: ChildProcess, signal: NodeJS.Signals): void {
if (process.platform === 'win32' || typeof child.pid !== 'number') {
child.kill(signal);
return;
}
try {
process.kill(-child.pid, signal);
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
if (code === 'ESRCH') return; // group already gone
if (code !== 'EPERM') throw err;
// Observed on macOS after a SIGKILLed group is reaped: signalling the
// now-empty group id returns EPERM, not ESRCH. Throwing here loses the
// shard's real outcome (a timeout gets recorded as a failure) and, from
// the timeout timer, leaves the shard promise unsettled — a hang, which
// is the exact failure class this runner exists to kill. Fall back to the
// direct pid so a genuinely-live child is still signalled.
try {
child.kill(signal);
} catch {
// Best-effort reap: nothing actionable is left if this fails too.
}
}
}
export async function runPaidShard(
files: string[],
shardNumber: number,
totalShards: number,
options: RunShardsOptions = {},
): Promise<ShardOutcome> {
if (files.length === 0) throw new Error('Cannot run an empty paid-test shard.');
const rootDir = options.rootDir ?? ROOT;
const timeoutMs = options.timeoutMs ?? DEFAULT_SHARD_TIMEOUT_MS;
const streamLive = (options.jobs ?? DEFAULT_JOBS) === 1;
const log = options.log ?? ((line: string) => console.log(line));
const label = `[test:paid] shard ${shardNumber}/${totalShards}`;
const { command, args } = options.commandFor
? options.commandFor(files)
: {
command: process.execPath,
args: buildPaidShardArgs(exactTestFileSelectors(files, rootDir), timeoutMs),
};
const env = { ...(options.env ?? process.env) };
if (options.evalDirBase) {
env.GSTACK_EVAL_DIR = path.join(options.evalDirBase, 'shards', shardSlug(files));
}
const startedAt = Date.now();
log(`${label} START ${files.join(' ')} (timeout ${Math.round(timeoutMs / 1000)}s)`);
const child = spawn(command, args, {
cwd: rootDir,
env,
stdio: ['ignore', 'pipe', 'pipe'],
detached: process.platform !== 'win32',
windowsHide: true,
});
const groupPid = child.pid ?? null;
// Group-kill on parent SIGINT/SIGTERM too, not just on timeout.
const forwarding = installChildSignalForwarding({
kill: (signal?: NodeJS.Signals | number) => {
killProcessGroup(child, (signal as NodeJS.Signals) ?? 'SIGTERM');
return true;
},
});
const classifier = new BunTestOutputClassifier();
const buffered: Buffer[] = [];
const sink = (destination: NodeJS.WriteStream): NodeJS.WriteStream => (streamLive
? destination
: ({ write: (chunk: Buffer | string) => buffered.push(Buffer.from(chunk)) } as unknown as NodeJS.WriteStream));
let timedOut = false;
const killTimer = setTimeout(() => {
timedOut = true;
killProcessGroup(child, 'SIGKILL');
}, timeoutMs);
let exitCode: number | null = null;
try {
const streams: Array<Promise<void>> = [];
if (child.stdout) streams.push(forwardAndClassify(child.stdout, sink(process.stdout), classifier));
if (child.stderr) streams.push(forwardAndClassify(child.stderr, sink(process.stderr), classifier));
exitCode = await new Promise<number | null>((resolve, reject) => {
child.once('error', reject);
child.once('close', (code) => resolve(code));
});
await Promise.all(streams);
} finally {
clearTimeout(killTimer);
forwarding.dispose();
// Reap survivors of this shard even on the clean path.
killProcessGroup(child, 'SIGKILL');
}
const summary = classifier.end();
if (!streamLive && buffered.length > 0) process.stdout.write(Buffer.concat(buffered));
// Pass expectedFiles so a shard whose bun child ran fewer files than planned
// (or zero, all self-skipped) with exit 0 is NOT recorded 'passed' — the
// invisible-non-execution class this runner exists to kill. bun prints
// "Ran N tests across M files" with M = selected files even when every test
// self-skips, so terminalFileCounts must include files.length. Only enforced
// on the real bun path: an injected commandFor (tests) isn't bun and emits no
// terminal summary, so there's no file count to check against.
const expectedFiles = options.commandFor ? undefined : files.length;
const status: ShardStatus = timedOut
? 'timed-out'
: strictTestExitCode(exitCode ?? 1, summary, expectedFiles) === 0 ? 'passed' : 'failed';
const elapsedMs = Date.now() - startedAt;
log(`${label} ${status.toUpperCase()} in ${Math.round(elapsedMs / 1000)}s (exit ${exitCode ?? 'signal'})`);
return { shard: shardNumber, files, status, exitCode, elapsedMs, groupPid };
}
export interface RunSummary {
total: number;
executed: number;
passed: number;
failed: number;
timedOut: number;
neverStarted: number;
outcomes: ShardOutcome[];
}
export function summarize(outcomes: ShardOutcome[]): RunSummary {
const count = (status: ShardStatus) => outcomes.filter((o) => o.status === status).length;
return {
total: outcomes.length,
executed: outcomes.length - count('never-started'),
passed: count('passed'),
failed: count('failed'),
timedOut: count('timed-out'),
neverStarted: count('never-started'),
outcomes,
};
}
/** Run every shard in its own process. A timeout or failure never aborts the run. */
export async function runPaidShards(
shards: string[][],
options: RunShardsOptions = {},
): Promise<RunSummary> {
const jobs = Math.max(1, options.jobs ?? DEFAULT_JOBS);
const outcomes: ShardOutcome[] = shards.map((files, index) => ({
shard: index + 1,
files,
status: 'never-started',
exitCode: null,
elapsedMs: 0,
groupPid: null,
}));
let next = 0;
const worker = async (): Promise<void> => {
while (true) {
const index = next;
next += 1;
if (index >= shards.length) return;
try {
outcomes[index] = await runPaidShard(shards[index], index + 1, shards.length, { ...options, jobs });
} catch (error) {
outcomes[index] = {
shard: index + 1,
files: shards[index],
status: 'failed',
exitCode: null,
elapsedMs: 0,
groupPid: null,
};
console.error(`[test:paid] shard ${index + 1} could not run: ${error instanceof Error ? error.message : String(error)}`);
}
}
};
await Promise.all(Array.from({ length: Math.min(jobs, shards.length) }, worker));
return summarize(outcomes);
}
export function formatSummary(summary: RunSummary): string[] {
const lines = [
'',
`[test:paid] ${summary.executed}/${summary.total} shards executed — `
+ `${summary.passed} passed, ${summary.failed} failed, `
+ `${summary.timedOut} timed out, ${summary.neverStarted} never started`,
];
for (const outcome of summary.outcomes) {
lines.push(
` ${outcome.status.padEnd(13)} ${String(Math.round(outcome.elapsedMs / 1000)).padStart(5)}s `
+ outcome.files.join(' '),
);
}
return lines;
}
type CliOptions = {
tier: PaidTier;
listOnly: boolean;
timeoutMs: number;
jobs: number;
maxFilesPerShard: number;
};
function parsePositiveInt(value: string | undefined, flag: string): number {
const parsed = Number.parseInt(value ?? '', 10);
if (!Number.isInteger(parsed) || parsed <= 0) throw new Error(`${flag} needs a positive integer. Received: ${value}`);
return parsed;
}
function validatedTier(value: string | undefined, source: string): PaidTier {
if (value === undefined || value === '') return DEFAULT_TIER;
// A typo'd EVALS_TIER (e.g. 'e2e', the tier string eval-store uses) would
// otherwise cast through unchecked, match nothing in the runtime E2E_TIERS
// filter, self-skip every test, and exit 0 with all shards 'passed' — the
// exact 0%-execution-looks-like-a-pass class this runner exists to kill.
if (value !== 'gate' && value !== 'periodic') {
throw new Error(`${source} must be gate or periodic. Received: ${value}`);
}
return value;
}
export function parseCliOptions(argv: string[], env: NodeJS.ProcessEnv = process.env): CliOptions {
const options: CliOptions = {
tier: validatedTier(env.EVALS_TIER, 'EVALS_TIER'),
listOnly: false,
timeoutMs: env.EVALS_SHARD_TIMEOUT_MS
? parsePositiveInt(env.EVALS_SHARD_TIMEOUT_MS, 'EVALS_SHARD_TIMEOUT_MS')
: DEFAULT_SHARD_TIMEOUT_MS,
jobs: env.EVALS_CONCURRENCY ? parsePositiveInt(env.EVALS_CONCURRENCY, 'EVALS_CONCURRENCY') : DEFAULT_JOBS,
maxFilesPerShard: DEFAULT_MAX_FILES_PER_SHARD,
};
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
if (arg === '--list') { options.listOnly = true; continue; }
if (arg === '--tier') {
const value = argv[index += 1];
if (value !== 'gate' && value !== 'periodic') throw new Error(`--tier must be gate or periodic. Received: ${value}`);
options.tier = value;
continue;
}
if (arg === '--timeout') { options.timeoutMs = parsePositiveInt(argv[index += 1], '--timeout') * 1000; continue; }
if (arg === '--jobs') { options.jobs = parsePositiveInt(argv[index += 1], '--jobs'); continue; }
if (arg === '--files-per-shard') { options.maxFilesPerShard = parsePositiveInt(argv[index += 1], '--files-per-shard'); continue; }
throw new Error(`Unknown argument: ${arg}`);
}
return options;
}
async function main(): Promise<number> {
const options = parseCliOptions(process.argv.slice(2));
const discovered = collectPaidTestFiles();
if (discovered.length === 0) throw new Error('No paid test files were discovered.');
const { selected, excluded } = selectPaidTestFiles(discovered, options.tier);
const shards = planPaidShards(selected, { maxFilesPerShard: options.maxFilesPerShard });
console.log(
`[test:paid] tier=${options.tier}: ${selected.length}/${discovered.length} files, `
+ `${shards.length} shards, jobs=${options.jobs}, timeout=${Math.round(options.timeoutMs / 1000)}s`,
);
if (options.listOnly) {
for (let index = 0; index < shards.length; index += 1) {
console.log(` shard ${index + 1}/${shards.length}: ${shards[index].join(' ')}`);
}
if (excluded.length > 0) {
console.log(`\nExcluded (${excluded.length}):`);
for (const { file, reason } of excluded) console.log(` - ${file} [${reason}]`);
}
return 0;
}
const summary = await runPaidShards(shards, {
// Tier reaches the children only via EVALS_TIER below; the runtime
// E2E_TIERS filter inside each child is the real selection mechanism.
timeoutMs: options.timeoutMs,
jobs: options.jobs,
env: { ...process.env, EVALS: '1', EVALS_TIER: options.tier },
evalDirBase: process.env.GSTACK_EVAL_DIR || getProjectEvalDir(),
});
for (const line of formatSummary(summary)) console.log(line);
return summary.passed === summary.total ? 0 : 1;
}
if (import.meta.main) {
try {
process.exitCode = await main();
} catch (error) {
console.error(`[test:paid] ${error instanceof Error ? error.message : String(error)}`);
process.exitCode = 1;
}
}

200
scripts/test-strict-output.ts Normal file
View File

@ -0,0 +1,200 @@
/**
* Strict Bun-test output classification + child lifecycle helpers.
*
* Works around a Bun test runner bug where failures can be printed even though
* the child exits successfully: output is forwarded byte-for-byte as it
* arrives, and only complete Bun result lines and terminal summaries are
* classified. `strictTestExitCode` then refuses to trust a zero exit when the
* output shows failures (or when fewer files ran than expected).
*
* Shared by the sharded paid-tier runner (scripts/test-paid-shards.ts) and any
* future strict wrapper around `bun test`.
*/
import { type ChildProcess } from 'node:child_process';
import { StringDecoder } from 'node:string_decoder';
import * as path from 'node:path';
const ROOT = path.resolve(import.meta.dir, '..');
const ANSI_ESCAPE = /\u001B\[[0-?]*[ -/]*[@-~]/g;
const BUN_FAIL_RESULT = /^\(fail\) .+ \[(?:\d+(?:\.\d+)?)(?:ns|us|µs|ms|s)\]$/;
const BUN_BETWEEN_TESTS_ERROR = '# Unhandled error between tests';
const BUN_TERMINAL_SUMMARY = /^Ran \d+ tests? across (\d+) files?\. \[(?:\d+(?:\.\d+)?)(?:ns|us|µs|ms|s)\]$/;
export type BunTestOutputFinding = 'failed-test' | 'unhandled-between-tests';
export interface BunTestOutputSummary {
failedTests: number;
unhandledBetweenTests: number;
terminalFileCounts: number[];
}
export type ForwardedTerminationSignal = 'SIGINT' | 'SIGTERM';
export interface TerminationSignalSource {
on(event: string, listener: () => void): unknown;
off(event: string, listener: () => void): unknown;
}
export interface TerminationTimerApi {
schedule(callback: () => void, delayMs: number): unknown;
cancel(handle: unknown): void;
}
export interface ChildSignalForwarding {
readonly receivedSignal: ForwardedTerminationSignal | null;
dispose(): void;
}
const DEFAULT_TERMINATION_TIMER: TerminationTimerApi = {
schedule: (callback, delayMs) => setTimeout(callback, delayMs),
cancel: (handle) => clearTimeout(handle as ReturnType<typeof setTimeout>),
};
/**
* Bind one active child to the parent's termination lifecycle. SIGINT and
* SIGTERM get a grace period so Bun can clean up; a repeated signal, timeout,
* or synchronous parent exit uses SIGKILL so the child cannot be orphaned.
*/
export function installChildSignalForwarding(
child: Pick<ChildProcess, 'kill'>,
source: TerminationSignalSource = process,
timer: TerminationTimerApi = DEFAULT_TERMINATION_TIMER,
graceMs = 5_000,
): ChildSignalForwarding {
let receivedSignal: ForwardedTerminationSignal | null = null;
let forceTimer: unknown = null;
let disposed = false;
const forward = (signal: ForwardedTerminationSignal): void => {
if (disposed) return;
if (receivedSignal !== null) {
child.kill('SIGKILL');
return;
}
receivedSignal = signal;
child.kill(signal);
forceTimer = timer.schedule(() => {
forceTimer = null;
child.kill('SIGKILL');
}, graceMs);
};
const onSigint = () => forward('SIGINT');
const onSigterm = () => forward('SIGTERM');
const onExit = () => { child.kill('SIGKILL'); };
source.on('SIGINT', onSigint);
source.on('SIGTERM', onSigterm);
source.on('exit', onExit);
return {
get receivedSignal() {
return receivedSignal;
},
dispose() {
if (disposed) return;
disposed = true;
source.off('SIGINT', onSigint);
source.off('SIGTERM', onSigterm);
source.off('exit', onExit);
if (forceTimer !== null) timer.cancel(forceTimer);
forceTimer = null;
},
};
}
export function classifyBunTestOutputLine(rawLine: string): BunTestOutputFinding | null {
const line = rawLine.replace(ANSI_ESCAPE, '').replace(/\r$/, '');
if (BUN_FAIL_RESULT.test(line)) return 'failed-test';
if (line === BUN_BETWEEN_TESTS_ERROR) return 'unhandled-between-tests';
return null;
}
export function parseBunTerminalSummaryLine(rawLine: string): number | null {
const line = rawLine.replace(ANSI_ESCAPE, '').replace(/\r$/, '');
const match = BUN_TERMINAL_SUMMARY.exec(line);
return match ? Number.parseInt(match[1], 10) : null;
}
/** Incrementally classifies output without assuming process chunks align to lines. */
export class BunTestOutputClassifier {
private readonly decoder = new StringDecoder('utf8');
private pending = '';
private failedTests = 0;
private unhandledBetweenTests = 0;
private terminalFileCounts: number[] = [];
write(chunk: Uint8Array | string): void {
this.pending += typeof chunk === 'string'
? chunk
: this.decoder.write(Buffer.from(chunk));
this.consumeCompleteLines();
}
end(): BunTestOutputSummary {
this.pending += this.decoder.end();
if (this.pending.length > 0) this.classify(this.pending);
this.pending = '';
return this.summary();
}
summary(): BunTestOutputSummary {
return {
failedTests: this.failedTests,
unhandledBetweenTests: this.unhandledBetweenTests,
terminalFileCounts: [...this.terminalFileCounts],
};
}
private consumeCompleteLines(): void {
let newline = this.pending.indexOf('\n');
while (newline !== -1) {
this.classify(this.pending.slice(0, newline));
this.pending = this.pending.slice(newline + 1);
newline = this.pending.indexOf('\n');
}
}
private classify(line: string): void {
const finding = classifyBunTestOutputLine(line);
if (finding === 'failed-test') this.failedTests += 1;
if (finding === 'unhandled-between-tests') this.unhandledBetweenTests += 1;
const terminalFileCount = parseBunTerminalSummaryLine(line);
if (terminalFileCount !== null) this.terminalFileCounts.push(terminalFileCount);
}
}
export function strictTestExitCode(
childExitCode: number,
summary: BunTestOutputSummary,
expectedFiles?: number,
): number {
if (childExitCode !== 0) return childExitCode;
if (summary.failedTests > 0 || summary.unhandledBetweenTests > 0) return 1;
if (expectedFiles !== undefined && !summary.terminalFileCounts.includes(expectedFiles)) return 1;
return 0;
}
/**
* Bun treats positional test paths as substring filters. Resolve every
* canonical relative path before spawning so `test/foo.test.ts` cannot also
* select `browse/test/foo.test.ts`.
*/
export function exactTestFileSelectors(files: string[], rootDir = ROOT): string[] {
return files.map((file) => path.isAbsolute(file) ? path.normalize(file) : path.resolve(rootDir, file));
}
export function forwardAndClassify(
stream: NodeJS.ReadableStream,
destination: NodeJS.WriteStream,
classifier: BunTestOutputClassifier,
): Promise<void> {
return new Promise((resolve, reject) => {
stream.on('data', (chunk: Buffer | string) => {
classifier.write(chunk);
destination.write(chunk);
});
stream.on('end', resolve);
stream.on('error', reject);
});
}

91
test/brain-sync.test.ts
View File

@ -34,7 +34,11 @@ function run(argv: string[], opts: { env?: Record<string, string>; input?: strin
const bin = argv[0];
const full = bin.startsWith('/') ? bin : path.join(BIN, bin);
const res = spawnSync(full, argv.slice(1), {
env: { ...process.env, GSTACK_HOME: tmpHome, ...(opts.env || {}) },
// HOME is overridden too: gstack-artifacts-init writes
// $HOME/.gstack-artifacts-remote.txt (plain $HOME, not GSTACK_HOME), so
// without this every free-suite run clobbers the operator's real
// artifacts-remote pointer. Keep it inside tmpHome, which afterEach removes.
env: { ...process.env, HOME: tmpHome, GSTACK_HOME: tmpHome, ...(opts.env || {}) },
encoding: 'utf-8',
input: opts.input,
cwd: ROOT,
@ -56,13 +60,18 @@ beforeEach(() => {
afterEach(() => {
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(bareRemote, { recursive: true, force: true });
// Clean up any remote-helper file init may have written.
const remoteFile = path.join(os.homedir(), '.gstack-brain-remote.txt');
// Only remove if it points at OUR bare remote (don't clobber a real user file).
try {
const contents = fs.readFileSync(remoteFile, 'utf-8').trim();
if (contents === bareRemote) fs.unlinkSync(remoteFile);
} catch {}
// Clean up any remote-helper file init may have written. run() now pins
// HOME to tmpHome so these land inside the removed temp dir, but scrub the
// real home too as defense in depth — and cover BOTH the legacy brain-remote
// name and the current artifacts-remote name (init writes the latter).
for (const name of ['.gstack-brain-remote.txt', '.gstack-artifacts-remote.txt']) {
const remoteFile = path.join(os.homedir(), name);
// Only remove if it points at OUR bare remote (don't clobber a real user file).
try {
const contents = fs.readFileSync(remoteFile, 'utf-8').trim();
if (contents === bareRemote) fs.unlinkSync(remoteFile);
} catch {}
}
});
// ---------------------------------------------------------------
@ -330,6 +339,72 @@ describe('gstack-brain-sync secret scan', () => {
});
});
// ---------------------------------------------------------------
// Egress receipt gate: receipt-before-commit, queue intact on refusal
// ---------------------------------------------------------------
describe('gstack-brain-sync egress receipt gate', () => {
test('refused receipt leaves the queue intact, makes no commit, and next run retries', () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return; // chmod is advisory there
run(['gstack-artifacts-init', '--remote', bareRemote]);
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
'{"skill":"x","insight":"y","ts":"2026-04-22T10:00:00Z"}\n');
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
const commitsBefore = git(['rev-list', '--count', 'HEAD']).stdout.trim();
// Make the receipt unwritable: security dir exists but is read-only.
// (artifacts-init may have created it already — mkdirSync's mode is a
// no-op on an existing dir, so chmod explicitly.)
fs.mkdirSync(path.join(tmpHome, 'security'), { recursive: true });
fs.chmodSync(path.join(tmpHome, 'security'), 0o500);
try {
const refused = run(['gstack-brain-sync', '--once']);
expect(refused.status).toBe(1);
// DX contract: problem + cause + fix, plain language.
expect(refused.stderr).toContain('NOT sent');
expect(refused.stderr).toContain('EGRESS_RECEIPT_FAILED');
expect(refused.stderr).toContain('Fix: chmod -R u+w');
expect(refused.stderr).toContain('ATTEMPTS to send off-machine');
// Queue intact (receipt is written BEFORE the commit consumes it).
const queue = fs.readFileSync(path.join(tmpHome, '.brain-queue.jsonl'), 'utf-8');
expect(queue).toContain('projects/p/learnings.jsonl');
// No local commit was created.
expect(git(['rev-list', '--count', 'HEAD']).stdout.trim()).toBe(commitsBefore);
// Nothing reached the remote.
const remoteLog = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
expect(remoteLog.stdout).not.toMatch(/sync: 1 file/);
const status = JSON.parse(fs.readFileSync(path.join(tmpHome, '.brain-sync-status.json'), 'utf-8'));
expect(status.status).toBe('push_failed');
expect(status.message).toContain('EGRESS_RECEIPT_FAILED');
} finally {
fs.chmodSync(path.join(tmpHome, 'security'), 0o700);
}
// Next run (ledger writable again) drains the intact queue and pushes.
const retry = run(['gstack-brain-sync', '--once']);
expect(retry.status).toBe(0);
const log = spawnSync('git', ['--git-dir=' + bareRemote, 'log', '--oneline'], { encoding: 'utf-8' });
expect(log.stdout).toMatch(/sync: 1 file/);
});
test('successful push writes a git-class receipt before the send', () => {
run(['gstack-artifacts-init', '--remote', bareRemote]);
run(['gstack-config', 'set', 'artifacts_sync_mode', 'full']);
fs.mkdirSync(path.join(tmpHome, 'projects', 'p'), { recursive: true });
fs.writeFileSync(path.join(tmpHome, 'projects/p/learnings.jsonl'),
'{"skill":"x","insight":"y","ts":"2026-04-22T10:00:00Z"}\n');
run(['gstack-brain-enqueue', 'projects/p/learnings.jsonl']);
const r = run(['gstack-brain-sync', '--once']);
expect(r.status).toBe(0);
const ledger = fs.readFileSync(path.join(tmpHome, 'security', 'egress.jsonl'), 'utf-8');
const records = ledger.trim().split('\n').map((l) => JSON.parse(l));
const pushReceipt = records.find((rec) => rec.sink === 'brain-sync' && rec.payload_class === 'curated-memory-git-push');
expect(pushReceipt).toBeTruthy();
expect(pushReceipt.sha256).toBeNull(); // git owns the bytes
});
});
// ---------------------------------------------------------------
// Uninstall preserves user data
// ---------------------------------------------------------------

122
test/catalog-budget.test.ts Normal file
View File

@ -0,0 +1,122 @@
import { describe, expect, test } from 'bun:test';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { skillCensus } from './helpers/skill-census';
/**
* Aggregate discovery-surface budget: the sum of every skill's frontmatter
* `name` + `description` is what EVERY host loads at discovery, every session.
*
* This is the missing enforcement layer over the existing catalog-trim
* mechanism: `applyCatalogTrim` in scripts/gen-skill-docs.ts (~line 865)
* shapes each description, and the 160KB per-file warn (~line 1015) covers
* BODY size — neither caps the aggregate frontmatter the catalog is made of.
*
* Import-free by design: parses skills' SKILL.md frontmatter directly. Do not
* import gen-skill-docs internals here — this test must survive generator
* refactors.
*
* Budget derivation (re-derive it, do not trust the number):
* ref this commit
* method for each authored skill (test/helpers/skill-census.ts
* authoredSkills — symlink-deduped, root router excluded) plus the
* root router's `_gstack-command` alias frontmatter as one separate
* line item, run parseFrontmatter() below and sum
* Buffer.byteLength(name) + Buffer.byteLength(description);
* token-equivalents = ceil(bytes / 4).
* result 53 authored skills = 4,371 bytes (1,093 token-equivalents);
* + root router alias 49 bytes = 4,420 bytes total
* = 1,105 token-equivalents (measured 2026-08-12)
* Ceiling is 1,150 token-equivalents (4,600 bytes), so headroom is 180 bytes
* (~4%). Dominant skill: design-consultation at 229 bytes name+description.
*/
const CATALOG_BUDGET_TOKEN_EQUIVALENTS = 1_150;
// Largest today: design-consultation at 229 bytes. A description that needs
// more than 260 bytes is a body paragraph, not a catalog entry.
const PER_SKILL_BYTE_CAP = 260;
const RATCHET_PROTOCOL =
'Adding a skill? Re-measure with: bun test test/catalog-budget.test.ts ' +
'(the failure prints the new total). Update CATALOG_BUDGET_TOKEN_EQUIVALENTS ' +
'AND the derivation comment (ref/date/value/which skill moved it) in the ' +
'SAME commit. Growing an existing description? Trim it instead — the ' +
'catalog is what every host loads at discovery, every session.';
const ROOT = join(import.meta.dir, '..');
function parseFrontmatter(body: string): { name: string; description: string } {
const name = body.match(/^name:\s*(.+)$/m)?.[1]?.trim() ?? '';
// Folded block scalar (description: >-) with two-space-indented continuation
// lines, falling back to a single-line description.
const folded = body.match(/^description:\s*>-?\r?\n((?: .*\r?\n)+)/m)?.[1];
const description = folded
? folded.split(/\r?\n/).map((line) => line.trim()).filter(Boolean).join(' ')
: body.match(/^description:\s*(?!>-?\s*$)(.+)$/m)?.[1]?.trim() ?? '';
return { name, description };
}
interface CatalogEntry {
skill: string;
name: string;
description: string;
bytes: number;
}
function catalogEntries(): CatalogEntry[] {
const entries: CatalogEntry[] = [];
for (const skill of skillCensus(ROOT).authoredSkills) {
const body = readFileSync(join(ROOT, skill, 'SKILL.md'), 'utf8');
const { name, description } = parseFrontmatter(body);
entries.push({
skill,
name,
description,
bytes: Buffer.byteLength(name) + Buffer.byteLength(description),
});
}
// The root SKILL.md is a router, registered by setup as the
// `_gstack-command` alias — not an authored skill, but its frontmatter
// still ships in the catalog, so it counts as one line item.
const router = parseFrontmatter(readFileSync(join(ROOT, 'SKILL.md'), 'utf8'));
if (router.name && router.description) {
entries.push({
skill: '(root router)',
name: router.name,
description: router.description,
bytes: Buffer.byteLength(router.name) + Buffer.byteLength(router.description),
});
}
return entries;
}
describe('catalog discovery-surface budget', () => {
test(`aggregate frontmatter stays within ${CATALOG_BUDGET_TOKEN_EQUIVALENTS} token-equivalents`, () => {
const entries = catalogEntries();
const totalBytes = entries.reduce((sum, e) => sum + e.bytes, 0);
const estimatedTokens = Math.ceil(totalBytes / 4);
const delta = estimatedTokens - CATALOG_BUDGET_TOKEN_EQUIVALENTS;
expect(
estimatedTokens,
`Catalog is ${estimatedTokens} token-equivalents (${totalBytes} bytes), ` +
`${delta} over the ${CATALOG_BUDGET_TOKEN_EQUIVALENTS} budget. ${RATCHET_PROTOCOL}`
).toBeLessThanOrEqual(CATALOG_BUDGET_TOKEN_EQUIVALENTS);
});
test(`every skill's name + description stays under ${PER_SKILL_BYTE_CAP} bytes`, () => {
for (const entry of catalogEntries()) {
expect(
entry.bytes,
`${entry.skill}: name + description is ${entry.bytes} bytes, ` +
`${entry.bytes - PER_SKILL_BYTE_CAP} over the ${PER_SKILL_BYTE_CAP}-byte ` +
`per-skill cap. ${RATCHET_PROTOCOL}`
).toBeLessThanOrEqual(PER_SKILL_BYTE_CAP);
}
});
test('every skill has a non-empty description', () => {
for (const entry of catalogEntries()) {
expect(entry.description, `${entry.skill}: empty or missing frontmatter description`).not.toBe('');
}
});
});

13
test/codex-e2e.test.ts
View File

@ -37,15 +37,24 @@ const CODEX_AVAILABLE = (() => {
const evalsEnabled = !!process.env.EVALS;
// Skip all tests if codex is not available or EVALS is not set.
// External-service tests are periodic-tier (CLAUDE.md tiering rule 3):
// "Requires external service (Codex, Gemini)? -> periodic". The positive
// form below is the canonical whole-file guard shape — the sharded runner's
// classifyPaidTestFile greps for it to exclude this file from gate.
const tierOk = process.env.EVALS_TIER === 'periodic';
// Skip all tests if codex is not available, EVALS is not set, or we're in
// the gate tier.
// Note: Codex uses its own auth from ~/.codex/ config — no OPENAI_API_KEY env var needed.
const SKIP = !CODEX_AVAILABLE || !evalsEnabled;
const SKIP = !CODEX_AVAILABLE || !evalsEnabled || !tierOk;
const describeCodex = SKIP ? describe.skip : describe;
// Log why we're skipping (helpful for debugging CI)
if (!evalsEnabled) {
// Silent — same as Claude E2E tests, EVALS=1 required
} else if (!tierOk) {
process.stderr.write('\nCodex E2E: SKIPPED — external-service test, periodic tier only (EVALS_TIER === \'periodic\')\n');
} else if (!CODEX_AVAILABLE) {
process.stderr.write('\nCodex E2E: SKIPPED — codex binary not found (install: npm i -g @openai/codex)\n');
}

679
test/context-bill.test.ts Normal file
View File

@ -0,0 +1,679 @@
/**
* gstack-context-bill — token bill-of-materials for an installed skills tree.
*
* Free tier, no network, no API keys. Covers the STRIPPED port:
* - ALWAYS-ON ledger: exact frontmatter byte sums, dead-key flag against
* the upstream router-key contract, foreign-host file flag
* - EAGER ledger: SKILL.md + forced-read refs from the "for every
* invocation" phrase; stripped tiers stay zero/empty (shape preserved)
* - the three upstream fixes: (a) root-as-container walking +
* node_modules/dotdir exclusion in walkMd, (b) repo-checkout subdir skip
* for installed trees, (c) widened ROUTER_KEYS
* - token estimate calibration, --json shape, --diff, --budget exit codes
* - --exact via injected fetch: envelope subtraction, measurement
* replacement, typed failures, egress receipt-before-send + fail-open
* - ground truth against THIS repo via test/helpers/skill-census.ts
*/
import { describe, it, expect, beforeAll, afterAll } from "bun:test";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import {
buildBill,
calibrationTable,
checkBudget,
contentClass,
contextBillMain,
diffBills,
estimateTokens,
findSkillDirs,
measureExactTokens,
renderBill,
tokenDisclaimer,
walkMd,
ExactModeError,
TOKEN_DIVISOR,
TOKEN_DIVISORS,
} from "../lib/context-bill";
import { listReceipts, sha256Hex } from "../lib/egress-receipt";
import { skillCensus } from "./helpers/skill-census";
const ROOT = path.join(import.meta.dir, "..");
const TREE_A = path.join(import.meta.dir, "fixtures", "context-bill", "tree-a");
function fileBytes(...segments: string[]): number {
return fs.statSync(path.join(...segments)).size;
}
/** Frontmatter block bytes, computed independently of the implementation. */
function frontmatterBytes(file: string): number {
const text = fs.readFileSync(file, "utf8");
const close = text.indexOf("\n---\n", 3);
return Buffer.byteLength(text.slice(0, close + 5), "utf8");
}
function capture() {
let buf = "";
return {
stream: { write: (s: string) => ((buf += s), true) } as unknown as NodeJS.WriteStream,
text: () => buf,
};
}
describe("always-on ledger", () => {
const bill = buildBill(TREE_A);
it("sums per-skill frontmatter bytes exactly", () => {
const alpha = bill.skills.find((s) => s.name === "alpha")!;
const beta = bill.skills.find((s) => s.name === "beta")!;
expect(alpha.frontmatterBytes).toBe(frontmatterBytes(path.join(TREE_A, "alpha", "SKILL.md")));
expect(beta.frontmatterBytes).toBe(frontmatterBytes(path.join(TREE_A, "beta", "SKILL.md")));
expect(bill.totals.alwaysOnBytes).toBe(alpha.frontmatterBytes + beta.frontmatterBytes);
});
it("flags only keys outside the upstream router contract (fix c: widened ROUTER_KEYS)", () => {
const alpha = bill.skills.find((s) => s.name === "alpha")!;
// `triggers` is part of the upstream frontmatter contract now — flagging
// it was the fork's contract, not this repo's.
expect(alpha.deadKeys).toEqual(["x-dead-key"]);
expect(bill.skills.find((s) => s.name === "beta")!.deadKeys).toEqual([]);
});
it("upstream contract keys are never dead: name/description/version/allowed-tools/triggers/preamble-tier", () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-keys-"));
fs.mkdirSync(path.join(tmp, "s"));
fs.writeFileSync(
path.join(tmp, "s", "SKILL.md"),
"---\nname: s\ndescription: d\nversion: 1\nallowed-tools: Bash\ntriggers: t\npreamble-tier: 2\n---\n\n# S\n",
);
const s = buildBill(tmp).skills[0];
expect(s.deadKeys).toEqual([]);
fs.rmSync(tmp, { recursive: true, force: true });
});
it("flags foreign-host skill-shaped files in scanner scope", () => {
const beta = bill.skills.find((s) => s.name === "beta")!;
expect(beta.foreignFiles.map((f) => f.path)).toEqual(["agents.md"]);
expect(bill.skills.find((s) => s.name === "alpha")!.foreignFiles).toEqual([]);
});
});
describe("eager ledger", () => {
const bill = buildBill(TREE_A);
const alpha = bill.skills.find((s) => s.name === "alpha")!;
it("eager = SKILL.md + only the forced-read references", () => {
expect(alpha.forcedRefs.map((r) => r.path)).toEqual([
"references/CORE.md",
"references/POLICY.md",
]);
const expected =
fileBytes(TREE_A, "alpha", "SKILL.md") +
fileBytes(TREE_A, "alpha", "references", "CORE.md") +
fileBytes(TREE_A, "alpha", "references", "POLICY.md");
expect(alpha.eagerBytes).toBe(expected);
});
it("a skill with no forced reads bills only its SKILL.md", () => {
const beta = bill.skills.find((s) => s.name === "beta")!;
expect(beta.eagerBytes).toBe(fileBytes(TREE_A, "beta", "SKILL.md"));
});
it("stripped tiers stay zero/empty but keep their shape (re-adding is additive)", () => {
// OPTIONAL.md is mandated under a condition and the mode table routes two
// legacy modules — the fork billed those in CONDITIONAL and LAZY. The
// stripped port must not bill them anywhere NOR lose the fields.
expect(alpha.conditionalRefs).toEqual([]);
expect(alpha.conditionalBytes).toBe(0);
expect(alpha.transitiveRefs).toEqual([]);
expect(alpha.transitiveBytes).toBe(0);
expect(alpha.lazy).toEqual([]);
expect(alpha.orphans).toEqual([]);
expect(alpha.fastPath).toBeNull();
expect(alpha.routeCeiling).toBeNull();
// With those tiers stripped, per-invocation == eager.
expect(alpha.perInvocationBytes).toBe(alpha.eagerBytes);
expect(alpha.perInvocationTokens).toBe(alpha.eagerTokens);
});
});
describe("upstream fix a: root-as-container + walkMd exclusions", () => {
it("a root with its own SKILL.md is billed AND walked into (router + children)", () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-root-"));
fs.writeFileSync(path.join(tmp, "SKILL.md"), "---\nname: router\ndescription: r\n---\n# Router\n");
fs.mkdirSync(path.join(tmp, "qa"));
fs.writeFileSync(path.join(tmp, "qa", "SKILL.md"), "---\nname: qa\ndescription: q\n---\n# QA\n");
const dirs = findSkillDirs(tmp).map((d) => path.relative(fs.realpathSync(tmp), fs.realpathSync(d)) || ".");
expect(dirs.sort()).toEqual([".", "qa"]);
// A NON-root skill dir is still a leaf: nothing nested under qa/ counts.
fs.mkdirSync(path.join(tmp, "qa", "nested"));
fs.writeFileSync(path.join(tmp, "qa", "nested", "SKILL.md"), "# not a skill\n");
expect(findSkillDirs(tmp).length).toBe(2);
fs.rmSync(tmp, { recursive: true, force: true });
});
it("walkMd skips node_modules and dot-directories", () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-walk-"));
fs.writeFileSync(path.join(tmp, "real.md"), "x");
fs.mkdirSync(path.join(tmp, "node_modules", "pkg"), { recursive: true });
fs.writeFileSync(path.join(tmp, "node_modules", "pkg", "README.md"), "y".repeat(5000));
fs.mkdirSync(path.join(tmp, ".git"), { recursive: true });
fs.writeFileSync(path.join(tmp, ".git", "notes.md"), "z");
const files = walkMd(tmp).map((f) => path.basename(f));
expect(files).toEqual(["real.md"]);
fs.rmSync(tmp, { recursive: true, force: true });
});
});
describe("upstream fix b: installed-tree layout (repo-checkout subdir skip)", () => {
it("skips a subdir that is its own repo checkout (gstack/ inside ~/.claude/skills)", () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-install-"));
// Flat installed skill dirs.
for (const name of ["qa", "ship"]) {
fs.mkdirSync(path.join(tmp, name));
fs.writeFileSync(path.join(tmp, name, "SKILL.md"), `---\nname: ${name}\ndescription: d\n---\n# ${name}\n`);
}
// A full repo checkout dropped into the tree: has .git and its own router
// SKILL.md plus nested skill sources. None of it is an installed skill.
fs.mkdirSync(path.join(tmp, "gstack", ".git"), { recursive: true });
fs.writeFileSync(path.join(tmp, "gstack", "SKILL.md"), "---\nname: _router\ndescription: d\n---\n# router\n");
fs.mkdirSync(path.join(tmp, "gstack", "review"));
fs.writeFileSync(path.join(tmp, "gstack", "review", "SKILL.md"), "---\nname: review\ndescription: d\n---\n# r\n");
const names = buildBill(tmp).skills.map((s) => s.name).sort();
expect(names).toEqual(["qa", "ship"]);
fs.rmSync(tmp, { recursive: true, force: true });
});
it("follows a directory symlink to a sibling skill (connect-chrome shape)", () => {
if (process.platform === "win32") return;
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-symlink-"));
fs.mkdirSync(path.join(tmp, "open-browser"));
fs.writeFileSync(path.join(tmp, "open-browser", "SKILL.md"), "---\nname: ob\ndescription: d\n---\n# ob\n");
fs.symlinkSync(path.join(tmp, "open-browser"), path.join(tmp, "connect-chrome"));
const names = buildBill(tmp).skills.map((s) => s.name).sort();
// Both entries are real scanner load, so both are billed.
expect(names).toEqual(["connect-chrome", "open-browser"]);
fs.rmSync(tmp, { recursive: true, force: true });
});
it("prefers ./skills, then .agents/skills, then .claude/skills, project before user", async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-detect-"));
const home = path.join(tmp, "home");
const proj = path.join(tmp, "proj");
const plant = (dir: string, name: string) => {
fs.mkdirSync(path.join(dir, name), { recursive: true });
fs.cpSync(TREE_A, path.join(dir, name), { recursive: true });
return path.join(dir, name);
};
const homeClaude = plant(home, path.join(".claude", "skills"));
const homeAgents = plant(home, path.join(".agents", "skills"));
const projClaude = plant(proj, path.join(".claude", "skills"));
const projAgents = plant(proj, path.join(".agents", "skills"));
const run = async () => {
const out = capture();
const code = await contextBillMain([], { cwd: proj, homeDir: home, stdout: out.stream, stderr: out.stream });
expect(code).toBe(0);
return out.text().split("\n")[0];
};
expect(await run()).toContain(projAgents);
fs.rmSync(projAgents, { recursive: true, force: true });
expect(await run()).toContain(projClaude);
fs.rmSync(projClaude, { recursive: true, force: true });
expect(await run()).toContain(homeAgents);
fs.rmSync(homeAgents, { recursive: true, force: true });
expect(await run()).toContain(homeClaude);
fs.rmSync(tmp, { recursive: true, force: true });
});
it("names every candidate it tried when no tree exists", async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-none-"));
const out = capture();
const code = await contextBillMain([], { cwd: tmp, homeDir: tmp, stdout: out.stream, stderr: out.stream });
expect(code).toBe(2);
expect(out.text()).toContain(path.join(tmp, ".agents", "skills"));
expect(out.text()).toContain(path.join(tmp, ".claude", "skills"));
fs.rmSync(tmp, { recursive: true, force: true });
});
});
describe("token estimate calibration", () => {
it("uses the corpus-wide divisor for path-less callers, rounded", () => {
expect(estimateTokens(3900)).toBe(1000);
expect(estimateTokens(TOKEN_DIVISOR * 2)).toBe(2);
});
it("charges each content class its own measured divisor", () => {
expect(contentClass("/x/plan/SKILL.md")).toBe("skillmd");
expect(contentClass("/x/plan/references/legacy/office.md")).toBe("legacy");
expect(contentClass("/x/plan/references/RUNTIME.md")).toBe("reference");
expect(contentClass("/x/plan/references/artifacts/qa/t.md")).toBe("artifact");
expect(contentClass("/x/plan/SKILL.md#frontmatter")).toBe("frontmatter");
expect(TOKEN_DIVISORS.legacy).toBeLessThan(TOKEN_DIVISORS.skillmd);
const alpha = buildBill(TREE_A).skills.find((s) => s.name === "alpha")!;
expect(alpha.skillMdTokens).toBeCloseTo(alpha.skillMdBytes / TOKEN_DIVISORS.skillmd, 6);
expect(alpha.forcedRefs[0].tokens).toBeCloseTo(alpha.forcedRefs[0].bytes / TOKEN_DIVISORS.reference, 6);
});
it("never claims more precision than it has: no divisor is a measurement", () => {
const bill = buildBill(TREE_A);
expect(bill.tokenEstimateErrorPct).toBeGreaterThan(0);
expect(bill.tokenSource).toContain("estimate");
expect(bill.calibration).toBeUndefined();
});
it("names the measured error band rather than a vague 'estimates'", () => {
const text = tokenDisclaimer(buildBill(TREE_A));
expect(text).toContain("ESTIMATES");
expect(text).toMatch(/worst single file 40%/);
expect(text).toContain("--exact");
expect(text).toContain("Bytes are always exact");
const exactText = tokenDisclaimer({
tokenEstimateErrorPct: 0,
tokenSource: "count_tokens (claude-opus-4-5)",
});
expect(exactText).toContain("measured with count_tokens");
expect(exactText).not.toContain("ESTIMATES");
});
});
describe("rendering", () => {
it("text output carries the live ledgers plus flags", () => {
const text = renderBill(buildBill(TREE_A));
expect(text).toContain("ALWAYS-ON (every session): 2 skills");
expect(text).toContain("EAGER (per invocation)");
expect(text).toContain("frontmatter key(s) the router never reads: x-dead-key");
expect(text).toContain("foreign-host file in scanner scope: agents.md");
expect(text).toContain("TOTAL on disk:");
expect(text).toContain("Token source: estimate");
expect(text).toContain("Token counts are ESTIMATES");
});
it("states the always-on row's exclusion of the host's per-skill wrapper", () => {
const text = renderBill(buildBill(TREE_A));
const alwaysOn = text.slice(text.indexOf("ALWAYS-ON"), text.indexOf("EAGER ("));
expect(alwaysOn).toContain("excludes the host's per-skill available_skills XML wrapper");
});
it("--json shape is stable (stripped tiers keep their fields)", async () => {
const out = capture();
const code = await contextBillMain([TREE_A, "--json"], { stdout: out.stream, stderr: out.stream });
expect(code).toBe(0);
const bill = JSON.parse(out.text());
expect(bill.tokenEstimate).toEqual(TOKEN_DIVISORS);
expect(bill.tokenEstimateErrorPct).toBe(40);
expect(Object.keys(bill.totals).sort()).toEqual([
"alwaysOnBytes",
"alwaysOnTokens",
"eagerBytesBySkill",
"eagerTokensBySkill",
"perInvocationBytesBySkill",
"perInvocationTokensBySkill",
"skillCount",
"totalMdBytes",
"totalMdTokens",
]);
const skill = bill.skills[0];
for (const key of ["name", "frontmatterBytes", "frontmatterTokens", "deadKeys", "skillMdBytes", "skillMdTokens", "forcedRefs", "eagerBytes", "eagerTokens", "fastPath", "conditionalRefs", "conditionalBytes", "conditionalTokens", "transitiveRefs", "transitiveBytes", "transitiveTokens", "perInvocationBytes", "perInvocationTokens", "routeCeiling", "lazy", "orphans", "foreignFiles", "totalMdTokens"]) {
expect(skill).toHaveProperty(key);
}
for (const r of skill.forcedRefs) expect(typeof r.tokens).toBe("number");
});
});
describe("--diff", () => {
it("reports exactly the grown row and exits 2", async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-"));
const treeB = path.join(tmp, "tree-b");
fs.cpSync(TREE_A, treeB, { recursive: true });
fs.appendFileSync(path.join(treeB, "alpha", "SKILL.md"), "x".repeat(43000));
const diff = diffBills(buildBill(TREE_A), buildBill(treeB));
expect(diff.rows).toHaveLength(1);
expect(diff.rows[0]).toMatchObject({ ledger: "eager", label: "alpha", delta: 43000 });
expect(diff.grew).toBe(true);
const out = capture();
const code = await contextBillMain(["--diff", TREE_A, treeB], { stdout: out.stream, stderr: out.stream });
expect(code).toBe(2);
expect(out.text()).toContain("eager");
expect(out.text()).toContain("GREW");
fs.rmSync(tmp, { recursive: true, force: true });
});
it("identical trees diff clean and exit 0", async () => {
const out = capture();
const code = await contextBillMain(["--diff", TREE_A, TREE_A], { stdout: out.stream, stderr: out.stream });
expect(code).toBe(0);
expect(out.text()).toContain("No context-cost changes");
});
});
describe("--budget", () => {
const alphaEagerTok = Math.round(buildBill(TREE_A).skills.find((s) => s.name === "alpha")!.eagerTokens);
it("exits 0 under budget, 2 over budget with offending files listed", async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-budget-"));
const okBudget = path.join(tmp, "ok.json");
const tightBudget = path.join(tmp, "tight.json");
fs.writeFileSync(okBudget, JSON.stringify({ alwaysOnTotal: 10000, eagerPerInvocation: { alpha: alphaEagerTok } }));
fs.writeFileSync(tightBudget, JSON.stringify({ eagerPerInvocation: { alpha: alphaEagerTok - 1 } }));
const ok = capture();
expect(await contextBillMain([TREE_A, "--budget", okBudget], { stdout: ok.stream, stderr: ok.stream })).toBe(0);
expect(ok.text()).toContain("Within budget");
const over = capture();
expect(await contextBillMain([TREE_A, "--budget", tightBudget], { stdout: over.stream, stderr: over.stream })).toBe(2);
expect(over.text()).toContain("OVER BUDGET: eagerPerInvocation.alpha");
expect(over.text()).toContain("alpha/SKILL.md");
expect(over.text()).toContain("references/CORE.md");
fs.rmSync(tmp, { recursive: true, force: true });
});
it("perInvocation and routeCeiling keys stay accepted (they gate the eager figure while tiers are stripped)", () => {
const bill = buildBill(TREE_A);
expect(checkBudget(bill, { perInvocation: { alpha: alphaEagerTok } })).toEqual([]);
expect(checkBudget(bill, { perInvocation: { alpha: alphaEagerTok - 1 } })).toHaveLength(1);
expect(checkBudget(bill, { routeCeiling: { alpha: alphaEagerTok } })).toEqual([]);
});
it("checkBudget flags a budgeted skill missing from the tree", () => {
const violations = checkBudget(buildBill(TREE_A), { eagerPerInvocation: { ghost: 100 } });
expect(violations).toHaveLength(1);
expect(violations[0].ceiling).toBe("eagerPerInvocation.ghost");
});
});
describe("--exact (opt-in measurement; offline here via an injected fetch)", () => {
let egressHome: string;
beforeAll(() => {
egressHome = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-egress-"));
});
afterAll(() => {
try { fs.chmodSync(path.join(egressHome, "security"), 0o700); } catch {}
fs.rmSync(egressHome, { recursive: true, force: true });
});
const ENVELOPE = 7;
/** Deterministic stand-in for count_tokens: 1 token per 3 chars, plus envelope. */
function fakeFetch(calls: { body: string }[] = []) {
return (async (_url: string, init: { body: string }) => {
calls.push({ body: init.body });
const { messages } = JSON.parse(init.body);
const text: string = messages[0].content;
return {
ok: true,
json: async () => ({ input_tokens: Math.ceil(text.length / 3) + ENVELOPE }),
};
}) as never;
}
it("writes the egress receipt BEFORE the first count_tokens POST", async () => {
const home = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-receipt-"));
let receiptsAtFirstPost = -1;
const fetchImpl = (async (_url: string, init: { body: string }) => {
if (receiptsAtFirstPost === -1) receiptsAtFirstPost = listReceipts(home).length;
const { messages } = JSON.parse(init.body);
return { ok: true, json: async () => ({ input_tokens: Math.ceil(messages[0].content.length / 3) + ENVELOPE }) };
}) as never;
await measureExactTokens(TREE_A, { model: "m", apiKey: "sk-test", fetchImpl, egressHome: home });
expect(receiptsAtFirstPost).toBe(1); // the receipt existed before any POST
const receipts = listReceipts(home);
expect(receipts).toHaveLength(1);
expect(receipts[0].sink).toBe("context-bill-exact");
expect(receipts[0].host).toBe("api.anthropic.com");
expect(receipts[0].sha256).toBeNull();
expect(receipts[0].bytes).toBeGreaterThan(0);
fs.rmSync(home, { recursive: true, force: true });
});
it("fail-open: an unwritable ledger degrades --exact to the offline estimate, sending nothing", async () => {
if (process.platform === "win32" || process.getuid?.() === 0) return;
const home = fs.mkdtempSync(path.join(os.tmpdir(), "context-bill-refuse-"));
fs.mkdirSync(path.join(home, "security"), { recursive: true, mode: 0o500 });
let called = false;
const stdout = capture();
const stderr = capture();
const code = await contextBillMain([TREE_A, "--exact", "--json"], {
stdout: stdout.stream,
stderr: stderr.stream,
apiKey: "sk-test",
fetchImpl: (() => ((called = true), Promise.reject(new Error("must not send")))) as never,
egressHome: home,
});
expect(code).toBe(0); // the run still answers, with the estimate
expect(called).toBe(false); // NOTHING was sent unrecorded
expect(stderr.text()).toContain("exact_egress_receipt_failed");
expect(stderr.text()).toContain("Falling back to the offline estimate");
expect(JSON.parse(stdout.text()).tokenSource).toContain("estimate");
fs.chmodSync(path.join(home, "security"), 0o700);
fs.rmSync(home, { recursive: true, force: true });
});
it("subtracts the request envelope so small files are not overcharged", async () => {
const calls: { body: string }[] = [];
const measured = await measureExactTokens(TREE_A, {
model: "test-model",
apiKey: "sk-test",
fetchImpl: fakeFetch(calls),
egressHome,
});
expect(measured.tokenSource).toBe("count_tokens (test-model)");
const core = path.join(TREE_A, "alpha", "references", "CORE.md");
const text = fs.readFileSync(core, "utf8");
expect(measured.tokensOf(core, text.length)).toBe(Math.ceil(text.length / 3));
// One probe call for the envelope, then one per measured text.
expect(calls.length).toBe(measured.measuredFiles + 1);
});
it("measures the frontmatter block, not the whole SKILL.md, for the always-on row", async () => {
const measured = await measureExactTokens(TREE_A, {
model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(), egressHome,
});
const skillMd = path.join(TREE_A, "alpha", "SKILL.md");
const fmTokens = measured.tokensOf(`${skillMd}#frontmatter`, 0);
const bodyTokens = measured.tokensOf(skillMd, 0);
expect(fmTokens).toBeGreaterThan(0);
expect(fmTokens).toBeLessThan(bodyTokens);
});
it("exact tokens replace the estimate everywhere the bill prices content", async () => {
const measured = await measureExactTokens(TREE_A, {
model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(), egressHome,
});
const bill = buildBill(TREE_A, { tokensOf: measured.tokensOf, tokenSource: measured.tokenSource });
const alpha = bill.skills.find((s) => s.name === "alpha")!;
expect(alpha.eagerTokens).toBeGreaterThanOrEqual(alpha.eagerBytes / 3);
expect(alpha.eagerTokens).toBeLessThan(alpha.eagerBytes / 3 + 4);
expect(alpha.eagerTokens / (alpha.eagerBytes / TOKEN_DIVISOR)).toBeGreaterThan(1.2);
expect(alpha.eagerTokens).toBe(alpha.skillMdTokens + alpha.forcedRefs.reduce((n, r) => n + r.tokens, 0));
expect(bill.tokenEstimateErrorPct).toBe(0);
expect(renderBill(bill)).toContain("Token counts measured with count_tokens");
expect(renderBill(bill)).not.toMatch(/\(~\d/);
});
it("measures against a relative root too, instead of silently estimating", async () => {
const relative = path.relative(process.cwd(), TREE_A);
const measured = await measureExactTokens(relative, {
model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(), egressHome,
});
const bill = buildBill(relative, { tokensOf: measured.tokensOf, tokenSource: measured.tokenSource });
for (const s of bill.skills) {
expect(s.totalMdBytes / s.totalMdTokens).toBeLessThan(3.2);
}
expect(measured.missedKeys.size).toBe(0);
});
it("counts anything it could not measure instead of passing it off as measured", async () => {
const measured = await measureExactTokens(TREE_A, {
model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(), egressHome,
});
expect(measured.missedKeys.size).toBe(0);
const ghost = path.join(TREE_A, "alpha", "nope.md");
expect(measured.tokensOf(ghost, 4150)).toBeGreaterThan(0);
expect(measured.missedKeys.has(ghost)).toBe(true);
});
it("grades its own estimate: calibrationTable reports the residual per file", async () => {
const measured = await measureExactTokens(TREE_A, {
model: "m", apiKey: "sk-test", fetchImpl: fakeFetch(), egressHome,
});
const table = calibrationTable(measured.counts, TREE_A);
expect(table.rows.length).toBeGreaterThan(0);
for (const row of table.rows) {
expect(row).toHaveProperty("contentClass");
expect(row.estimatedTokens).toBeGreaterThan(0);
expect(row.tokens).toBeGreaterThan(0);
expect(row.errorPct).toBeCloseTo(((row.estimatedTokens - row.tokens) / row.tokens) * 100, 1);
}
expect(typeof table.worstErrorPct).toBe("number");
expect(typeof table.biasPct).toBe("number");
});
it("refuses to go to the network without a key, and says nothing was sent", async () => {
let called = false;
await expect(
measureExactTokens(TREE_A, {
model: "m",
apiKey: "",
fetchImpl: (() => ((called = true), Promise.reject(new Error("should not run")))) as never,
egressHome,
}),
).rejects.toMatchObject({ code: "exact_missing_api_key" });
expect(called).toBe(false);
});
it("maps HTTP failures to typed codes", async () => {
const status = (code: number) => (async () => ({ ok: false, status: code, text: async () => "nope" })) as never;
for (const [code, expected] of [
[401, "exact_auth_rejected"],
[403, "exact_auth_rejected"],
[500, "exact_request_failed"],
] as const) {
await expect(
measureExactTokens(TREE_A, { model: "m", apiKey: "k", fetchImpl: status(code), egressHome }),
).rejects.toMatchObject({ code: expected });
}
await expect(
measureExactTokens(TREE_A, {
model: "m", apiKey: "k",
fetchImpl: (() => Promise.reject(new Error("offline"))) as never,
egressHome,
}),
).rejects.toBeInstanceOf(ExactModeError);
});
it("offline by default: no --exact means no network call at all", async () => {
const out = capture();
let called = false;
const code = await contextBillMain([TREE_A, "--json"], {
stdout: out.stream,
stderr: out.stream,
fetchImpl: (() => ((called = true), Promise.reject(new Error("no")))) as never,
apiKey: "sk-test",
egressHome,
});
expect(code).toBe(0);
expect(called).toBe(false);
expect(JSON.parse(out.text()).tokenSource).toContain("estimate");
});
it("discloses what --exact sends before sending it", async () => {
const stdout = capture();
const stderr = capture();
const code = await contextBillMain([TREE_A, "--exact", "--json"], {
stdout: stdout.stream,
stderr: stderr.stream,
apiKey: "sk-test",
fetchImpl: fakeFetch(),
egressHome,
});
expect(code).toBe(0);
expect(stderr.text()).toContain("api.anthropic.com");
expect(stderr.text()).toMatch(/sending the content of \d+ \.md file\(s\)/);
const bill = JSON.parse(stdout.text());
expect(bill.tokenSource).toContain("count_tokens");
expect(bill.calibration.rows.length).toBeGreaterThan(0);
});
it("degrades to the estimate when exact mode is unavailable, naming the code", async () => {
const stdout = capture();
const stderr = capture();
const code = await contextBillMain([TREE_A, "--exact", "--json"], {
stdout: stdout.stream,
stderr: stderr.stream,
apiKey: "",
fetchImpl: (() => Promise.reject(new Error("should not run"))) as never,
egressHome,
});
expect(code).toBe(0);
expect(stderr.text()).toContain("exact_missing_api_key");
expect(JSON.parse(stdout.text()).tokenSource).toContain("estimate");
});
it("--help discloses --exact's egress and recalibration", async () => {
const out = capture();
expect(await contextBillMain(["--help"], { stdout: out.stream, stderr: out.stream })).toBe(0);
expect(out.text()).toContain("api.anthropic.com");
expect(out.text()).toContain("egress receipt");
expect(out.text()).toContain("recalibrates");
});
});
describe("ground truth against THIS repo (skill-census parity)", () => {
const census = skillCensus(ROOT);
const dirs = findSkillDirs(ROOT);
const rels = dirs.map((d) => path.relative(ROOT, d) || ".");
it("the walker reaches every physical SKILL.md the census counts", () => {
// physicalSkillFiles is the walker's expectation: every depth-1 skill dir
// (symlinked dirs included) plus the root router.
for (const rel of census.physicalSkillFiles) {
const dirRel = rel === "SKILL.md" ? "." : path.dirname(rel);
expect(rels, `walker missed ${rel}`).toContain(dirRel);
}
});
it("the root router SKILL.md is billed (fix a live on the real repo)", () => {
expect(census.physicalSkillFiles).toContain("SKILL.md"); // guard the guard
expect(rels).toContain(".");
});
it("a real skill's frontmatter bytes match an independent computation", () => {
const qaDir = dirs.find((d) => path.relative(ROOT, d) === "qa")!;
expect(qaDir).toBeTruthy();
const bill = buildBill(qaDir);
expect(bill.skills).toHaveLength(1);
expect(bill.skills[0].frontmatterBytes).toBe(frontmatterBytes(path.join(qaDir, "SKILL.md")));
expect(bill.skills[0].frontmatterBytes).toBeGreaterThan(0);
});
it("skill count is at least the census count (deeper trees may add more, never fewer)", () => {
expect(dirs.length).toBeGreaterThanOrEqual(census.physicalSkillFiles.length);
});
it("no skill dir is billed twice", () => {
expect(new Set(rels).size).toBe(rels.length);
});
});
describe("CLI plumbing", () => {
it("unknown flag and missing tree are usage errors (exit 2)", async () => {
const a = capture();
expect(await contextBillMain(["--nope"], { stdout: a.stream, stderr: a.stream })).toBe(2);
const b = capture();
expect(await contextBillMain(["--diff", TREE_A], { stdout: b.stream, stderr: b.stream })).toBe(2);
const c = capture();
expect(await contextBillMain([path.join(os.tmpdir(), "does-not-exist-xyz")], { stdout: c.stream, stderr: c.stream })).toBe(1);
});
it("bin/gstack-context-bill runs standalone", () => {
const result = Bun.spawnSync([path.join(ROOT, "bin", "gstack-context-bill"), TREE_A]);
expect(result.exitCode).toBe(0);
expect(result.stdout.toString()).toContain("ALWAYS-ON");
expect(result.stdout.toString()).toContain("EAGER");
});
});

185
test/egress-lib.test.ts Normal file
View File

@ -0,0 +1,185 @@
/**
* gstack-egress-lib.sh — shared shell receipt helpers, tested end-to-end
* against a local listener. Free tier, loopback only.
*
* Pins the shell-sink contract:
* - receipt sha256 == sha256 of the exact bytes the listener received
* (same file is hashed and handed to curl via --data-binary @file)
* - fail-closed refusal never touches the network and its stderr carries
* problem + cause + fix in plain language
* - fail-open warns on stderr and proceeds unrecorded
* - payload temp files are consumed immediately (no EXIT traps)
*/
import { describe, test, expect, beforeEach, afterEach, afterAll } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { listReceipts, sha256Hex } from '../lib/egress-receipt';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
const LIB = path.join(ROOT, 'bin', 'gstack-egress-lib.sh');
const received: string[] = [];
const server = Bun.serve({
port: 0,
fetch: async (req) => {
received.push(await req.text());
return new Response('ok');
},
});
const URL_BASE = `http://127.0.0.1:${server.port}`;
afterAll(() => {
server.stop(true);
});
let home: string;
beforeEach(() => {
home = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-egress-lib-'));
received.length = 0;
});
afterEach(() => {
try { fs.chmodSync(path.join(home, 'security'), 0o700); } catch {}
fs.rmSync(home, { recursive: true, force: true });
});
// Async spawn: spawnSync would block Bun's event loop, deadlocking the
// in-process listener the script curls against.
async function runBash(script: string) {
const proc = Bun.spawn(['bash', '-c', script], {
env: { ...process.env, GSTACK_HOME: home },
stdout: 'pipe',
stderr: 'pipe',
});
const [stdout, stderr, status] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
return { status, stdout, stderr };
}
describe('_receipted_curl', () => {
test('receipt sha256 equals sha256 of the body the listener received; payload file consumed', async () => {
const body = '[{"event":"lib-test","v":1}]';
const result = await runBash(`
set -uo pipefail
. "${LIB}"
payload="$(mktemp)"
printf '%s' '${body}' > "$payload"
echo "PAYLOAD_PATH=$payload"
_receipted_curl closed lib-test 127.0.0.1:${server.port} test-events "telemetry=community" "$payload" \\
curl -s --max-time 5 -X POST "${URL_BASE}/ingest"
echo "CURL_EXIT=$?"
[ -e "$payload" ] && echo "PAYLOAD_STILL_EXISTS" || echo "PAYLOAD_GONE"
`);
expect(result.status).toBe(0);
expect(result.stdout).toContain('CURL_EXIT=0');
expect(result.stdout).toContain('PAYLOAD_GONE');
expect(received.length).toBe(1);
expect(received[0]).toBe(body);
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].sha256).toBe(sha256Hex(received[0]));
expect(receipts[0].bytes).toBe(Buffer.byteLength(body));
expect(receipts[0].sink).toBe('lib-test');
expect(receipts[0].status).toBe('exit:0'); // best-effort outcome joined
});
test('fail-closed refusal never hits the network; stderr is problem + cause + fix', async () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return;
fs.mkdirSync(path.join(home, 'security'), { recursive: true, mode: 0o500 });
const result = await runBash(`
set -uo pipefail
. "${LIB}"
payload="$(mktemp)"
printf '%s' 'secret payload' > "$payload"
_receipted_curl closed lib-test 127.0.0.1:${server.port} test-events "telemetry=community" "$payload" \\
curl -s --max-time 5 -X POST "${URL_BASE}/ingest"
status=$?
[ -e "$payload" ] && echo "PAYLOAD_STILL_EXISTS" || echo "PAYLOAD_GONE"
exit $status
`);
expect(result.status).toBe(3);
expect(received.length).toBe(0); // the send was refused, not attempted
expect(result.stdout).toContain('PAYLOAD_GONE'); // cleaned up on refusal too
// Problem: what did not happen.
expect(result.stderr).toContain('lib-test NOT sent');
// Cause: the bridge's typed error is quoted.
expect(result.stderr).toContain('EGRESS_RECEIPT_FAILED');
// Fix: an actionable command.
expect(result.stderr).toContain('Fix: chmod -R u+w');
// What this is: the ledger explained.
expect(result.stderr).toContain('ATTEMPTS to send off-machine');
expect(result.stderr).toContain('gstack-egress');
});
test('fail-open warns and proceeds when the receipt cannot be written', async () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return;
fs.mkdirSync(path.join(home, 'security'), { recursive: true, mode: 0o500 });
const result = await runBash(`
set -uo pipefail
. "${LIB}"
_receipted_curl open lib-open 127.0.0.1:${server.port} version-fetch "user ran --force" --no-payload \\
curl -s --max-time 5 "${URL_BASE}/VERSION"
`);
expect(result.status).toBe(0);
expect(received.length).toBe(1); // the send proceeded unrecorded
expect(result.stderr).toContain('sending anyway');
expect(result.stderr).toContain('lib-open');
});
test('--no-payload records sha256:null and does not append --data-binary', async () => {
const result = await runBash(`
set -uo pipefail
. "${LIB}"
_receipted_curl closed lib-get 127.0.0.1:${server.port} version-fetch "user ran --force" --no-payload \\
curl -s --max-time 5 "${URL_BASE}/VERSION"
`);
expect(result.status).toBe(0);
expect(received.length).toBe(1);
expect(received[0]).toBe(''); // GET, no body
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].sha256).toBeNull();
expect(receipts[0].bytes).toBe(0);
});
});
describe('_receipted_git', () => {
test('writes a git-class sha256:null receipt then runs the command unmodified', async () => {
const result = await runBash(`
set -uo pipefail
. "${LIB}"
_receipted_git closed brain-sync github.com curated-memory-git-push "artifacts_sync_mode!=off" \\
bash -c 'echo GIT_RAN "$@"' _
`);
expect(result.status).toBe(0);
expect(result.stdout).toContain('GIT_RAN');
expect(result.stdout).not.toContain('--data-binary');
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].sha256).toBeNull();
expect(receipts[0].payload_class).toBe('curated-memory-git-push');
expect(receipts[0].status).toBe('exit:0');
});
test('fail-closed git refusal returns 3 without running the command', async () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return;
fs.mkdirSync(path.join(home, 'security'), { recursive: true, mode: 0o500 });
const marker = path.join(os.tmpdir(), `gstack-egress-git-${process.pid}`);
fs.rmSync(marker, { force: true });
const result = await runBash(`
set -uo pipefail
. "${LIB}"
_receipted_git closed brain-sync github.com curated-memory-git-push "artifacts_sync_mode!=off" \\
touch "${marker}"
`);
expect(result.status).toBe(3);
expect(fs.existsSync(marker)).toBe(false); // command never ran
expect(result.stderr).toContain('brain-sync NOT sent');
});
});

380
test/egress-receipt-wiring.test.ts Normal file
View File

@ -0,0 +1,380 @@
/**
* Static-grep tripwire for egress-receipt wiring. Free tier — no API.
*
* THREAT MODEL: the egress ledger is forensic observability — it records
* ATTEMPTED egress so accidents are auditable; it is not an exfiltration
* control. Receipts are written before send, outcomes are best-effort, and
* fail-open classes can send unrecorded with a warning.
*
* Every enumerated off-machine sink must route its send through the receipt
* ledger (lib/egress-receipt.ts), receipt BEFORE send. A future egress call
* site added without a receipt fails CI here instead of becoming a
* user-filed issue. The NEW-SINK SCANNER at the bottom sweeps the whole
* tree for outbound network ops and requires every hit to be either wired
* or in the REASONED exemption list — there is no KNOWN_UNWIRED bucket.
*
* Out of scope, documented here on purpose: the preamble-generated brain
* sync block (scripts/resolvers/preamble/generate-brain-sync-block.ts)
* renders a `git fetch` into skill PROSE that the agent executes — it is
* agent-executed instructions, not a gstack binary, so it is covered by the
* skill-prose exemption below rather than a receipt.
*
* Pattern mirrors test/hermetic-wiring.test.ts: read source files as text,
* assert invariants on their contents. Brittle by design — renaming a
* helper must force the author to look here.
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
function read(rel: string): string {
return fs.readFileSync(path.join(ROOT, rel), 'utf-8');
}
function exists(rel: string): boolean {
return fs.existsSync(path.join(ROOT, rel));
}
// ── POLARITY TABLE (amendments T3/C8) ──────────────────────────────────────
// Pinned as data: which sinks refuse the send when the receipt cannot be
// written (fail-closed) vs warn and proceed (fail-open). Changing a sink's
// polarity is a security decision — update this table deliberately.
const POLARITY: Record<string, 'fail-closed' | 'fail-open'> = {
// fail-closed: gstack state leaving the machine unrecorded is worse than
// the operation failing.
'brain-sync': 'fail-closed',
'memory-ingest': 'fail-closed',
'gbrain-sync': 'fail-closed',
'telemetry-sync': 'fail-closed',
'browse-tunnel (ngrok)': 'fail-closed',
'gbrain-mcp-verify': 'fail-closed',
'supabase-provision': 'fail-closed',
// fail-open: user-facing operations that must not die over an audit-log
// hiccup; they warn on stderr and proceed.
'design-openai': 'fail-open',
'update-check': 'fail-open',
'security-dashboard': 'fail-open',
'community-dashboard': 'fail-open',
'git-class user ops (artifacts-init, brain-restore, session-update)': 'fail-open',
'context-bill --exact': 'fail-open',
};
/** TS sinks: must import the canonical helper and call writeReceipt(). */
const MODULE_SINKS = [
'bin/gstack-gbrain-sync.ts',
'bin/gstack-memory-ingest.ts',
'browse/src/server.ts',
// Unconditional: context-bill ships in the same tree as this tripwire. A
// missing file must fail loudly (a rename/move that drops its receipt wiring
// is exactly what this pins), not silently soften the assertion.
'lib/context-bill.ts',
];
/** Shell sinks: must source the shared lib; every network op receipted. */
const SHELL_SINKS = [
'bin/gstack-telemetry-sync',
'bin/gstack-update-check',
'bin/gstack-brain-sync',
'bin/gstack-gbrain-mcp-verify',
'bin/gstack-security-dashboard',
'bin/gstack-community-dashboard',
'bin/gstack-gbrain-supabase-provision',
'bin/gstack-artifacts-init',
'bin/gstack-brain-restore',
'bin/gstack-session-update',
];
/** design files that talk to api.openai.com — all must use receiptedFetch. */
const DESIGN_SINKS = [
'design/src/generate.ts',
'design/src/variants.ts',
'design/src/iterate.ts',
'design/src/evolve.ts',
'design/src/check.ts',
'design/src/diff.ts',
'design/src/design-to-code.ts',
'design/src/memory.ts',
];
// ── NEW-SINK SCANNER exemptions ────────────────────────────────────────────
// Every entry carries its reason. An unexplained network op anywhere in the
// swept tree fails the scanner — add real sinks to the wired lists above,
// not here.
const SCANNER_EXEMPT: Record<string, string> = {
'bin/gstack-team-init':
'every git clone is inside an echoed instruction string (install docs); the script executes no network ops',
'bin/gstack-gbrain-install':
'user-invoked installer: bodyless HEAD reachability probe to github.com + clone of the public gbrain repo (user-directed install; no gstack state leaves the machine)',
'bin/gstack-next-version':
'fetches the user\'s own repo\'s base branch for version-claim freshness — a user-repo dev-workflow op, not gstack-state egress',
'bin/gstack-version-bump':
'git fetch appears only in an error-message string',
'bin/gstack-redact-prepush':
'git push mentions are hook documentation strings (bypass instructions)',
'browse/src/security-classifier.ts':
'HF model download: bodyless GET of a public classifier model (variable URL)',
'browse/src/write-commands.ts':
'user-directed page fetch — the browser command surface fetches what the user asked for',
'browse/src/cli.ts':
'health probe of the user\'s own pair-agent tunnel URL (reachability probe)',
'browse/src/commands.ts':
'git pull appears only in an upgrade-hint message string',
'browse/src/cookie-picker-ui.ts':
'served-page JS talking to its own loopback server (same-origin relative fetch)',
'design/src/compare.ts':
'served-page JS talking to its own loopback server (relative ./api fetch)',
// Skill prose templates: these render agent-executed instructions (the
// agent runs git in the USER\'S repo at the user\'s direction), they are
// not gstack binaries. Includes the preamble-generated brain-sync block —
// see the header.
'scripts/resolvers':
'skill prose templates — agent-executed instructions rendered into SKILL.md, not gstack binaries',
};
function isExempt(rel: string): string | undefined {
for (const [key, reason] of Object.entries(SCANNER_EXEMPT)) {
if (rel === key || rel.startsWith(`${key}/`)) return reason;
}
return undefined;
}
// Receipt markers that make a nearby network op "wired".
const RECEIPT_MARKER =
/_receipted_(curl|git|version_fetch)\b|gstack-egress-receipt["']?\s+write\b|writeReceipt\(|receiptedFetch\(/;
/** Was a receipt marker present on this line or the 30 preceding lines? */
function guarded(lines: string[], i: number): boolean {
for (let j = i; j >= Math.max(0, i - 30); j--) {
if (RECEIPT_MARKER.test(lines[j])) return true;
}
return false;
}
// git as a COMMAND followed by a remote op. Local ops (rev-parse, remote
// get-url, add, commit, merge) never match; neither does prose like
// "curated-memory-git-push" (hyphenated) or "'git fetch'" (quoted).
const GIT_REMOTE_OP = /(^|[;|&`($!]|\s)git(\s+-C\s+\S+)?\s+(push|pull|fetch|clone|ls-remote)\b/;
// git spawn-array form in TS: spawn("git", ["push", ...]).
const GIT_SPAWN_OP = /["'`]git["'`]\s*,\s*\[\s*["'`](push|pull|fetch|clone|ls-remote)/;
// curl as a command token.
const CURL_OP = /(^|[|&;(`]|\s|\$\()curl\s/;
// fetch() with an absolute http(s) URL (loopback filtered separately).
const FETCH_ABS = /(^|[^A-Za-z])fetch(Fn|Impl)?\(\s*[`'"]https?:\/\//;
function isTextFile(full: string): boolean {
try {
const buf = fs.readFileSync(full);
return !buf.subarray(0, 1024).includes(0);
} catch {
return false;
}
}
function* walk(dir: string): Generator<string> {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
if (entry.name === 'node_modules' || entry.name === 'dist' || entry.name.startsWith('.')) continue;
const full = path.join(dir, entry.name);
if (entry.isDirectory()) yield* walk(full);
else if (entry.isFile()) yield full;
}
}
/**
* Collect un-receipted outbound network ops in a file. Skips comments,
* loopback lines, `command -v` probes, message-emitting lines, and shell
* heredoc bodies (echoed instructions are not executed ops).
*/
function scanFile(rel: string): string[] {
const src = read(rel);
const isTs = /\.(ts|js|mjs|tsx)$/.test(rel);
const lines = src.split('\n');
const offenders: string[] = [];
let heredocEnd: string | null = null;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (heredocEnd !== null) {
if (line.trim() === heredocEnd) heredocEnd = null;
continue;
}
if (!isTs) {
const heredoc = line.match(/<<-?\s*['"]?([A-Za-z_][A-Za-z0-9_]*)['"]?/);
if (heredoc) heredocEnd = heredoc[1];
}
const trimmed = line.trimStart();
if (/^(#|\/\/|\*|\/\*)/.test(trimmed)) continue;
if (line.includes('127.0.0.1') || line.includes('localhost')) continue;
if (/command -v/.test(line)) continue;
if (/^(echo|printf|emit|die|fail|log)\b/.test(trimmed)) continue;
// TS: shell-style git ops only count on lines that actually execute
// something (spawn/exec markers) — template-literal prose does not.
const tsExecGit =
GIT_SPAWN_OP.test(line) ||
(GIT_REMOTE_OP.test(line) && /\b(spawn|spawnSync|exec|execSync|execFileSync|runCommand)\b/.test(line));
const isNetOp = isTs
? FETCH_ABS.test(line) || tsExecGit
: CURL_OP.test(line) || GIT_REMOTE_OP.test(line);
if (!isNetOp) continue;
if (guarded(lines, i)) continue;
offenders.push(`${rel}:${i + 1}: ${line.trim().slice(0, 100)}`);
}
return offenders;
}
describe('egress receipt wiring tripwire', () => {
test('every TS sink imports lib/egress-receipt and calls writeReceipt()', () => {
for (const rel of MODULE_SINKS) {
const src = read(rel);
expect(src.includes('egress-receipt'), `${rel}: must import lib/egress-receipt`).toBe(true);
expect(src.includes('writeReceipt('), `${rel}: must call writeReceipt() before its send`).toBe(true);
}
});
test('every shell sink sources gstack-egress-lib.sh', () => {
for (const rel of SHELL_SINKS) {
const src = read(rel);
expect(
src.includes('gstack-egress-lib.sh'),
`${rel}: must source bin/gstack-egress-lib.sh for _receipted_* helpers`,
).toBe(true);
}
});
test('every network op in a wired shell sink sits under a receipt', () => {
const offenders = SHELL_SINKS.flatMap((rel) => scanFile(rel));
expect(
offenders,
'un-receipted network call(s) — wrap in _receipted_curl/_receipted_git or write the receipt first:\n' +
offenders.join('\n'),
).toEqual([]);
});
test('browse tunnel: every ngrok.forward() has a writeReceipt in the 30 preceding lines', () => {
const lines = read('browse/src/server.ts').split('\n');
const offenders: string[] = [];
let sawForward = false;
for (let i = 0; i < lines.length; i++) {
if (!lines[i].includes('ngrok.forward(')) continue;
if (/^\s*(\/\/|\*)/.test(lines[i])) continue;
sawForward = true;
const context = lines.slice(Math.max(0, i - 30), i).join('\n');
if (!context.includes('writeReceipt(')) offenders.push(`browse/src/server.ts:${i + 1}`);
}
expect(sawForward, 'expected ngrok.forward call sites in server.ts').toBe(true);
expect(offenders, 'tunnel session opened without a receipt: ' + offenders.join(', ')).toEqual([]);
});
test('design: every api.openai.com call routes through receiptedFetch', () => {
for (const rel of DESIGN_SINKS) {
const src = read(rel);
expect(
src.includes('receipted-fetch'),
`${rel}: must import design/src/receipted-fetch`,
).toBe(true);
const lines = src.split('\n');
for (let i = 0; i < lines.length; i++) {
if (!lines[i].includes('api.openai.com')) continue;
if (/^\s*(\/\/|\*)/.test(lines[i])) continue;
if (/\bfetch(Fn|Impl)?\(/.test(lines[i])) {
throw new Error(
`${rel}:${i + 1}: raw fetch to api.openai.com — route it through receiptedFetch()`,
);
}
}
}
});
test('deprecated dead-endpoint brain consumer/reader scripts stay deleted', () => {
// lstat (not existsSync) so a dangling symlink also fails.
for (const rel of ['bin/gstack-brain-consumer', 'bin/gstack-brain-reader']) {
let present = true;
try {
fs.lstatSync(path.join(ROOT, rel));
} catch {
present = false;
}
expect(present, `${rel} was deleted (dead /ingest-repo egress sink) — do not resurrect`).toBe(false);
}
});
test('polarity table names every wired sink exactly once per polarity', () => {
const closed = Object.entries(POLARITY).filter(([, p]) => p === 'fail-closed').map(([s]) => s);
const open = Object.entries(POLARITY).filter(([, p]) => p === 'fail-open').map(([s]) => s);
expect(closed.sort()).toEqual([
'brain-sync',
'browse-tunnel (ngrok)',
'gbrain-mcp-verify',
'gbrain-sync',
'memory-ingest',
'supabase-provision',
'telemetry-sync',
]);
expect(open.sort()).toEqual([
'community-dashboard',
'context-bill --exact',
'design-openai',
'git-class user ops (artifacts-init, brain-restore, session-update)',
'security-dashboard',
'update-check',
]);
});
test('polarity spot-checks: closed sinks refuse, open sinks warn', () => {
// telemetry-sync (closed): the wrapped POST uses the `closed` policy.
expect(read('bin/gstack-telemetry-sync')).toMatch(/_receipted_curl closed telemetry-sync/);
// brain-sync (closed): refusal exits before the commit consumes the queue.
expect(read('bin/gstack-brain-sync')).toMatch(/gstack-egress-receipt["']? write/);
// update-check (open).
expect(read('bin/gstack-update-check')).toMatch(/_receipted_curl open update-check/);
// dashboards (open).
expect(read('bin/gstack-security-dashboard')).toMatch(/_receipted_curl open security-dashboard/);
expect(read('bin/gstack-community-dashboard')).toMatch(/_receipted_curl open community-dashboard/);
// mcp-verify + provision (closed).
expect(read('bin/gstack-gbrain-mcp-verify')).toMatch(/_receipted_curl closed gbrain-mcp-verify/);
expect(read('bin/gstack-gbrain-supabase-provision')).toMatch(/_receipted_curl closed supabase-provision/);
// design (open): the wrapper catches receipt errors and proceeds.
const rf = read('design/src/receipted-fetch.ts');
expect(rf).toContain('fail-open');
expect(rf.indexOf('writeReceipt(')).toBeLessThan(rf.indexOf('fetchImpl(url, init)'));
});
test('NEW-SINK SCANNER: every outbound network op in the tree is wired or reasoned-exempt', () => {
const SWEEP = ['bin', 'lib', 'scripts', 'design/src', 'browse/src'];
const offenders: string[] = [];
for (const dirRel of SWEEP) {
const dir = path.join(ROOT, dirRel);
if (!fs.existsSync(dir)) continue;
for (const full of walk(dir)) {
const rel = path.relative(ROOT, full).split(path.sep).join('/');
if (!/\.(ts|js|mjs|sh|tsx)$/.test(rel) && !isTextFile(full)) continue;
if (isExempt(rel)) continue;
offenders.push(...scanFile(rel));
}
}
expect(
offenders,
'unwired outbound network op(s). Wire each through the receipt helpers ' +
'(_receipted_curl/_receipted_git in shell, writeReceipt/receiptedFetch in TS) ' +
'or add a REASONED exemption with the honest why:\n' +
offenders.join('\n'),
).toEqual([]);
});
test('shebang tripwire: no bin/gstack-* file carries a node shebang (amendment 2A)', () => {
const offenders: string[] = [];
for (const entry of fs.readdirSync(path.join(ROOT, 'bin'))) {
if (!entry.startsWith('gstack-')) continue;
const full = path.join(ROOT, 'bin', entry);
if (!fs.lstatSync(full).isFile()) continue;
if (!isTextFile(full)) continue;
const firstLine = fs.readFileSync(full, 'utf-8').split('\n', 1)[0];
if (firstLine.startsWith('#!') && /\bnode\b/.test(firstLine)) {
offenders.push(`bin/${entry}: ${firstLine}`);
}
}
expect(offenders, 'node shebangs in bin/ (use #!/usr/bin/env bun): ' + offenders.join(', ')).toEqual([]);
});
});

258
test/egress-receipt.test.ts Normal file
View File

@ -0,0 +1,258 @@
/**
* Egress receipts — chain, fail-closed, verify, shell bridge. Free tier, no network.
*
* THREAT MODEL: the egress ledger is forensic observability — it records
* ATTEMPTED egress so accidents are auditable; it is not an exfiltration
* control.
*
* Pins the auditor contract:
* - receipt-before-send fail-closed (EGRESS_RECEIPT_FAILED, no ledger = no send)
* - content-free lines chained by prev = sha256(previous raw line)
* - tail-read correctness (last line found without loading the whole file)
* - abandoned-lock reclaim (stale lock dir >10s old is removed, not fatal)
* - WARN-at-size (one self-explanatory stderr warning per process >25MB)
*/
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { spawnSync } from 'child_process';
import {
EGRESS_RECEIPT_FAILED,
LEDGER_WARN_BYTES,
egressLedgerPath,
ledgerSizeWarning,
listReceipts,
resetLedgerSizeWarningForTests,
sha256Hex,
verifyLedger,
writeOutcome,
writeReceipt,
} from '../lib/egress-receipt';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
let home: string;
beforeEach(() => {
home = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-egress-'));
});
afterEach(() => {
try { fs.chmodSync(path.join(home, 'security'), 0o700); } catch {} // undo fail-closed fixtures
fs.rmSync(home, { recursive: true, force: true });
});
describe('egress receipt library', () => {
test('receipts chain: prev = sha256 of the previous raw line, "" for line 1', () => {
writeReceipt({ home, sink: 'a', host: 'h1', payloadClass: 'c', bytes: 3, sha256: sha256Hex('abc'), consent: 'k=v' });
writeReceipt({ home, sink: 'b', host: 'h2', payloadClass: 'c', bytes: 0, sha256: null, consent: 'k=v' });
const lines = fs.readFileSync(egressLedgerPath(home), 'utf-8').trim().split('\n');
expect(lines.length).toBe(2);
const first = JSON.parse(lines[0]);
const second = JSON.parse(lines[1]);
expect(first.prev).toBe('');
expect(second.prev).toBe(sha256Hex(lines[0]));
expect(first.sha256).toBe(sha256Hex('abc'));
expect(second.sha256).toBeNull();
expect(verifyLedger(home)).toMatchObject({ ok: true, count: 2 });
});
test('ledger is 0600 and the security dir 0700', () => {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
const ledger = egressLedgerPath(home);
expect(fs.statSync(ledger).mode & 0o777).toBe(0o600);
expect(fs.statSync(path.dirname(ledger)).mode & 0o777).toBe(0o700);
});
test('fail-closed: unwritable security dir throws typed EGRESS_RECEIPT_FAILED', () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return; // chmod is advisory there
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
fs.chmodSync(path.join(home, 'security'), 0o500);
try {
expect(() =>
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' }),
).toThrow();
try {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
} catch (err: any) {
expect(err.code).toBe(EGRESS_RECEIPT_FAILED);
}
} finally {
fs.chmodSync(path.join(home, 'security'), 0o700);
}
});
test('outcome records join back onto their receipt in listReceipts', () => {
const { id } = writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
writeOutcome({ home, receipt: id, status: 204 });
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].status).toBe('204');
expect(verifyLedger(home)).toMatchObject({ ok: true, count: 2 });
});
test('tampering with a middle line breaks verification at that line', () => {
for (let i = 0; i < 3; i += 1) {
writeReceipt({ home, sink: `s${i}`, host: 'h', payloadClass: 'c', consent: 'telemetry=community' });
}
const ledger = egressLedgerPath(home);
const lines = fs.readFileSync(ledger, 'utf-8').trim().split('\n');
lines[1] = lines[1].replace('community', 'communitX');
fs.writeFileSync(ledger, `${lines.join('\n')}\n`);
// Line 2's edited bytes no longer hash to line 3's recorded prev.
expect(verifyLedger(home)).toMatchObject({ ok: false, brokenLine: 3 });
});
test('validation rejects garbage before touching the ledger', () => {
expect(() => writeReceipt({ home, sink: '', host: 'h', payloadClass: 'c', consent: 'k' } as any)).toThrow();
expect(() => writeReceipt({ home, sink: 's', host: 'h', payloadClass: 'c', consent: 'k', bytes: -1 })).toThrow();
expect(() => writeReceipt({ home, sink: 's', host: 'h', payloadClass: 'c', consent: 'k', sha256: 'nothex' })).toThrow();
expect(fs.existsSync(egressLedgerPath(home))).toBe(false);
});
test('abandoned lock: a stale lock dir (>10s-old mtime) is reclaimed by the next writer', () => {
// First write creates the security dir so the lock path's parent exists.
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
const lock = `${egressLedgerPath(home)}.lock`;
fs.mkdirSync(lock);
const old = new Date(Date.now() - 60_000);
fs.utimesSync(lock, old, old);
const started = Date.now();
const { id } = writeReceipt({ home, sink: 'b', host: 'h', payloadClass: 'c', consent: 'k=v' });
expect(id).toMatch(/^[0-9a-f]{64}$/);
// Spin budget is 2.5s; reclaim happens right after budget exhaustion.
expect(Date.now() - started).toBeLessThan(10_000);
expect(fs.existsSync(lock)).toBe(false);
expect(verifyLedger(home)).toMatchObject({ ok: true, count: 2 });
});
test('a fresh (recent-mtime) lock held past the budget fails closed instead of being stolen', () => {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
const lock = `${egressLedgerPath(home)}.lock`;
fs.mkdirSync(lock);
// Keep the mtime fresh so the reclaim path never fires: refresh it in the
// background while the writer spins out its 2.5s budget.
const refresher = setInterval(() => {
const now = new Date();
try { fs.utimesSync(lock, now, now); } catch { /* test teardown race */ }
}, 1000);
try {
expect(() =>
writeReceipt({ home, sink: 'b', host: 'h', payloadClass: 'c', consent: 'k=v' }),
).toThrow(/locked/);
} finally {
clearInterval(refresher);
fs.rmdirSync(lock);
}
}, 15_000);
test('tail-read: last line is found correctly on a multi-record ledger larger than the tail window', () => {
// 30 records ≈ 9KB > the 4KB tail window, so the append path must find
// the true last line from a partial read.
for (let i = 0; i < 30; i += 1) {
writeReceipt({
home,
sink: `sink-${i}`,
host: 'h',
payloadClass: `class-${'x'.repeat(200)}-${i}`,
consent: 'k=v',
});
}
const ledger = egressLedgerPath(home);
expect(fs.statSync(ledger).size).toBeGreaterThan(4096);
const result = verifyLedger(home);
expect(result).toMatchObject({ ok: true, count: 30 });
});
test('chain verify stays intact across 100+ records', () => {
for (let i = 0; i < 120; i += 1) {
writeReceipt({ home, sink: `s${i}`, host: 'h', payloadClass: 'c', consent: 'k=v' });
if (i % 10 === 0) writeOutcome({ home, receipt: 'f'.repeat(64), status: 200 });
}
const result = verifyLedger(home);
expect(result.ok).toBe(true);
expect(result.count).toBe(132);
expect(listReceipts(home).length).toBe(120);
});
test('WARN-at-size: >25MB ledger emits one self-explanatory stderr warning per process', () => {
const ledger = egressLedgerPath(home);
fs.mkdirSync(path.dirname(ledger), { recursive: true, mode: 0o700 });
// Grow the file past the threshold with valid-looking filler; the warning
// keys off file size only.
const filler = `${JSON.stringify({ type: 'egress', pad: 'x'.repeat(1024) })}\n`;
const chunk = filler.repeat(1024); // ~1MB
const writes = Math.ceil(LEDGER_WARN_BYTES / chunk.length) + 1;
for (let i = 0; i < writes; i += 1) fs.appendFileSync(ledger, chunk);
expect(fs.statSync(ledger).size).toBeGreaterThan(LEDGER_WARN_BYTES);
resetLedgerSizeWarningForTests();
const captured: string[] = [];
const originalWrite = process.stderr.write.bind(process.stderr);
(process.stderr as any).write = (chunk: string) => { captured.push(String(chunk)); return true; };
try {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
writeReceipt({ home, sink: 'b', host: 'h', payloadClass: 'c', consent: 'k=v' });
} finally {
(process.stderr as any).write = originalWrite;
}
const warnings = captured.filter((c) => c.includes('egress ledger is large'));
expect(warnings.length).toBe(1); // once per process, not per write
// Self-explanatory shape: what the ledger is, how to inspect, what's coming.
expect(warnings[0]).toContain('ATTEMPTS to send off-machine');
expect(warnings[0]).toContain('gstack-egress list');
expect(warnings[0]).toContain('rotation');
expect(warnings[0]).toContain(ledger);
// verifyLedger surfaces the same warning as data.
const message = ledgerSizeWarning(ledger, fs.statSync(ledger).size);
expect(message).toContain('MB');
expect(verifyLedger(home).sizeWarning).toBe(message);
});
});
describe('gstack-egress-receipt shell bridge', () => {
const bin = path.join(ROOT, 'bin', 'gstack-egress-receipt');
test('write hashes the exact payload file, prints the receipt id; outcome joins', () => {
const payload = path.join(home, 'payload.json');
fs.writeFileSync(payload, '[{"v":1}]');
const write = spawnSync(bin, ['write', '--sink', 'telemetry-sync', '--host', '127.0.0.1:8399',
'--class', 'telemetry-events', '--payload-file', payload, '--consent', 'telemetry=community'],
{ encoding: 'utf-8', env: { ...process.env, GSTACK_HOME: home } });
expect(write.status).toBe(0);
const id = write.stdout.trim();
expect(id).toMatch(/^[0-9a-f]{64}$/);
const outcome = spawnSync(bin, ['outcome', id, '204'],
{ encoding: 'utf-8', env: { ...process.env, GSTACK_HOME: home } });
expect(outcome.status).toBe(0);
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].bytes).toBe(9);
expect(receipts[0].sha256).toBe(sha256Hex('[{"v":1}]'));
expect(receipts[0].status).toBe('204');
});
test('--no-payload records sha256:null (git-class: a subprocess owns the bytes)', () => {
const write = spawnSync(bin, ['write', '--sink', 'brain-sync', '--host', 'github.com',
'--class', 'git-push', '--no-payload', '--consent', 'artifacts_sync_mode=auto'],
{ encoding: 'utf-8', env: { ...process.env, GSTACK_HOME: home } });
expect(write.status).toBe(0);
const receipts = listReceipts(home);
expect(receipts.length).toBe(1);
expect(receipts[0].sha256).toBeNull();
expect(receipts[0].bytes).toBe(0);
});
test('write exits 3 with EGRESS_RECEIPT_FAILED when the ledger is unwritable', () => {
if (process.platform === 'win32' || process.getuid?.() === 0) return;
fs.mkdirSync(path.join(home, 'security'), { recursive: true, mode: 0o500 });
const write = spawnSync(bin, ['write', '--sink', 's', '--host', 'h', '--class', 'c', '--no-payload'],
{ encoding: 'utf-8', env: { ...process.env, GSTACK_HOME: home } });
expect(write.status).toBe(3);
expect(write.stderr).toContain('EGRESS_RECEIPT_FAILED');
fs.chmodSync(path.join(home, 'security'), 0o700);
});
});

21
test/fixtures/context-bill/tree-a/alpha/SKILL.md vendored Normal file
View File

@ -0,0 +1,21 @@
---
name: alpha
description: Fixture dispatcher with a mode table and forced-read references.
triggers: a-key-the-upstream-router-DOES-read
x-dead-key: a-key-no-router-reads
---
# Alpha
## Dispatch protocol
1. Infer the mode from the request.
2. Read `references/CORE.md` and `references/POLICY.md` for every invocation. Read `references/OPTIONAL.md` before public-web work.
3. When the target is a repository, read `references/CORE.md` once before specialist work; it is already billed eagerly.
## Top-level modes
| Mode | Infer when | Candidate internal specialists |
|---|---|---|
| `Discovery` | The idea is fluid. | `references/legacy/office.md` |
| `Full chain` | Everything at once. | `references/legacy/auto.md` |

3
test/fixtures/context-bill/tree-a/alpha/references/CORE.md vendored Normal file
View File

@ -0,0 +1,3 @@
# Core
Forced-read reference number one. Read on every invocation of alpha.

3
test/fixtures/context-bill/tree-a/alpha/references/OPTIONAL.md vendored Normal file
View File

@ -0,0 +1,3 @@
# Optional
Conditional reference. Not part of the eager bill.

3
test/fixtures/context-bill/tree-a/alpha/references/POLICY.md vendored Normal file
View File

@ -0,0 +1,3 @@
# Policy
Forced-read reference number two.

8
test/fixtures/context-bill/tree-a/beta/SKILL.md vendored Normal file
View File

@ -0,0 +1,8 @@
---
name: beta
description: Clean fixture tool skill with no forced reads and no mode table.
---
# Beta
One-command tool skill. Nothing eager beyond this file, nothing lazy.

3
test/fixtures/context-bill/tree-a/beta/agents.md vendored Normal file
View File

@ -0,0 +1,3 @@
# Foreign host file
A skill-shaped file another host dropped into scanner scope (#1694).

13
test/gbrain-refresh-install-render.test.ts
View File

@ -9,13 +9,18 @@ import * as fs from 'fs';
const ROOT = path.resolve(import.meta.dir, '..');
const SRC = fs.readFileSync(path.join(ROOT, 'bin', 'gstack-config'), 'utf-8');
// Pull out just the gbrain-refresh `ok)` branch so assertions can't be
// satisfied by unrelated text elsewhere in the file.
// Pull out just the gbrain-refresh healthy-status branch so assertions can't
// be satisfied by unrelated text elsewhere in the file. The case label grew
// from `ok)` to `ok|timeout|thin-client)` (#1964, #2051) and may grow again,
// so match any label that STARTS with `ok` followed by alternations.
function okBranch(): string {
const start = SRC.indexOf('gbrain-refresh)');
const ok = SRC.indexOf('ok)', start);
if (start < 0) throw new Error('Could not locate gbrain-refresh case');
const labelMatch = /^\s*ok(?:\|[\w-]+)*\)/m.exec(SRC.slice(start));
if (!labelMatch) throw new Error('Could not locate gbrain-refresh ok) branch');
const ok = start + labelMatch.index;
const end = SRC.indexOf(';;', ok);
if (start < 0 || ok < 0 || end < 0) throw new Error('Could not locate gbrain-refresh ok) branch');
if (end < 0) throw new Error('Could not locate gbrain-refresh ok) branch terminator');
return SRC.slice(ok, end);
}

28
test/gemini-e2e.test.ts
View File

@ -33,18 +33,42 @@ const GEMINI_AVAILABLE = (() => {
} catch { return false; }
})();
// A binary on PATH is not enough: the CLI can be present but UNUSABLE — the
// individual code-assist auth path was deprecated upstream ("migrate to the
// Antigravity suite"), which fails every run before any model call, and flag
// churn (--skip-trust removed in 0.34) errors at argv parse. Probe with a
// bare --help: a CLI that can't even print usage is unusable, and a working
// one is cheap to confirm. Deeper auth failures are classified per-run.
const GEMINI_USABLE = GEMINI_AVAILABLE && (() => {
try {
const result = Bun.spawnSync(['gemini', '--help'], { timeout: 15_000 });
return result.exitCode === 0;
} catch { return false; }
})();
const evalsEnabled = !!process.env.EVALS;
// Skip all tests if gemini is not available or EVALS is not set.
const SKIP = !GEMINI_AVAILABLE || !evalsEnabled;
// External-service tests are periodic-tier (CLAUDE.md tiering rule 3):
// "Requires external service (Codex, Gemini)? -> periodic". The positive
// form below is the canonical whole-file guard shape — the sharded runner's
// classifyPaidTestFile greps for it to exclude this file from gate.
const tierOk = process.env.EVALS_TIER === 'periodic';
// Skip all tests if gemini is not available/usable, EVALS is not set, or
// we're in the gate tier.
const SKIP = !GEMINI_USABLE || !evalsEnabled || !tierOk;
const describeGemini = SKIP ? describe.skip : describe;
// Log why we're skipping (helpful for debugging CI)
if (!evalsEnabled) {
// Silent — same as Claude E2E tests, EVALS=1 required
} else if (!tierOk) {
process.stderr.write('\nGemini E2E: SKIPPED — external-service test, periodic tier only (EVALS_TIER === \'periodic\')\n');
} else if (!GEMINI_AVAILABLE) {
process.stderr.write('\nGemini E2E: SKIPPED — gemini binary not found (install: npm i -g @google/gemini-cli)\n');
} else if (!GEMINI_USABLE) {
process.stderr.write('\nGemini E2E: SKIPPED — gemini CLI present but unusable (auth path deprecated upstream or CLI broken; try updating @google/gemini-cli)\n');
}
// --- Diff-based test selection ---

151
test/gstack-egress-cli.test.ts Normal file
View File

@ -0,0 +1,151 @@
/**
* gstack-egress CLI — list | verify | grants smoke tests. Free tier.
*
* Spawns the real bin against a temp GSTACK_HOME: list filters, verify
* exit-3-on-tamper (naming the first broken line), sizeWarning surfacing,
* and grants against the upstream config keys (telemetry,
* artifacts_sync_mode, redact_repo_visibility, redact_prepush_hook).
*/
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { spawnSync } from 'child_process';
import {
LEDGER_WARN_BYTES,
egressLedgerPath,
sha256Hex,
writeReceipt,
} from '../lib/egress-receipt';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
const BIN = path.join(ROOT, 'bin', 'gstack-egress');
let home: string;
beforeEach(() => {
home = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-egress-cli-'));
});
afterEach(() => {
fs.rmSync(home, { recursive: true, force: true });
});
function run(args: string[]) {
const result = spawnSync(BIN, args, {
encoding: 'utf-8',
env: { ...process.env, GSTACK_HOME: home },
});
return { code: result.status ?? -1, stdout: result.stdout || '', stderr: result.stderr || '' };
}
describe('gstack-egress list', () => {
test('fresh home prints "no receipts" and the ledger path, exit 0', () => {
const r = run(['list']);
expect(r.code).toBe(0);
expect(r.stdout).toContain('no receipts');
expect(r.stdout).toContain(egressLedgerPath(home));
});
test('--json returns receipts and honors --sink/--host/--since filters', () => {
writeReceipt({ home, sink: 'telemetry-sync', host: '10.0.0.1:8399', payloadClass: 'telemetry-events', bytes: 2, sha256: sha256Hex('[]'), consent: 'telemetry=community' });
writeReceipt({ home, sink: 'design-openai', host: 'api.openai.com', payloadClass: 'generate-image-request', consent: 'user ran design command' });
const all = run(['list', '--json']);
expect(all.code).toBe(0);
expect(JSON.parse(all.stdout).length).toBe(2);
const filtered = run(['list', '--json', '--sink', 'telemetry-sync']);
const rows = JSON.parse(filtered.stdout);
expect(rows.length).toBe(1);
expect(rows[0].host).toBe('10.0.0.1:8399');
expect(rows[0].sha256).toBe(sha256Hex('[]'));
const none = run(['list', '--json', '--since', '2999-01-01T00:00:00Z']);
expect(JSON.parse(none.stdout).length).toBe(0);
});
test('unknown option exits 2 with usage', () => {
const r = run(['list', '--bogus']);
expect(r.code).toBe(2);
expect(r.stderr).toContain('Usage');
});
});
describe('gstack-egress verify', () => {
test('exits 0 on an intact chain and 3 naming the first broken line on tamper', () => {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'telemetry=community' });
writeReceipt({ home, sink: 'b', host: 'h', payloadClass: 'c', consent: 'telemetry=community' });
const ok = run(['verify']);
expect(ok.code).toBe(0);
expect(ok.stdout).toContain('chain intact: 2');
const ledger = egressLedgerPath(home);
const lines = fs.readFileSync(ledger, 'utf-8').trim().split('\n');
lines[0] = lines[0].replace('community', 'communitX');
fs.writeFileSync(ledger, `${lines.join('\n')}\n`);
const tampered = run(['verify']);
expect(tampered.code).toBe(3);
expect(tampered.stdout).toContain('line 2');
});
test('prints the sizeWarning when the ledger exceeds the threshold', () => {
writeReceipt({ home, sink: 'a', host: 'h', payloadClass: 'c', consent: 'k=v' });
const ledger = egressLedgerPath(home);
// verify keys the warning off file size only — pad with a trailing
// comment-free blank region by appending to a side channel is not
// possible in JSONL, so grow via many valid-shaped junk lines and
// assert on sizeWarning presence (chain will break; both surface).
const filler = `${JSON.stringify({ type: 'junk', pad: 'x'.repeat(1024) })}\n`.repeat(1024);
while (fs.statSync(ledger).size <= LEDGER_WARN_BYTES) fs.appendFileSync(ledger, filler);
const r = run(['verify', '--json']);
expect(r.code).toBe(3); // filler breaks the chain — expected
const parsed = JSON.parse(r.stdout);
expect(parsed.sizeWarning).toContain('egress ledger is large');
expect(parsed.sizeWarning).toContain('gstack-egress list');
const human = run(['verify']);
expect(human.stdout).toContain('egress ledger is large');
});
});
describe('gstack-egress grants', () => {
test('fresh home shows the four upstream grants off, each naming file and revoke command', () => {
const r = run(['grants']);
expect(r.code).toBe(0);
for (const grant of ['telemetry', 'brain-sync', 'redact_repo_visibility', 'redact_prepush_hook']) {
expect(r.stdout).toContain(grant);
}
expect(r.stdout).not.toContain('[GRANTED]');
expect(r.stdout).toContain(path.join(home, 'config.yaml'));
expect(r.stdout).toContain('revoke:');
});
test('--json flips granted=true when telemetry and sync mode are enabled', () => {
const config = spawnSync(path.join(ROOT, 'bin', 'gstack-config'), ['set', 'telemetry', 'community'], {
encoding: 'utf-8',
env: { ...process.env, GSTACK_HOME: home },
});
expect(config.status).toBe(0);
spawnSync(path.join(ROOT, 'bin', 'gstack-config'), ['set', 'artifacts_sync_mode', 'full'], {
encoding: 'utf-8',
env: { ...process.env, GSTACK_HOME: home },
});
const r = run(['grants', '--json']);
expect(r.code).toBe(0);
const grants = JSON.parse(r.stdout);
const telemetry = grants.find((g: any) => g.grant === 'telemetry');
expect(telemetry.granted).toBe(true);
expect(telemetry.revoke).toContain('telemetry off');
const sync = grants.find((g: any) => g.grant === 'brain-sync');
expect(sync.granted).toBe(true);
expect(sync.value).toBe('full');
const hook = grants.find((g: any) => g.grant === 'redact_prepush_hook');
expect(hook.granted).toBe(false);
});
});
describe('gstack-egress usage', () => {
test('no subcommand exits 2 with usage', () => {
const r = run([]);
expect(r.code).toBe(2);
expect(r.stderr).toContain('Usage');
});
});

11
test/gstack-gbrain-mcp-verify.test.ts
View File

@ -51,13 +51,20 @@ function makeFakeCurl(opts: {
printf 'CURL_CALL '"'"'%s'"'"' ' "$@" >> "${curlCallLog}"
echo "" >> "${curlCallLog}"
# Walk argv to find -o <out> and -d <data>.
# Walk argv to find -o <out> and the request body (-d <data> or the
# receipted --data-binary @<payload-file> shape).
out=""
data=""
while [ $# -gt 0 ]; do
case "$1" in
-o) out="$2"; shift 2 ;;
-d) data="$2"; shift 2 ;;
--data-binary)
case "$2" in
@*) data="$(cat "\${2#@}" 2>/dev/null)" ;;
*) data="$2" ;;
esac
shift 2 ;;
*) shift ;;
esac
done
@ -83,6 +90,8 @@ function runVerify(token: string, url: string): { code: number; stdout: string;
...process.env,
PATH: `${fakeBinDir}:${process.env.PATH}`,
GBRAIN_MCP_TOKEN: token,
// The probe writes egress receipts — keep them in the temp home.
GSTACK_HOME: tmpDir,
},
encoding: 'utf-8',
});

85
test/helpers/claude-pty-runner.ts
View File

@ -24,7 +24,7 @@
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { hermeticChildEnv, isHermeticEnabled } from './hermetic-env';
import { hermeticChildEnv, hermeticSkillsConfigDir, isHermeticEnabled } from './hermetic-env';
/** Strip ANSI escapes for pattern-matching against visible text. */
export function stripAnsi(s: string): string {
@ -61,6 +61,11 @@ export function resolveClaudeBinary(): string | null {
}
export interface ClaudePtyOptions {
/** Register the repo's shipped skills in the child's user scope via
* hermeticSkillsConfigDir(). Required by any test that types a /skill
* slash command; without it hermetic claude rejects the command as
* Unknown before any model turn. No effect when EVALS_HERMETIC=0. */
seedSkills?: boolean;
/**
* Permission mode for the session.
* - 'plan' (default) — launches with --permission-mode plan
@ -300,6 +305,17 @@ export function isPermissionDialogVisible(visible: string): boolean {
}
/** Detect any AskUserQuestion-shaped numbered option list with cursor. */
/**
* Strip terminal residue that survives ANSI-stripping and can interleave
* with AUQ text: DEC cursor-visibility fragments (`[?25l` / `[?25h` — the ESC
* byte is gone but the bracket sequence remains) and the spinner frames
* rendered between them. Observed in plan-design-with-ui's failure buffer,
* where `[?25l✻Sprouting…[?25h` fragments sat inside the option lines.
*/
export function stripPtyResidue(visible: string): string {
return visible.replace(/\[\?25[lh]/g, '');
}
export function isNumberedOptionListVisible(visible: string): boolean {
// ❯ cursor + at least two numbered options 1-9.
// Matches the trust dialog AND plan-ready prompt AND skill questions.
@ -311,7 +327,8 @@ export function isNumberedOptionListVisible(visible: string): boolean {
// because `t-2` is a word-to-word transition. We use the weaker
// `[^0-9]2\.` to require a non-digit before `2` (so we don't match
// `12.0`) without requiring whitespace.
return /❯\s*1\./.test(visible) && /(^|[^0-9])2\./.test(visible);
const cleaned = stripPtyResidue(visible);
return /❯\s*1\./.test(cleaned) && /(^|[^0-9])2\./.test(cleaned);
}
// ────────────────────────────────────────────────────────────────────────────
@ -687,6 +704,7 @@ export function isScopeGateAutoSelectVisible(visible: string): boolean {
export function parseNumberedOptions(
visible: string,
): Array<{ index: number; label: string }> {
visible = stripPtyResidue(visible);
const tail = visible.length > 4096 ? visible.slice(-4096) : visible;
// Split on lines, look for `❯ N.` or ` N.` patterns. Up to N=9.
// The `\s*` after `.` (not `\s+`) is required because stripAnsi removes
@ -728,30 +746,41 @@ export function parseNumberedOptions(
const seenIndices = new Set<number>();
// Cursor line: option 1 may be inline after box dividers + prompt header
// (`...divider...header...❯1. label`). Use a non-anchored regex that
// captures `❯N. label` from anywhere on the line through end-of-line.
// Only used for the cursor line — subsequent options are parsed with the
// start-of-line `optionRe`.
// (`...divider...header...❯1. label`) — and, when the PTY reflows the whole
// AUQ onto ONE logical line, options 2..N sit on the SAME line after it
// (observed with /plan-design-review's Step-0 scope gate: `❯1.Branch diff
// ... 2.Plan or design doc ... 5.Chat about this ... Enter to select`).
// Parse the cursor line as a STREAM: find every `N.` token (not preceded
// by a digit, not followed by one — excludes "12." and "1.5"), require
// ascending indices starting from the cursor's option, and take each
// label as the text between successive number tokens.
const cursorLine = lines[cursorLineIdx] ?? '';
const cursorInlineRe = /❯\s*([1-9])\.\s*(\S.*?)\s*$/;
const inlineMatch = cursorInlineRe.exec(cursorLine);
if (inlineMatch) {
const idx = Number(inlineMatch[1]);
const label = (inlineMatch[2] ?? '').trim();
if (label.length > 0 && !seenIndices.has(idx)) {
seenIndices.add(idx);
found.push({ index: idx, label });
}
} else {
// No inline cursor match — fall back to start-of-line regex.
const startMatch = optionRe.exec(cursorLine);
if (startMatch) {
const idx = Number(startMatch[1]);
const label = (startMatch[2] ?? '').trim();
if (label.length > 0 && !seenIndices.has(idx)) {
seenIndices.add(idx);
found.push({ index: idx, label });
}
const cursorStart = cursorLine.indexOf('❯');
const cursorSegment = cursorStart >= 0 ? cursorLine.slice(cursorStart) : cursorLine;
const tokenRe = /(?:^|[^0-9])([1-9])\.(?!\d)\s*/g;
const tokens: Array<{ idx: number; labelStart: number; matchStart: number }> = [];
for (let m = tokenRe.exec(cursorSegment); m !== null; m = tokenRe.exec(cursorSegment)) {
tokens.push({
idx: Number(m[1]),
labelStart: m.index + m[0].length,
matchStart: m.index === 0 ? 0 : m.index + 1, // skip the [^0-9] guard char
});
}
// Keep only the ascending run that starts the sequence (1, 2, 3, ...);
// stray numbers inside labels break ascension and end the run.
let expected = 1;
for (let t = 0; t < tokens.length; t++) {
const token = tokens[t]!;
if (token.idx !== expected) continue;
const next = tokens
.slice(t + 1)
.find((candidate) => candidate.idx === expected + 1 && candidate.matchStart > token.labelStart);
const labelEnd = next ? next.matchStart : cursorSegment.length;
const label = cursorSegment.slice(token.labelStart, labelEnd).trim();
if (label.length > 0 && !seenIndices.has(token.idx)) {
seenIndices.add(token.idx);
found.push({ index: token.idx, label });
expected += 1;
}
}
@ -1285,6 +1314,9 @@ export async function launchClaudePty(
// Hermetic by default (test/helpers/hermetic-env.ts): operator session
// context never reaches the child; per-test opts.env merges last.
const childEnv = hermeticChildEnv(opts.env);
if (opts.seedSkills && hermetic && !opts.env?.CLAUDE_CONFIG_DIR) {
childEnv.CLAUDE_CONFIG_DIR = hermeticSkillsConfigDir();
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const proc = (Bun as any).spawn([claudePath, ...args], {
@ -1670,6 +1702,7 @@ export async function runPlanSkillObservation(opts: {
extraArgs: opts.extraArgs,
env: opts.env,
model: opts.model,
seedSkills: true,
});
try {
@ -1966,6 +1999,7 @@ export async function runPlanSkillCounting(opts: {
timeoutMs: timeoutMs + 60_000,
env: opts.env,
model: opts.model,
seedSkills: true,
});
const fingerprints: AskUserQuestionFingerprint[] = [];
@ -2199,6 +2233,7 @@ export async function runPlanSkillFloorCheck(opts: {
timeoutMs: timeoutMs + 60_000,
env: opts.env,
model: opts.model,
seedSkills: true,
});
try {

8
test/helpers/codex-session-runner.ts
View File

@ -199,8 +199,12 @@ export async function runCodexSkill(opts: {
}
}
// Build codex exec command
const args = ['exec', prompt, '--json', '-s', sandbox];
// Build codex exec command.
// --skip-git-repo-check: newer codex CLIs refuse exec in an untrusted
// non-git directory ("Not inside a trusted directory and
// --skip-git-repo-check was not specified") — our temp skill dirs are
// exactly that. Empirically verified against codex on this machine.
const args = ['exec', prompt, '--json', '-s', sandbox, '--skip-git-repo-check'];
// Spawn codex with temp HOME so it discovers our installed skill.
// Hermetic scrub (test/helpers/hermetic-env.ts) with codex's auth surface

292
test/helpers/eval-store.test.ts
View File

@ -6,6 +6,9 @@ import {
EvalCollector,
extractToolSummary,
findPreviousRun,
findLatestFinalizedRun,
isPartialEval,
listEvalJsonFiles,
compareEvalResults,
formatComparison,
generateCommentary,
@ -58,6 +61,19 @@ function makeResult(overrides?: Partial<EvalResult>): EvalResult {
};
}
/** Capture everything a block writes to stderr (finalize prints there). */
async function captureStderr(fn: () => Promise<void>): Promise<string> {
const original = process.stderr.write.bind(process.stderr);
let captured = '';
(process.stderr as any).write = (chunk: any) => { captured += String(chunk); return true; };
try {
await fn();
} finally {
(process.stderr as any).write = original;
}
return captured;
}
// --- EvalCollector tests ---
describe('EvalCollector', () => {
@ -119,6 +135,41 @@ describe('EvalCollector', () => {
expect(fs.readdirSync(tmpDir).filter(f => f.endsWith('.json') && !f.startsWith('_partial'))).toHaveLength(1);
});
test('with no completed prior run, says NO BASELINE instead of comparing against its own partial', async () => {
// addTest writes the in-progress accumulator into the same dir. If that
// counted as a baseline, the run would compare against itself and print a
// reassuring all-clear forever.
const collector = new EvalCollector('e2e', tmpDir);
collector.addTest(makeEntry({ name: 'test-1', passed: true }));
const output = await captureStderr(async () => { await collector.finalize(); });
expect(fs.existsSync(path.join(tmpDir, '_partial-e2e.json'))).toBe(true); // the trap exists
expect(output).toContain('NO BASELINE');
expect(output).not.toContain('vs previous');
expect(output).not.toContain('Stable run');
});
test('with a genuine prior run, reports the real delta', async () => {
fs.writeFileSync(
path.join(tmpDir, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({
timestamp: '2026-03-12T10:00:00Z',
tests: [makeEntry({ name: 'test-1', passed: true, turns_used: 5 })],
})),
);
const collector = new EvalCollector('e2e', tmpDir);
collector.addTest(makeEntry({ name: 'test-1', passed: false, turns_used: 5 }));
const output = await captureStderr(async () => { await collector.finalize(); });
expect(output).toContain('vs previous');
expect(output).toContain('REGRESSION');
expect(output).toContain('1 regressed');
expect(output).not.toContain('NO BASELINE');
});
test('empty collector writes valid file', async () => {
const collector = new EvalCollector('llm-judge', tmpDir);
const filepath = await collector.finalize();
@ -131,6 +182,59 @@ describe('EvalCollector', () => {
});
});
// --- GSTACK_EVAL_DIR + shard slug tests ---
describe('EvalCollector eval-dir resolution', () => {
const savedEnv = process.env.GSTACK_EVAL_DIR;
afterEach(() => {
if (savedEnv === undefined) delete process.env.GSTACK_EVAL_DIR;
else process.env.GSTACK_EVAL_DIR = savedEnv;
});
test('honors GSTACK_EVAL_DIR set after import — no --preload needed', async () => {
// The default eval dir must resolve lazily at construction, not at module
// load: the sharded runner sets GSTACK_EVAL_DIR in each shard child's env
// and shard tests import this module long before any collector exists.
const envDir = path.join(tmpDir, 'env-dir');
process.env.GSTACK_EVAL_DIR = envDir;
const collector = new EvalCollector('e2e');
collector.addTest(makeEntry());
await captureStderr(async () => { await collector.finalize(); });
expect(fs.readdirSync(envDir).filter(f => !f.startsWith('_partial'))).toHaveLength(1);
});
test('explicit constructor arg beats GSTACK_EVAL_DIR', async () => {
process.env.GSTACK_EVAL_DIR = path.join(tmpDir, 'env-dir');
const explicit = path.join(tmpDir, 'explicit');
const collector = new EvalCollector('e2e', explicit);
collector.addTest(makeEntry());
await captureStderr(async () => { await collector.finalize(); });
expect(fs.existsSync(path.join(tmpDir, 'env-dir'))).toBe(false);
expect(fs.readdirSync(explicit).length).toBeGreaterThan(0);
});
test('writes the shard slug when the eval dir is a shards/ subdir', async () => {
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
const collector = new EvalCollector('e2e', shardDir);
collector.addTest(makeEntry());
await captureStderr(async () => { await collector.finalize(); });
const partial = JSON.parse(fs.readFileSync(path.join(shardDir, '_partial-e2e.json'), 'utf-8'));
expect(partial.shard).toBe('skill-e2e-qa');
const final = fs.readdirSync(shardDir).find(f => !f.startsWith('_partial'))!;
expect(JSON.parse(fs.readFileSync(path.join(shardDir, final), 'utf-8')).shard).toBe('skill-e2e-qa');
});
test('writes no shard slug for a flat eval dir', async () => {
const collector = new EvalCollector('e2e', tmpDir);
collector.addTest(makeEntry());
await captureStderr(async () => { await collector.finalize(); });
const final = fs.readdirSync(tmpDir).find(f => f.endsWith('.json') && !f.startsWith('_partial'))!;
expect(JSON.parse(fs.readFileSync(path.join(tmpDir, final), 'utf-8')).shard).toBeUndefined();
});
});
// --- judgePassed tests ---
describe('judgePassed', () => {
@ -259,6 +363,34 @@ describe('findPreviousRun', () => {
expect(result).toBeNull(); // only file is excluded
});
test('never returns the in-progress accumulator as a baseline', () => {
// The current run's own partial carries the current tier + branch and the
// freshest timestamp. If it were a candidate, every run would compare
// against itself and report "no regressions" forever.
fs.writeFileSync(
path.join(tmpDir, '_partial-e2e.json'),
JSON.stringify(makeResult({ branch: 'main', timestamp: '2026-03-14T10:00:00Z', _partial: true })),
);
const result = findPreviousRun(tmpDir, 'e2e', 'main', path.join(tmpDir, 'current.json'));
expect(result).toBeNull();
});
test('prefers a completed run over a newer in-progress accumulator', () => {
fs.writeFileSync(
path.join(tmpDir, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({ branch: 'main', timestamp: '2026-03-12T10:00:00Z' })),
);
// Newer, same tier + branch, but in-progress — must lose to the older completed run.
fs.writeFileSync(
path.join(tmpDir, '_partial-e2e.json'),
JSON.stringify(makeResult({ branch: 'main', timestamp: '2026-03-14T10:00:00Z', _partial: true })),
);
const result = findPreviousRun(tmpDir, 'e2e', 'main', path.join(tmpDir, 'current.json'));
expect(result).toContain('0.3.5-main-e2e');
});
test('filters by tier', () => {
fs.writeFileSync(
path.join(tmpDir, '0.3.6-main-llm-judge-20260314-100000.json'),
@ -268,6 +400,143 @@ describe('findPreviousRun', () => {
const result = findPreviousRun(tmpDir, 'e2e', 'main', 'current.json');
expect(result).toBeNull(); // only llm-judge file, looking for e2e
});
test('a shard run prefers its own shard history over newer other-shard or flat priors', () => {
const mine = path.join(tmpDir, 'shards', 'skill-e2e-qa');
const other = path.join(tmpDir, 'shards', 'codex-e2e');
fs.mkdirSync(mine, { recursive: true });
fs.mkdirSync(other, { recursive: true });
// Same-shard prior — oldest of the three, must still win.
fs.writeFileSync(
path.join(mine, '0.3.4-main-e2e-20260311-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-11T10:00:00Z' })),
);
fs.writeFileSync(
path.join(other, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-12T10:00:00Z' })),
);
fs.writeFileSync(
path.join(tmpDir, '0.3.6-main-e2e-20260313-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-13T10:00:00Z' })),
);
const result = findPreviousRun(tmpDir, 'e2e', 'main', path.join(mine, 'current.json'));
expect(result).toContain(path.join('shards', 'skill-e2e-qa'));
});
test('a flat run prefers flat history over a newer shard prior', () => {
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
fs.mkdirSync(shardDir, { recursive: true });
fs.writeFileSync(
path.join(shardDir, '0.3.6-main-e2e-20260314-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-14T10:00:00Z' })),
);
fs.writeFileSync(
path.join(tmpDir, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-12T10:00:00Z' })),
);
const result = findPreviousRun(tmpDir, 'e2e', 'main', path.join(tmpDir, 'current.json'));
expect(result).toContain('0.3.5-main-e2e');
});
test('falls back to a shard prior when the flat dir has no candidate', () => {
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
fs.mkdirSync(shardDir, { recursive: true });
fs.writeFileSync(
path.join(shardDir, '0.3.6-main-e2e-20260314-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-14T10:00:00Z' })),
);
const result = findPreviousRun(tmpDir, 'e2e', 'main', path.join(tmpDir, 'current.json'));
expect(result).toContain(path.join('shards', 'skill-e2e-qa'));
});
});
// --- isPartialEval tests ---
describe('isPartialEval', () => {
test('flag set but file renamed — still partial', () => {
expect(isPartialEval({ _partial: true }, 'renamed-to-look-final.json')).toBe(true);
});
test('name matches but no flag — still partial', () => {
expect(isPartialEval({}, '_partial-e2e.json')).toBe(true);
expect(isPartialEval(null, path.join('/some/dir', '_partial-e2e.json'))).toBe(true);
});
test('finalized run is not partial', () => {
expect(isPartialEval(makeResult(), '0.3.6-main-e2e-20260314-100000.json')).toBe(false);
});
});
// --- listEvalJsonFiles / findLatestFinalizedRun tests ---
describe('findLatestFinalizedRun', () => {
test('listEvalJsonFiles recurses exactly one shards/*/ level', () => {
fs.writeFileSync(path.join(tmpDir, 'flat.json'), '{}');
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
fs.mkdirSync(shardDir, { recursive: true });
fs.writeFileSync(path.join(shardDir, 'sharded.json'), '{}');
// Nested one level deeper than the contract — must NOT be picked up.
const tooDeep = path.join(shardDir, 'shards', 'nested');
fs.mkdirSync(tooDeep, { recursive: true });
fs.writeFileSync(path.join(tooDeep, 'too-deep.json'), '{}');
fs.writeFileSync(path.join(tmpDir, 'not-json.txt'), '');
const files = listEvalJsonFiles(tmpDir).map(f => path.basename(f)).sort();
expect(files).toEqual(['flat.json', 'sharded.json']);
});
test('finds the newest finalized run across flat dir and shard subdirs', () => {
fs.writeFileSync(
path.join(tmpDir, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-12T10:00:00Z' })),
);
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
fs.mkdirSync(shardDir, { recursive: true });
fs.writeFileSync(
path.join(shardDir, '0.3.6-main-e2e-20260314-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-14T10:00:00Z' })),
);
const latest = findLatestFinalizedRun(tmpDir, 'e2e');
expect(latest?.filepath).toContain('skill-e2e-qa');
expect(latest?.result.timestamp).toBe('2026-03-14T10:00:00Z');
});
test('skips partials by flag and by name, filters by tier', () => {
// Newest by timestamp, but partial by flag under a shard dir.
const shardDir = path.join(tmpDir, 'shards', 'skill-e2e-qa');
fs.mkdirSync(shardDir, { recursive: true });
fs.writeFileSync(
path.join(shardDir, 'flagged.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-16T10:00:00Z', _partial: true })),
);
// Partial by name only.
fs.writeFileSync(
path.join(tmpDir, '_partial-e2e.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-15T10:00:00Z' })),
);
// Wrong tier.
fs.writeFileSync(
path.join(tmpDir, '0.3.6-main-llm-judge-20260317-100000.json'),
JSON.stringify(makeResult({ tier: 'llm-judge', timestamp: '2026-03-17T10:00:00Z' })),
);
// The genuine baseline.
fs.writeFileSync(
path.join(tmpDir, '0.3.5-main-e2e-20260312-100000.json'),
JSON.stringify(makeResult({ timestamp: '2026-03-12T10:00:00Z' })),
);
const latest = findLatestFinalizedRun(tmpDir, 'e2e');
expect(latest?.result.timestamp).toBe('2026-03-12T10:00:00Z');
});
test('returns null for missing dir or no finalized runs', () => {
expect(findLatestFinalizedRun('/nonexistent/path', 'e2e')).toBeNull();
expect(findLatestFinalizedRun(tmpDir, 'e2e')).toBeNull();
});
});
// --- compareEvalResults tests ---
@ -524,6 +793,29 @@ describe('generateCommentary', () => {
expect(notes.some(n => n.includes('No regressions'))).toBe(true);
});
test('says NO BASELINE instead of "stable" when nothing matched the prior run', () => {
// A baseline file existed but shares no test names (renamed/retired suite),
// so zero tests were actually compared. Claiming stability here is a lie.
const c: ComparisonResult = {
before_file: 'a.json', after_file: 'b.json',
before_branch: 'main', after_branch: 'main',
before_timestamp: '', after_timestamp: '',
deltas: [
{ name: 'a', before: { passed: false, cost_usd: 0 }, after: { passed: true, cost_usd: 0.10 }, status_change: 'unchanged' },
{ name: 'b', before: { passed: false, cost_usd: 0 }, after: { passed: true, cost_usd: 0.10 }, status_change: 'unchanged' },
{ name: 'c', before: { passed: false, cost_usd: 0 }, after: { passed: true, cost_usd: 0.10 }, status_change: 'unchanged' },
],
total_cost_delta: 0.30, total_duration_delta: 0,
improved: 0, regressed: 0, unchanged: 3,
tool_count_before: 0, tool_count_after: 0,
matched: 0,
};
const notes = generateCommentary(c);
expect(notes.some(n => n.includes('NO BASELINE'))).toBe(true);
expect(notes.some(n => n.includes('Stable run'))).toBe(false);
});
test('returns empty for stable run with no significant changes', () => {
const c: ComparisonResult = {
before_file: 'a.json', after_file: 'b.json',

174
test/helpers/eval-store.ts
View File

@ -39,7 +39,16 @@ export function getProjectEvalDir(): string {
return LEGACY_EVAL_DIR;
}
const DEFAULT_EVAL_DIR = getProjectEvalDir();
/**
* Lazy + memoized so importing this module never spawns the gstack-slug
* subprocess. Callers that pass an explicit dir or set GSTACK_EVAL_DIR
* (the sharded paid runner does, per shard) never pay for slug detection.
*/
let memoizedDefaultEvalDir: string | null = null;
function defaultEvalDir(): string {
if (memoizedDefaultEvalDir === null) memoizedDefaultEvalDir = getProjectEvalDir();
return memoizedDefaultEvalDir;
}
// --- Interfaces ---
@ -104,6 +113,8 @@ export interface EvalResult {
total_duration_ms: number;
wall_clock_ms?: number; // wall-clock from collector creation to finalization (shows parallelism)
tests: EvalTestEntry[];
/** Shard slug when the run was collected under <evalDir>/shards/<slug>/. */
shard?: string;
_partial?: boolean; // true for incremental saves, absent in final
}
@ -131,10 +142,94 @@ export interface ComparisonResult {
unchanged: number;
tool_count_before: number;
tool_count_after: number;
/** After-tests that had a same-named entry in the before run. 0 = nothing was
* actually compared, so no stability claim is warranted. */
matched?: number;
}
// --- Shared helpers ---
/**
* Is this eval file an in-progress accumulator rather than a finalized run?
*
* True on either signal: the `_partial` flag inside the JSON (the authoritative
* role marker) OR a filename starting with `_partial` (catches accumulators
* whose body predates the flag, and flagged files that were renamed keep being
* caught by the flag). Every baseline lookup must exclude these — an
* accumulator carries the current run's tier, branch, and freshest timestamp,
* so treating it as a baseline makes the run compare against itself.
*/
export function isPartialEval(data: unknown, filename: string): boolean {
if (path.basename(filename).startsWith('_partial')) return true;
return Boolean((data as { _partial?: unknown } | null)?._partial);
}
/**
* List eval JSON files in `evalDir` plus one level of `<evalDir>/shards/<slug>/`
* subdirectories (where the sharded paid runner points each shard's collector).
* Returns absolute paths. Missing dirs yield [].
*/
export function listEvalJsonFiles(evalDir: string): string[] {
const jsonIn = (dir: string): string[] => {
let names: string[];
try {
names = fs.readdirSync(dir);
} catch {
return [];
}
return names.filter(f => f.endsWith('.json')).map(f => path.join(dir, f));
};
const files = jsonIn(evalDir);
const shardsRoot = path.join(evalDir, 'shards');
let shardDirs: fs.Dirent[];
try {
shardDirs = fs.readdirSync(shardsRoot, { withFileTypes: true });
} catch {
return files;
}
for (const entry of shardDirs) {
if (!entry.isDirectory()) continue;
files.push(...jsonIn(path.join(shardsRoot, entry.name)));
}
return files;
}
/**
* Shard slug for an eval dir: when the dir is directly under a `shards/`
* directory (the sharded paid runner's per-shard GSTACK_EVAL_DIR layout),
* the dir name is the slug; otherwise null.
*/
export function shardSlugOfEvalDir(evalDir: string): string | null {
const normalized = path.resolve(evalDir);
return path.basename(path.dirname(normalized)) === 'shards' ? path.basename(normalized) : null;
}
/**
* Find the most recent finalized (non-partial) eval file for a tier, scanning
* `evalDir` and one level of `shards/<slug>/` subdirs. Shared by the budget
* regression gate and any tooling that needs "the latest real run".
*/
export function findLatestFinalizedRun(
evalDir: string,
tier: 'e2e' | 'llm-judge',
): { filepath: string; result: EvalResult } | null {
let latest: { filepath: string; result: EvalResult; timestamp: string } | null = null;
for (const filepath of listEvalJsonFiles(evalDir)) {
let data: EvalResult;
try {
data = JSON.parse(fs.readFileSync(filepath, 'utf-8')) as EvalResult;
} catch { continue; }
if (isPartialEval(data, filepath)) continue;
if (data.tier !== tier) continue;
const timestamp = data.timestamp ?? '';
if (!latest || timestamp.localeCompare(latest.timestamp) > 0) {
latest = { filepath, result: data, timestamp };
}
}
return latest ? { filepath: latest.filepath, result: latest.result } : null;
}
/**
* Determine if a planted-bug eval passed based on judge results vs ground truth thresholds.
* Centralizes the pass/fail logic so all planted-bug tests use the same criteria.
@ -171,8 +266,16 @@ export function extractToolSummary(transcript: any[]): Record<string, number> {
}
/**
* Find the most recent prior eval file for comparison.
* Prefers same branch, falls back to any branch.
* Find the most recent prior COMPLETED eval file for comparison.
* Scans the eval dir plus one level of `shards/<slug>/` subdirs. Prefers
* same shard slug (a shard's own history over another shard's or the flat
* dir's), then same branch, then falls back to anything.
*
* In-progress accumulators (`_partial: true`, written by savePartial after every
* test) are never candidates: the current run's own partial carries the current
* tier + branch and the freshest timestamp, so including it made every run
* compare against itself and report "no regressions" unconditionally. The
* exclusion is by role (the `_partial` flag), not by filename.
*/
export function findPreviousRun(
evalDir: string,
@ -180,24 +283,22 @@ export function findPreviousRun(
branch: string,
excludeFile: string,
): string | null {
let files: string[];
try {
files = fs.readdirSync(evalDir).filter(f => f.endsWith('.json'));
} catch {
return null; // dir doesn't exist
}
// Parse top-level fields from each file (cheap — no full tests array needed)
const entries: Array<{ file: string; branch: string; timestamp: string }> = [];
for (const file of files) {
if (file === path.basename(excludeFile)) continue;
const fullPath = path.join(evalDir, file);
const entries: Array<{ file: string; branch: string; timestamp: string; shard: string | null }> = [];
for (const fullPath of listEvalJsonFiles(evalDir)) {
if (path.resolve(fullPath) === path.resolve(excludeFile)) continue;
try {
const raw = fs.readFileSync(fullPath, 'utf-8');
// Quick parse — only grab the fields we need
const data = JSON.parse(raw);
if (isPartialEval(data, fullPath)) continue; // in-progress run, not a baseline
if (data.tier !== tier) continue;
entries.push({ file: fullPath, branch: data.branch || '', timestamp: data.timestamp || '' });
entries.push({
file: fullPath,
branch: data.branch || '',
timestamp: data.timestamp || '',
shard: data.shard || shardSlugOfEvalDir(path.dirname(fullPath)),
});
} catch { continue; }
}
@ -206,11 +307,17 @@ export function findPreviousRun(
// Sort by timestamp descending
entries.sort((a, b) => b.timestamp.localeCompare(a.timestamp));
// Prefer same branch
const sameBranch = entries.find(e => e.branch === branch);
if (sameBranch) return sameBranch.file;
// Fallback: any branch
// Prefer same shard slug (null = the flat dir), then same branch, then any.
const targetShard = shardSlugOfEvalDir(path.dirname(excludeFile));
const preferences: Array<(e: typeof entries[number]) => boolean> = [
e => e.shard === targetShard && e.branch === branch,
e => e.shard === targetShard,
e => e.branch === branch,
];
for (const matches of preferences) {
const hit = entries.find(matches);
if (hit) return hit.file;
}
return entries[0].file;
}
@ -226,6 +333,7 @@ export function compareEvalResults(
const deltas: TestDelta[] = [];
let improved = 0, regressed = 0, unchanged = 0;
let toolCountBefore = 0, toolCountAfter = 0;
let matched = 0;
// Index before tests by name
const beforeMap = new Map<string, EvalTestEntry>();
@ -246,6 +354,7 @@ export function compareEvalResults(
let statusChange: TestDelta['status_change'] = 'unchanged';
if (beforeTest) {
matched++;
if (!beforeTest.passed && afterTest.passed) { statusChange = 'improved'; improved++; }
else if (beforeTest.passed && !afterTest.passed) { statusChange = 'regressed'; regressed++; }
else { unchanged++; }
@ -314,6 +423,7 @@ export function compareEvalResults(
unchanged,
tool_count_before: toolCountBefore,
tool_count_after: toolCountAfter,
matched,
};
}
@ -512,7 +622,17 @@ export function generateCommentary(c: ComparisonResult): string[] {
}
}
// 4. Overall summary
// 4. No baseline — say so. A run with nothing to compare against must never
// read as "stable"; silence or a false all-clear is worse than no output.
if (c.matched === 0 && c.deltas.length > 0) {
notes.push(
`NO BASELINE: none of these ${c.deltas.length} test(s) appear in ${path.basename(c.before_file)}. ` +
'Nothing was compared, so this run says nothing about regressions.',
);
return notes;
}
// 5. Overall summary
if (c.deltas.length >= 3 && regressions.length === 0) {
const overallParts: string[] = [];
@ -649,11 +769,13 @@ export class EvalCollector {
private tests: EvalTestEntry[] = [];
private finalized = false;
private evalDir: string;
private shard: string | null;
private createdAt = Date.now();
constructor(tier: 'e2e' | 'llm-judge', evalDir?: string) {
this.tier = tier;
this.evalDir = evalDir || DEFAULT_EVAL_DIR;
this.evalDir = evalDir || process.env.GSTACK_EVAL_DIR || defaultEvalDir();
this.shard = shardSlugOfEvalDir(this.evalDir);
}
addTest(entry: EvalTestEntry): void {
@ -684,6 +806,7 @@ export class EvalCollector {
total_cost_usd: Math.round(totalCost * 100) / 100,
total_duration_ms: totalDuration,
tests: this.tests,
...(this.shard ? { shard: this.shard } : {}),
_partial: true,
};
@ -721,6 +844,7 @@ export class EvalCollector {
total_duration_ms: totalDuration,
wall_clock_ms: Date.now() - this.createdAt,
tests: this.tests,
...(this.shard ? { shard: this.shard } : {}),
};
// Write eval file
@ -742,7 +866,11 @@ export class EvalCollector {
const comparison = compareEvalResults(prevResult, result, prevFile, filepath);
process.stderr.write(formatComparison(comparison) + '\n');
} else {
process.stderr.write('\nFirst run — no comparison available.\n');
process.stderr.write(
`\nNO BASELINE: no completed prior ${this.tier} run found in ${this.evalDir}` +
' (the in-progress accumulator is not a baseline). Nothing compared —' +
' this run says nothing about regressions.\n',
);
}
} catch (err: any) {
process.stderr.write(`\nCompare error: ${err.message}\n`);

38
test/helpers/gemini-session-runner.ts
View File

@ -8,8 +8,8 @@
* Key differences from Codex session-runner:
* - Uses `gemini -p` instead of `codex exec`
* - Output is NDJSON with event types: init, message, tool_use, tool_result, result
* - Uses `--output-format stream-json --yolo --skip-trust` instead of `--json -s read-only`
* (`--skip-trust` required for headless/untrusted cwds; see gemini trusted-folders docs)
* - Uses `--output-format stream-json --yolo` instead of `--json -s read-only`
* (`--skip-trust` was removed in gemini-cli 0.34; folder trust is settings-driven now)
* - No temp HOME needed — Gemini discovers skills from `.agents/skills/` in cwd
* - Message events are streamed with `delta: true` — must concatenate
*/
@ -121,10 +121,11 @@ export async function runGeminiSkill(opts: {
};
}
// Build gemini command
// --skip-trust: headless/CI and temp cwds aren't in ~/.gemini/trustedFolders.json;
// without it gemini exits FatalUntrustedWorkspaceError before any model call.
const args = ['-p', prompt, '--output-format', 'stream-json', '--yolo', '--skip-trust'];
// Build gemini command.
// --skip-trust was REMOVED in gemini-cli 0.34 ("Unknown arguments:
// skip-trust"); folder trust moved to settings and no longer needs a flag
// for headless runs. --yolo still auto-approves tool actions.
const args = ['-p', prompt, '--output-format', 'stream-json', '--yolo'];
// Spawn gemini — uses real HOME for auth (~/.gemini; HOME is allowlisted),
// cwd for skill discovery. Hermetic scrub with gemini's auth surface
@ -198,6 +199,31 @@ export async function runGeminiSkill(opts: {
process.stderr.write(` [gemini stderr] ${stderr.trim().slice(0, 200)}\n`);
}
// Environment-unusable classification: these are Google-side conditions no
// test assertion can act on — the deprecated individual code-assist auth
// path ("migrate to the Antigravity suite") and argv drift on older/newer
// CLIs. Return the same SKIP shape as binary-not-found so callers report
// SKIPPED instead of a false FAIL.
const unusableMarkers = [
'no longer supported for Gemini Code Assist',
'antigravity',
'Unknown arguments: skip-trust',
];
if (exitCode !== 0 && parsed.tokens === 0) {
const marker = unusableMarkers.find((m) => stderr.toLowerCase().includes(m.toLowerCase()));
if (marker) {
return {
output: `SKIP: gemini CLI unusable (${marker})`,
toolCalls: [],
tokens: 0,
exitCode: -1,
durationMs,
sessionId: null,
rawLines: collectedLines,
};
}
}
return {
output: parsed.output,
toolCalls: parsed.toolCalls,

70
test/helpers/hermetic-env.ts
View File

@ -36,7 +36,8 @@ import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { promotedEnv } from '../../lib/conductor-env-shim';
import { isProcessAlive } from '../../browse/src/error-handling';
import { isProcessAlive, safeUnlink } from '../../browse/src/error-handling';
import { skillCensus, frontmatterName } from './skill-census';
/** Exact env names a hermetic child keeps. Everything not listed (or matched
* by a prefix rule below) is dropped. */
@ -225,6 +226,73 @@ export function getHermeticDirs(): HermeticDirs {
return cachedDirs;
}
let cachedSkillsConfigDir: string | null = null;
/**
* A hermetic CLAUDE_CONFIG_DIR with the repo's shipped skills REGISTERED in
* user scope, mirroring ./setup's registration exactly: each discovered skill
* gets a REAL directory `<configDir>/skills/<registryName>/` containing a
* SYMLINK to that skill's SKILL.md (absolute path), plus a `sections/`
* symlink when the skill has one. registryName is the frontmatter `name:`
* (dir-name fallback), NO gstack- prefix; the root SKILL.md router registers
* as `_gstack-command`. skillCensus().registryEntries is the authoritative
* set of what must appear here.
*
* The default hermetic dir deliberately seeds no skills — correct for
* children that install their own or probe setup behavior — but a PTY test
* that TYPES `/office-hours` needs the slash command to exist, or claude
* rejects it as Unknown command before any model turn and the gate measures
* nothing. Separate dir, same runRoot: opt-in per session, never contaminates
* the default-config children, and the existing exit teardown + pid-aware GC
* cover it. Ends in `/.claude` for the same plan-path anchoring reason as
* HermeticDirs.configDir.
*
* Two intentional non-hermetic edges:
* - Seeding reads the LIVE repo tree BY DESIGN — the skills ARE the subject
* under test; a snapshot would measure stale copies.
* - HOME is not hermeticized, so the ~64 absolute
* `~/.claude/skills/gstack/...` preamble references inside each SKILL.md
* still resolve to the operator install (same limitation as CI).
*/
export function hermeticSkillsConfigDir(): string {
if (cachedSkillsConfigDir) return cachedSkillsConfigDir;
const { runRoot } = getHermeticDirs();
const configDir = path.join(runRoot, 'with-skills', '.claude');
const skillsDir = path.join(configDir, 'skills');
fs.mkdirSync(skillsDir, { recursive: true });
fs.writeFileSync(
path.join(configDir, '.claude.json'),
JSON.stringify(buildSeedConfig({
apiKey: process.env.ANTHROPIC_API_KEY ?? process.env.GSTACK_ANTHROPIC_API_KEY,
trustedDirs: [repoRoot()],
}), null, 2),
);
const root = repoRoot();
for (const rel of skillCensus(root).physicalSkillFiles) {
const skillMd = path.join(root, rel);
const skillDir = path.dirname(rel);
const registryName = rel === 'SKILL.md'
? '_gstack-command'
: frontmatterName(skillMd) || skillDir;
const target = path.join(skillsDir, registryName);
// Idempotent overwrite mirrors setup's re-link: connect-chrome (a dir
// symlink to open-gstack-browser) shares its target's frontmatter name,
// so the two walk entries collapse to one registry dir.
fs.mkdirSync(target, { recursive: true });
safeUnlink(path.join(target, 'SKILL.md'));
fs.symlinkSync(skillMd, path.join(target, 'SKILL.md'));
if (rel !== 'SKILL.md') {
const sections = path.join(root, skillDir, 'sections');
if (fs.existsSync(sections)) {
safeUnlink(path.join(target, 'sections'));
fs.symlinkSync(sections, path.join(target, 'sections'));
}
}
}
cachedSkillsConfigDir = configDir;
return configDir;
}
/** A dir younger than this is never GC'd even if its pid looks dead — guards
* against PID reuse deleting a freshly-created dir whose original pid exited
* and was recycled to an unrelated live process between create and GC. */

25
test/helpers/paid-test-set.ts Normal file
View File

@ -0,0 +1,25 @@
/**
* The ONE definition of which test files are paid (API spend, external
* services, e2e harnesses). package.json's test:gate/test:evals globs, the
* free-suite exclusion in scripts/test-free-shards.ts, and the sharded paid
* runner in scripts/test-paid-shards.ts all derive from this list — a file
* added to one and not the others either burns money in the free suite or
* silently never runs in the paid tier.
*/
import { matchGlob } from './touchfiles';
/** The exact globs package.json's `test:gate` passes to `bun test`. */
export const PAID_TEST_GLOBS = [
'test/skill-llm-eval.test.ts',
'test/skill-e2e-*.test.ts',
'test/skill-routing-e2e.test.ts',
'test/codex-e2e.test.ts',
'test/gemini-e2e.test.ts',
] as const;
/** True when a repo-relative path (either slash style) is a paid test file. */
export function isPaidTestFile(relativePath: string): boolean {
const normalized = relativePath.replace(/\\/g, '/');
return PAID_TEST_GLOBS.some((glob) => matchGlob(normalized, glob));
}

15
test/helpers/providers/gemini.ts
View File

@ -103,10 +103,10 @@ export function resultFromGeminiStream(
* Headless flags always passed:
* --output-format stream-json — NDJSON events (message/tool_use/result)
* --yolo — auto-approve tools (non-interactive)
* --skip-trust — trust cwd for this session; required when
* workdir is a temp/untrusted folder (benchmarks
* use mkdtemp). Without it headless gemini exits
* before calling the model.
*
* --skip-trust is gone: gemini-cli 0.34 removed the flag ("Unknown arguments:
* skip-trust") — folder trust is settings-driven now and headless runs no
* longer need a flag for temp workdirs.
*/
export class GeminiAdapter implements ProviderAdapter {
readonly name = 'gemini';
@ -136,10 +136,9 @@ export class GeminiAdapter implements ProviderAdapter {
async run(opts: RunOpts): Promise<RunResult> {
const start = Date.now();
// Default to --yolo (non-interactive) and stream-json output so we can parse
// tokens + tool calls. --skip-trust is required for headless/temp workdirs
// (gemini CLI otherwise exits: "not running in a trusted directory").
// Callers can override via extraArgs.
const args = ['-p', opts.prompt, '--output-format', 'stream-json', '--yolo', '--skip-trust'];
// tokens + tool calls. Callers can override via extraArgs. (--skip-trust was
// removed in gemini-cli 0.34; passing it errors at argv parse.)
const args = ['-p', opts.prompt, '--output-format', 'stream-json', '--yolo'];
if (opts.model) args.push('--model', opts.model);
if (opts.extraArgs) args.push(...opts.extraArgs);

96
test/helpers/skill-census.ts Normal file
View File

@ -0,0 +1,96 @@
/**
* Shared skill census — the ONE place that counts skills (C11).
*
* Three consumers (hermetic seeding, context-bill ground truth, catalog-budget
* test) each need a DIFFERENT count of "the skills", and hand-rolled walks
* encode the wrong one somewhere. The counts diverge because of two facts:
*
* 1. `connect-chrome/` is a directory SYMLINK to `open-gstack-browser/`.
* `Dirent.isDirectory()` is false for it (scripts/discover-skills.ts
* skips it) while setup's trailing-slash shell glob follows it.
* 2. The root `SKILL.md` is a router, registered by `setup` under the
* alias name `_gstack-command`, not as an authored skill.
*
* So: use `physicalSkillFiles` when you mean "every SKILL.md a filesystem
* walk can reach" (context-bill's walker), `authoredSkills` when you mean
* "distinct skills a human maintains" (catalog budget), and
* `registryEntries` when you mean "what a host discovers after ./setup"
* (hermetic seeding must mirror this exactly).
*/
import * as fs from 'fs';
import * as path from 'path';
const SKIP = new Set(['node_modules', '.git', 'dist']);
export interface SkillCensus {
/** Every reachable `<dir>/SKILL.md` (symlinked dirs INCLUDED) plus the
* root `SKILL.md` router. Relative paths from root. */
physicalSkillFiles: string[];
/** Distinct authored skills: symlink-deduped by realpath, root router
* EXCLUDED. Each entry is the canonical directory name. */
authoredSkills: string[];
/** What `setup` registers into `~/.claude/skills/` with prefixing off:
* one entry per unique frontmatter `name:` (falling back to dir name),
* plus the `_gstack-command` root alias when the root router exists.
* Symlinked dirs collapse here because they share a frontmatter name. */
registryEntries: string[];
}
/** First `name:` from SKILL.md frontmatter, mirroring `setup`'s
* `grep -m1 '^name:'` (whitespace stripped). Empty string if absent. */
export function frontmatterName(skillMdPath: string): string {
let body: string;
try {
body = fs.readFileSync(skillMdPath, 'utf-8');
} catch {
return '';
}
const m = body.match(/^name:\s*(.+)$/m);
return m ? m[1].replace(/\s+/g, '') : '';
}
export function skillCensus(root: string): SkillCensus {
const physicalSkillFiles: string[] = [];
const authoredByRealpath = new Map<string, string>();
const registrySet = new Set<string>();
if (fs.existsSync(path.join(root, 'SKILL.md'))) {
physicalSkillFiles.push('SKILL.md');
registrySet.add('_gstack-command');
}
for (const entry of fs.readdirSync(root, { withFileTypes: true })) {
if (entry.name.startsWith('.') || SKIP.has(entry.name)) continue;
const dirPath = path.join(root, entry.name);
// Follow directory symlinks like setup's shell glob does; Dirent alone
// reports symlinks as non-directories.
let isDir = entry.isDirectory();
if (!isDir && entry.isSymbolicLink()) {
try {
isDir = fs.statSync(dirPath).isDirectory();
} catch {
continue; // dangling symlink
}
}
if (!isDir) continue;
const skillMd = path.join(dirPath, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
physicalSkillFiles.push(`${entry.name}/SKILL.md`);
const real = fs.realpathSync(dirPath);
if (!authoredByRealpath.has(real)) {
authoredByRealpath.set(real, path.basename(real));
}
registrySet.add(frontmatterName(skillMd) || entry.name);
}
return {
physicalSkillFiles: physicalSkillFiles.sort(),
authoredSkills: [...authoredByRealpath.values()].sort(),
registryEntries: [...registrySet].sort(),
};
}

84
test/hermetic-skills-seeding.test.ts Normal file
View File

@ -0,0 +1,84 @@
/**
* Unit tests for hermeticSkillsConfigDir() — the opt-in hermetic config dir
* that registers the repo's shipped skills for PTY slash-command children.
* Free tier — no API calls; exercises the real seeder against the live repo
* tree (that's the seeder's contract: the skills ARE the subject under test).
*
* Pins four contracts:
* 1. The seeded dir is a valid CLAUDE_CONFIG_DIR (.claude.json present,
* /.claude suffix, under the hermetic runRoot).
* 2. Registration mirrors ./setup exactly: one entry per
* skillCensus().registryEntries, each a REAL dir with a SKILL.md symlink
* resolving to a real file (plus sections/ when the skill has one).
* 3. connect-chrome (dir symlink) collapses into open-gstack-browser — no
* duplicate, no connect-chrome entry.
* 4. Per-process idempotence: the second call returns the cached dir.
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import {
hermeticSkillsConfigDir,
getHermeticDirs,
buildSeedConfig,
} from './helpers/hermetic-env';
import { skillCensus } from './helpers/skill-census';
const ROOT = path.resolve(__dirname, '..');
const configDir = hermeticSkillsConfigDir();
const skillsDir = path.join(configDir, 'skills');
describe('hermeticSkillsConfigDir', () => {
test('seeded dir contains .claude.json and ends in /.claude under runRoot', () => {
expect(fs.existsSync(path.join(configDir, '.claude.json'))).toBe(true);
expect(path.basename(configDir)).toBe('.claude');
expect(configDir.startsWith(getHermeticDirs().runRoot + path.sep)).toBe(true);
});
test('one registry entry per skillCensus registryEntries, nothing extra', () => {
const seeded = fs.readdirSync(skillsDir).sort();
expect(seeded).toEqual(skillCensus(ROOT).registryEntries);
});
test('every SKILL.md is a symlink resolving to a real file', () => {
for (const entry of fs.readdirSync(skillsDir)) {
const link = path.join(skillsDir, entry, 'SKILL.md');
expect(fs.lstatSync(link).isSymbolicLink()).toBe(true);
expect(fs.statSync(link).isFile()).toBe(true); // follows the link
}
});
test('sections/ symlink registered for skills that ship one', () => {
// ship/ is a carved skill with a sections/ dir — the registered entry
// must expose it or runtime "Read sections/<name>.md" 404s.
expect(fs.existsSync(path.join(ROOT, 'ship', 'sections'))).toBe(true);
const link = path.join(skillsDir, 'ship', 'sections');
expect(fs.lstatSync(link).isSymbolicLink()).toBe(true);
expect(fs.statSync(link).isDirectory()).toBe(true);
});
test('connect-chrome collapses into a single open-gstack-browser entry', () => {
const seeded = fs.readdirSync(skillsDir);
expect(seeded.filter((n) => n === 'open-gstack-browser')).toHaveLength(1);
expect(seeded).not.toContain('connect-chrome');
});
test('root router registered as _gstack-command pointing at the root SKILL.md', () => {
const link = path.join(skillsDir, '_gstack-command', 'SKILL.md');
expect(fs.realpathSync(link)).toBe(fs.realpathSync(path.join(ROOT, 'SKILL.md')));
});
test('second call returns the cached dir', () => {
expect(hermeticSkillsConfigDir()).toBe(configDir);
});
test('buildSeedConfig with undefined apiKey omits customApiKeyResponses', () => {
// The seeder passes process.env keys straight through; when the operator
// has no key exported the seed must stay valid (child fails auth later,
// not here).
const seed = buildSeedConfig({ apiKey: undefined, trustedDirs: [ROOT] });
expect(seed).not.toHaveProperty('customApiKeyResponses');
expect(seed.hasCompletedOnboarding).toBe(true);
});
});

21
test/hermetic-wiring.test.ts
View File

@ -16,6 +16,8 @@
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { getHermeticDirs, hermeticSkillsConfigDir } from './helpers/hermetic-env';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
@ -110,4 +112,23 @@ describe('hermetic wiring tripwire', () => {
offenders.join(', '),
).toEqual([]);
});
test('skill seeding stays under runRoot and reads the live repo tree, never operator ~/.claude', () => {
// hermeticSkillsConfigDir() is a BLESSED non-hermetic edge: it registers
// the LIVE repo tree's skills (the skills are the subject under test).
// What it must never do is hand children the operator's ~/.claude — the
// seeded CLAUDE_CONFIG_DIR lives under the hermetic runRoot, and every
// registered symlink resolves into the repo checkout.
const configDir = hermeticSkillsConfigDir();
const { runRoot } = getHermeticDirs();
const operatorClaude = path.join(os.homedir(), '.claude') + path.sep;
expect(configDir.startsWith(runRoot + path.sep)).toBe(true);
expect(configDir.startsWith(operatorClaude)).toBe(false);
const skillsDir = path.join(configDir, 'skills');
for (const entry of fs.readdirSync(skillsDir)) {
const target = fs.readlinkSync(path.join(skillsDir, entry, 'SKILL.md'));
expect(target.startsWith(operatorClaude), `${entry}: symlink escapes to ${target}`).toBe(false);
expect(fs.realpathSync(target).startsWith(fs.realpathSync(ROOT) + path.sep), `${entry}: symlink outside repo: ${target}`).toBe(true);
}
});
});

117
test/paid-shards.test.ts Normal file
View File

@ -0,0 +1,117 @@
/**
* Pins the paid-tier sharded runner (scripts/test-paid-shards.ts).
*
* Two properties matter, and both are why `test:gate` has never finished a run:
* 1. Enumeration + sharding — every file `test:gate`'s globs expand to gets
* its own process, and tier exclusion only ever fires on explicit evidence.
* 2. A spinning shard is killed externally and the run CONTINUES. The fake
* command here is a real busy loop, so an in-process timer could not save
* it — exactly the failure mode `sample` caught on the wedged run.
*/
import { describe, test, expect } from 'bun:test';
import {
PAID_TEST_GLOBS,
classifyPaidTestFile,
collectPaidTestFiles,
isPaidTestFile,
planPaidShards,
runPaidShards,
summarize,
type ShardOutcome,
} from '../scripts/test-paid-shards';
describe('paid test enumeration', () => {
test('matches the globs package.json test:gate expands', () => {
expect(isPaidTestFile('test/skill-e2e-qa-workflow.test.ts')).toBe(true);
expect(isPaidTestFile('test/skill-llm-eval.test.ts')).toBe(true);
expect(isPaidTestFile('test/codex-e2e.test.ts')).toBe(true);
expect(isPaidTestFile('test/skill-e2e-triage-audit.test.ts')).toBe(true);
// Outside the globs: no dash, extra suffix, or a free test.
expect(isPaidTestFile('test/skill-e2e.test.ts')).toBe(false);
expect(isPaidTestFile('test/codex-e2e-recommendation-substance.test.ts')).toBe(false);
expect(isPaidTestFile('test/paid-shards.test.ts')).toBe(false);
});
test('discovers files and gives each one its own shard', () => {
const files = collectPaidTestFiles();
expect(files.length).toBeGreaterThan(0);
expect(files.every(isPaidTestFile)).toBe(true);
expect(PAID_TEST_GLOBS.length).toBe(5);
const shards = planPaidShards(files);
expect(shards.flat().sort()).toEqual([...files].sort());
expect(shards.every((shard) => shard.length === 1)).toBe(true);
});
});
describe('tier classification', () => {
test('excludes only on an explicit other-tier guard', () => {
const gateGuard = "const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'gate';";
const periodicGuard = "const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'periodic';";
expect(classifyPaidTestFile(gateGuard, 'gate').included).toBe(true);
expect(classifyPaidTestFile(periodicGuard, 'gate').included).toBe(false);
expect(classifyPaidTestFile(gateGuard, 'periodic').included).toBe(false);
expect(classifyPaidTestFile(periodicGuard, 'periodic').included).toBe(true);
});
test('keeps files whose tier is decided per-test at runtime', () => {
// Naming an E2E_TIERS key is not evidence — 'retro' appears in the
// LLM-judge file, which test:gate does run.
const noGuard = "runSkillTest('retro', async () => {});";
expect(classifyPaidTestFile(noGuard, 'gate').included).toBe(true);
expect(classifyPaidTestFile(noGuard, 'periodic').included).toBe(true);
expect(classifyPaidTestFile('', 'gate').included).toBe(true);
});
});
describe('shard execution', () => {
const BUSY_LOOP = 'const end = Date.now() + 600000; while (Date.now() < end) {}';
const commandFor = (files: string[]) => {
if (files[0] === 'spin') return { command: process.execPath, args: ['-e', BUSY_LOOP] };
if (files[0] === 'fail') return { command: process.execPath, args: ['-e', 'process.exit(3)'] };
return { command: process.execPath, args: ['-e', 'console.log("ok")'] };
};
test('a spinning shard times out, is killed, and the run continues', async () => {
const lines: string[] = [];
const summary = await runPaidShards([['spin'], ['fail'], ['pass']], {
timeoutMs: 1_200,
jobs: 1,
commandFor,
log: (line) => lines.push(line),
});
const byName = (name: string) => summary.outcomes.find((o) => o.files[0] === name) as ShardOutcome;
expect(byName('spin').status).toBe('timed-out');
expect(byName('fail').status).toBe('failed');
expect(byName('pass').status).toBe('passed');
// The run never aborted: every shard reports, none is 'never-started'.
expect(summary).toMatchObject({
total: 3, executed: 3, passed: 1, failed: 1, timedOut: 1, neverStarted: 0,
});
// The spinner was killed at the deadline, not left to burn a core.
expect(byName('spin').elapsedMs).toBeLessThan(30_000);
expect(byName('spin').groupPid).toBeGreaterThan(0);
if (process.platform !== 'win32') {
expect(() => process.kill(byName('spin').groupPid as number, 0)).toThrow();
}
// Heartbeat: a START and a terminal line per shard, with elapsed seconds.
expect(lines.filter((l) => l.includes(' START ')).length).toBe(3);
expect(lines.some((l) => /TIMED-OUT in \d+s/.test(l))).toBe(true);
expect(lines.some((l) => /PASSED in \d+s/.test(l))).toBe(true);
}, 30_000);
test('summarize reports shards that never ran', () => {
const summary = summarize([
{ shard: 1, files: ['a'], status: 'passed', exitCode: 0, elapsedMs: 1, groupPid: 1 },
{ shard: 2, files: ['b'], status: 'never-started', exitCode: null, elapsedMs: 0, groupPid: null },
]);
expect(summary).toMatchObject({ total: 2, executed: 1, passed: 1, neverStarted: 1 });
});
});

67
test/pty-askuserquestion-single-line.test.ts Normal file
View File

@ -0,0 +1,67 @@
/**
* Pins single-logical-line AskUserQuestion detection (the plan-design-with-ui
* gate-timeout class). When the PTY reflows a boxed AskUserQuestion, ALL
* options land on ONE logical line after stripAnsi — the per-line option
* parser then finds only option 1 and the >= 2 check fails forever while the
* (correct) question sits on screen. Both fixture strings below are condensed
* from REAL observed failure buffers of
* test/skill-e2e-plan-design-with-ui.test.ts.
*/
import { describe, expect, it } from 'bun:test';
import {
isNumberedOptionListVisible,
parseNumberedOptions,
stripPtyResidue,
} from './helpers/claude-pty-runner';
// Condensed from the 2026-08-13 failure buffer: whole AskUserQuestion on one
// logical line — dividers, cursor option, options 2-5, footer. Note the
// missing spaces ("2.Planordesigndoc") from stripped cursor-positioning
// escapes.
const SINGLE_LINE_QUESTION =
'────────────Planning: /tmp/x/.claude/plans/soft-questing-jellyfish.md──────────' +
' ☐ Review target What should I review? <gstack-qid:plan-design-review-scope-gate>' +
'❯1.Branch diff (current WIP) Review the design implications of what changed. ' +
'Recommendation: A when a branch diff exists.2.Planordesigndoc Paste or point me to a plan file. ' +
'3. Spcific page, file, r pah Name a specific file. 4. Type something.' +
'────────────5.ChataboutthisEnter to select · ↑/↓ o navigate · Escto cancel';
// The same AskUserQuestion with DEC cursor-visibility residue + spinner
// frames interleaved, as captured before stripPtyResidue existed.
const RESIDUE_QUESTION =
'[?25l✻Sprouting…[?25h[?25l✶[?25h' + SINGLE_LINE_QUESTION.slice(0, 200) +
'[?25l·still thinking[?25h' + SINGLE_LINE_QUESTION.slice(200);
describe('single-logical-line AskUserQuestion detection', () => {
it('isNumberedOptionListVisible matches the reflowed one-line AskUserQuestion', () => {
expect(isNumberedOptionListVisible(SINGLE_LINE_QUESTION)).toBe(true);
});
it('parseNumberedOptions finds the full ascending option run on one line', () => {
const options = parseNumberedOptions(SINGLE_LINE_QUESTION);
expect(options.length).toBeGreaterThanOrEqual(2);
expect(options[0]).toEqual({ index: 1, label: expect.stringContaining('Branch diff') });
expect(options[1]?.index).toBe(2);
});
it('survives DEC residue + spinner interleave', () => {
expect(isNumberedOptionListVisible(RESIDUE_QUESTION)).toBe(true);
expect(parseNumberedOptions(RESIDUE_QUESTION).length).toBeGreaterThanOrEqual(2);
});
it('stripPtyResidue removes cursor-visibility fragments only', () => {
expect(stripPtyResidue('[?25la[?25hb')).toBe('ab');
expect(stripPtyResidue('keep [?250] this')).toBe('keep [?250] this');
});
it('still rejects prose numbered lists (no cursor sigil)', () => {
expect(parseNumberedOptions('steps: 1. Read the file 2. Edit it 3. Done')).toEqual([]);
});
it('still parses classic multi-line AskUserQuestions', () => {
const multiLine = 'What should I do?\n❯ 1. First option\n 2. Second option\n 3. Third option\n';
const options = parseNumberedOptions(multiLine);
expect(options.map((o) => o.index)).toEqual([1, 2, 3]);
});
});

77
test/pty-skill-seeding-wiring.test.ts Normal file
View File

@ -0,0 +1,77 @@
/**
* Static-grep tripwire for PTY slash-command skill seeding. Free tier — no API.
*
* The default hermetic config dir registers NO skills, so a PTY test that
* TYPES a /skill slash command against it gets "Unknown command" before any
* model turn — the test still runs, still spends money, and measures nothing.
* Every test file that sends a slash command over the PTY must therefore
* either route through a runPlanSkill* helper (which opts in for you) or pass
* `seedSkills: true` in its own launchClaudePty options.
*
* Pattern mirrors test/hermetic-wiring.test.ts: read sources as text, assert
* invariants on their contents. Brittle by design — changing the seeding
* wiring must force the author to look here.
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
/** A PTY send whose payload starts with a slash command (`/name`, optionally
* followed by `\r`, whitespace, or the closing quote). A second slash right
* after the name (a file path like '/tmp/x') does NOT match. */
const SLASH_SEND = /\.send\(\s*(['"`])\/[a-z][a-z0-9-]*(\\r|\s|\1)/;
const RUN_PLAN_HELPER = /\brunPlanSkill(Observation|Counting|FloorCheck)\s*\(/;
function testFiles(dir: string): string[] {
const out: string[] = [];
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) out.push(...testFiles(full));
else if (entry.name.endsWith('.test.ts')) out.push(full);
}
return out;
}
describe('PTY skill-seeding tripwire', () => {
test('every slash-command PTY test seeds skills (helper or seedSkills: true)', () => {
const offenders: string[] = [];
for (const full of testFiles(path.join(ROOT, 'test'))) {
if (path.basename(full) === 'pty-skill-seeding-wiring.test.ts') continue;
const src = fs.readFileSync(full, 'utf-8');
const lines = src.split('\n');
const sendLine = lines.findIndex((l) => SLASH_SEND.test(l));
if (sendLine === -1) continue;
if (RUN_PLAN_HELPER.test(src)) continue;
if (src.includes('seedSkills: true')) continue;
offenders.push(`${path.relative(ROOT, full)}:${sendLine + 1}`);
}
expect(
offenders,
'These tests type a /skill slash command into a hermetic PTY child that has ' +
'no skills registered — claude rejects it as Unknown command and the test ' +
'measures nothing. Pass seedSkills: true to launchClaudePty (or route ' +
'through a runPlanSkill* helper): ' + offenders.join(', '),
).toEqual([]);
});
test('the runPlanSkill* helpers all opt in via seedSkills: true', () => {
// The helper family types slash commands on behalf of ~20 test files;
// dropping the opt-in there silently un-measures all of them at once.
const src = fs.readFileSync(path.join(ROOT, 'test/helpers/claude-pty-runner.ts'), 'utf-8');
const optIns = src.match(/seedSkills: true/g) ?? [];
expect(optIns.length).toBeGreaterThanOrEqual(3);
});
test('launchClaudePty wires seedSkills to hermeticSkillsConfigDir()', () => {
const src = fs.readFileSync(path.join(ROOT, 'test/helpers/claude-pty-runner.ts'), 'utf-8');
// Gated on hermetic (EVALS_HERMETIC=0 must keep the operator config) and
// on the per-test env override (explicit CLAUDE_CONFIG_DIR wins).
const wired =
/if\s*\(opts\.seedSkills && hermetic && !opts\.env\?\.CLAUDE_CONFIG_DIR\)\s*\{\s*\n\s*childEnv\.CLAUDE_CONFIG_DIR = hermeticSkillsConfigDir\(\);/.test(src);
expect(wired, 'launchClaudePty must set CLAUDE_CONFIG_DIR from hermeticSkillsConfigDir() when seedSkills && hermetic && no per-test override').toBe(true);
});
});

39
test/skill-budget-regression.test.ts
View File

@ -27,10 +27,10 @@
import { describe, test } from 'bun:test';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import {
getProjectEvalDir,
findPreviousRun,
findLatestFinalizedRun,
compareEvalResults,
assertNoBudgetRegression,
type EvalResult,
@ -72,42 +72,9 @@ function currentGitBranch(): string {
}
}
interface LatestRun {
filepath: string;
result: EvalResult;
}
/** Find the most recent finalized (non-_partial) eval file for a tier. */
function findLatestRun(evalDir: string, tier: 'e2e' | 'llm-judge'): LatestRun | null {
let entries: string[];
try {
entries = fs.readdirSync(evalDir);
} catch {
return null;
}
const candidates: Array<{ filepath: string; timestamp: string }> = [];
for (const f of entries) {
if (!f.endsWith('.json')) continue;
if (f.startsWith('_partial')) continue;
const fullPath = path.join(evalDir, f);
try {
const data = JSON.parse(fs.readFileSync(fullPath, 'utf-8')) as EvalResult;
if (data.tier !== tier) continue;
candidates.push({ filepath: fullPath, timestamp: data.timestamp ?? '' });
} catch { /* ignore corrupt */ }
}
if (candidates.length === 0) return null;
candidates.sort((a, b) => b.timestamp.localeCompare(a.timestamp));
const top = candidates[0]!;
return {
filepath: top.filepath,
result: JSON.parse(fs.readFileSync(top.filepath, 'utf-8')) as EvalResult,
};
}
function checkTier(tier: 'e2e' | 'llm-judge'): void {
const evalDir = getProjectEvalDir();
const latest = findLatestRun(evalDir, tier);
const latest = findLatestFinalizedRun(evalDir, tier);
if (!latest) {
// eslint-disable-next-line no-console
console.log(`[budget-regression:${tier}] no current run in ${evalDir} — skipping`);
@ -165,7 +132,7 @@ function checkTier(tier: 'e2e' | 'llm-judge'): void {
/** Enforce a hard dollar cap on per-run eval cost. */
function checkHardCap(tier: 'e2e' | 'llm-judge'): void {
const evalDir = getProjectEvalDir();
const latest = findLatestRun(evalDir, tier);
const latest = findLatestFinalizedRun(evalDir, tier);
if (!latest) return;
const cap = TIER_CAPS[tier];
const cost = latest.result.total_cost_usd;

69
test/skill-census.test.ts Normal file
View File

@ -0,0 +1,69 @@
/**
* Pins the three-count contract of test/helpers/skill-census.ts (C11).
*
* No hardcoded totals here — the catalog-budget test owns the ratchet.
* This file pins the STRUCTURAL relationships that make the three counts
* mean different things, using the live repo as the fixture.
*/
import { describe, expect, it } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import { frontmatterName, skillCensus } from './helpers/skill-census';
const ROOT = path.resolve(new URL(import.meta.url).pathname, '..', '..');
const census = skillCensus(ROOT);
describe('skillCensus', () => {
it('physicalSkillFiles includes the root router and the symlinked dir', () => {
expect(census.physicalSkillFiles).toContain('SKILL.md');
expect(census.physicalSkillFiles).toContain('connect-chrome/SKILL.md');
expect(census.physicalSkillFiles).toContain('open-gstack-browser/SKILL.md');
});
it('authoredSkills dedupes the connect-chrome symlink and excludes the root router', () => {
expect(census.authoredSkills).toContain('open-gstack-browser');
expect(census.authoredSkills).not.toContain('connect-chrome');
// Root router is not an authored skill; its dir entry would be '' anyway.
for (const name of census.authoredSkills) expect(name.length).toBeGreaterThan(0);
});
it('registryEntries carries the root alias and collapses shared frontmatter names', () => {
expect(census.registryEntries).toContain('_gstack-command');
expect(
census.registryEntries.filter((n) => n === 'open-gstack-browser'),
).toHaveLength(1);
});
it('count relationships hold: physical = authored + root + symlink dups', () => {
const symlinkDups = census.physicalSkillFiles.length - 1 - census.authoredSkills.length;
expect(symlinkDups).toBeGreaterThanOrEqual(1); // connect-chrome today
// Registry = unique frontmatter names + root alias. It can only collapse
// entries relative to physical, never invent them.
expect(census.registryEntries.length).toBeLessThanOrEqual(census.physicalSkillFiles.length);
expect(census.registryEntries.length).toBeGreaterThan(census.authoredSkills.length - 1);
});
it('frontmatterName mirrors setup: first ^name: line, whitespace stripped', () => {
const qa = frontmatterName(path.join(ROOT, 'qa', 'SKILL.md'));
expect(qa).toBe('qa');
const alias = frontmatterName(path.join(ROOT, 'connect-chrome', 'SKILL.md'));
expect(alias).toBe('open-gstack-browser');
expect(frontmatterName(path.join(ROOT, 'no-such-dir', 'SKILL.md'))).toBe('');
});
it('every registry entry a host would see resolves back to a physical SKILL.md', () => {
const names = new Set(
census.physicalSkillFiles
.filter((p) => p !== 'SKILL.md')
.map((p) => frontmatterName(path.join(ROOT, p)) || path.dirname(p)),
);
for (const entry of census.registryEntries) {
if (entry === '_gstack-command') {
expect(fs.existsSync(path.join(ROOT, 'SKILL.md'))).toBe(true);
} else {
expect(names.has(entry)).toBe(true);
}
}
});
});

1
test/skill-e2e-autoplan-chain.test.ts
View File

@ -71,6 +71,7 @@ describeE2E('/autoplan chain ordering (periodic)', () => {
permissionMode: 'plan',
cwd: tempDir,
timeoutMs: 1_080_000, // 18 min, slightly above test budget
seedSkills: true,
});
const hits: PhaseHit[] = [];

19
test/skill-e2e-benchmark-providers.test.ts
View File

@ -30,7 +30,15 @@ import * as os from 'os';
// --- Prerequisites / gating ---
const evalsEnabled = !!process.env.EVALS;
const describeIfEvals = evalsEnabled ? describe : describe.skip;
// External-service tests are periodic-tier (CLAUDE.md tiering rule 3) —
// the header above says so, but without a whole-file guard the sharded gate
// runner still selects this file into gate. The positive form below is the
// canonical guard shape classifyPaidTestFile greps for.
const tierOk = process.env.EVALS_TIER === 'periodic';
const describeIfEvals = evalsEnabled && tierOk ? describe : describe.skip;
if (evalsEnabled && !tierOk) {
process.stderr.write('\nbenchmark-providers: SKIPPED — external-service test, periodic tier only\n');
}
const PROMPT = 'Reply with exactly this text and nothing else: ok';
@ -127,6 +135,15 @@ describeIfEvals('multi-provider benchmark adapters (live)', () => {
}
const result = await gemini.run({ prompt: PROMPT, workdir, timeoutMs: 120_000 });
if (result.error) {
// auth / rate_limit are ENVIRONMENT conditions the test can't act on
// (e.g. Google deprecated the individual code-assist auth path — the
// adapter classifies "no longer supported" as auth). A live smoke
// reports them as a skip, not a false adapter failure. timeout/unknown
// still fail: those are the drift classes this test exists to catch.
if (result.error.code === 'auth' || result.error.code === 'rate_limit') {
process.stderr.write(`\ngemini live smoke: SKIPPED — ${result.error.code}: ${result.error.reason.slice(0, 160)}\n`);
return;
}
throw new Error(`gemini errored: ${result.error.code} — ${result.error.reason}`);
}
// Adapter must never report empty-success (#2159). After content/stats

8
test/skill-e2e-hermetic-canary.test.ts
View File

@ -84,9 +84,13 @@ describeIfSelected('hermetic isolation canaries', ['hermetic-canary', 'hermetic-
try {
const result = await runSkillTest({
// ${VAR:-} expansion, not bare $VAR: when scrubbing WORKS the planted
// vars are unset, and under a nounset shell (set -u in the operator's
// shell snapshot) a bare expansion of an unset var errors the whole
// command — making the canary fail exactly when isolation succeeds.
prompt: 'Run exactly this bash command and then stop: ' +
'echo "CFG=$CLAUDE_CONFIG_DIR"; echo "GH=$GSTACK_HOME"; ' +
'echo "CW=$CONDUCTOR_WORKSPACE_PATH"; echo "GP=$GBRAIN_POISON_PROBE"',
'echo "CFG=${CLAUDE_CONFIG_DIR:-}"; echo "GH=${GSTACK_HOME:-}"; ' +
'echo "CW=${CONDUCTOR_WORKSPACE_PATH:-}"; echo "GP=${GBRAIN_POISON_PROBE:-}"',
workingDirectory: workDir,
maxTurns: 3,
allowedTools: ['Bash'],

1
test/skill-e2e-plan-ceo-mode-routing.test.ts
View File

@ -152,6 +152,7 @@ describeE2E('/plan-ceo-review mode routing (gate)', () => {
const session = await launchClaudePty({
permissionMode: 'plan',
timeoutMs: 540_000,
seedSkills: true,
});
try {
await Bun.sleep(8000);

Some files were not shown because too many files have changed in this diff Show More