gstack/scripts
Garry Tan 4e0725479f
feat(evals): sharded paid tier runner
scripts/test-paid-shards.ts runs the gate/periodic tier one Bun process
per test file, with an EXTERNAL wall-clock timeout that SIGKILLs the
shard's detached process group and an aggregate that distinguishes
passed / failed / timed-out / never-started — partial execution can no
longer read as a pass. Bun's native --shard/--isolate covers none of
this: no process-group kill (hung claude/codex PTY grandchildren
survive in-process isolation), no never-started taxonomy, no per-shard
env. Each shard child gets GSTACK_EVAL_DIR=<evalDir>/shards/<slug>/
(slug = test filename sans extension, stable across runs) so shard
baselines compare against their own prior runs.

Output classification lives in scripts/test-strict-output.ts (strict
exit-code derivation, incremental fail-line classifier, child signal
forwarding) so the runner and any future strict bun-test wrapper share
one implementation. Enumeration derives from the shared paid-test-set
module; tier exclusion fires only on an explicit whole-file
EVALS_TIER === '<other>' guard.

package.json gains test:gate:sharded / test:periodic:sharded, and
eval:bg:gate / eval:bg:periodic now run the sharded scripts with detach
timeouts sized to the worst case (gate: 49 shards x 30min / 4 jobs ~
6.2h -> 25200s; periodic: 59 -> 28800s).

test/paid-shards.test.ts pins enumeration, tier classification, and the
kill-and-continue property with a real busy-loop shard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 5e76bd5931836257f896cedfe4e93912cb759c70)
2026-08-12 15:31:47 -07:00
..
app feat: GStack Browser — double-click AI browser with anti-bot stealth (#695) 2026-04-04 10:17:05 -07:00
host-adapters feat: declarative multi-host platform + OpenCode, Slate, Cursor, OpenClaw (v0.15.5.0) (#793) 2026-04-04 15:32:20 -07:00
resolvers v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533) 2026-08-12 11:12:28 -07:00
analytics.ts feat: safety hook skills + skill usage telemetry (v0.7.1) (#189) 2026-03-18 23:57:59 -05:00
archetypes.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
brain-cache-spec.ts v1.52.1.0 feat: brain-aware planning — 5 skills read structured gbrain context before asking (#1742) 2026-05-29 08:35:00 -07:00
build-app.sh v1.41.1.0 fix wave: 7 HIGH bugs from external audit + regression tests (PR #1169 follow-up) (#1592) 2026-05-20 06:56:41 -07:00
build.sh v1.42.0.0 Daegu wave: 23 community-filed bugs + PTY classifier enforcement (24 bisect commits) (#1594) 2026-05-20 07:35:01 -07:00
capture-baseline.ts v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712) 2026-05-26 16:50:03 -07:00
compare-pr-version.ts v1.16.0.0 feat: tunnel allowlist 17→26 + canDispatchOverTunnel pure function (#1253) 2026-04-28 00:57:28 -07:00
declared-annotation.ts v1.52.0.0 feat(plan-tune): explicit consent + first-run setup wizard for contributors (#1741) 2026-05-28 18:21:09 -07:00
detect-bump.ts v1.11.0.0 feat(ship): workspace-aware version allocation (#1168) 2026-04-23 23:03:27 -07:00
dev-skill.ts v1.57.9.0 feat: source-clean gbrain render (dev-setup --out-dir + machine-wide gbrain-refresh) (#1951) 2026-06-09 22:29:23 -07:00
discover-skills.ts v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806) 2026-05-30 12:09:10 -07:00
eval-compare.ts feat(evals): env-driven lazy eval dir + shard-aware store and tooling 2026-08-12 15:31:47 -07:00
eval-list.ts feat(evals): env-driven lazy eval dir + shard-aware store and tooling 2026-08-12 15:31:47 -07:00
eval-select.ts feat: diff-based test selection for E2E and LLM-judge evals (v0.6.1.0) (#139) 2026-03-17 18:45:41 -05:00
eval-summary.ts feat(evals): env-driven lazy eval dir + shard-aware store and tooling 2026-08-12 15:31:47 -07:00
eval-watch.ts feat: /land-and-deploy, /canary, /benchmark + perf review (v0.7.0) (#183) 2026-03-21 14:31:36 -07:00
garry-output-comparison.ts fix: remove hardcoded author emails from throughput script 2026-04-18 15:36:50 +08:00
gen-llms-txt.ts v1.28.0.0 feat: browse --headed/--proxy/--navigate + gstack/llms.txt + webdriver-only stealth (#1363) 2026-05-07 20:14:59 -07:00
gen-skill-docs.ts v1.61.0.0 fix wave: guards failing open / silent failures (9 fixes, 4 community PRs absorbed) (#2472) 2026-08-08 09:28:45 -07:00
gstack-schema-pack.ts v1.52.1.0 feat: brain-aware planning — 5 skills read structured gbrain context before asking (#1742) 2026-05-29 08:35:00 -07:00
host-config-export.ts fix(hosts): validate suppressed resolver names 2026-07-14 12:56:10 -07:00
host-config.ts fix(hosts): validate suppressed resolver names 2026-07-14 12:56:10 -07:00
jargon-list.json feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
models.ts feat(v1.5.2.0): Opus 4.7 migration — model overlay, voice, routing (#1117) 2026-04-22 01:06:22 -07:00
one-way-doors.ts v1.61.0.0 fix wave: guards failing open / silent failures (9 fixes, 4 community PRs absorbed) (#2472) 2026-08-08 09:28:45 -07:00
preflight-agent-sdk.ts v1.39.2.0 feat: GSTACK_* env-shim for Conductor + gbrain/gstack setup docs (#1534) 2026-05-16 12:32:33 -07:00
proactive-suggestions.json v1.61.0.0 fix wave: guards failing open / silent failures (9 fixes, 4 community PRs absorbed) (#2472) 2026-08-08 09:28:45 -07:00
psychographic-signals.ts v1.52.0.0 feat(plan-tune): explicit consent + first-run setup wizard for contributors (#1741) 2026-05-28 18:21:09 -07:00
question-registry.ts v1.52.0.0 feat(plan-tune): explicit consent + first-run setup wizard for contributors (#1741) 2026-05-28 18:21:09 -07:00
setup-scc.sh feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
skill-check.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
slop-diff.ts security: tunnel dual-listener + SSRF + envelope + path wave (v1.6.0.0) (#1137) 2026-04-21 21:58:27 -07:00
task-emission-schema.ts v1.38.1.0 fix wave: surrogate-safe page captures (#1440), Implementation Tasks across review skills (#1454), root-level artifact patterns (#1452) (#1504) 2026-05-14 21:46:50 -07:00
test-free-shards.ts refactor(evals): single shared paid-test-set module 2026-08-12 15:31:47 -07:00
test-paid-shards.ts feat(evals): sharded paid tier runner 2026-08-12 15:31:47 -07:00
test-strict-output.ts feat(evals): sharded paid tier runner 2026-08-12 15:31:47 -07:00
update-readme-throughput.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
write-version-files.sh v1.42.0.0 Daegu wave: 23 community-filed bugs + PTY classifier enforcement (24 bisect commits) (#1594) 2026-05-20 07:35:01 -07:00