RomanNum3ral
/
gstack
mirror of https://github.com/garrytan/gstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
gstack/browse/test
History
Garry Tan b86ef06706
test(security): live + fixture-replay bench harness with 500-case capture 
Adds two new benches that permanently guard the v2 tuning:

- security-bench-ensemble-live.test.ts (opt-in via GSTACK_BENCH_ENSEMBLE=1).
  Runs full ensemble on BrowseSafe-Bench smoke with real Haiku calls.
  Worker-pool concurrency (default 8, tunable via
  GSTACK_BENCH_ENSEMBLE_CONCURRENCY) cuts wall clock from ~2hr to
  ~25min on 500 cases. Captures Haiku responses to fixture for replay.
  Subsampling via GSTACK_BENCH_ENSEMBLE_CASES for faster iteration.
  Stop-loss iterations write to ~/.gstack-dev/evals/stop-loss-iter-N-*
  WITHOUT overwriting canonical fixture.

- security-bench-ensemble.test.ts (CI gate, deterministic replay).
  Replays captured fixture through combineVerdict, asserts
  detection >= 55% AND FP <= 25%. Fail-closed when fixture is missing
  AND security-layer files changed in branch diff. Uses
  `git diff --name-only base` (two-dot) to catch both committed
  and working-tree changes — `git diff base...HEAD` would silently
  skip in CI after fixture lands.

- browse/test/fixtures/security-bench-haiku-responses.json — 500 cases
  × 3 classifier signals each. Header includes schema_version, pinned
  model, component hashes (prompt, exemplars, thresholds, combiner,
  dataset version). Any change invalidates the fixture and forces
  fresh live capture.

- docs/evals/security-bench-ensemble-v2.json — durable PR artifact
  with measured TP/FN/FP/TN, 95% CIs, knob state, v1 baseline delta.
  Checked in so reviewers can see the numbers that justified the ship.

Measured baseline on the new harness:
  TP=146 FN=114 FP=55 TN=185 → 56.2% / 22.9% → GATE PASS

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 		2026-04-21 20:32:15 -07:00
..
fixtures test(security): live + fixture-replay bench harness with 500-case capture 2026-04-21 20:32:15 -07:00
activity.test.ts feat: headed mode + sidebar agent + Chrome extension (v0.12.0) (#517) 2026-03-26 11:15:24 -06:00
adversarial-security.test.ts fix: security audit remediation — 12 fixes, 20 tests (v0.13.1.0) (#595) 2026-03-28 08:35:24 -06:00
batch.test.ts refactor: extract TabSession for per-tab state isolation (v0.15.16.0) (#873) 2026-04-07 00:23:36 -07:00
browser-manager-unit.test.ts feat: headed mode + sidebar agent + Chrome extension (v0.12.0) (#517) 2026-03-26 11:15:24 -06:00
build.test.ts fix: ngrok Windows build + close CI error-swallowing gap (v0.18.0.1) (#1024) 2026-04-16 13:49:04 -07:00
bun-polyfill.test.ts fix: Windows support — Node.js server fallback for Playwright (#255) 2026-03-20 12:22:11 -07:00
commands.test.ts feat(browse): Puppeteer parity — load-html, screenshot --selector, viewport --scale, file:// (v1.1.0.0) (#1062) 2026-04-18 23:25:33 +08:00
compare-board.test.ts refactor: extract TabSession for per-tab state isolation (v0.15.16.0) (#873) 2026-04-07 00:23:36 -07:00
config.test.ts fix: Windows browse — health-check-first ensureServer, detached startServer, Windows process mgmt (v0.11.11.0) (#431) 2026-03-24 00:38:10 -07:00
content-security.test.ts feat: content security — 4-layer prompt injection defense for pair-agent (#815) 2026-04-06 14:41:06 -07:00
cookie-import-browser.test.ts feat: Wave 3 — community bug fixes & platform support (v0.11.6.0) (#359) 2026-03-23 22:15:23 -07:00
cookie-picker-routes.test.ts community wave: 6 PRs + hardening (v0.18.1.0) (#1028) 2026-04-17 00:45:13 -07:00
data-platform.test.ts feat: browser data platform for AI agents (v0.16.0.0) (#907) 2026-04-08 00:41:55 -07:00
dx-polish.test.ts feat(browse): Puppeteer parity — load-html, screenshot --selector, viewport --scale, file:// (v1.1.0.0) (#1062) 2026-04-18 23:25:33 +08:00
error-handling.test.ts refactor: AI slop reduction with cross-model quality review (v0.16.3.0) (#941) 2026-04-10 17:13:15 -10:00
file-drop.test.ts feat: headed mode + sidebar agent + Chrome extension (v0.12.0) (#517) 2026-03-26 11:15:24 -06:00
find-browse.test.ts feat: multi-agent support — gstack works on Codex, Gemini CLI, and Cursor (v0.9.0) (#226) 2026-03-19 18:20:50 -07:00
findport.test.ts feat: community PRs — faster install, skill namespacing, uninstall, Codex fallback, Windows fix, Python patterns (v0.12.9.0) (#561) 2026-03-27 00:44:37 -06:00
gstack-config.test.ts feat: composable skills — INVOKE_SKILL resolver + factoring infrastructure (v0.13.7.0) (#644) 2026-03-29 23:35:17 -06:00
gstack-update-check.test.ts fix: community PRs + security hardening + E2E stability (v0.12.7.0) (#552) 2026-03-26 23:21:27 -06:00
handoff.test.ts refactor: extract TabSession for per-tab state isolation (v0.15.16.0) (#873) 2026-04-07 00:23:36 -07:00
learnings-injection.test.ts fix: community security wave — 8 PRs, 4 contributors (v0.15.13.0) (#847) 2026-04-06 00:47:04 -07:00
path-validation.test.ts fix: community security wave — 8 PRs, 4 contributors (v0.15.13.0) (#847) 2026-04-06 00:47:04 -07:00
pdf-flags.test.ts feat(v1.4.0.0): /make-pdf — markdown to publication-quality PDFs (#1086) 2026-04-20 13:20:30 +08:00
platform.test.ts fix: Windows support — Node.js server fallback for Playwright (#255) 2026-03-20 12:22:11 -07:00
security-adversarial-fixes.test.ts feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK 2026-04-21 20:31:53 -07:00
security-adversarial.test.ts feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK 2026-04-21 20:31:53 -07:00
security-audit-r2.test.ts feat(browse): Puppeteer parity — load-html, screenshot --selector, viewport --scale, file:// (v1.1.0.0) (#1062) 2026-04-18 23:25:33 +08:00
security-bench-ensemble-live.test.ts test(security): live + fixture-replay bench harness with 500-case capture 2026-04-21 20:32:15 -07:00
security-bench-ensemble.test.ts test(security): live + fixture-replay bench harness with 500-case capture 2026-04-21 20:32:15 -07:00
security-bench.test.ts feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK 2026-04-21 20:31:53 -07:00
security-bunnative.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-classifier.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-e2e-fullstack.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-integration.test.ts feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK 2026-04-21 20:31:53 -07:00
security-live-playwright.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-review-flow.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-review-fullstack.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-review-sidepanel-e2e.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-sidepanel-dom.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security-source-contracts.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
security.test.ts feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK 2026-04-21 20:31:53 -07:00
server-auth.test.ts fix: cookie picker auth token leak (v0.15.17.0) (#904) 2026-04-08 10:10:13 -07:00
sidebar-agent-roundtrip.test.ts fix: sidebar agent uses real tab URL instead of stale Playwright URL (v0.12.6.0) (#544) 2026-03-26 22:07:03 -06:00
sidebar-agent.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
sidebar-integration.test.ts fix: sidebar agent uses real tab URL instead of stale Playwright URL (v0.12.6.0) (#544) 2026-03-26 22:07:03 -06:00
sidebar-security.test.ts feat(security): ML prompt injection defense for sidebar (v1.4.0.0) (#1089) 2026-04-20 22:18:37 +08:00
sidebar-unit.test.ts fix: sidebar agent uses real tab URL instead of stale Playwright URL (v0.12.6.0) (#544) 2026-03-26 22:07:03 -06:00
sidebar-ux.test.ts fix: community security wave — 8 PRs, 4 contributors (v0.15.13.0) (#847) 2026-04-06 00:47:04 -07:00
snapshot.test.ts refactor: extract TabSession for per-tab state isolation (v0.15.16.0) (#873) 2026-04-07 00:23:36 -07:00
state-ttl.test.ts fix: security audit remediation — 12 fixes, 20 tests (v0.13.1.0) (#595) 2026-03-28 08:35:24 -06:00
tab-isolation.test.ts feat: browser data platform for AI agents (v0.16.0.0) (#907) 2026-04-08 00:41:55 -07:00
test-server.ts feat: Phase 3.5 — cookie import, QA testing, team retro (v0.3.1) (#29) 2026-03-13 00:31:41 -07:00
token-registry.test.ts feat: browser data platform for AI agents (v0.16.0.0) (#907) 2026-04-08 00:41:55 -07:00
url-validation.test.ts feat(browse): Puppeteer parity — load-html, screenshot --selector, viewport --scale, file:// (v1.1.0.0) (#1062) 2026-04-18 23:25:33 +08:00
watch.test.ts feat: headed mode + sidebar agent + Chrome extension (v0.12.0) (#517) 2026-03-26 11:15:24 -06:00
watchdog.test.ts community wave: 6 PRs + hardening (v0.18.1.0) (#1028) 2026-04-17 00:45:13 -07:00
welcome-page.test.ts feat: GStack Browser — double-click AI browser with anti-bot stealth (#695) 2026-04-04 10:17:05 -07:00