gstack/test/helpers
Garry Tan 28deff3d00
fix: harden E2E tests — server lifecycle, timeouts, preamble budget, skip flaky
Cross-cutting fixes:
- Pre-seed ~/.gstack/.completeness-intro-seen and ~/.gstack/.telemetry-prompted
  so preamble doesn't burn 3-7 turns on lake intro + telemetry in every test
- Each describe block creates its own test server instance instead of sharing
  a global that dies between suites

Test fixes (5 tests):
- /qa quick: own server instance + preamble skip
- /review SQL injection: timeout 90→180s, maxTurns 15→20, added assertion
  that review output actually mentions SQL injection
- /review design-lite: maxTurns 25→35 + preamble skip (now detects 7/7)
- ship-base-branch: both timeouts 90→150/180s + preamble skip
- plan-eng artifact: clean stale state in beforeAll, maxTurns 20→25

Skipped (4 flaky/redundant tests):
- contributor-mode: tests prompt compliance, not skill functionality
- design-consultation-research: WebSearch-dependent, redundant with core
- design-consultation-preview: redundant with core test
- /qa bootstrap: too ambitious (65 turns, installs vitest)

Also: preamble skip added to qa-only, qa-fix-loop, design-consultation-core,
and design-consultation-existing prompts. Updated touchfiles entries and
touchfiles.test.ts. Added honest comment to codex-review-findings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 22:54:56 -07:00
..
codex-session-runner.ts feat: multi-agent support — gstack works on Codex, Gemini CLI, and Cursor (v0.9.0) (#226) 2026-03-19 18:20:50 -07:00
eval-store.test.ts feat: QA restructure, browser ref staleness, eval efficiency metrics (v0.4.0) (#83) 2026-03-15 23:55:39 -05:00
eval-store.ts feat: QA restructure, browser ref staleness, eval efficiency metrics (v0.4.0) (#83) 2026-03-15 23:55:39 -05:00
llm-judge.ts feat: 3-tier eval suite with planted-bug outcome testing (EVALS=1) 2026-03-14 01:17:36 -05:00
observability.test.ts fix: never clean up observability artifacts — partial file persists after finalize 2026-03-14 12:37:38 -05:00
session-runner.test.ts feat: stream-json NDJSON parser for real-time E2E progress 2026-03-14 03:49:36 -05:00
session-runner.ts fix: auto-clear stale heartbeat when process is dead 2026-03-14 12:55:40 -05:00
skill-parser.ts feat: 3-tier eval suite with planted-bug outcome testing (EVALS=1) 2026-03-14 01:17:36 -05:00
touchfiles.ts fix: harden E2E tests — server lifecycle, timeouts, preamble budget, skip flaky 2026-03-20 22:54:56 -07:00