Garry Tan f692da35d7 evals: judge default back to Sonnet — Haiku regressed the rubric family (A/B receipts) ... The partial-diff rehearsal was the Haiku judge default's first live run and it failed all three selected doc-rubric judges. Controlled A/B on the identical health-rubric prompt: Haiku 2/2/2 vs Sonnet 4/3/4, both with coherent reasoning — Haiku is simply a harsher grader on long-document rubrics, and every >=4 threshold in skill-llm-eval was calibrated against months of Sonnet baselines. Per D1a's pin-on-regressors protocol the default reverts; a new GSTACK_EVAL_MODEL_JUDGE override makes future recalibration a one-var experiment. Haiku keeps the classifier-grade duties (pty hung/working, warmup, distill via lib/eval-model.ts) and D1a's capture->Sonnet stands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>		2026-08-15 16:14:01 -07:00
..
providers	v1.65.0.0 feat: fork port wave 2 — feature fixes, session persistence, Apple releases, supply-chain CI (#2577)	2026-08-15 11:42:19 -07:00
agent-sdk-runner.ts	evals: SDK runner default Opus -> Sonnet (D1a)	2026-08-15 09:14:36 -07:00
anthropic-preflight.ts	evals: preflight ping once in the sharded parent, not per shard	2026-08-15 08:13:30 -07:00
auq-sdk-capture.ts	v1.65.0.0 feat: fork port wave 2 — feature fixes, session persistence, Apple releases, supply-chain CI (#2577)	2026-08-15 11:42:19 -07:00
benchmark-judge.ts	feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)	2026-04-19 17:50:31 +08:00
benchmark-runner.ts	feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)	2026-04-19 17:50:31 +08:00
budget-override.test.ts	v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)	2026-05-26 16:50:03 -07:00
budget-override.ts	v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)	2026-05-26 16:50:03 -07:00
capture-parity-baseline.test.ts	v1.46.0.0 feat: gstack v2 foundation — catalog tokens drop 56%, eval-first floor covers all 51 skills (#1712)	2026-05-26 16:50:03 -07:00
capture-parity-baseline.ts	Merge origin/main (v1.64.1.0 code-smell wave) into test-evals-ci-speedup	2026-08-15 10:29:25 -07:00
carve-guard-checks.ts	v1.57.0.0 feat: carve-guard system + carve cso/document-release/design-consultation (#1907)	2026-06-07 19:13:24 -07:00
carve-guards.ts	v1.65.0.0 feat: fork port wave 2 — feature fixes, session persistence, Apple releases, supply-chain CI (#2577)	2026-08-15 11:42:19 -07:00
claude-pty-runner.scope-gate-floor.unit.test.ts	v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)	2026-08-12 11:12:28 -07:00
claude-pty-runner.ts	v1.65.0.0 feat: fork port wave 2 — feature fixes, session persistence, Apple releases, supply-chain CI (#2577)	2026-08-15 11:42:19 -07:00
claude-pty-runner.unit.test.ts	v1.62.0.0 feat: plan-mode auto-select at the review scope gate (#2533)	2026-08-12 11:12:28 -07:00
codex-session-runner.ts	test(evals): codex runner gains sections option — review variant 88% smaller	2026-08-15 08:50:03 -07:00
e2e-gate.ts	v1.64.1.0 v1.64.1.0: the code-smell fix wave — every pipeline guard now provably fires (net −24,943 lines) (#2572)	2026-08-15 09:37:04 -07:00
e2e-gate.unit.test.ts	v1.64.1.0 v1.64.1.0: the code-smell fix wave — every pipeline guard now provably fires (net −24,943 lines) (#2572)	2026-08-15 09:37:04 -07:00
e2e-helpers.ts	Merge origin/main (v1.64.1.0 code-smell wave) into test-evals-ci-speedup	2026-08-15 10:29:25 -07:00
eval-store.test.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
eval-store.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
gemini-session-runner.test.ts	feat: Gemini CLI E2E tests (v0.9.2.0) (#252)	2026-03-20 08:30:09 -07:00
gemini-session-runner.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
hermetic-env.test.ts	v1.58.1.0 feat: hermetic local E2E + Conductor prose AskUserQuestion (#2004)	2026-06-14 11:40:57 -07:00
hermetic-env.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
llm-judge.ts	evals: judge default back to Sonnet — Haiku regressed the rubric family (A/B receipts)	2026-08-15 16:14:01 -07:00
observability.test.ts	v1.60.2.0 fix: free-suite drift on dev machines (eval-list cwd, gemini regex, observability floor) (#2470)	2026-08-07 15:08:12 -07:00
paid-test-set.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
parity-harness.ts	v1.65.0.0 feat: fork port wave 2 — feature fixes, session persistence, Apple releases, supply-chain CI (#2577)	2026-08-15 11:42:19 -07:00
pricing.ts	feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)	2026-04-19 17:50:31 +08:00
required-reads.ts	v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806)	2026-05-30 12:09:10 -07:00
secret-sink-harness.ts	v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183)	2026-04-24 01:38:21 -07:00
session-runner.test.ts	feat: stream-json NDJSON parser for real-time E2E progress	2026-03-14 03:49:36 -05:00
session-runner.ts	fix(test-harness): spawn claude without shell interpolation	2026-07-14 12:56:07 -07:00
skill-census.ts	v1.63.0.0 feat: GStack 2 fork port wave — egress receipts, context-bill, sharded gate, /health fix (#2541)	2026-08-14 09:28:56 -07:00
skill-fixture.ts	test: skill-fixture helper — extract SKILL.md sections, don't copy files	2026-08-15 08:50:02 -07:00
skill-parser.ts	feat: content security — 4-layer prompt injection defense for pair-agent (#815)	2026-04-06 14:41:06 -07:00
test-selection.ts	evals: selection sees uncommitted work; git errors fail closed	2026-08-15 08:34:21 -07:00
tool-map.ts	feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040)	2026-04-19 17:50:31 +08:00
touchfiles-data.ts	Merge origin/main (v1.64.1.0 code-smell wave) into test-evals-ci-speedup	2026-08-15 10:29:25 -07:00
touchfiles.ts	v1.64.1.0 v1.64.1.0: the code-smell fix wave — every pipeline guard now provably fires (net −24,943 lines) (#2572)	2026-08-15 09:37:04 -07:00
transcript-section-logger.ts	v1.54.0.0 feat: carve /ship into skeleton + on-demand sections (-59% always-loaded) (#1806)	2026-05-30 12:09:10 -07:00