mirror of https://github.com/garrytan/gstack.git
84f52f3bad
EvalCollector accumulates test results during eval runs, writes JSON to
~/.gstack-dev/evals/{version}-{branch}-{tier}-{timestamp}.json, prints
a summary table, and automatically compares against the previous run.
- EvalCollector class with addTest() / finalize() / summary table
- findPreviousRun() prefers same branch, falls back to any branch
- compareEvalResults() matches tests by name, detects improved/regressed
- extractToolSummary() counts tool types from transcript events
- formatComparison() renders delta table with per-test + aggregate diffs
- Wire into skill-e2e.test.ts (recordE2E helper) and skill-llm-eval.test.ts
- 19 unit tests for collector + comparison functions
- schema_version: 1 for forward compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| fixtures | ||
| helpers | ||
| gen-skill-docs.test.ts | ||
| skill-e2e.test.ts | ||
| skill-llm-eval.test.ts | ||
| skill-parser.test.ts | ||
| skill-validation.test.ts | ||