RomanNum3ral
/
gstack
mirror of https://github.com/garrytan/gstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

fix(eval-harness): eng-review qid boundary matches the real skill-name prefix 

Live qids render as gstack-qid:plan-eng-review-<slug> ({skill}-{slug}
convention); the boundary anchored eng-review- immediately after the
colon and never matched, leaving the batching counter blind while the
transcript showed per-finding questions being asked one by one.
This commit is contained in:
Garry Tan 2026-08-15 00:09:51 -07:00
parent 5a6a686635
commit 9889f27491
No known key found for this signature in database
GPG Key ID: C1F69E85C74EFE1D
1 changed files with 5 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
test/helpers/claude-pty-runner.ts
View File

@ -1257,10 +1257,12 @@ export const engStep0Boundary: Step0BoundaryPredicate = (fp) =>
) ||
// plan-eng-review's Step 0 may legitimately end with NO scope-reduction /
// learnings AUQ. When it does, the first answered review-phase question —
// tagged <gstack-qid:eng-review-...> by the question-tuning resolver —
// tagged <gstack-qid:plan-eng-review-...> ({skill}-{slug} convention) —
// must fire the boundary, or every per-finding AUQ stays classified
// preReview and the multi-finding batching counter reads 0.
/gstack-qid:\s*eng-review-/i.test(fp.promptSnippet);
// preReview and the multi-finding batching counter reads 0. Anchor allows
// the skill-name prefix; live qids observed: plan-eng-review-jitter,
// plan-eng-review-idempotency, plan-eng-review-todos-e2e-concurrent.
/gstack-qid:\s*(?:plan-)?eng-review-/i.test(fp.promptSnippet);
export const designStep0Boundary: Step0BoundaryPredicate = (fp) =>
/design system|design posture|design score|first dimension/i.test(