RomanNum3ral
/
gstack
mirror of https://github.com/garrytan/gstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

fix(eval-harness): eng-review phase boundary fires on qid-tagged questions 

The Step 0 boundary only matched two prose phrases, but plan-eng-review
may legitimately reach the review phase without either — every
per-finding AskUserQuestion then counted as pre-review and the batching
regression test read 0 questions while watching the agent ask them one
by one. The boundary now also fires on the first answered question
carrying a gstack-qid:eng-review- marker. Additive only; 119 runner
unit tests green.
This commit is contained in:
Garry Tan 2026-08-14 21:22:37 -07:00
parent fa783470a6
commit f6a9cd841f
No known key found for this signature in database
GPG Key ID: C1F69E85C74EFE1D
1 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
test/helpers/claude-pty-runner.ts
View File

@ -1254,7 +1254,13 @@ export const ceoStep0Boundary: Step0BoundaryPredicate = (fp) =>
export const engStep0Boundary: Step0BoundaryPredicate = (fp) =>
/scope reduction recommendation|cross[\s-]?project learnings/i.test(
fp.promptSnippet,
);
) ||
// plan-eng-review's Step 0 may legitimately end with NO scope-reduction /
// learnings AUQ. When it does, the first answered review-phase question —
// tagged <gstack-qid:eng-review-...> by the question-tuning resolver —
// must fire the boundary, or every per-finding AUQ stays classified
// preReview and the multi-finding batching counter reads 0.
/gstack-qid:\s*eng-review-/i.test(fp.promptSnippet);
export const designStep0Boundary: Step0BoundaryPredicate = (fp) =>
/design system|design posture|design score|first dimension/i.test(