RomanNum3ral
/
gstack
mirror of https://github.com/garrytan/gstack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

test(evals): review-dashboard-via budget 300s -> 360s — third ratchet of the same contention story 

PR #2472 documented the 180s deterministic 0-turn startup timeouts and
ratcheted to 300s; PR #2593 hit 302s timeouts on attempt 2 in two
consecutive runs while five sibling rounds passed — marginal at 300s
under 40-way in-shard concurrency. Same headroom its contention-class
sibling (retro-base-branch) carries; outer bun timeout rises to 480s.
This commit is contained in:
Garry Tan 2026-08-15 21:51:56 -07:00
parent c12ca0696d
commit 18698b964e
No known key found for this signature in database
GPG Key ID: C1F69E85C74EFE1D
1 changed files with 9 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
test/skill-e2e-review-attribution.test.ts
View File

@ -239,13 +239,14 @@ Skip the preamble, lake intro, telemetry, and all other ship steps.
Write the dashboard output to ${dashDir}/dashboard-output.md`,
workingDirectory: dashDir,
maxTurns: 12,
// 300s, not 180s: on a saturated CI runner this file's concurrent
// sessions queue behind each other and session STARTUP can eat the
// whole budget — observed as deterministic timeout at 0 turns/$0.00
// for exactly 180s across 3 attempts (PR #2472 CI + its baseline),
// while the 240s-budget tests in the same job passed. Outer bun
// timeout below rises to 360s to keep headroom over the inner budget.
timeout: 300_000,
// 360s, third ratchet of the same contention story: 180s deterministic
// 0-turn timeouts (PR #2472) → 300s; then PR #2593 hit 302s timeouts
// on attempt 2 in two consecutive runs while five sibling rounds
// passed — the queue-behind-siblings startup tax under 40-way in-shard
// concurrency is real and marginal at 300s. Same headroom its
// contention-class sibling (retro-base-branch) already carries. Outer
// bun timeout below rises to 480s to keep headroom over the inner.
timeout: 360_000,
testName: 'review-dashboard-via',
runId,
});
@ -279,7 +280,7 @@ Write the dashboard output to ${dashDir}/dashboard-output.md`,
);
// Ship dashboard should not gate when eng review is clear
expect(gateQuestions).toHaveLength(0);
}, 360_000);
}, 480_000);
});
// Module-level afterAll — finalize eval collector after all tests complete