From c7bf129b4de869acdb9e3478a5a8fc2ec6cd459b Mon Sep 17 00:00:00 2001 From: Garry Tan Date: Fri, 7 Aug 2026 15:18:11 -0700 Subject: [PATCH] fix(e2e): review-dashboard-via session budget survives runner contention (#2473) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The test failed on CI (and its baseline run) with the timeout signature: 0 turns, $0.00, exactly 183s, 3/3 attempts — the spawned claude -p session never emitted a single stream event before the 180s inner timeout. The file's tests run concurrently on one runner; session startup queues behind sibling sessions, and this test had the tightest budget in the file (the 240s-budget tests in the same job passed). A clean local run takes 270s wall for 4 turns, confirming 180s was too tight even without contention. Inner timeout 180s -> 300s; outer bun timeout 240s -> 360s to keep headroom over the inner budget. Verified: PASS locally post-fix (4 turns, 270s). Co-Authored-By: Claude Fable 5 --- test/skill-e2e-review.test.ts | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/test/skill-e2e-review.test.ts b/test/skill-e2e-review.test.ts index 1adbe25c7..b6915b57e 100644 --- a/test/skill-e2e-review.test.ts +++ b/test/skill-e2e-review.test.ts @@ -614,7 +614,13 @@ Skip the preamble, lake intro, telemetry, and all other ship steps. Write the dashboard output to ${dashDir}/dashboard-output.md`, workingDirectory: dashDir, maxTurns: 12, - timeout: 180_000, + // 300s, not 180s: on a saturated CI runner this file's concurrent + // sessions queue behind each other and session STARTUP can eat the + // whole budget — observed as deterministic timeout at 0 turns/$0.00 + // for exactly 180s across 3 attempts (PR #2472 CI + its baseline), + // while the 240s-budget tests in the same job passed. Outer bun + // timeout below rises to 360s to keep headroom over the inner budget. + timeout: 300_000, testName: 'review-dashboard-via', runId, }); @@ -648,7 +654,7 @@ Write the dashboard output to ${dashDir}/dashboard-output.md`, ); // Ship dashboard should not gate when eng review is clear expect(gateQuestions).toHaveLength(0); - }, 240_000); + }, 360_000); }); // Module-level afterAll — finalize eval collector after all tests complete