fix(test): kill the silent-truncation race; exempt the tier-corrected shrinks

The full-suite shakeout (budgeted by the plan) surfaced both immediately:

1. server-embedder-terminal-port.test.ts stubbed process.exit and restored
   the REAL exit in its finally — but shutdown() schedules async work that
   can call process.exit AFTER restoration, killing the entire bun process
   mid-suite with exit 0 and NO summary. This is the silent-truncation class
   the new free-suite CI job guards against, reproduced locally on the first
   full run. Exit now stays a logging no-op between tests (late async exits
   become visible stderr lines, not process death); the true exit returns in
   afterAll.

2. The 80%-of-baseline shrink guard correctly flagged the six tier-corrected
   skills — their baseline was measured at the silent tier-4 default. Added
   to INTENTIONAL_SHRINKS with the reason, joining spec's double-preamble
   entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan 2026-08-14 20:42:39 -07:00
parent 593be14c49
commit be2f3c286a
No known key found for this signature in database
GPG Key ID: C1F69E85C74EFE1D
2 changed files with 32 additions and 3 deletions

View File

@ -78,10 +78,30 @@ function readIfExists(p: string): string | null {
* machine. Returns the captured kill calls so tests can assert kill
* scope.
*/
// The TRUE process.exit, restored only in afterAll. withStubs used to restore
// it in its finally — but shutdown() schedules async work (timers,
// fire-and-forget promises) that can call process.exit AFTER the stub was
// restored, killing the entire bun test process mid-suite with exit 0 and no
// summary (the silent-truncation class the free-suite CI job guards against;
// this file was the killer). Between tests, exit stays a logging no-op so a
// late async exit is visible instead of fatal.
const TRUE_EXIT = process.exit;
const lateExitGuard = ((code: number) => {
console.error(`[test-guard] late process.exit(${code}) swallowed (async shutdown work after stub restore)`);
}) as any;
afterAll(async () => {
// Drain shutdown()'s pending async work before restoring the real exit:
// disposeSession escalates SIGINT -> SIGKILL on a 3s timer, and a timer
// firing after this file's afterAll would otherwise hit the REAL
// process.exit and kill the whole multi-file bun run (observed: the free
// suite died at file 47 with exit 0 and no summary — twice).
await new Promise((r) => setTimeout(r, 3500));
(process as any).exit = TRUE_EXIT;
});
async function withStubs(
cb: (killCalls: Array<[number, NodeJS.Signals | number]>) => Promise<void>
): Promise<Array<[number, NodeJS.Signals | number]>> {
const origExit = process.exit;
const origKill = process.kill;
const killCalls: Array<[number, NodeJS.Signals | number]> = [];
(process as any).exit = ((code: number) => {
@ -101,7 +121,7 @@ async function withStubs(
try {
await cb(killCalls);
} finally {
(process as any).exit = origExit;
(process as any).exit = lateExitGuard;
(process as any).kill = origKill;
}
return killCalls;

View File

@ -171,7 +171,16 @@ describe('SKILL.md size budget regression (gate, free)', () => {
// {{PREAMBLE}} literally, so the generator expanded the ENTIRE preamble a
// second time mid-sentence (~47 KB of duplication). Fixed by rewording the
// prose; spec/SKILL.md now carries exactly one preamble (~80.9 KB, ×0.79).
const INTENTIONAL_SHRINKS = new Set<string>(['spec']);
// - scrape/diagram/open-gstack-browser/landing-report/pair-agent/skillify:
// the baseline measured these at the silent tier-4 default (a missing
// preamble-tier frontmatter fell through `?? 4`). Their tiers are now
// declared correctly (1-2), shedding the tier-2..4 onboarding prose they
// never should have carried (-271 lines each for tier 1).
const INTENTIONAL_SHRINKS = new Set<string>([
'spec',
'scrape', 'diagram', 'open-gstack-browser',
'landing-report', 'pair-agent', 'skillify',
]);
const undershoots: Array<{
skill: string; beforeBytes: number; afterBytes: number; ratio: number;