From 5cc778498644fad5b49fa7ecd53e68e1772a5ed9 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Fri, 11 Sep 2026 17:39:57 -0700 Subject: [PATCH] test: remove cold executable reads from runner integrity deadlines (#13301) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runner protocol tests verify that invalid authenticated input fails closed. > - Cloud readiness requires those tests to pass before a source commit can deploy. > - Two test fixtures hash the host Node executable twice even though its process launcher is synthetic. > - Cold reads of a large Linux executable consume time unrelated to protocol failure handling. > - This PR gives both synthetic runner fixtures tiny real artifacts and keeps their existing deadlines. > - Real authentication, encrypted frames, and request and notification failures remain covered. ## Linked Issues or Issue Description **What happened?** Cloud readiness repeatedly failed in `DurablePrpControlPlane > promptly fails the real transport request and notification paths on authenticated bad semantic input (throwing observer: false)` with `Test timed out in 5000ms`. The image built, but the failed source check prevented deployment. See runs [34656885170](https://github.com/paperclipai/paperclip/actions/runs/34656885170) and [34657111560](https://github.com/paperclipai/paperclip/actions/runs/34657111560). The first case took 5.6–11.4 seconds; the following case took about 0.4 seconds. The original test reads and hashes `process.execPath` once itself and once through the real transport. The Node executable is 122,678,944 bytes in the local Linux Node 24 container, versus 68,672 bytes on the development Mac. Instrumented Mac runs pass; cold executable reads are a likely cause of the CI-only timeout. **Expected behavior** The deadline should measure real protocol rejection and consumer failure, without reading a large unrelated executable as test fixture data. **Steps to reproduce** 1. Run the named test with the real transport and synthetic process launcher. 2. For a deterministic probe, inject a six-second delay into the first `readFileSync(process.execPath)` call. 3. The original test exceeds its existing five-second deadline. Both cases pass after this change because neither reads the host executable. The probe is temporary instrumentation, not part of this commit. **Paperclip version or commit** f12b647ae; the same failure also occurred on a38ccf997 and 9031516a7. **Deployment mode** Cloud readiness in GitHub Actions on Linux AWS runners. No duplicate open PR found after searching runner timeout and durable PRP changes. ## What Changed - Write a 26-byte synthetic runner artifact inside each existing temporary test directory. - Bind both the real authority and test client to that artifact's actual SHA-256 digest. - Retain synthetic process launchers, real wire exchange, all observer and composed-runtime cases, integrity assertions, cleanup, and existing five- and fifteen-second timeouts. ## Verification - Passed all 107 tests across `src/control-plane/durable-prp-control-plane.test.ts` and `src/drivers/codex/codex-protocol-integrity.test.ts` using the package Vitest configuration. - With a temporary six-second host-executable read delay, the original case times out. Both fixed cases pass in 235 ms combined under the same fault injection. - Passed `git diff --check`. - Full workspace typecheck and build passed. The full local test command stopped after the general server group: 10,694 tests passed and 15 failed. Thirteen failures involve macOS directory rename permissions in the skill-cache suites. Two failures involve a missing local AgentMail skill path. Later groups did not run in that local invocation. The targeted runner tests passed; all Linux CI test groups passed. - [Current-head CI run 34661107658](https://github.com/paperclipai/paperclip/actions/runs/34661107658) passed. All 32 checks are green or intentionally skipped at `e7053594e647fd8e9b71fc569cab6b7a3e9cd210`. Native runner verification passed in 15m25s. Greptile is 5/5 on that head, with no unresolved review findings. ## Risks Low risk: this changes test fixture data only. The launcher never executes the file. Artifact hashing and authentication still use real bytes. Protocol behavior and existing test deadlines do not change. The current Linux CI run passed both fixtures and the full native runner verification. Further post-merge runs will test repeatability. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact serving model ID and context window are not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted runner tests; full local suite limitations are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- .../control-plane/durable-prp-control-plane.test.ts | 7 ++++--- .../drivers/codex/codex-protocol-integrity.test.ts | 11 ++++++++--- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts index 24d83f8617..6c8274fd10 100644 --- a/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts +++ b/packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts @@ -1542,9 +1542,10 @@ describe.sequential("DurablePrpControlPlane", () => { let authority: DurablePrpControlPlane | undefined; let launched = false; const diagnostics: string[] = []; - // The launcher below is synthetic; use the current executable only as - // its artifact identity, without depending on a staged Rust build. - const runnerBinary = process.execPath; + // The launcher never executes this file. Use a small artifact so cold + // reads of the Linux Node executable do not consume the failure deadline. + const runnerBinary = resolve(root, "synthetic-runner"); + writeFileSync(runnerBinary, "synthetic runner artifact\n", { mode: 0o600 }); const runnerDigest = `sha256:${createHash("sha256").update(readFileSync(runnerBinary)).digest("hex")}`; const handler = vi.fn(async () => ({ success: true, contentItems: [] })); const bundle = createCapabilityRunnerdCodexTransport({ diff --git a/packages/paperclip-runner/src/drivers/codex/codex-protocol-integrity.test.ts b/packages/paperclip-runner/src/drivers/codex/codex-protocol-integrity.test.ts index ff9bf710ed..b3932975df 100644 --- a/packages/paperclip-runner/src/drivers/codex/codex-protocol-integrity.test.ts +++ b/packages/paperclip-runner/src/drivers/codex/codex-protocol-integrity.test.ts @@ -41,6 +41,7 @@ import { async function authenticatedRunner( core: DurablePrpControlPlane, identity: DurableRecoveryIdentity, + runnerBinary: string, ) { const framed = (domain: string, parts: Buffer[]) => { const values = [Buffer.from(domain), Buffer.from([0])]; @@ -84,7 +85,7 @@ async function authenticatedRunner( protocolMax: 1, ...identity, runnerVersion: "0.3.0", - runnerDigest: `sha256:${createHash("sha256").update(readFileSync(process.execPath)).digest("hex")}`, + runnerDigest: `sha256:${createHash("sha256").update(readFileSync(runnerBinary)).digest("hex")}`, }, }), ); @@ -427,6 +428,10 @@ describe("Codex protocol integrity propagation", () => { const directory = mkdtempSync( join(tmpdir(), "paperclip-composed-integrity-"), ); + // Only the process launcher is synthetic; hash a small fixture instead + // of cold-reading the host Node executable during the protocol deadline. + const runnerBinary = join(directory, "synthetic-runner"); + writeFileSync(runnerBinary, "synthetic runner artifact\n", { mode: 0o600 }); const identity: DurableRecoveryIdentity = { runnerInstanceId: `composed-runner-${scenario}`, environmentLeaseId: `composed-lease-${scenario}`, @@ -526,7 +531,7 @@ describe("Codex protocol integrity propagation", () => { const bundle = createCapabilityRunnerdCodexTransport({ stateDirectory: directory, prpIdentity: identity, - runnerBinary: process.execPath, + runnerBinary, codexCommand: process.execPath, codexArgs: [], sourceCodexHome: null, @@ -575,7 +580,7 @@ describe("Codex protocol integrity propagation", () => { try { await vi.waitFor(() => expect(launch).toHaveBeenCalledTimes(1)); const core = authority!; - client = await authenticatedRunner(core, identity); + client = await authenticatedRunner(core, identity, runnerBinary); const commandResult = async ( type: string, result: Record = {},