From cf5ba4bbea9d9e21ced6e7e3e662cccee95249f4 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Fri, 17 Jul 2026 14:18:01 -0500 Subject: [PATCH] feat(adapter-utils): generic per-asset lifecycle-contribution seam (#9778) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip's sandbox managed runtime is responsible for provisioning the agent's execution environment — it extracts a home directory asset into the sandbox before the adapter runs. > - The sandbox runtime core was directly branching on the adapter key (`codex`) to decide which merge scripts to stage and which merge-extract command to run, coupling generic infrastructure to a specific adapter's credential-merge protocol. > - This makes it harder to add, remove, or modify per-adapter asset provisioning without touching the runtime core; it also prevents other adapters from contributing staged files or a custom extract command at all. > - The fix is to move the adapter-specific knowledge into the adapter itself: the asset descriptor gains optional `provision` (stageFiles + extractCommand) and `restore` contribution fields that any adapter can populate, and the runtime core consumes them generically. > - This pull request introduces those contribution fields, wires the Codex adapter's inbound credential-merge as a `provision` contribution, and removes the adapter-specific branching from the runtime core. > - The benefit is a clean seam: the runtime core is now adapter-agnostic for asset provisioning, the inbound behavior is unchanged (same merge matrix, same scripts), and other adapters can attach custom staged files or extract commands without modifying shared infrastructure. ## Linked Issues or Issue Description No pre-existing public GitHub issue. Describing the problem inline per the feature template: **Problem or motivation** The sandbox managed-runtime asset provisioning in `sandbox-managed-runtime.ts` branched directly on the adapter key (`codex`) to decide which merge scripts to stage and which shell command to use during asset extraction. This tight coupling prevents other adapters from customizing their provisioning without modifying the runtime core, and it means the runtime core must import and know about adapter-specific merge scripts. **Proposed solution** Add an optional `provision` contribution (array of `stageFiles` entries + an `extractCommand` string) and an optional `restore` contribution to the asset descriptor returned by adapters. The runtime core now consumes these generically — if a `provision` contribution is present, it stages those files and uses the supplied command; otherwise it falls back to the default `tar -xf` extraction. The Codex adapter populates the `provision` contribution where it previously depended on core branching. **Alternatives considered** Keeping the adapter-specific logic in the core as a documented exception; rejected because it makes the seam inextensible. **Roadmap alignment** Decoupling — removes a latent coupling between the runtime core and a specific adapter. ## What Changed - Added `provision` contribution field (`stageFiles: Array<{src, dest}>` + `extractCommand: string`) to the `SandboxManagedRuntimeAsset` descriptor type in `adapter-utils`. - Added `restore` contribution field (hook for post-restore logic, populated in a later phase) to the descriptor. - Removed adapter-key branching (`if adapterKey === 'codex'`) from the runtime core in `sandbox-managed-runtime.ts`; the core now reads `provision.stageFiles` and `provision.extractCommand` generically. - Extracted Codex-specific merge-script paths and the merge-extract command into `codex-auth-merge-scripts.ts` in `adapter-utils`; the Codex adapter's `execute.ts` now attaches them as a `provision` contribution when it builds its managed-home asset descriptor. - Updated `execution-target.ts` to pass the extended asset type through to the adapter call site so the new fields are load-bearing end-to-end. - Added seam-proving unit tests in `sandbox-managed-runtime.test.ts`: contribution-less asset uses the default path; a non-adapter asset round-trips the generic provision+restore seam; a structural assertion verifies the runtime core carries no Codex-specific string literals. - Added one test in `workspace-restore-merge.test.ts` confirming the inbound merge matrix is unaffected. ## Verification ```bash # Unit tests (20 pass): npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/workspace-restore-merge.test.ts # Type-check both affected packages: cd packages/adapter-utils && npx tsc --noEmit cd packages/adapters/codex-local && npx tsc --noEmit # Structural: runtime core carries no adapter string literals grep -n 'codex\|auth\.json' packages/adapter-utils/src/sandbox-managed-runtime.ts # Expected: zero matches ``` ## Risks **Low risk.** This is a behavior-preserving refactor: the inbound provisioning output (which files get staged, which command runs) is identical to before, now driven by the adapter-supplied contribution instead of core branching. The existing inbound merge matrix tests are the regression guard. No change to which bytes cross the sandbox boundary. The SSH transport is untouched. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200 K tokens - **Capabilities used:** tool use (file read/edit, bash execution, Paperclip API), extended reasoning over multi-file TypeScript refactor - **Mode:** agentic (Paperclip ACPX platform) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip Co-authored-by: Harold Kim --- .../src/codex-auth-merge-scripts.ts | 42 ++++ .../adapter-utils/src/execution-target.ts | 9 +- .../src/sandbox-managed-runtime.test.ts | 216 ++++++++++++++++++ .../src/sandbox-managed-runtime.ts | 128 +++++++---- .../src/workspace-restore-merge.test.ts | 6 + .../codex-local/src/server/execute.ts | 7 + ...eartbeat-workspace-finalize-branch.test.ts | 2 +- .../heartbeat-worktree-suppression.test.ts | 2 +- ...ermissions-upgrade-boundary-routes.test.ts | 2 +- 9 files changed, 364 insertions(+), 50 deletions(-) create mode 100644 packages/adapter-utils/src/codex-auth-merge-scripts.ts diff --git a/packages/adapter-utils/src/codex-auth-merge-scripts.ts b/packages/adapter-utils/src/codex-auth-merge-scripts.ts new file mode 100644 index 0000000000..ef6b79f5aa --- /dev/null +++ b/packages/adapter-utils/src/codex-auth-merge-scripts.ts @@ -0,0 +1,42 @@ +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { shellQuote } from "./ssh.js"; +import type { SandboxManagedRuntimeAssetProvision } from "./sandbox-managed-runtime.js"; + +// Codex-specific inbound auth-merge assets. These physically live in +// `adapter-utils/src` in Phase 1 of the generic-asset-lifecycle-seam work; +// a follow-on phase will relocate this module and the two script files +// into the `codex-local` adapter. The sandbox runtime *core* +// (`sandbox-managed-runtime.ts`) is intentionally free of any Codex knowledge — +// the adapter supplies this contribution through the generic `provision` seam. + +export const CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME = "codex-auth-merge-extract.sh"; +export const CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME = "codex-auth-merge-decision.cjs"; + +const CODEX_AUTH_MERGE_EXTRACT_SCRIPT_BYTES = readFileSync( + new URL(`./${CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME}`, import.meta.url), +); +const CODEX_AUTH_MERGE_DECISION_SCRIPT_BYTES = readFileSync( + new URL(`./${CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME}`, import.meta.url), +); + +/** + * Builds the inbound (host→sandbox) provisioning contribution for the Codex + * managed-home asset: stage the two merge scripts into the runtime root and run + * the merge-extract script instead of a plain `tar -xf`, so a sandbox that + * already carries a Codex `auth.json` keeps whichever credential is newer. + * + * This is behaviour-identical to the extraction the sandbox core previously + * hardcoded for `adapterKey === "codex" && assetKey === "home"`. + */ +export function buildCodexAuthInboundProvision(): SandboxManagedRuntimeAssetProvision { + return { + stageFiles: [ + { name: CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME, contents: CODEX_AUTH_MERGE_EXTRACT_SCRIPT_BYTES }, + { name: CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME, contents: CODEX_AUTH_MERGE_DECISION_SCRIPT_BYTES }, + ], + extractCommand: ({ assetTarPath, assetDir, runtimeRootDir }) => + `sh ${shellQuote(path.posix.join(runtimeRootDir, CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME))} ` + + `${shellQuote(assetDir)} ${shellQuote(assetTarPath)}`, + }; +} diff --git a/packages/adapter-utils/src/execution-target.ts b/packages/adapter-utils/src/execution-target.ts index bd56cafbe9..f70e8e01b4 100644 --- a/packages/adapter-utils/src/execution-target.ts +++ b/packages/adapter-utils/src/execution-target.ts @@ -6,13 +6,13 @@ import { randomUUID } from "node:crypto"; import type { SshRemoteExecutionSpec } from "./ssh.js"; import { prepareCommandManagedRuntime, + type CommandManagedRuntimeAsset, type CommandManagedRuntimeRunner, } from "./command-managed-runtime.js"; import { buildRemoteExecutionSessionIdentity, prepareRemoteManagedRuntime, remoteExecutionSessionMatches, - type RemoteManagedRuntimeAsset, } from "./remote-managed-runtime.js"; import { createCommandManagedSandboxCallbackBridgeQueueClient, @@ -83,7 +83,12 @@ export type AdapterExecutionTarget = export type AdapterRemoteExecutionSpec = SshRemoteExecutionSpec; -export type AdapterManagedRuntimeAsset = RemoteManagedRuntimeAsset; +// The adapter-facing managed-runtime asset type. Aliased to the sandbox/command +// asset descriptor so the per-asset lifecycle contributions (`provision` / +// `restore`) declared on the sandbox core are load-bearing all the way from the +// adapter call site through to the sandbox runtime. The SSH transport consumes +// the subset of fields it understands and ignores the rest. +export type AdapterManagedRuntimeAsset = CommandManagedRuntimeAsset; export interface PreparedAdapterExecutionTargetRuntime { target: AdapterExecutionTarget; diff --git a/packages/adapter-utils/src/sandbox-managed-runtime.test.ts b/packages/adapter-utils/src/sandbox-managed-runtime.test.ts index a101fc817c..e2f7237a92 100644 --- a/packages/adapter-utils/src/sandbox-managed-runtime.test.ts +++ b/packages/adapter-utils/src/sandbox-managed-runtime.test.ts @@ -824,4 +824,220 @@ describe("sandbox managed runtime", () => { expect(emptyArchiveCommand).toBeDefined(); expect(emptyArchiveCommand).not.toContain("/dev/null"); }); + + it("provisions a contribution-less asset via a plain tar extract and restores it as a no-op", async () => { + const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-default-asset-")); + cleanupDirs.push(rootDir); + const localWorkspaceDir = path.join(rootDir, "local-workspace"); + const remoteWorkspaceDir = path.join(rootDir, "remote-workspace"); + const localAssetsDir = path.join(rootDir, "local-assets"); + await mkdir(localWorkspaceDir, { recursive: true }); + await mkdir(localAssetsDir, { recursive: true }); + await writeFile(path.join(localWorkspaceDir, "README.md"), "workspace\n", "utf8"); + await writeFile(path.join(localAssetsDir, "plain.txt"), "plain asset\n", "utf8"); + + const stagedWrites: string[] = []; + const runCommands: string[] = []; + const client: SandboxManagedRuntimeClient = { + makeDir: async (remotePath) => { + await mkdir(remotePath, { recursive: true }); + }, + writeFile: async (remotePath, bytes) => { + await mkdir(path.dirname(remotePath), { recursive: true }); + if (!remotePath.endsWith("-upload.tar")) stagedWrites.push(path.basename(remotePath)); + await writeFile(remotePath, Buffer.from(bytes)); + }, + readFile: async (remotePath) => await readFile(remotePath), + listFiles: async () => [], + remove: async (remotePath) => { + await rm(remotePath, { recursive: true, force: true }); + }, + run: async (command) => { + runCommands.push(command); + await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 }); + }, + }; + + const prepared = await prepareSandboxManagedRuntime({ + spec: { + transport: "sandbox", + provider: "test", + sandboxId: "sandbox-1", + remoteCwd: remoteWorkspaceDir, + timeoutMs: 30_000, + apiKey: null, + }, + adapterKey: "test-adapter", + client, + workspaceLocalDir: localWorkspaceDir, + // No `provision` / `restore` on the asset: it must ride the default path. + assets: [{ key: "plain", localDir: localAssetsDir }], + }); + + // Extracted through the default `tar -xf` path. + await expect(readFile(path.join(prepared.assetDirs.plain, "plain.txt"), "utf8")).resolves.toBe("plain asset\n"); + // A contribution-less asset stages no extra files beyond its own tar. + expect(stagedWrites.filter((name) => name.includes("plain"))).toEqual([]); + // The extract command is the generic tar path, not an adapter-specific script. + const assetExtract = runCommands.find((command) => command.includes(`${path.posix.basename(prepared.assetDirs.plain)}-upload.tar`)); + expect(assetExtract).toBeDefined(); + expect(assetExtract).toContain("tar -xf"); + expect(assetExtract).not.toMatch(/\.sh|\.cjs/); + + // Restore is a clean no-op for a contribution-less asset (no throw, asset dir untouched). + await expect(prepared.restoreWorkspace()).resolves.toBeUndefined(); + await expect(readFile(path.join(prepared.assetDirs.plain, "plain.txt"), "utf8")).resolves.toBe("plain asset\n"); + }); + + it("round-trips a non-codex asset through generic provision + restore contributions", async () => { + const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-seam-")); + cleanupDirs.push(rootDir); + const localWorkspaceDir = path.join(rootDir, "local-workspace"); + const remoteWorkspaceDir = path.join(rootDir, "remote-workspace"); + const localAssetsDir = path.join(rootDir, "local-assets"); + await mkdir(localWorkspaceDir, { recursive: true }); + await mkdir(localAssetsDir, { recursive: true }); + await writeFile(path.join(localWorkspaceDir, "README.md"), "workspace\n", "utf8"); + await writeFile(path.join(localAssetsDir, "seed.txt"), "seed\n", "utf8"); + + const client: SandboxManagedRuntimeClient = { + makeDir: async (remotePath) => { + await mkdir(remotePath, { recursive: true }); + }, + writeFile: async (remotePath, bytes) => { + await mkdir(path.dirname(remotePath), { recursive: true }); + await writeFile(remotePath, Buffer.from(bytes)); + }, + readFile: async (remotePath) => await readFile(remotePath), + listFiles: async () => [], + remove: async (remotePath) => { + await rm(remotePath, { recursive: true, force: true }); + }, + run: async (command) => { + await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 }); + }, + }; + + // A minimal shell quoter local to the test's custom extract command; the seam + // itself carries no adapter knowledge — the fake asset supplies everything. + const q = (value: string) => `'${value.replace(/'/g, `'\"'\"'`)}'`; + const restored: string[] = []; + const stagedContentSeen: string[] = []; + + const prepared = await prepareSandboxManagedRuntime({ + spec: { + transport: "sandbox", + provider: "test", + sandboxId: "sandbox-1", + remoteCwd: remoteWorkspaceDir, + timeoutMs: 30_000, + apiKey: null, + }, + adapterKey: "generic-adapter", + client, + workspaceLocalDir: localWorkspaceDir, + assets: [{ + key: "widget", + localDir: localAssetsDir, + provision: { + stageFiles: [{ name: "widget-helper.txt", contents: "helper-bytes\n" }], + // Extract the asset AND consume the staged helper file, proving both + // stageFiles and extractCommand flow through the core generically. + extractCommand: ({ assetTarPath, assetDir, runtimeRootDir }) => + `rm -rf ${q(assetDir)} && mkdir -p ${q(assetDir)} && ` + + `tar -xf ${q(assetTarPath)} -C ${q(assetDir)} && rm -f ${q(assetTarPath)} && ` + + `cp ${q(path.posix.join(runtimeRootDir, "widget-helper.txt"))} ${q(path.posix.join(assetDir, "helper.copied.txt"))}`, + }, + restore: async ({ assetDir, readFile: readRemote }) => { + const bytes = await readRemote(path.posix.join(assetDir, "refreshed.txt")); + restored.push(bytes.toString("utf8")); + }, + }], + }); + + // provision: the asset's own content extracted... + await expect(readFile(path.join(prepared.assetDirs.widget, "seed.txt"), "utf8")).resolves.toBe("seed\n"); + // ...the staged helper file was written to the runtime root and consumed by the custom extract command. + await expect(readFile(path.join(prepared.assetDirs.widget, "helper.copied.txt"), "utf8")).resolves.toBe("helper-bytes\n"); + stagedContentSeen.push("provisioned"); + + // Simulate the sandbox refreshing a file inside the asset dir, then restore. + await writeFile(path.join(prepared.assetDirs.widget, "refreshed.txt"), "refreshed-by-sandbox\n", "utf8"); + await prepared.restoreWorkspace(); + + // restore contribution was invoked with a working remote readFile against assetDir. + expect(restored).toEqual(["refreshed-by-sandbox\n"]); + expect(stagedContentSeen).toEqual(["provisioned"]); + }); + + it("rejects a provision stageFile.name that is not a simple basename", async () => { + const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-traversal-")); + cleanupDirs.push(rootDir); + const localWorkspaceDir = path.join(rootDir, "local-workspace"); + const remoteWorkspaceDir = path.join(rootDir, "remote-workspace"); + const localAssetsDir = path.join(rootDir, "local-assets"); + await mkdir(localWorkspaceDir, { recursive: true }); + await mkdir(localAssetsDir, { recursive: true }); + await writeFile(path.join(localWorkspaceDir, "README.md"), "workspace\n", "utf8"); + await writeFile(path.join(localAssetsDir, "seed.txt"), "seed\n", "utf8"); + + const writtenPaths: string[] = []; + const client: SandboxManagedRuntimeClient = { + makeDir: async (remotePath) => { + await mkdir(remotePath, { recursive: true }); + }, + writeFile: async (remotePath, bytes) => { + writtenPaths.push(remotePath); + await mkdir(path.dirname(remotePath), { recursive: true }); + await writeFile(remotePath, Buffer.from(bytes)); + }, + readFile: async (remotePath) => await readFile(remotePath), + listFiles: async () => [], + remove: async (remotePath) => { + await rm(remotePath, { recursive: true, force: true }); + }, + run: async (command) => { + await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 }); + }, + }; + + // A compromised adapter supplying a traversal name must be rejected before + // the core ever writes outside the runtime root. + for (const maliciousName of ["../evil.txt", "..", "nested/child.txt", "back\\slash.txt", "../../etc/passwd"]) { + writtenPaths.length = 0; + await expect( + prepareSandboxManagedRuntime({ + spec: { + transport: "sandbox", + provider: "test", + sandboxId: "sandbox-1", + remoteCwd: remoteWorkspaceDir, + timeoutMs: 30_000, + apiKey: null, + }, + adapterKey: "generic-adapter", + client, + workspaceLocalDir: localWorkspaceDir, + assets: [{ + key: "widget", + localDir: localAssetsDir, + provision: { + stageFiles: [{ name: maliciousName, contents: "payload\n" }], + }, + }], + }), + ).rejects.toThrow(/must be a simple basename/); + + // The guard fires before the offending write, so nothing landed under the runtime root. + expect(writtenPaths.some((p) => p.endsWith("evil.txt") || p.endsWith("passwd") || p.endsWith("child.txt"))).toBe(false); + } + }); + + it("keeps the sandbox runtime core free of Codex-specific string literals", async () => { + const coreSource = await readFile(new URL("./sandbox-managed-runtime.ts", import.meta.url), "utf8"); + // The seam must be generic: no adapter (Codex) knowledge may live in the core. + expect(coreSource).not.toMatch(/codex/i); + expect(coreSource).not.toMatch(/auth\.json/i); + expect(coreSource).not.toMatch(/merge-extract|merge-decision/i); + }); }); diff --git a/packages/adapter-utils/src/sandbox-managed-runtime.ts b/packages/adapter-utils/src/sandbox-managed-runtime.ts index 10dc8ab40a..ccce1d1312 100644 --- a/packages/adapter-utils/src/sandbox-managed-runtime.ts +++ b/packages/adapter-utils/src/sandbox-managed-runtime.ts @@ -1,5 +1,5 @@ import { execFile as execFileCallback } from "node:child_process"; -import { constants as fsConstants, promises as fs, readFileSync } from "node:fs"; +import { constants as fsConstants, promises as fs } from "node:fs"; import os from "node:os"; import path from "node:path"; import { promisify } from "node:util"; @@ -27,14 +27,6 @@ import { import { isRelativePathOrDescendant, shouldExcludePath } from "./exclude-patterns.js"; const execFile = promisify(execFileCallback); -const CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME = "codex-auth-merge-extract.sh"; -const CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME = "codex-auth-merge-decision.cjs"; -const CODEX_AUTH_MERGE_EXTRACT_SCRIPT_BYTES = readFileSync( - new URL(`./${CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME}`, import.meta.url), -); -const CODEX_AUTH_MERGE_DECISION_SCRIPT_BYTES = readFileSync( - new URL(`./${CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME}`, import.meta.url), -); const SANDBOX_WORKSPACE_HEAVY_DIR_NAMES = [ "node_modules", "vendor", @@ -62,11 +54,61 @@ export interface SandboxRemoteExecutionSpec { apiKey: string | null; } +/** + * Remote paths handed to an asset's `provision.extractCommand`. All are POSIX + * paths inside the sandbox: `assetTarPath` is the uploaded asset tarball, + * `assetDir` is where the asset should be materialized, and `runtimeRootDir` + * is the directory any `stageFiles` were written into. + */ +export interface SandboxManagedRuntimeAssetProvisionContext { + assetTarPath: string; + assetDir: string; + runtimeRootDir: string; +} + +/** + * Per-asset inbound provisioning contribution. The core is adapter-agnostic: + * an asset that supplies neither `stageFiles` nor `extractCommand` is extracted + * with a plain `tar -xf`. An adapter that needs custom provisioning (e.g. a + * credential merge) supplies helper files via `stageFiles` and the shell + * command that consumes them via `extractCommand`. + */ +export interface SandboxManagedRuntimeAssetProvision { + /** + * Extra files written into `runtimeRootDir` (alongside the asset tar) before + * the extract command runs — typically helper scripts the extract command + * invokes. Contents may be raw bytes or a UTF-8 string. + */ + stageFiles?: { name: string; contents: Buffer | string }[]; + /** + * Builds the shell command that materializes the uploaded asset tar into + * `assetDir`. Defaults to a plain `tar -xf` extraction when omitted. + */ + extractCommand?: (ctx: SandboxManagedRuntimeAssetProvisionContext) => string; +} + +/** + * Context passed to an asset's `restore` contribution during teardown. + * `assetDir` is the asset's directory inside the sandbox and `readFile` reads + * a file back from the sandbox as raw bytes. + */ +export interface SandboxManagedRuntimeAssetRestoreContext { + assetDir: string; + readFile: (remotePath: string) => Promise; +} + export interface SandboxManagedRuntimeAsset { key: string; localDir: string; followSymlinks?: boolean; exclude?: string[]; + /** Optional inbound provisioning contribution (staged files + extract command). */ + provision?: SandboxManagedRuntimeAssetProvision; + /** + * Optional teardown/outbound contribution, invoked once per asset during + * `restoreWorkspace`. Defaults to a no-op when omitted. + */ + restore?: (ctx: SandboxManagedRuntimeAssetRestoreContext) => Promise; } /** @@ -118,26 +160,14 @@ function shellQuote(value: string) { return `'${value.replace(/'/g, `'\"'\"'`)}'`; } -function buildExtractRuntimeAssetCommand(input: { - adapterKey: string; - assetKey: string; +function buildDefaultExtractRuntimeAssetCommand(input: { remoteAssetDir: string; remoteAssetTar: string; - remoteCodexAuthMergeExtractScript?: string; }): string { - if (input.adapterKey !== "codex" || input.assetKey !== "home") { - return `rm -rf ${shellQuote(input.remoteAssetDir)} && ` + - `mkdir -p ${shellQuote(input.remoteAssetDir)} && ` + - `tar -xf ${shellQuote(input.remoteAssetTar)} -C ${shellQuote(input.remoteAssetDir)} && ` + - `rm -f ${shellQuote(input.remoteAssetTar)}`; - } - - if (!input.remoteCodexAuthMergeExtractScript) { - throw new Error("Codex auth merge extract script path is required for codex home assets"); - } - - return `sh ${shellQuote(input.remoteCodexAuthMergeExtractScript)} ` + - `${shellQuote(input.remoteAssetDir)} ${shellQuote(input.remoteAssetTar)}`; + return `rm -rf ${shellQuote(input.remoteAssetDir)} && ` + + `mkdir -p ${shellQuote(input.remoteAssetDir)} && ` + + `tar -xf ${shellQuote(input.remoteAssetTar)} -C ${shellQuote(input.remoteAssetDir)} && ` + + `rm -f ${shellQuote(input.remoteAssetTar)}`; } export function parseSandboxRemoteExecutionSpec(value: unknown): SandboxRemoteExecutionSpec | null { @@ -594,9 +624,6 @@ export async function prepareSandboxManagedRuntime(input: { const assetTarBytes = await fs.readFile(assetTarPath); const remoteAssetDir = path.posix.join(runtimeRootDir, asset.key); const remoteAssetTar = path.posix.join(runtimeRootDir, `${asset.key}-upload.tar`); - const remoteCodexAuthMergeExtractScript = input.adapterKey === "codex" && asset.key === "home" - ? path.posix.join(runtimeRootDir, CODEX_AUTH_MERGE_EXTRACT_SCRIPT_NAME) - : undefined; const assetUpload = makeTransferProgress( input.onProgress, "Syncing", @@ -606,26 +633,26 @@ export async function prepareSandboxManagedRuntime(input: { ); await input.client.writeFile(remoteAssetTar, toArrayBuffer(assetTarBytes), assetUpload.options); await assetUpload.finish(assetTarBytes.byteLength, assetTarBytes.byteLength); - if (remoteCodexAuthMergeExtractScript) { + for (const stageFile of asset.provision?.stageFiles ?? []) { + const stageBytes = typeof stageFile.contents === "string" + ? Buffer.from(stageFile.contents) + : stageFile.contents; + const safeName = stageFile.name; + if (/[\\/]|\.\.(\.|$)/.test(safeName) || safeName === "..") { + throw new Error(`provision stageFile.name must be a simple basename, got: ${safeName}`); + } await input.client.writeFile( - remoteCodexAuthMergeExtractScript, - toArrayBuffer(CODEX_AUTH_MERGE_EXTRACT_SCRIPT_BYTES), - ); - await input.client.writeFile( - path.posix.join(runtimeRootDir, CODEX_AUTH_MERGE_DECISION_SCRIPT_NAME), - toArrayBuffer(CODEX_AUTH_MERGE_DECISION_SCRIPT_BYTES), + path.posix.join(runtimeRootDir, safeName), + toArrayBuffer(stageBytes), ); } + const extractCommand = asset.provision?.extractCommand?.({ + assetTarPath: remoteAssetTar, + assetDir: remoteAssetDir, + runtimeRootDir, + }) ?? buildDefaultExtractRuntimeAssetCommand({ remoteAssetDir, remoteAssetTar }); await input.client.run( - `sh -c ${shellQuote( - buildExtractRuntimeAssetCommand({ - adapterKey: input.adapterKey, - assetKey: asset.key, - remoteAssetDir, - remoteAssetTar, - remoteCodexAuthMergeExtractScript, - }), - )}`, + `sh -c ${shellQuote(extractCommand)}`, { timeoutMs: input.spec.timeoutMs }, ); } @@ -741,6 +768,17 @@ export async function prepareSandboxManagedRuntime(input: { } : undefined, }); + + // Per-asset teardown/outbound contributions. Generic: an asset with + // no `restore` is a no-op. The contribution reads back from the + // sandbox (e.g. a refreshed credential) via the provided `readFile`. + for (const asset of input.assets ?? []) { + if (!asset.restore) continue; + await asset.restore({ + assetDir: path.posix.join(runtimeRootDir, asset.key), + readFile: async (remotePath) => toBuffer(await input.client.readFile(remotePath)), + }); + } } finally { await emitRuntimeStatus(input.onRuntimeProgress, "finalize", "Finalizing sandbox workspace"); if (importedRef) { diff --git a/packages/adapter-utils/src/workspace-restore-merge.test.ts b/packages/adapter-utils/src/workspace-restore-merge.test.ts index e7e907c278..a34e9326ba 100644 --- a/packages/adapter-utils/src/workspace-restore-merge.test.ts +++ b/packages/adapter-utils/src/workspace-restore-merge.test.ts @@ -10,6 +10,7 @@ import { prepareSandboxManagedRuntime, type SandboxManagedRuntimeClient, } from "./sandbox-managed-runtime.js"; +import { buildCodexAuthInboundProvision } from "./codex-auth-merge-scripts.js"; import { captureDirectorySnapshot, mergeDirectoryWithBaseline } from "./workspace-restore-merge.js"; const execFile = promisify(execFileCallback); @@ -187,6 +188,11 @@ describe("codex home auth merge on sandbox asset extract", () => { key: "home", localDir: localHomeDir, followSymlinks: true, + // The Codex inbound auth-merge now rides the generic per-asset + // `provision` seam. This matrix drives the sandbox core directly, so it + // supplies the same contribution the codex adapter (`execute.ts`) + // attaches in production — proving the seam reproduces inbound behavior. + provision: buildCodexAuthInboundProvision(), }], }); diff --git a/packages/adapters/codex-local/src/server/execute.ts b/packages/adapters/codex-local/src/server/execute.ts index a6f47ba3ea..cb8c882e53 100644 --- a/packages/adapters/codex-local/src/server/execute.ts +++ b/packages/adapters/codex-local/src/server/execute.ts @@ -2,6 +2,7 @@ import fs from "node:fs/promises"; import path from "node:path"; import { fileURLToPath } from "node:url"; import { inferOpenAiCompatibleBiller, type AdapterExecutionContext, type AdapterExecutionResult } from "@paperclipai/adapter-utils"; +import { buildCodexAuthInboundProvision } from "@paperclipai/adapter-utils/codex-auth-merge-scripts"; import { adapterExecutionTargetIsRemote, adapterExecutionTargetRemoteCwd, @@ -629,6 +630,12 @@ export async function execute(ctx: AdapterExecutionContext): Promise afterAll(async () => { await db.$client.end(); await tempDb?.cleanup(); - }); + }, 60_000); it("repairs clean unrecorded branch drift before recording workspace finalization", async () => { const repoRoot = await createGitRepo(); diff --git a/server/src/__tests__/heartbeat-worktree-suppression.test.ts b/server/src/__tests__/heartbeat-worktree-suppression.test.ts index c02a339721..b1998f9584 100644 --- a/server/src/__tests__/heartbeat-worktree-suppression.test.ts +++ b/server/src/__tests__/heartbeat-worktree-suppression.test.ts @@ -83,7 +83,7 @@ describeEmbeddedPostgres("heartbeat worktree suppression", () => { afterAll(async () => { await tempDb?.cleanup(); - }); + }, 60_000); async function insertAgentAndIssue() { const companyId = randomUUID(); diff --git a/server/src/__tests__/permissions-upgrade-boundary-routes.test.ts b/server/src/__tests__/permissions-upgrade-boundary-routes.test.ts index 6dffd0e907..d0e7222891 100644 --- a/server/src/__tests__/permissions-upgrade-boundary-routes.test.ts +++ b/server/src/__tests__/permissions-upgrade-boundary-routes.test.ts @@ -254,7 +254,7 @@ describeEmbeddedPostgres("permissions upgrade visibility and route boundaries", expect(activity.body).toEqual(expect.arrayContaining([expect.objectContaining({ action: "issue.updated" })])); expect(workProducts.status, JSON.stringify(workProducts.body)).toBe(200); expect(workProducts.body).toEqual(expect.arrayContaining([expect.objectContaining({ title: "Preview" })])); - }); + }, 20_000); it("denies cross-company issue reads before private-agent grant evaluation can matter", async () => { const sourceCompany = await seedCompany(db, "Source");