From 2ceb630b6cac5a68835cbc7fff413f99597619cb Mon Sep 17 00:00:00 2001
From: Dotta <34892728+cryppadotta@users.noreply.github.com>
Date: Wed, 9 Sep 2026 10:14:58 -0500
Subject: [PATCH] fix(workspaces): preserve dependency provisioning failures
 (#13093)

## Thinking Path

> - Paperclip runs agent work in isolated worktrees.
> - Each worktree needs dependencies that match its source and patches.
> - A failed install currently loses its exit status after an `if`
statement.
> - The provisioner can then record a successful dependency fingerprint.
> - This pull request preserves failures and bounds lockfile recovery.
> - Agents receive a usable workspace or an accurate provisioning
failure.

## Linked Issues or Issue Description

**What happened?** A nonzero pnpm install could return success and save
a fingerprint. Patch changes alone also did not invalidate the
fingerprint.

**Expected behavior:** Fail provisioning on an unsuccessful install.
Retry known frozen-lockfile mismatches once and record success only
after installation succeeds.

**Steps to reproduce:** Run the provisioner in a worktree with a pnpm
install that exits nonzero. The regression suite uses real shell
execution and a controlled pnpm fixture.

**Paperclip version or commit:** Reproduced on master before this
change. **Deployment mode:** Self-hosted. **Installation method:** Git
checkout. **Agent adapters involved:** Core workspace provisioning.
**Database mode:** Not relevant. **Access context:** Execution host.
**Node.js version:** 26.4.0 locally; supported minimum remains
unchanged. **Operating system:** macOS locally and Linux execution
hosts.

**Relevant logs or output:** `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`,
`ERR_PNPM_OUTDATED_LOCKFILE`, and ordinary installation failures.

**Additional context:** Related lockfile maintenance: #13061. No
lockfile or workflow changes are included.

## What Changed

- Capture the failed install status inside the `else` branch.
- Use the existing single retry for both frozen-lockfile mismatch
errors.
- Include patch contents in the dependency fingerprint.
- Add executable regression coverage and document the behavior.

## Verification

- All CI checks passed, including build, typecheck, tests, browser
suites, canary dry run, and security scans. Greptile: 5/5 with no
remaining findings.

- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`:
19 passed; one existing test requires Linux flock and was skipped on
macOS.
- `bash -n scripts/provision-worktree.sh` and `git diff --check` passed.
- Full workspace typecheck and build passed in the companion runner-fix
worktree at the same base revision. This change only touches shell
provisioning, its tests, and documentation.
- CI and review are pending.

## Risks

The existing non-frozen recovery can update a worktree-local lockfile.
Committed lockfile updates remain bot-owned. Ordinary failures now
correctly stop provisioning and may expose previously hidden
installation problems.

## Model Used

OpenAI GPT-6 through Codex, with code editing, shell execution, tests,
and browser inspection. The exact model variant and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
---
 doc/DEVELOPING.md                             |  7 +++
 .../provision-worktree-self-heal.test.mjs     | 57 ++++++++++++++++++-
 scripts/provision-worktree.sh                 | 15 ++++-
 3 files changed, 73 insertions(+), 6 deletions(-)

diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md
index bd458a0668..15dbd92e6e 100644
--- a/doc/DEVELOPING.md
+++ b/doc/DEVELOPING.md
@@ -131,6 +131,13 @@ pnpm dev:stop
 
 `pnpm dev:once` now tracks backend-relevant file changes and pending migrations. When the current boot is stale, the board UI shows a `Restart required` banner. You can also enable guarded auto-restart in `Instance Settings > Experimental`, which waits for queued/running local agent runs to finish before restarting the dev server.
 
+Worktree dependency provisioning records its fingerprint only after a successful
+install. Frozen installs with outdated lockfiles or patched-dependency hash
+mismatches retry once without `--frozen-lockfile`; other failures retain their
+exit status. Patch contents are part of the install fingerprint. Generated
+lockfile changes remain local to the worktree; the repository's lockfile bot
+owns committed updates.
+
 ## Hot-Restart Deploys
 
 Primary-instance rebuilds that restart `paperclip.service` can request one-shot live-run adoption instead of using the normal graceful shutdown drain. Before restarting the service, write the marker from the newly staged app with the current service PID:
diff --git a/scripts/__tests__/provision-worktree-self-heal.test.mjs b/scripts/__tests__/provision-worktree-self-heal.test.mjs
index c74c480b49..943a15f48b 100644
--- a/scripts/__tests__/provision-worktree-self-heal.test.mjs
+++ b/scripts/__tests__/provision-worktree-self-heal.test.mjs
@@ -16,7 +16,7 @@ const testPath = [path.dirname(process.execPath), "/usr/bin", "/bin"].join(":");
 const cleanupDirs = [];
 
 function makeTempDir(prefix) {
-  const dir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
+  const dir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), prefix)));
   cleanupDirs.push(dir);
   return dir;
 }
@@ -102,8 +102,9 @@ process.exit(0);
   return baseCwd;
 }
 
-function runProvision(baseCwd, { pathPrefix } = {}) {
-  const worktreeCwd = makeTempDir("paperclip-provision-worktree-");
+function runProvision(baseCwd, { pathPrefix, setupWorktree, existingWorktree } = {}) {
+  const worktreeCwd = existingWorktree ?? makeTempDir("paperclip-provision-worktree-");
+  setupWorktree?.(worktreeCwd);
   const worktreesHome = makeTempDir("paperclip-provision-home-");
   const paperclipHome = makeInstanceHome();
   const result = spawnSync("bash", [script], {
@@ -524,3 +525,53 @@ test("every pnpm install call site silences DEP0169 without overwriting NODE_OPT
     }
   }
 });
+
+for (const failure of ["ERR_PNPM_LOCKFILE_CONFIG_MISMATCH", "ERR_PNPM_OUTDATED_LOCKFILE", "ENOTFOUND", "retry-fails"]) {
+  test(`dependency provisioning preserves failures and bounds recovery: ${failure}`, () => {
+    const baseCwd = makeBaseWorkspace({ helpExit: 0, initExit: 0 });
+    const bin = makeTempDir("paperclip-fake-pnpm-");
+    fs.writeFileSync(path.join(bin, "pnpm"), `#!/bin/sh
+printf '%s\\n' "$*" >> pnpm-calls
+case "$*" in
+  *--frozen-lockfile*) echo '${failure === "retry-fails" ? "ERR_PNPM_LOCKFILE_CONFIG_MISMATCH" : failure}' >&2; exit 42 ;;
+  *) ${failure === "retry-fails" ? "exit 43" : "mkdir -p node_modules; exit 0"} ;;
+esac
+`, { mode: 0o700 });
+    const { result, worktreeCwd } = runProvision(baseCwd, {
+      pathPrefix: bin,
+      setupWorktree(root) {
+        fs.writeFileSync(path.join(root, "package.json"), "{}\n");
+        fs.writeFileSync(path.join(root, "pnpm-lock.yaml"), "lockfileVersion: '9.0'\n");
+      },
+    });
+    const recovers = failure.startsWith("ERR_PNPM_");
+    assert.equal(result.status, recovers ? 0 : failure === "ENOTFOUND" ? 42 : 1, result.stderr);
+    assert.equal(fs.existsSync(path.join(worktreeCwd, ".paperclip/pnpm-install-fingerprint")), recovers);
+    const calls = fs.readFileSync(path.join(worktreeCwd, "pnpm-calls"), "utf8").trim().split("\n").filter((call) => call.startsWith("install "));
+    assert.equal(calls.length, failure === "ENOTFOUND" ? 1 : 2);
+    if (calls.length === 2) assert.match(calls[1], /--no-frozen-lockfile/);
+  });
+}
+
+test("patch content changes invalidate an otherwise matching install fingerprint", () => {
+  const baseCwd = makeBaseWorkspace({ helpExit: 0, initExit: 0 });
+  const bin = makeTempDir("paperclip-patch-pnpm-");
+  fs.writeFileSync(path.join(bin, "pnpm"), '#!/bin/sh\ncase "$1" in install) echo install >> pnpm-calls; mkdir -p node_modules cli/node_modules ;; esac\n', { mode: 0o700 });
+  const first = runProvision(baseCwd, { pathPrefix: bin, setupWorktree(root) {
+    fs.writeFileSync(path.join(root, "package.json"), JSON.stringify({ pnpm: { patchedDependencies: { "dependency@1": "patches/dependency.diff" } } }));
+    fs.writeFileSync(path.join(root, "pnpm-lock.yaml"), "lockfileVersion: '9.0'\n");
+    fs.mkdirSync(path.join(root, "patches"));
+    fs.writeFileSync(path.join(root, "patches/dependency.diff"), "first patch");
+  } });
+  assert.equal(first.result.status, 0, first.result.stderr);
+  const options = { pathPrefix: bin, existingWorktree: first.worktreeCwd };
+  assert.equal(runProvision(baseCwd, options).result.status, 0);
+  const callsPath = path.join(first.worktreeCwd, "pnpm-calls");
+  assert.equal(fs.readFileSync(callsPath, "utf8"), "install\n");
+  fs.writeFileSync(path.join(first.worktreeCwd, "unrelated.patch"), "unrelated change");
+  assert.equal(runProvision(baseCwd, options).result.status, 0);
+  assert.equal(fs.readFileSync(callsPath, "utf8"), "install\n");
+  fs.writeFileSync(path.join(first.worktreeCwd, "patches/dependency.diff"), "changed patch");
+  assert.equal(runProvision(baseCwd, options).result.status, 0);
+  assert.equal(fs.readFileSync(callsPath, "utf8"), "install\ninstall\n");
+});
diff --git a/scripts/provision-worktree.sh b/scripts/provision-worktree.sh
index 1f41b03082..af7c61b1a8 100644
--- a/scripts/provision-worktree.sh
+++ b/scripts/provision-worktree.sh
@@ -692,6 +692,14 @@ function walk(dir) {
 }
 
 walk(root);
+// package.json is the pnpm 9 patch manifest for this repository. Hash the
+// declared paths, including non-.patch filenames and patches outside patches/.
+const manifest = JSON.parse(fs.readFileSync(path.join(root, "package.json"), "utf8"));
+for (const patch of Object.values(manifest.pnpm?.patchedDependencies ?? {})) {
+  if (typeof patch !== "string") throw new Error("Invalid pnpm patch path");
+  const file = path.resolve(root, patch);
+  if (!files.includes(file)) files.push(file);
+}
 files.sort((left, right) => path.relative(root, left).localeCompare(path.relative(root, right)));
 
 const hash = crypto.createHash("sha256");
@@ -768,7 +776,7 @@ if [[ -f "$worktree_cwd/package.json" && -f "$worktree_cwd/pnpm-lock.yaml" ]]; t
     }
 
     run_pnpm_install() {
-      local stdout_path stderr_path
+      local stdout_path stderr_path exit_code
       stdout_path="$(mktemp)"
       stderr_path="$(mktemp)"
 
@@ -783,12 +791,13 @@ if [[ -f "$worktree_cwd/package.json" && -f "$worktree_cwd/pnpm-lock.yaml" ]]; t
         cat "$stderr_path" >&2
         rm -f "$stdout_path" "$stderr_path"
         return 0
+      else
+        exit_code=$?
       fi
 
-      local exit_code=$?
       cat "$stdout_path"
       cat "$stderr_path" >&2
-      if grep -q "ERR_PNPM_OUTDATED_LOCKFILE" "$stdout_path" "$stderr_path"; then
+      if grep -Eq "ERR_PNPM_(OUTDATED_LOCKFILE|LOCKFILE_CONFIG_MISMATCH)" "$stdout_path" "$stderr_path"; then
         rm -f "$stdout_path" "$stderr_path"
         return 90
       fi