Add auto-forward execution workspace branch reconciliation (#9172)
## Thinking Path > - Paperclip manages execution workspaces for AI agent runs, each associated with a git branch so the agent always works in a known code state. > - Workspace runtime reconciles an agent's checkout branch against the workspace's recorded branch when a heartbeat resumes; without forward-ancestry detection, any divergence fails closed and blocks the run. > - When the workspace branch has moved forward (e.g. after a feature merge), the recorded branch is an ancestor of the current HEAD — a safe, forward-only case that the previous implementation refused even though it carries no safety risk. > - The gap means legitimate forward-advancing deployments require manual operator intervention to unblock agents every time, creating operational friction and interrupting automated workflows. > - This pull request adds a flag-gated reconcile-forward path that detects when the current branch is a strict forward descendant of the workspace branch and auto-reconciles, while preserving fail-closed behavior for all non-forward or flag-off cases. > - It also threads the active execution workspace id through restore and finalize call sites so the reconcile verdict can be persisted durably across heartbeats. > - The benefit is that agents resume automatically from forward-advancing workspace branches without operator intervention, while adversarial and backward branch changes continue to fail closed. ## Linked Issues or Issue Description This PR adds an auto-forward reconcile path for execution workspace branch tracking. When a workspace's recorded branch is a strict ancestor of the current HEAD (a forward-only advancement), the runtime now auto-reconciles rather than hard-blocking. The feature is gated behind an explicit runtime flag, defaults to off, and falls back to fail-closed behavior for all non-forward or flag-off cases. The prior implementation treated all branch divergences identically: any mismatch between the recorded workspace branch and the current HEAD failed closed. This prevented agents from resuming after routine forward deployments (e.g. after a feature branch merges into the workspace branch), requiring manual operator action to unblock every affected run. ## What Changed - Added `reconcileForward` flag-gated path in workspace runtime reconciliation logic that allows auto-reconciliation when the workspace branch is a strict ancestor of the current HEAD. - Threaded active execution workspace id through `restore` and `finalize` call sites so reconcile verdicts are persisted durably. - Added `plainLanguageReason` and `ancestryVerdict` evidence fields to the reconciliation result structure for operator visibility. - Stabilized a branch containment test that exposed a late run-linked activity FK cleanup race during the focused Vitest rerun. - All new paths remain fail-closed when the flag is off or when the branch relationship is not strictly forward. ## Verification ```bash pnpm --filter @paperclipai/shared typecheck pnpm --filter @paperclipai/server typecheck pnpm exec vitest run \ server/src/__tests__/workspace-runtime.test.ts \ server/src/__tests__/heartbeat-workspace-branch-containment.test.ts \ server/src/__tests__/execution-workspaces-service.test.ts git diff --check origin/master..HEAD ``` All 3 test files / 98 tests pass. Typecheck passes for both shared and server packages. > **Note:** This is a stacked PR on top of PR #9170 (Add execution workspace branch reconciliation route). The diff shown targets that branch; the combined change builds on the reconciliation route infrastructure it provides. ## Risks - **Flag-off default:** The reconcile-forward path is off by default. No behavior change for existing workspaces unless the flag is explicitly enabled by an operator. - **Ancestry check correctness:** The forward-only guard uses git ancestry verification; a branch that is not a strict ancestor of HEAD remains fail-closed. Adversarial or concurrent branch resets are not auto-reconciled. - **FK cleanup race (stabilized):** A late run-linked activity FK cleanup race in the containment test was exposed during the Vitest rerun. The stabilization commit addresses the non-deterministic ordering without changing production behavior. - **Stacking dependency:** This PR must not be merged before PR #9170 merges, as it is built on top of the reconciliation route infrastructure. ## Model Used - Provider: Anthropic - Model: Claude Sonnet 4.6 (`claude-sonnet-4-6`) - Context: 200k token context window - Mode: Agentic tool use with code execution and git operations ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky Leach
GitHub
parent
f17202b571
commit
bb87047fe6
|
|
@ -1,5 +1,5 @@
|
|||
import { execFile } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
|
@ -40,6 +40,42 @@ import { instanceSettingsService } from "../services/instance-settings.ts";
|
|||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
function stableStringifyForTest(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map((entry) => stableStringifyForTest(entry)).join(",")}]`;
|
||||
if (value && typeof value === "object") {
|
||||
const rec = value as Record<string, unknown>;
|
||||
return `{${Object.keys(rec).sort().map((key) => `${JSON.stringify(key)}:${stableStringifyForTest(rec[key])}`).join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function fingerprintWorkspaceBranchIncoherenceForTest(input: {
|
||||
sourceIssueId: string | null;
|
||||
executionWorkspaceId: string | null;
|
||||
worktreePath: string;
|
||||
expectedBranch: string;
|
||||
actualBranch: string | null;
|
||||
cleanliness: "clean" | "dirty" | "unknown";
|
||||
expectedHeadSha: string | null;
|
||||
actualHeadSha: string | null;
|
||||
}) {
|
||||
const digest = createHash("sha256")
|
||||
.update(stableStringifyForTest({
|
||||
version: 1,
|
||||
reason: "git_worktree_branch_incoherence",
|
||||
sourceIssueId: input.sourceIssueId,
|
||||
executionWorkspaceId: input.executionWorkspaceId,
|
||||
worktreePath: path.resolve(input.worktreePath),
|
||||
expectedBranch: input.expectedBranch,
|
||||
actualBranch: input.actualBranch,
|
||||
cleanliness: input.cleanliness,
|
||||
expectedHeadSha: input.expectedHeadSha,
|
||||
actualHeadSha: input.actualHeadSha,
|
||||
}))
|
||||
.digest("hex");
|
||||
return `workspace_incoherence:v1:sha256:${digest}`;
|
||||
}
|
||||
|
||||
const adapterExecute = vi.hoisted(() =>
|
||||
vi.fn(async () => ({
|
||||
exitCode: 0,
|
||||
|
|
@ -83,6 +119,10 @@ async function runGit(cwd: string, args: string[]) {
|
|||
await execFileAsync("git", args, { cwd });
|
||||
}
|
||||
|
||||
async function readGit(cwd: string, args: string[]) {
|
||||
return (await execFileAsync("git", args, { cwd })).stdout.trim();
|
||||
}
|
||||
|
||||
async function createGitRepo() {
|
||||
const repoRoot = await mkdtemp(path.join(os.tmpdir(), "paperclip-branch-containment-repo-"));
|
||||
await runGit(repoRoot, ["init"]);
|
||||
|
|
@ -231,7 +271,12 @@ function readAdapterWorkspace(input: unknown) {
|
|||
return { cwd, branchName, executionWorkspaceId };
|
||||
}
|
||||
|
||||
async function seedBranchContainmentRun(db: Db, repoRoot: string, callSite: BranchContainmentCallSite) {
|
||||
async function seedBranchContainmentRun(
|
||||
db: Db,
|
||||
repoRoot: string,
|
||||
callSite: BranchContainmentCallSite,
|
||||
opts: { enableWorkspaceBranchReconcileForward?: boolean } = {},
|
||||
) {
|
||||
const companyId = randomUUID();
|
||||
const projectId = randomUUID();
|
||||
const projectWorkspaceId = randomUUID();
|
||||
|
|
@ -254,6 +299,7 @@ async function seedBranchContainmentRun(db: Db, repoRoot: string, callSite: Bran
|
|||
|
||||
await instanceSettingsService(db).updateExperimental({
|
||||
enableIsolatedWorkspaces: true,
|
||||
enableWorkspaceBranchReconcileForward: opts.enableWorkspaceBranchReconcileForward === true,
|
||||
});
|
||||
await db.insert(companies).values({
|
||||
id: companyId,
|
||||
|
|
@ -476,6 +522,14 @@ async function seedBranchContainmentRun(db: Db, repoRoot: string, callSite: Bran
|
|||
},
|
||||
]);
|
||||
|
||||
await db
|
||||
.update(executionWorkspaces)
|
||||
.set({
|
||||
sourceIssueId,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(eq(executionWorkspaces.id, sourceExecutionWorkspaceId));
|
||||
|
||||
return {
|
||||
companyId,
|
||||
agentId,
|
||||
|
|
@ -487,6 +541,7 @@ async function seedBranchContainmentRun(db: Db, repoRoot: string, callSite: Bran
|
|||
otherExecutionWorkspaceId,
|
||||
expectedBranch,
|
||||
actualBranch,
|
||||
worktreePath,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -609,6 +664,163 @@ async function expectContainedWorkspaceBranchFailure(input: {
|
|||
expect(comments.filter((comment) => comment.issueId === input.otherWorkspaceSiblingId)).toHaveLength(0);
|
||||
}
|
||||
|
||||
async function expectForwardBranchReconciled(input: {
|
||||
db: Db;
|
||||
heartbeat: Heartbeat;
|
||||
runId: string;
|
||||
sourceIssueId: string;
|
||||
sourceExecutionWorkspaceId: string;
|
||||
expectedBranch: string;
|
||||
actualBranch: string;
|
||||
expectedWorktreeStateAfterReconcile: {
|
||||
head: string;
|
||||
status: string;
|
||||
};
|
||||
worktreePath: string;
|
||||
expectsExistingRecordUpdate: boolean;
|
||||
expectedResolvedRecoveryActionFingerprint?: string | null;
|
||||
}) {
|
||||
const finishedRun = await waitForRunToFinish(input.heartbeat, input.runId, 10_000);
|
||||
expect(finishedRun).toMatchObject({
|
||||
status: "succeeded",
|
||||
errorCode: null,
|
||||
});
|
||||
|
||||
expect(input.expectedWorktreeStateAfterReconcile.head).toEqual(expect.stringMatching(/^[a-f0-9]{40}$/));
|
||||
await expect(readGit(input.worktreePath, ["rev-parse", "HEAD"])).resolves.toBe(input.expectedWorktreeStateAfterReconcile.head);
|
||||
await expect(readGit(input.worktreePath, ["status", "--porcelain", "--untracked-files=all"])).resolves.toBe(input.expectedWorktreeStateAfterReconcile.status);
|
||||
|
||||
const [sourceIssue] = await input.db
|
||||
.select({
|
||||
status: issues.status,
|
||||
executionWorkspaceId: issues.executionWorkspaceId,
|
||||
})
|
||||
.from(issues)
|
||||
.where(eq(issues.id, input.sourceIssueId));
|
||||
expect(sourceIssue?.status).toBe("done");
|
||||
expect(sourceIssue?.executionWorkspaceId).toEqual(expect.any(String));
|
||||
|
||||
const activeWorkspaceId = sourceIssue?.executionWorkspaceId!;
|
||||
const [activeWorkspace] = await input.db
|
||||
.select({
|
||||
id: executionWorkspaces.id,
|
||||
name: executionWorkspaces.name,
|
||||
branchName: executionWorkspaces.branchName,
|
||||
providerRef: executionWorkspaces.providerRef,
|
||||
})
|
||||
.from(executionWorkspaces)
|
||||
.where(eq(executionWorkspaces.id, activeWorkspaceId));
|
||||
expect(activeWorkspace).toMatchObject({
|
||||
name: input.actualBranch,
|
||||
branchName: input.actualBranch,
|
||||
providerRef: input.worktreePath,
|
||||
});
|
||||
|
||||
const recoveryRows = await input.db
|
||||
.select()
|
||||
.from(issueRecoveryActions)
|
||||
.where(eq(issueRecoveryActions.sourceIssueId, input.sourceIssueId));
|
||||
if (input.expectedResolvedRecoveryActionFingerprint) {
|
||||
expect(recoveryRows).toEqual([
|
||||
expect.objectContaining({
|
||||
status: "resolved",
|
||||
outcome: "restored",
|
||||
fingerprint: input.expectedResolvedRecoveryActionFingerprint,
|
||||
resolutionNote: expect.stringContaining("Execution workspace branch record reconciled"),
|
||||
resolvedAt: expect.any(Date),
|
||||
}),
|
||||
]);
|
||||
} else {
|
||||
expect(recoveryRows).toHaveLength(0);
|
||||
}
|
||||
|
||||
const operations = await input.db
|
||||
.select()
|
||||
.from(workspaceOperations)
|
||||
.where(eq(workspaceOperations.heartbeatRunId, input.runId));
|
||||
expect(operations).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
status: "succeeded",
|
||||
metadata: expect.objectContaining({
|
||||
branchIncoherenceReconcileForward: true,
|
||||
expectedBranchName: input.expectedBranch,
|
||||
actualBranchName: input.actualBranch,
|
||||
fingerprint: expect.stringMatching(/^workspace_incoherence:v1:sha256:/),
|
||||
}),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
|
||||
if (input.expectsExistingRecordUpdate) {
|
||||
const [updatedWorkspace] = await input.db
|
||||
.select({
|
||||
name: executionWorkspaces.name,
|
||||
branchName: executionWorkspaces.branchName,
|
||||
})
|
||||
.from(executionWorkspaces)
|
||||
.where(eq(executionWorkspaces.id, activeWorkspaceId));
|
||||
expect(updatedWorkspace).toMatchObject({
|
||||
name: input.actualBranch,
|
||||
branchName: input.actualBranch,
|
||||
});
|
||||
if (activeWorkspaceId !== input.sourceExecutionWorkspaceId) {
|
||||
const [sourceWorkspace] = await input.db
|
||||
.select({ branchName: executionWorkspaces.branchName })
|
||||
.from(executionWorkspaces)
|
||||
.where(eq(executionWorkspaces.id, input.sourceExecutionWorkspaceId));
|
||||
expect(sourceWorkspace?.branchName).toBe(input.expectedBranch);
|
||||
}
|
||||
|
||||
const comments = await readContainmentComments(input.db, [input.sourceIssueId]);
|
||||
const resolvedRecoveryActionId = recoveryRows.length === 1 ? recoveryRows[0]?.id : null;
|
||||
expect(comments).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
authorType: "system",
|
||||
body: expect.stringContaining("Execution workspace branch reconciled."),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
if (resolvedRecoveryActionId) {
|
||||
expect(comments).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
authorType: "system",
|
||||
body: expect.stringContaining(`Recovery action: \`${resolvedRecoveryActionId}\``),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
}
|
||||
|
||||
const activities = await input.db
|
||||
.select()
|
||||
.from(activityLog)
|
||||
.where(eq(activityLog.entityId, activeWorkspaceId));
|
||||
expect(activities).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
actorType: "system",
|
||||
actorId: "workspace_runtime",
|
||||
action: "execution_workspace.branch_reconciled",
|
||||
details: expect.objectContaining({
|
||||
mode: "forward",
|
||||
fromBranch: input.expectedBranch,
|
||||
toBranch: input.actualBranch,
|
||||
ancestryVerdict: "ancestor",
|
||||
}),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
} else {
|
||||
const [sourceWorkspace] = await input.db
|
||||
.select({ branchName: executionWorkspaces.branchName })
|
||||
.from(executionWorkspaces)
|
||||
.where(eq(executionWorkspaces.id, input.sourceExecutionWorkspaceId));
|
||||
expect(sourceWorkspace?.branchName).toBe(input.expectedBranch);
|
||||
}
|
||||
}
|
||||
|
||||
describeEmbeddedPostgres("heartbeat workspace branch containment", () => {
|
||||
let db!: Db;
|
||||
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
||||
|
|
@ -643,6 +855,9 @@ describeEmbeddedPostgres("heartbeat workspace branch containment", () => {
|
|||
await db.delete(environmentLeases);
|
||||
await db.delete(activityLog);
|
||||
await db.delete(heartbeatRunEvents);
|
||||
// Heartbeat failure/finalization paths can emit run-linked activity after
|
||||
// the first cleanup pass observes all runs as non-active.
|
||||
await db.delete(activityLog);
|
||||
await db.delete(heartbeatRuns);
|
||||
await db.delete(issueComments);
|
||||
await db.delete(issues);
|
||||
|
|
@ -722,4 +937,113 @@ describeEmbeddedPostgres("heartbeat workspace branch containment", () => {
|
|||
});
|
||||
expect(adapterExecute).toHaveBeenCalledTimes(callSite === "finalize" ? 1 : 0);
|
||||
}, 30_000);
|
||||
|
||||
it.each([
|
||||
["workspace-runtime fresh worktree reuse", "fresh_realize" as const, true],
|
||||
["workspace-runtime persisted restore", "persisted_restore" as const, true],
|
||||
["heartbeat finalization", "finalize" as const, true],
|
||||
])("auto-reconciles forward branch divergence at %s when the flag is enabled", async (_name, callSite, expectsExistingRecordUpdate) => {
|
||||
const repoRoot = await createGitRepo();
|
||||
tempRoots.push(repoRoot);
|
||||
const seeded = await seedBranchContainmentRun(db, repoRoot, callSite, {
|
||||
enableWorkspaceBranchReconcileForward: true,
|
||||
});
|
||||
|
||||
const expectedWorktreeStateAfterReconcile = {
|
||||
head: callSite === "finalize" ? "" : await readGit(seeded.worktreePath, ["rev-parse", "HEAD"]),
|
||||
status: callSite === "finalize" ? "" : await readGit(seeded.worktreePath, ["status", "--porcelain", "--untracked-files=all"]),
|
||||
};
|
||||
let expectedResolvedRecoveryActionFingerprint: string | null = null;
|
||||
if (callSite === "fresh_realize") {
|
||||
const expectedHeadSha = await readGit(seeded.worktreePath, ["rev-parse", seeded.expectedBranch]);
|
||||
const actualHeadSha = await readGit(seeded.worktreePath, ["rev-parse", seeded.actualBranch]);
|
||||
expectedResolvedRecoveryActionFingerprint = fingerprintWorkspaceBranchIncoherenceForTest({
|
||||
sourceIssueId: seeded.sourceIssueId,
|
||||
executionWorkspaceId: null,
|
||||
worktreePath: seeded.worktreePath,
|
||||
expectedBranch: seeded.expectedBranch,
|
||||
actualBranch: seeded.actualBranch,
|
||||
cleanliness: "clean",
|
||||
expectedHeadSha,
|
||||
actualHeadSha,
|
||||
});
|
||||
const now = new Date("2026-07-07T00:00:01.000Z");
|
||||
await db.insert(issueRecoveryActions).values({
|
||||
id: randomUUID(),
|
||||
companyId: seeded.companyId,
|
||||
sourceIssueId: seeded.sourceIssueId,
|
||||
kind: "workspace_validation",
|
||||
status: "active",
|
||||
ownerType: "agent",
|
||||
ownerAgentId: seeded.agentId,
|
||||
cause: "workspace_validation_failed",
|
||||
fingerprint: expectedResolvedRecoveryActionFingerprint,
|
||||
evidence: {},
|
||||
nextAction: "Retry after fresh worktree branch adoption can be audited.",
|
||||
attemptCount: 1,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
});
|
||||
}
|
||||
|
||||
adapterExecute.mockImplementationOnce(async (adapterInput) => {
|
||||
if (callSite === "finalize") {
|
||||
const workspace = readAdapterWorkspace(adapterInput);
|
||||
const actualBranch = `${workspace.branchName.replace(/-recorded$/, "")}-actual`;
|
||||
await db
|
||||
.update(issues)
|
||||
.set({
|
||||
executionWorkspaceId: workspace.executionWorkspaceId,
|
||||
executionWorkspacePreference: "reuse_existing",
|
||||
executionWorkspaceSettings: { mode: "isolated_workspace" },
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(issues.id, seeded.sameWorkspaceSiblingId));
|
||||
await runGit(workspace.cwd, ["checkout", "-b", actualBranch]);
|
||||
await writeFile(path.join(workspace.cwd, "actual-branch.txt"), "actual branch work\n", "utf8");
|
||||
await runGit(workspace.cwd, ["add", "actual-branch.txt"]);
|
||||
await runGit(workspace.cwd, ["commit", "-m", "Add actual branch work"]);
|
||||
expectedWorktreeStateAfterReconcile.head = await readGit(workspace.cwd, ["rev-parse", "HEAD"]);
|
||||
expectedWorktreeStateAfterReconcile.status = await readGit(workspace.cwd, ["status", "--porcelain", "--untracked-files=all"]);
|
||||
}
|
||||
await db
|
||||
.update(issues)
|
||||
.set({
|
||||
status: "done",
|
||||
completedAt: new Date(),
|
||||
checkoutRunId: null,
|
||||
executionRunId: null,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(eq(issues.id, seeded.sourceIssueId));
|
||||
return {
|
||||
exitCode: 0,
|
||||
signal: null,
|
||||
timedOut: false,
|
||||
summary: callSite === "finalize"
|
||||
? "Adapter completed after switching to an unrecorded branch."
|
||||
: "Adapter completed after branch reconciliation.",
|
||||
provider: "test",
|
||||
model: "test-model",
|
||||
};
|
||||
});
|
||||
|
||||
const heartbeat = heartbeatService(db);
|
||||
await heartbeat.resumeQueuedRuns();
|
||||
|
||||
await expectForwardBranchReconciled({
|
||||
db,
|
||||
heartbeat,
|
||||
runId: seeded.runId,
|
||||
sourceIssueId: seeded.sourceIssueId,
|
||||
sourceExecutionWorkspaceId: seeded.sourceExecutionWorkspaceId,
|
||||
expectedBranch: seeded.expectedBranch,
|
||||
actualBranch: seeded.actualBranch,
|
||||
expectedWorktreeStateAfterReconcile,
|
||||
worktreePath: seeded.worktreePath,
|
||||
expectsExistingRecordUpdate,
|
||||
expectedResolvedRecoveryActionFingerprint,
|
||||
});
|
||||
expect(adapterExecute).toHaveBeenCalledTimes(1);
|
||||
}, 30_000);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -122,6 +122,78 @@ async function createTempRepo(defaultBranch = "main") {
|
|||
return repoRoot;
|
||||
}
|
||||
|
||||
async function expectPersistedBranchMismatchRejected(input: {
|
||||
repoRoot: string;
|
||||
worktreePath: string;
|
||||
expectedBranch: string;
|
||||
actualBranch: string;
|
||||
issueId: string;
|
||||
executionWorkspaceId: string;
|
||||
expectedAncestryVerdict: "diverged" | "unknown";
|
||||
expectedReason?: string;
|
||||
}) {
|
||||
let error: unknown = null;
|
||||
try {
|
||||
await ensurePersistedExecutionWorkspaceAvailable({
|
||||
base: {
|
||||
baseCwd: input.repoRoot,
|
||||
source: "project_primary",
|
||||
projectId: "project-1",
|
||||
workspaceId: "workspace-1",
|
||||
repoUrl: null,
|
||||
repoRef: "HEAD",
|
||||
},
|
||||
workspace: {
|
||||
id: input.executionWorkspaceId,
|
||||
mode: "isolated_workspace",
|
||||
strategyType: "git_worktree",
|
||||
cwd: input.worktreePath,
|
||||
providerRef: input.worktreePath,
|
||||
projectId: "project-1",
|
||||
projectWorkspaceId: "workspace-1",
|
||||
repoUrl: null,
|
||||
baseRef: "HEAD",
|
||||
branchName: input.expectedBranch,
|
||||
},
|
||||
issue: {
|
||||
id: input.issueId,
|
||||
identifier: "PAP-459",
|
||||
title: "Reject unsafe forward branch reconciliation",
|
||||
},
|
||||
agent: {
|
||||
id: "agent-1",
|
||||
name: "Codex Coder",
|
||||
companyId: "company-1",
|
||||
},
|
||||
enableWorkspaceBranchReconcileForward: true,
|
||||
});
|
||||
} catch (err) {
|
||||
error = err;
|
||||
}
|
||||
|
||||
expect(error).toMatchObject({
|
||||
code: "workspace_validation_failed",
|
||||
resultJson: {
|
||||
workspaceValidation: expect.objectContaining({
|
||||
reason: "git_worktree_branch_incoherence",
|
||||
sourceIssueId: input.issueId,
|
||||
executionWorkspaceId: input.executionWorkspaceId,
|
||||
expectedBranch: input.expectedBranch,
|
||||
actualBranch: input.actualBranch,
|
||||
provenance: expect.objectContaining({
|
||||
ancestryVerdict: input.expectedAncestryVerdict,
|
||||
}),
|
||||
safeRepair: expect.objectContaining({
|
||||
eligible: false,
|
||||
attempted: false,
|
||||
succeeded: false,
|
||||
...(input.expectedReason ? { reason: input.expectedReason } : {}),
|
||||
}),
|
||||
}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
async function createClonedRepoWithRemote() {
|
||||
const sourceRepo = await createTempRepo("master");
|
||||
const remoteDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-worktree-remote-"));
|
||||
|
|
@ -2558,6 +2630,7 @@ describe("realizeExecutionWorkspace", () => {
|
|||
name: "Codex Coder",
|
||||
companyId: "company-1",
|
||||
},
|
||||
enableWorkspaceBranchReconcileForward: true,
|
||||
});
|
||||
} catch (err) {
|
||||
error = err;
|
||||
|
|
@ -2630,6 +2703,7 @@ describe("realizeExecutionWorkspace", () => {
|
|||
name: "Codex Coder",
|
||||
companyId: "company-1",
|
||||
},
|
||||
enableWorkspaceBranchReconcileForward: true,
|
||||
});
|
||||
} catch (err) {
|
||||
error = err;
|
||||
|
|
@ -2665,6 +2739,93 @@ describe("realizeExecutionWorkspace", () => {
|
|||
});
|
||||
}, 15_000);
|
||||
|
||||
it("keeps forward reconciliation fail-closed for same-content rewritten history", async () => {
|
||||
const repoRoot = await createTempRepo();
|
||||
const expectedBranch = "PAP-459-recorded-content";
|
||||
const actualBranch = "PAP-459-rewritten-content";
|
||||
const worktreePath = path.join(repoRoot, ".paperclip", "worktrees", expectedBranch);
|
||||
|
||||
await runGit(repoRoot, ["checkout", "-b", expectedBranch]);
|
||||
await fs.writeFile(path.join(repoRoot, "same-content.txt"), "same content\n", "utf8");
|
||||
await runGit(repoRoot, ["add", "same-content.txt"]);
|
||||
await runGit(repoRoot, ["commit", "-m", "Add content on recorded branch"]);
|
||||
await runGit(repoRoot, ["checkout", "main"]);
|
||||
|
||||
await fs.mkdir(path.dirname(worktreePath), { recursive: true });
|
||||
await runGit(repoRoot, ["worktree", "add", "-b", actualBranch, worktreePath, "main"]);
|
||||
await fs.writeFile(path.join(worktreePath, "same-content.txt"), "same content\n", "utf8");
|
||||
await runGit(worktreePath, ["add", "same-content.txt"]);
|
||||
await runGit(worktreePath, ["commit", "-m", "Add content on rewritten branch"]);
|
||||
|
||||
await expectPersistedBranchMismatchRejected({
|
||||
repoRoot,
|
||||
worktreePath,
|
||||
expectedBranch,
|
||||
actualBranch,
|
||||
issueId: "issue-rewritten-history",
|
||||
executionWorkspaceId: "execution-workspace-rewritten-history",
|
||||
expectedAncestryVerdict: "diverged",
|
||||
expectedReason: "expected branch and current HEAD differ",
|
||||
});
|
||||
}, 15_000);
|
||||
|
||||
it("keeps forward reconciliation fail-closed for an unrelated task branch", async () => {
|
||||
const repoRoot = await createTempRepo();
|
||||
const expectedBranch = "PAP-459-recorded-task";
|
||||
const actualBranch = "PAP-999-unrelated-task";
|
||||
const worktreePath = path.join(repoRoot, ".paperclip", "worktrees", expectedBranch);
|
||||
|
||||
await runGit(repoRoot, ["checkout", "-b", expectedBranch]);
|
||||
await fs.writeFile(path.join(repoRoot, "recorded-task.txt"), "recorded task work\n", "utf8");
|
||||
await runGit(repoRoot, ["add", "recorded-task.txt"]);
|
||||
await runGit(repoRoot, ["commit", "-m", "Add recorded task work"]);
|
||||
await runGit(repoRoot, ["checkout", "main"]);
|
||||
|
||||
await fs.mkdir(path.dirname(worktreePath), { recursive: true });
|
||||
await runGit(repoRoot, ["worktree", "add", "-b", actualBranch, worktreePath, "main"]);
|
||||
await fs.writeFile(path.join(worktreePath, "unrelated-task.txt"), "unrelated task work\n", "utf8");
|
||||
await runGit(worktreePath, ["add", "unrelated-task.txt"]);
|
||||
await runGit(worktreePath, ["commit", "-m", "Add unrelated task work"]);
|
||||
|
||||
await expectPersistedBranchMismatchRejected({
|
||||
repoRoot,
|
||||
worktreePath,
|
||||
expectedBranch,
|
||||
actualBranch,
|
||||
issueId: "issue-unrelated-task",
|
||||
executionWorkspaceId: "execution-workspace-unrelated-task",
|
||||
expectedAncestryVerdict: "diverged",
|
||||
expectedReason: "expected branch and current HEAD differ",
|
||||
});
|
||||
}, 15_000);
|
||||
|
||||
it("keeps forward reconciliation fail-closed when the live branch is behind the recorded branch", async () => {
|
||||
const repoRoot = await createTempRepo();
|
||||
const expectedBranch = "PAP-459-recorded-ahead";
|
||||
const actualBranch = "PAP-459-live-behind";
|
||||
const worktreePath = path.join(repoRoot, ".paperclip", "worktrees", expectedBranch);
|
||||
|
||||
await fs.mkdir(path.dirname(worktreePath), { recursive: true });
|
||||
await runGit(repoRoot, ["branch", expectedBranch]);
|
||||
await runGit(repoRoot, ["worktree", "add", "-b", actualBranch, worktreePath, expectedBranch]);
|
||||
|
||||
await runGit(repoRoot, ["checkout", expectedBranch]);
|
||||
await fs.writeFile(path.join(repoRoot, "recorded-ahead.txt"), "recorded branch moved ahead\n", "utf8");
|
||||
await runGit(repoRoot, ["add", "recorded-ahead.txt"]);
|
||||
await runGit(repoRoot, ["commit", "-m", "Move recorded branch ahead"]);
|
||||
|
||||
await expectPersistedBranchMismatchRejected({
|
||||
repoRoot,
|
||||
worktreePath,
|
||||
expectedBranch,
|
||||
actualBranch,
|
||||
issueId: "issue-live-behind",
|
||||
executionWorkspaceId: "execution-workspace-live-behind",
|
||||
expectedAncestryVerdict: "diverged",
|
||||
expectedReason: "expected branch and current HEAD differ",
|
||||
});
|
||||
}, 15_000);
|
||||
|
||||
it("does not reuse a missing persisted local filesystem workspace", async () => {
|
||||
const baseCwd = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-workspace-base-"));
|
||||
const missingCwd = path.join(baseCwd, "missing-workspace");
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ const WORKSPACE_VALIDATION_RECOVERY_CAUSE = "workspace_validation_failed";
|
|||
export type ExecutionWorkspaceBranchReconcileMode = "forward" | "override";
|
||||
|
||||
export type ExecutionWorkspaceBranchReconcileActor = {
|
||||
actorType: "agent" | "user";
|
||||
actorType: "agent" | "user" | "system";
|
||||
actorId: string;
|
||||
agentId: string | null;
|
||||
runId: string | null;
|
||||
|
|
@ -299,8 +299,10 @@ function assertBranchReconcileWorkspaceIsSafe(input: {
|
|||
workspaceStatus: ExecutionWorkspace["status"];
|
||||
inspection: ExecutionWorkspaceBranchReconcileInspection;
|
||||
runtimeServices: WorkspaceRuntimeService[];
|
||||
allowActiveWorkspace?: boolean;
|
||||
}) {
|
||||
if (input.workspaceStatus !== "idle") {
|
||||
const allowedStatuses = input.allowActiveWorkspace ? ["idle", "active"] : ["idle"];
|
||||
if (!allowedStatuses.includes(input.workspaceStatus)) {
|
||||
throw unprocessable("Execution workspace branch reconciliation requires the workspace to be idle", {
|
||||
workspaceStatus: input.workspaceStatus,
|
||||
inspection: input.inspection,
|
||||
|
|
@ -1336,6 +1338,7 @@ export function executionWorkspaceService(db: Db) {
|
|||
mode: ExecutionWorkspaceBranchReconcileMode;
|
||||
reason?: string | null;
|
||||
actor: ExecutionWorkspaceBranchReconcileActor;
|
||||
alternateRecoveryFingerprints?: string[] | null;
|
||||
},
|
||||
): Promise<ExecutionWorkspaceBranchReconcileResult> => {
|
||||
const existingRow = await db
|
||||
|
|
@ -1360,6 +1363,11 @@ export function executionWorkspaceService(db: Db) {
|
|||
|
||||
const reason = readNullableString(input.reason);
|
||||
const now = new Date();
|
||||
const allowActiveWorkspace =
|
||||
input.mode === "forward" &&
|
||||
input.actor.actorType === "system" &&
|
||||
input.actor.actorId === "workspace_runtime" &&
|
||||
Boolean(input.actor.runId);
|
||||
return db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
// Runtime-service activation takes this same row lock before spawning
|
||||
|
|
@ -1422,6 +1430,7 @@ export function executionWorkspaceService(db: Db) {
|
|||
workspaceStatus: lockedWorkspace.status,
|
||||
inspection,
|
||||
runtimeServices: lockedRuntimeServices,
|
||||
allowActiveWorkspace,
|
||||
});
|
||||
if (lockedWorkspace.branchName !== inspection.fromBranch) {
|
||||
throw unprocessable("Execution workspace branch changed during reconciliation; retry with a fresh inspection", {
|
||||
|
|
@ -1444,7 +1453,9 @@ export function executionWorkspaceService(db: Db) {
|
|||
.where(
|
||||
and(
|
||||
eq(executionWorkspaces.id, lockedWorkspace.id),
|
||||
eq(executionWorkspaces.status, "idle"),
|
||||
allowActiveWorkspace
|
||||
? inArray(executionWorkspaces.status, ["idle", "active"])
|
||||
: eq(executionWorkspaces.status, "idle"),
|
||||
eq(executionWorkspaces.branchName, inspection.fromBranch),
|
||||
noActiveRuntimeServicesForWorkspaceCondition(lockedRow),
|
||||
),
|
||||
|
|
@ -1461,13 +1472,14 @@ export function executionWorkspaceService(db: Db) {
|
|||
workspaceStatus: lockedWorkspace.status,
|
||||
inspection,
|
||||
runtimeServices: latestRuntimeServices,
|
||||
allowActiveWorkspace,
|
||||
});
|
||||
throw unprocessable("Execution workspace branch reconciliation requires the workspace to stay idle with stopped runtime services during the update", {
|
||||
inspection,
|
||||
});
|
||||
}
|
||||
|
||||
const recoveryAction = await recoveryActionsSvc.resolveActiveForIssue(
|
||||
let recoveryAction = await recoveryActionsSvc.resolveActiveForIssue(
|
||||
{
|
||||
companyId: lockedWorkspace.companyId,
|
||||
sourceIssueId: lockedWorkspace.sourceIssueId,
|
||||
|
|
@ -1480,6 +1492,25 @@ export function executionWorkspaceService(db: Db) {
|
|||
},
|
||||
tx,
|
||||
);
|
||||
if (!recoveryAction) {
|
||||
for (const alternateFingerprint of input.alternateRecoveryFingerprints ?? []) {
|
||||
if (!alternateFingerprint || alternateFingerprint === inspection.fingerprint) continue;
|
||||
recoveryAction = await recoveryActionsSvc.resolveActiveForIssue(
|
||||
{
|
||||
companyId: existing.companyId,
|
||||
sourceIssueId: existing.sourceIssueId!,
|
||||
kind: "workspace_validation",
|
||||
cause: WORKSPACE_VALIDATION_RECOVERY_CAUSE,
|
||||
fingerprint: alternateFingerprint,
|
||||
status: "resolved",
|
||||
outcome: "restored",
|
||||
resolutionNote: `Execution workspace branch record reconciled from "${inspection.fromBranch}" to "${inspection.toBranch}".`,
|
||||
},
|
||||
tx,
|
||||
);
|
||||
if (recoveryAction) break;
|
||||
}
|
||||
}
|
||||
|
||||
const [auditComment] = await tx
|
||||
.insert(issueComments)
|
||||
|
|
|
|||
|
|
@ -105,6 +105,7 @@ import {
|
|||
inspectManagedGitWorktreeBranch,
|
||||
persistAdapterManagedRuntimeServices,
|
||||
realizeExecutionWorkspace,
|
||||
reconcilePendingForwardBranchAfterPersistence,
|
||||
releaseRuntimeServicesForRun,
|
||||
type ExecutionWorkspaceInput,
|
||||
type RealizedExecutionWorkspace,
|
||||
|
|
@ -10779,6 +10780,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
workspaceConfigFreshness,
|
||||
restoreExistingWorkspace: reusableExistingExecutionWorkspace
|
||||
? () => ensurePersistedExecutionWorkspaceAvailable({
|
||||
db,
|
||||
base: executionWorkspaceBase,
|
||||
workspace: {
|
||||
id: reusableExistingExecutionWorkspace.id,
|
||||
|
|
@ -10806,10 +10808,14 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
name: agent.name,
|
||||
companyId: agent.companyId,
|
||||
},
|
||||
heartbeatRunId: run.id,
|
||||
enableWorkspaceBranchReconcileForward:
|
||||
resolvedInstanceSettings.experimental.enableWorkspaceBranchReconcileForward,
|
||||
recorder: workspaceOperationRecorder,
|
||||
})
|
||||
: null,
|
||||
realizeWorkspace: () => realizeExecutionWorkspace({
|
||||
db,
|
||||
base: executionWorkspaceBase,
|
||||
config: hostExecutionWorkspaceConfig,
|
||||
issue: issueRef,
|
||||
|
|
@ -10818,6 +10824,9 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
name: agent.name,
|
||||
companyId: agent.companyId,
|
||||
},
|
||||
heartbeatRunId: run.id,
|
||||
enableWorkspaceBranchReconcileForward:
|
||||
resolvedInstanceSettings.experimental.enableWorkspaceBranchReconcileForward,
|
||||
recorder: workspaceOperationRecorder,
|
||||
}),
|
||||
});
|
||||
|
|
@ -10839,13 +10848,16 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
baseRef: executionWorkspace.repoRef,
|
||||
baseRefSha: executionWorkspace.baseRefSha ?? null,
|
||||
});
|
||||
const pendingForwardBranchReconcile = executionWorkspace.pendingForwardBranchReconcile ?? null;
|
||||
const branchNameForInitialPersistence =
|
||||
pendingForwardBranchReconcile?.recordedBranchName ?? executionWorkspace.branchName;
|
||||
try {
|
||||
persistedExecutionWorkspace = resolvedWorkspaceReusePolicy.shouldRestoreExistingWorkspace && reusableExistingExecutionWorkspace
|
||||
? await executionWorkspacesSvc.update(reusableExistingExecutionWorkspace.id, {
|
||||
cwd: executionWorkspace.cwd,
|
||||
repoUrl: executionWorkspace.repoUrl,
|
||||
baseRef: executionWorkspace.repoRef,
|
||||
branchName: executionWorkspace.branchName,
|
||||
branchName: branchNameForInitialPersistence,
|
||||
providerType: executionWorkspace.strategy === "git_worktree" ? "git_worktree" : "local_fs",
|
||||
providerRef: executionWorkspace.worktreePath,
|
||||
status: "active",
|
||||
|
|
@ -10867,12 +10879,12 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
? "adapter_managed"
|
||||
: "shared_workspace",
|
||||
strategyType: executionWorkspace.strategy === "git_worktree" ? "git_worktree" : "project_primary",
|
||||
name: executionWorkspace.branchName ?? issueRef?.identifier ?? `workspace-${agent.id.slice(0, 8)}`,
|
||||
name: branchNameForInitialPersistence ?? issueRef?.identifier ?? `workspace-${agent.id.slice(0, 8)}`,
|
||||
status: "active",
|
||||
cwd: executionWorkspace.cwd,
|
||||
repoUrl: executionWorkspace.repoUrl,
|
||||
baseRef: executionWorkspace.repoRef,
|
||||
branchName: executionWorkspace.branchName,
|
||||
branchName: branchNameForInitialPersistence,
|
||||
providerType: executionWorkspace.strategy === "git_worktree" ? "git_worktree" : "local_fs",
|
||||
providerRef: executionWorkspace.worktreePath,
|
||||
lastUsedAt: new Date(),
|
||||
|
|
@ -10925,6 +10937,18 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
}
|
||||
throw error;
|
||||
}
|
||||
if (persistedExecutionWorkspace && pendingForwardBranchReconcile) {
|
||||
await workspaceOperationRecorder.attachExecutionWorkspaceId(persistedExecutionWorkspace.id);
|
||||
const reconcileResult = await reconcilePendingForwardBranchAfterPersistence({
|
||||
db,
|
||||
executionWorkspaceId: persistedExecutionWorkspace.id,
|
||||
pending: pendingForwardBranchReconcile,
|
||||
heartbeatRunId: run.id,
|
||||
reconcileOperationPhase: "worktree_prepare",
|
||||
recorder: workspaceOperationRecorder,
|
||||
});
|
||||
persistedExecutionWorkspace = reconcileResult.workspace;
|
||||
}
|
||||
await workspaceOperationRecorder.attachExecutionWorkspaceId(persistedExecutionWorkspace?.id ?? null);
|
||||
await recordWorkspaceConfigFreshnessOperation({
|
||||
recorder: workspaceOperationRecorder,
|
||||
|
|
@ -11568,8 +11592,10 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
let inspection = branchInspection.inspection;
|
||||
const initialManagedGitWorktreeBranch = formatManagedGitWorktreeBranchInspection(inspection);
|
||||
if (!inspection.valid && inspection.reasonCode === "branch_mismatch" && inspection.repoRoot) {
|
||||
let reconciledBranchName: string | null = null;
|
||||
try {
|
||||
await ensureGitWorktreeBranchCoherent({
|
||||
const coherence = await ensureGitWorktreeBranchCoherent({
|
||||
db,
|
||||
repoRoot: inspection.repoRoot,
|
||||
worktreePath: inspection.worktreePath,
|
||||
expectedBranchName: inspection.expectedBranchName,
|
||||
|
|
@ -11580,11 +11606,18 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
identifier: issueRef.identifier,
|
||||
title: issueRef.title,
|
||||
workMode: issueRef.workMode,
|
||||
}
|
||||
}
|
||||
: null,
|
||||
executionWorkspaceId: branchInspection.workspaceRecord.id,
|
||||
heartbeatRunId: run.id,
|
||||
enableWorkspaceBranchReconcileForward:
|
||||
resolvedInstanceSettings.experimental.enableWorkspaceBranchReconcileForward,
|
||||
reconcileOperationPhase: "workspace_finalize",
|
||||
recorder: workspaceOperationRecorder,
|
||||
});
|
||||
if (coherence.reconciledForward && coherence.branchName) {
|
||||
reconciledBranchName = coherence.branchName;
|
||||
}
|
||||
} catch (repairErr) {
|
||||
const workspaceValidationFailure = isWorkspaceValidationFailure(repairErr) ? repairErr : null;
|
||||
finalizeBranchMetadata = {
|
||||
|
|
@ -11621,7 +11654,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
|
|||
|
||||
const repairedInspection = await inspectManagedGitWorktreeBranch({
|
||||
worktreePath: inspection.worktreePath,
|
||||
expectedBranchName: inspection.expectedBranchName,
|
||||
expectedBranchName: reconciledBranchName ?? inspection.expectedBranchName,
|
||||
repoRoot: inspection.repoRoot,
|
||||
});
|
||||
finalizeBranchRepairMetadata = {
|
||||
|
|
|
|||
|
|
@ -29,7 +29,8 @@ import {
|
|||
writeLocalServiceRegistryRecord,
|
||||
} from "./local-service-supervisor.js";
|
||||
import type { WorkspaceOperationRecorder } from "./workspace-operations.js";
|
||||
import { readExecutionWorkspaceConfig } from "./execution-workspaces.js";
|
||||
import { executionWorkspaceService, readExecutionWorkspaceConfig } from "./execution-workspaces.js";
|
||||
import { logActivity } from "./activity-log.js";
|
||||
import { readProjectWorkspaceRuntimeConfig } from "./project-workspace-runtime-config.js";
|
||||
|
||||
export function resolveShell(): string {
|
||||
|
|
@ -70,6 +71,7 @@ export interface RealizedExecutionWorkspace extends ExecutionWorkspaceInput {
|
|||
warnings: string[];
|
||||
created: boolean;
|
||||
baseRefSha?: string | null;
|
||||
pendingForwardBranchReconcile?: PendingForwardBranchReconcile | null;
|
||||
}
|
||||
|
||||
export class WorkspaceRuntimeValidationFailure extends Error {
|
||||
|
|
@ -659,6 +661,19 @@ type GitWorktreeCleanliness = SharedGitWorktreeBranchIncoherenceEvidence["cleanl
|
|||
|
||||
type GitWorktreeBranchIncoherenceEvidence = SharedGitWorktreeBranchIncoherenceEvidence;
|
||||
|
||||
type GitWorktreeBranchCoherenceResult = {
|
||||
branchName: string | null;
|
||||
reconciledForward: boolean;
|
||||
pendingForwardBranchReconcile?: PendingForwardBranchReconcile | null;
|
||||
};
|
||||
|
||||
export type PendingForwardBranchReconcile = {
|
||||
recordedBranchName: string;
|
||||
adoptedBranchName: string;
|
||||
prePersistenceFingerprint: string;
|
||||
reason: string;
|
||||
};
|
||||
|
||||
function formatBranchForMessage(branch: string | null | undefined) {
|
||||
return branch && branch.length > 0 ? branch : "<detached>";
|
||||
}
|
||||
|
|
@ -845,22 +860,183 @@ function branchIncoherenceValidationFailure(evidence: GitWorktreeBranchIncoheren
|
|||
);
|
||||
}
|
||||
|
||||
async function recordForwardBranchReconcileOperation(input: {
|
||||
recorder?: WorkspaceOperationRecorder | null;
|
||||
phase?: "worktree_prepare" | "workspace_finalize";
|
||||
cwd: string;
|
||||
repoRoot: string;
|
||||
worktreePath: string;
|
||||
expectedBranchName: string;
|
||||
actualBranchName: string;
|
||||
executionWorkspaceId: string | null;
|
||||
sourceIssueId: string | null;
|
||||
fingerprint: string;
|
||||
expectedHeadSha: string | null;
|
||||
actualHeadSha: string | null;
|
||||
ancestryVerdict: GitWorktreeBranchAncestryVerdict;
|
||||
mode: "record_updated" | "adopt_for_realize";
|
||||
auditCommentId?: string | null;
|
||||
recoveryActionId?: string | null;
|
||||
}) {
|
||||
if (!input.recorder) return;
|
||||
|
||||
await input.recorder.recordOperation({
|
||||
phase: input.phase ?? "worktree_prepare",
|
||||
cwd: input.cwd,
|
||||
metadata: {
|
||||
repoRoot: input.repoRoot,
|
||||
worktreePath: input.worktreePath,
|
||||
expectedBranchName: input.expectedBranchName,
|
||||
actualBranchName: input.actualBranchName,
|
||||
branchIncoherenceReconcileForward: true,
|
||||
reconcileMode: input.mode,
|
||||
fingerprint: input.fingerprint,
|
||||
sourceIssueId: input.sourceIssueId,
|
||||
executionWorkspaceId: input.executionWorkspaceId,
|
||||
expectedHeadSha: input.expectedHeadSha,
|
||||
actualHeadSha: input.actualHeadSha,
|
||||
ancestryVerdict: input.ancestryVerdict,
|
||||
auditCommentId: input.auditCommentId ?? null,
|
||||
recoveryActionId: input.recoveryActionId ?? null,
|
||||
},
|
||||
run: async () => ({
|
||||
status: "succeeded",
|
||||
system:
|
||||
input.mode === "record_updated"
|
||||
? `Reconciled execution workspace branch record from ${input.expectedBranchName} to ${input.actualBranchName}; worktree left unchanged.\n`
|
||||
: `Adopted live git worktree branch ${input.actualBranchName} for this execution workspace realization; worktree left unchanged.\n`,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
async function logForwardBranchReconcileActivity(input: {
|
||||
db: Db;
|
||||
companyId: string;
|
||||
executionWorkspaceId: string;
|
||||
sourceIssueId: string | null;
|
||||
runId: string | null;
|
||||
mode: "forward";
|
||||
reason: string | null;
|
||||
fromBranch: string;
|
||||
toBranch: string;
|
||||
fromSha: string | null;
|
||||
toSha: string | null;
|
||||
ancestryVerdict: GitWorktreeBranchAncestryVerdict;
|
||||
fingerprint: string;
|
||||
auditCommentId: string | null;
|
||||
recoveryActionId: string | null;
|
||||
}) {
|
||||
await logActivity(input.db, {
|
||||
companyId: input.companyId,
|
||||
actorType: "system",
|
||||
actorId: "workspace_runtime",
|
||||
runId: input.runId,
|
||||
action: "execution_workspace.branch_reconciled",
|
||||
entityType: "execution_workspace",
|
||||
entityId: input.executionWorkspaceId,
|
||||
details: {
|
||||
mode: input.mode,
|
||||
reason: input.reason,
|
||||
fromBranch: input.fromBranch,
|
||||
toBranch: input.toBranch,
|
||||
fromSha: input.fromSha,
|
||||
toSha: input.toSha,
|
||||
ancestryVerdict: input.ancestryVerdict,
|
||||
fingerprint: input.fingerprint,
|
||||
sourceIssueId: input.sourceIssueId,
|
||||
auditCommentId: input.auditCommentId,
|
||||
recoveryActionId: input.recoveryActionId,
|
||||
actor: {
|
||||
type: "system",
|
||||
id: "workspace_runtime",
|
||||
source: "workspace_runtime",
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export async function reconcilePendingForwardBranchAfterPersistence(input: {
|
||||
db: Db;
|
||||
executionWorkspaceId: string;
|
||||
pending: PendingForwardBranchReconcile;
|
||||
heartbeatRunId?: string | null;
|
||||
reconcileOperationPhase?: "worktree_prepare" | "workspace_finalize";
|
||||
recorder?: WorkspaceOperationRecorder | null;
|
||||
}) {
|
||||
const result = await executionWorkspaceService(input.db).reconcileExecutionWorkspaceBranch(
|
||||
input.executionWorkspaceId,
|
||||
{
|
||||
mode: "forward",
|
||||
reason: input.pending.reason,
|
||||
alternateRecoveryFingerprints: [input.pending.prePersistenceFingerprint],
|
||||
actor: {
|
||||
actorType: "system",
|
||||
actorId: "workspace_runtime",
|
||||
agentId: null,
|
||||
runId: input.heartbeatRunId ?? null,
|
||||
},
|
||||
},
|
||||
);
|
||||
await logForwardBranchReconcileActivity({
|
||||
db: input.db,
|
||||
companyId: result.workspace.companyId,
|
||||
executionWorkspaceId: result.workspace.id,
|
||||
sourceIssueId: result.workspace.sourceIssueId,
|
||||
runId: input.heartbeatRunId ?? null,
|
||||
mode: "forward",
|
||||
reason: input.pending.reason,
|
||||
fromBranch: result.inspection.fromBranch,
|
||||
toBranch: result.inspection.toBranch,
|
||||
fromSha: result.inspection.fromSha,
|
||||
toSha: result.inspection.toSha,
|
||||
ancestryVerdict: result.inspection.ancestryVerdict,
|
||||
fingerprint: result.inspection.fingerprint,
|
||||
auditCommentId: result.auditCommentId,
|
||||
recoveryActionId: result.recoveryAction?.id ?? null,
|
||||
});
|
||||
await recordForwardBranchReconcileOperation({
|
||||
recorder: input.recorder,
|
||||
phase: input.reconcileOperationPhase,
|
||||
cwd: result.inspection.worktreePath,
|
||||
repoRoot: result.inspection.repoRoot,
|
||||
worktreePath: result.inspection.worktreePath,
|
||||
expectedBranchName: result.inspection.fromBranch,
|
||||
actualBranchName: result.inspection.toBranch,
|
||||
executionWorkspaceId: result.workspace.id,
|
||||
sourceIssueId: result.workspace.sourceIssueId,
|
||||
fingerprint: result.inspection.fingerprint,
|
||||
expectedHeadSha: result.inspection.fromSha,
|
||||
actualHeadSha: result.inspection.toSha,
|
||||
ancestryVerdict: result.inspection.ancestryVerdict,
|
||||
mode: "adopt_for_realize",
|
||||
auditCommentId: result.auditCommentId,
|
||||
recoveryActionId: result.recoveryAction?.id ?? null,
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function ensureGitWorktreeBranchCoherent(input: {
|
||||
db?: Db | null;
|
||||
repoRoot: string;
|
||||
worktreePath: string;
|
||||
expectedBranchName: string | null;
|
||||
sourceIssue: ExecutionWorkspaceIssueRef | null;
|
||||
executionWorkspaceId?: string | null;
|
||||
actualBranchName?: string | null;
|
||||
heartbeatRunId?: string | null;
|
||||
enableWorkspaceBranchReconcileForward?: boolean;
|
||||
reconcileOperationPhase?: "worktree_prepare" | "workspace_finalize";
|
||||
recorder?: WorkspaceOperationRecorder | null;
|
||||
}) {
|
||||
}): Promise<GitWorktreeBranchCoherenceResult> {
|
||||
const expectedBranchName = input.expectedBranchName?.trim();
|
||||
if (!expectedBranchName) return;
|
||||
if (!expectedBranchName) return { branchName: null, reconciledForward: false };
|
||||
|
||||
const currentBranch = input.actualBranchName !== undefined
|
||||
? input.actualBranchName
|
||||
: await runGit(["symbolic-ref", "--quiet", "--short", "HEAD"], input.worktreePath).catch(() => null);
|
||||
if (currentBranch === expectedBranchName) return;
|
||||
if (currentBranch === expectedBranchName) {
|
||||
return { branchName: expectedBranchName, reconciledForward: false };
|
||||
}
|
||||
|
||||
const evidence = await inspectGitWorktreeBranchIncoherence({
|
||||
repoRoot: input.repoRoot,
|
||||
|
|
@ -871,6 +1047,90 @@ export async function ensureGitWorktreeBranchCoherent(input: {
|
|||
executionWorkspaceId: input.executionWorkspaceId ?? null,
|
||||
});
|
||||
|
||||
if (
|
||||
input.enableWorkspaceBranchReconcileForward === true &&
|
||||
evidence.provenance.ancestryVerdict === "ancestor" &&
|
||||
currentBranch
|
||||
) {
|
||||
const reason = "Automatic forward reconciliation: recorded branch is an ancestor of the checked-out branch.";
|
||||
if (input.executionWorkspaceId) {
|
||||
if (!input.db) {
|
||||
evidence.safeRepair.reason = "forward reconciliation requires database access to update the execution workspace record";
|
||||
throw branchIncoherenceValidationFailure(evidence);
|
||||
}
|
||||
try {
|
||||
const result = await executionWorkspaceService(input.db).reconcileExecutionWorkspaceBranch(
|
||||
input.executionWorkspaceId,
|
||||
{
|
||||
mode: "forward",
|
||||
reason,
|
||||
actor: {
|
||||
actorType: "system",
|
||||
actorId: "workspace_runtime",
|
||||
agentId: null,
|
||||
runId: input.heartbeatRunId ?? null,
|
||||
},
|
||||
},
|
||||
);
|
||||
await logForwardBranchReconcileActivity({
|
||||
db: input.db,
|
||||
companyId: result.workspace.companyId,
|
||||
executionWorkspaceId: result.workspace.id,
|
||||
sourceIssueId: result.workspace.sourceIssueId ?? evidence.sourceIssueId ?? null,
|
||||
runId: input.heartbeatRunId ?? null,
|
||||
mode: "forward",
|
||||
reason,
|
||||
fromBranch: result.inspection.fromBranch,
|
||||
toBranch: result.inspection.toBranch,
|
||||
fromSha: result.inspection.fromSha,
|
||||
toSha: result.inspection.toSha,
|
||||
ancestryVerdict: result.inspection.ancestryVerdict,
|
||||
fingerprint: result.inspection.fingerprint,
|
||||
auditCommentId: result.auditCommentId,
|
||||
recoveryActionId: result.recoveryAction?.id ?? null,
|
||||
});
|
||||
await recordForwardBranchReconcileOperation({
|
||||
recorder: input.recorder,
|
||||
phase: input.reconcileOperationPhase,
|
||||
cwd: input.worktreePath,
|
||||
repoRoot: result.inspection.repoRoot,
|
||||
worktreePath: result.inspection.worktreePath,
|
||||
expectedBranchName: result.inspection.fromBranch,
|
||||
actualBranchName: result.inspection.toBranch,
|
||||
executionWorkspaceId: result.workspace.id,
|
||||
sourceIssueId: result.workspace.sourceIssueId ?? evidence.sourceIssueId ?? null,
|
||||
fingerprint: result.inspection.fingerprint,
|
||||
expectedHeadSha: result.inspection.fromSha,
|
||||
actualHeadSha: result.inspection.toSha,
|
||||
ancestryVerdict: result.inspection.ancestryVerdict,
|
||||
mode: "record_updated",
|
||||
auditCommentId: result.auditCommentId,
|
||||
recoveryActionId: result.recoveryAction?.id ?? null,
|
||||
});
|
||||
return { branchName: result.inspection.toBranch, reconciledForward: true };
|
||||
} catch (error) {
|
||||
evidence.safeRepair.reason =
|
||||
`forward reconciliation failed: ${error instanceof Error ? error.message : String(error)}`;
|
||||
throw branchIncoherenceValidationFailure(evidence);
|
||||
}
|
||||
}
|
||||
|
||||
if (!input.db) {
|
||||
evidence.safeRepair.reason = "forward reconciliation adoption requires database access to audit after workspace realization";
|
||||
throw branchIncoherenceValidationFailure(evidence);
|
||||
}
|
||||
return {
|
||||
branchName: currentBranch,
|
||||
reconciledForward: true,
|
||||
pendingForwardBranchReconcile: {
|
||||
recordedBranchName: expectedBranchName,
|
||||
adoptedBranchName: currentBranch,
|
||||
prePersistenceFingerprint: evidence.fingerprint,
|
||||
reason,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (!evidence.safeRepair.eligible) {
|
||||
throw branchIncoherenceValidationFailure(evidence);
|
||||
}
|
||||
|
|
@ -910,6 +1170,7 @@ export async function ensureGitWorktreeBranchCoherent(input: {
|
|||
|
||||
evidence.safeRepair.succeeded = true;
|
||||
evidence.safeRepair.reason = "clean worktree checked out the recorded branch";
|
||||
return { branchName: expectedBranchName, reconciledForward: false };
|
||||
}
|
||||
|
||||
// Resolve the authoritative base ref for a fresh worktree. A configured local
|
||||
|
|
@ -1603,10 +1864,13 @@ async function resolveGitRepoRootForWorkspaceCleanup(
|
|||
}
|
||||
|
||||
export async function realizeExecutionWorkspace(input: {
|
||||
db?: Db | null;
|
||||
base: ExecutionWorkspaceInput;
|
||||
config: Record<string, unknown>;
|
||||
issue: ExecutionWorkspaceIssueRef | null;
|
||||
agent: ExecutionWorkspaceAgentRef;
|
||||
heartbeatRunId?: string | null;
|
||||
enableWorkspaceBranchReconcileForward?: boolean;
|
||||
recorder?: WorkspaceOperationRecorder | null;
|
||||
}): Promise<RealizedExecutionWorkspace> {
|
||||
const rawStrategy = parseObject(input.config.workspaceStrategy);
|
||||
|
|
@ -1632,12 +1896,13 @@ export async function realizeExecutionWorkspace(input: {
|
|||
projectId: input.base.projectId,
|
||||
repoRef: input.base.repoRef,
|
||||
});
|
||||
const branchName = sanitizeBranchName(renderedBranch);
|
||||
let branchName = sanitizeBranchName(renderedBranch);
|
||||
const configuredParentDir = asString(rawStrategy.worktreeParentDir, "");
|
||||
const worktreeParentDir = configuredParentDir
|
||||
? resolveConfiguredPath(configuredParentDir, repoRoot)
|
||||
: path.join(repoRoot, ".paperclip", "worktrees");
|
||||
const worktreePath = path.join(worktreeParentDir, branchName);
|
||||
let pendingForwardBranchReconcile: PendingForwardBranchReconcile | null = null;
|
||||
const configuredBaseRef = typeof rawStrategy.baseRef === "string" && rawStrategy.baseRef.length > 0
|
||||
? rawStrategy.baseRef
|
||||
: input.base.repoRef ?? null;
|
||||
|
|
@ -1715,6 +1980,7 @@ export async function realizeExecutionWorkspace(input: {
|
|||
warnings: [...baseRefreshWarnings, ...baseDrift.warnings],
|
||||
created: false,
|
||||
baseRefSha: refresh.baseRefSha ?? baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha,
|
||||
pendingForwardBranchReconcile,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -1725,15 +1991,23 @@ export async function realizeExecutionWorkspace(input: {
|
|||
expectedBranchName: branchName,
|
||||
}).catch(() => null);
|
||||
if (validation && !validation.valid && validation.reasonCode === "branch_mismatch") {
|
||||
await ensureGitWorktreeBranchCoherent({
|
||||
const coherence = await ensureGitWorktreeBranchCoherent({
|
||||
db: input.db ?? null,
|
||||
repoRoot,
|
||||
worktreePath: reusablePath,
|
||||
expectedBranchName: branchName,
|
||||
actualBranchName: validation.actualBranchName ?? null,
|
||||
sourceIssue: input.issue,
|
||||
executionWorkspaceId: null,
|
||||
heartbeatRunId: input.heartbeatRunId ?? null,
|
||||
enableWorkspaceBranchReconcileForward: input.enableWorkspaceBranchReconcileForward === true,
|
||||
reconcileOperationPhase: "worktree_prepare",
|
||||
recorder: input.recorder ?? null,
|
||||
});
|
||||
if (coherence.reconciledForward && coherence.branchName) {
|
||||
branchName = coherence.branchName;
|
||||
pendingForwardBranchReconcile = coherence.pendingForwardBranchReconcile ?? null;
|
||||
}
|
||||
return await validateLinkedGitWorktree({
|
||||
repoRoot,
|
||||
worktreePath: reusablePath,
|
||||
|
|
@ -1837,6 +2111,7 @@ export async function realizeExecutionWorkspace(input: {
|
|||
}
|
||||
|
||||
export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
db?: Db | null;
|
||||
base: ExecutionWorkspaceInput;
|
||||
workspace: {
|
||||
id?: string | null;
|
||||
|
|
@ -1856,6 +2131,8 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
|||
};
|
||||
issue: ExecutionWorkspaceIssueRef | null;
|
||||
agent: ExecutionWorkspaceAgentRef;
|
||||
heartbeatRunId?: string | null;
|
||||
enableWorkspaceBranchReconcileForward?: boolean;
|
||||
recorder?: WorkspaceOperationRecorder | null;
|
||||
}): Promise<RealizedExecutionWorkspace | null> {
|
||||
const cwd = asString(input.workspace.cwd ?? input.workspace.providerRef, "").trim();
|
||||
|
|
@ -1891,14 +2168,21 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
|||
const reuseBaseRef = input.workspace.baseRef ?? input.base.repoRef ?? null;
|
||||
const reuseWorktreePath = realized.worktreePath ?? cwd;
|
||||
if (await isGitCheckout(reuseWorktreePath)) {
|
||||
await ensureGitWorktreeBranchCoherent({
|
||||
const coherence = await ensureGitWorktreeBranchCoherent({
|
||||
db: input.db ?? null,
|
||||
repoRoot,
|
||||
worktreePath: reuseWorktreePath,
|
||||
expectedBranchName: realized.branchName,
|
||||
sourceIssue: input.issue,
|
||||
executionWorkspaceId: input.workspace.id ?? null,
|
||||
heartbeatRunId: input.heartbeatRunId ?? null,
|
||||
enableWorkspaceBranchReconcileForward: input.enableWorkspaceBranchReconcileForward === true,
|
||||
reconcileOperationPhase: "worktree_prepare",
|
||||
recorder: input.recorder ?? null,
|
||||
});
|
||||
if (coherence.reconciledForward && coherence.branchName) {
|
||||
realized.branchName = coherence.branchName;
|
||||
}
|
||||
}
|
||||
const validation = await validateLinkedGitWorktree({
|
||||
repoRoot,
|
||||
|
|
|
|||
Loading…
Reference in New Issue