From 35aaaa0bd03063e0a7a79e8cb43f281f72823a35 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Mon, 10 Aug 2026 17:01:33 -0700 Subject: [PATCH] feat(server): preserve task timestamps and hierarchy through company import/export (#11193) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company export/import moves a whole company — agents, tasks, comments — between instances as a portable bundle > - The bundle never carried task timestamps or parent links: the export writes neither, the importer lets database defaults stamp "now", and sub-tasks arrive flattened > - Boards sort by recency, so every imported task showing "created just now" collapses the task list into import order, and the task hierarchy the user built is gone > - This pull request adds created/updated/started/completed/cancelled timestamps and a parent link to the bundle (schema v7), preserves them end to end on import, and keeps comment imports from clobbering a preserved updated time > - The benefit is that an imported company reads like the company the user left: same recency order, same task tree ## Linked Issues or Issue Description **What happened?** After a company import, every task showed as created at import time. Recency sorting collapsed to import order, and parent/child task nesting disappeared. The user called out losing "the meaningful task hierarchy and recency sorting". Cause: the export bundle has no fields for task timestamps or parent links, the importer lets `defaultNow()` win on insert, and the comment importer bumps every touched task's `updatedAt` to now. **Expected behavior** An imported company preserves each task's creation/update/start/completion times and its position in the task tree, so sorting and nesting on the destination match the source. **Steps to reproduce** 1. On a source instance, create tasks over several days, including sub-tasks nested under parents. 2. Export the company and import it into another instance. 3. Every task shows the import moment as its creation/update time and all tasks are top-level. ## What Changed - Export writes `createdAt`/`updatedAt`/`startedAt`/`completedAt`/`cancelledAt` (ISO, only when set) and `parent: ` into each task's bundle extension; a parent outside the export selection drops the edge with an aggregate warning, mirroring the existing blocker-edge warning (`server/src/services/company-portability.ts`). - Bundle schema version 6 → 7. All new fields are optional: v5/v6 bundles import unchanged with a version-aware downlevel warning; bundles newer than the board still fail closed. - Manifest parsing validates the new timestamps like comment timestamps (invalid → warn and ignore, never a hard failure); shared types and the zod validator carry the new optional fields. - Import resolves parent slugs to pre-generated destination ids, drops self-references and cycles from tampered bundles with warnings, and orders rows parents-first because the self-referencing FK is checked per insert chunk. - `importIssues` writes the preserved timestamps (falling back to insert time when absent; `startedAt` stays null unless bundle-carried, per #11191's semantics) and `parentId`. - `addImportedComments` no longer blanket-bumps `updatedAt = now()`; it takes `GREATEST(updated_at, newest imported comment createdAt)`, so a preserved update time never regresses while unpreserved rows keep the old behavior. ## Verification - `pnpm vitest run server/src/__tests__/company-portability.test.ts server/src/__tests__/company-portability-import-batching.test.ts server/src/__tests__/productivity-review-service.test.ts` — 102 passed, 1 pre-existing opt-in benchmark skip. Includes: full round-trip with exact timestamp equality and a 3-deep parent chain against embedded Postgres; v6 back-compat (defaults + warning); forward-compat rejection (v8); cycle/self-reference/invalid-timestamp tampered-bundle handling; comment-bump preserve-awareness in both directions. - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter @paperclipai/shared typecheck` — clean. ## Risks - **Rollout ordering**: a board on the previous build (max schema v6) refuses bundles exported by this build (stamped v7) — the existing newer-than-supported rejection, working as designed. Cross-instance moves need the importing board upgraded first. Called out here so operators aren't surprised during the transition window. - Parent edges from tampered bundles are dropped with warnings rather than failing the import; blocker relations already behave this way. - Timestamps are data-only; no destination schema migration. Stacked on #11191 (its commit is included here) — merge #11191 first; this PR then shows only the v7 changes. ## Model Used - Claude Fable 5 (`claude-fable-5`) via Claude Code CLI, with extended thinking and tool use (multi-agent implementation with independent verification). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- .../shared/src/types/company-portability.ts | 8 + .../src/validators/company-portability.ts | 6 + ...ompany-portability-import-batching.test.ts | 152 ++++++++++++- .../src/__tests__/company-portability.test.ts | 209 +++++++++++++++++- server/src/services/company-portability.ts | 131 ++++++++++- server/src/services/import-write-types.ts | 12 + server/src/services/issues.ts | 44 +++- 7 files changed, 543 insertions(+), 19 deletions(-) diff --git a/packages/shared/src/types/company-portability.ts b/packages/shared/src/types/company-portability.ts index 3cc0eb4a6f..a32486b42f 100644 --- a/packages/shared/src/types/company-portability.ts +++ b/packages/shared/src/types/company-portability.ts @@ -200,6 +200,14 @@ export interface CompanyPortabilityIssueManifestEntry { workProducts?: CompanyPortabilityIssueWorkProductManifestEntry[]; monitor?: CompanyPortabilityIssueMonitorManifestEntry | null; attachments?: CompanyPortabilityIssueAttachmentManifestEntry[]; + /** Slug of the parent task when it is part of the same bundle (schemaVersion >= 7). */ + parentSlug?: string | null; + /** Preserved source timestamps as ISO strings (schemaVersion >= 7); absent in older bundles. */ + createdAt?: string | null; + updatedAt?: string | null; + startedAt?: string | null; + completedAt?: string | null; + cancelledAt?: string | null; metadata: Record | null; } diff --git a/packages/shared/src/validators/company-portability.ts b/packages/shared/src/validators/company-portability.ts index eb766e8ecf..dcd1bdc4db 100644 --- a/packages/shared/src/validators/company-portability.ts +++ b/packages/shared/src/validators/company-portability.ts @@ -222,6 +222,12 @@ export const portabilityIssueManifestEntrySchema = z.object({ workProducts: z.array(portabilityIssueWorkProductManifestEntrySchema).default([]), monitor: portabilityIssueMonitorManifestEntrySchema.nullable().default(null), attachments: z.array(portabilityIssueAttachmentManifestEntrySchema).default([]), + parentSlug: z.string().min(1).nullable().optional(), + createdAt: z.string().datetime().nullable().optional(), + updatedAt: z.string().datetime().nullable().optional(), + startedAt: z.string().datetime().nullable().optional(), + completedAt: z.string().datetime().nullable().optional(), + cancelledAt: z.string().datetime().nullable().optional(), metadata: z.record(z.string(), z.unknown()).nullable(), }); diff --git a/server/src/__tests__/company-portability-import-batching.test.ts b/server/src/__tests__/company-portability-import-batching.test.ts index 53ed945bc1..0c7226aa74 100644 --- a/server/src/__tests__/company-portability-import-batching.test.ts +++ b/server/src/__tests__/company-portability-import-batching.test.ts @@ -157,7 +157,7 @@ interface SyntheticBundleOptions { documentsPerIssue: number; } -/** Build an inline import bundle with the requested shape, valid at schemaVersion 6. */ +/** Build an inline import bundle with the requested shape, valid at schemaVersion 7. */ function buildSyntheticBundle(options: SyntheticBundleOptions): { rootPath: string; files: Record; @@ -193,7 +193,7 @@ function buildSyntheticBundle(options: SyntheticBundleOptions): { }; } - files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 6, tasks }); + files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 7, tasks }); return { rootPath: "batching-bench", files }; } @@ -223,7 +223,7 @@ function buildTouchedIssuesBundle(issueCount: number): { }; } - files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 6, tasks }); + files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 7, tasks }); return { rootPath: "inbox-flood", files }; } @@ -423,6 +423,152 @@ describeEmbeddedPostgres("company import batches inserts", () => { expect(imported?.startedAt).toBeNull(); }); + it("preserves bundle timestamps and parent hierarchy through a real import", async () => { + const files: Record = {}; + files["COMPANY.md"] = ['---', 'schema: "agentcompanies/v1"', 'name: "Timestamp Fidelity Co"', '---', '', 'Synthetic import bundle.', ''].join("\n"); + const taskFile = (name: string) => ['---', `name: "${name}"`, "kind: task", '---', '', `${name} body.`, ''].join("\n"); + files["tasks/a-child/TASK.md"] = taskFile("Child"); + files["tasks/m-grandchild/TASK.md"] = taskFile("Grandchild"); + files["tasks/z-parent/TASK.md"] = taskFile("Parent"); + files[".paperclip.yaml"] = renderPaperclipYaml({ + schemaVersion: 7, + tasks: { + // The child slug sorts before its parent so the importer's + // parents-first row ordering (not manifest order) must satisfy the + // parent foreign key. + "a-child": { + status: "todo", + priority: "medium", + parent: "z-parent", + createdAt: "2026-01-05T00:00:00.000Z", + updatedAt: "2026-03-05T00:00:00.000Z", + comments: [{ + body: "Older than the preserved updatedAt.", + authorType: "system", + createdAt: "2026-02-01T00:00:00.000Z", + }], + }, + "m-grandchild": { + status: "todo", + priority: "medium", + parent: "a-child", + createdAt: "2026-01-06T00:00:00.000Z", + updatedAt: "2026-01-07T00:00:00.000Z", + }, + "z-parent": { + status: "done", + priority: "medium", + createdAt: "2026-01-01T00:00:00.000Z", + updatedAt: "2026-03-01T00:00:00.000Z", + completedAt: "2026-02-20T00:00:00.000Z", + comments: [{ + body: "Newer than the preserved updatedAt.", + authorType: "system", + createdAt: "2026-04-01T00:00:00.000Z", + }], + }, + }, + }); + + const portability = companyPortabilityService(db); + const result = await portability.importBundle( + { + source: { type: "inline", rootPath: "timestamp-fidelity", files }, + include: { company: true, agents: false, projects: false, issues: true }, + target: { mode: "new_company", newCompanyName: "Timestamp Fidelity Co" }, + collisionStrategy: "rename", + }, + "user-timestamp-fidelity", + ); + expect(result.warnings).toEqual([]); + + const rows = await db + .select({ + id: issues.id, + title: issues.title, + parentId: issues.parentId, + createdAt: issues.createdAt, + updatedAt: issues.updatedAt, + startedAt: issues.startedAt, + completedAt: issues.completedAt, + }) + .from(issues) + .where(eq(issues.companyId, result.company.id)); + const parent = rows.find((row) => row.title === "Parent")!; + const child = rows.find((row) => row.title === "Child")!; + const grandchild = rows.find((row) => row.title === "Grandchild")!; + + // The parent chain of three lands intact. + expect(parent.parentId).toBeNull(); + expect(child.parentId).toBe(parent.id); + expect(grandchild.parentId).toBe(child.id); + + expect(parent.createdAt).toEqual(new Date("2026-01-01T00:00:00.000Z")); + expect(parent.completedAt).toEqual(new Date("2026-02-20T00:00:00.000Z")); + expect(parent.startedAt).toBeNull(); + expect(child.createdAt).toEqual(new Date("2026-01-05T00:00:00.000Z")); + expect(grandchild.createdAt).toEqual(new Date("2026-01-06T00:00:00.000Z")); + expect(grandchild.updatedAt).toEqual(new Date("2026-01-07T00:00:00.000Z")); + + // A comment older than the preserved updatedAt must not regress it... + expect(child.updatedAt).toEqual(new Date("2026-03-05T00:00:00.000Z")); + // ...while a newer imported comment still bumps recency forward. + expect(parent.updatedAt).toEqual(new Date("2026-04-01T00:00:00.000Z")); + }); + + it("imports a v6 bundle without the timestamp fields using import-time defaults", async () => { + // A little slack absorbs clock drift between this process and Postgres. + const importStart = Date.now() - 1_000; + const files: Record = {}; + files["COMPANY.md"] = ['---', 'schema: "agentcompanies/v1"', 'name: "Legacy Shape Co"', '---', '', 'Synthetic import bundle.', ''].join("\n"); + files["tasks/legacy-task/TASK.md"] = ['---', 'name: "Legacy task"', "kind: task", '---', '', 'Legacy body.', ''].join("\n"); + files[".paperclip.yaml"] = renderPaperclipYaml({ + schemaVersion: 6, + tasks: { + "legacy-task": { + status: "todo", + priority: "medium", + comments: [{ + body: "Predates the import.", + authorType: "system", + createdAt: "2026-01-01T00:00:00.000Z", + }], + }, + }, + }); + + const portability = companyPortabilityService(db); + const result = await portability.importBundle( + { + source: { type: "inline", rootPath: "legacy-shape", files }, + include: { company: true, agents: false, projects: false, issues: true }, + target: { mode: "new_company", newCompanyName: "Legacy Shape Co" }, + collisionStrategy: "rename", + }, + "user-legacy-shape", + ); + expect(result.warnings).toContain( + "This package declares schemaVersion 6 and predates task timestamp and parent link transfer; that task data imports only if the bundle carries it.", + ); + + const [row] = await db + .select({ + parentId: issues.parentId, + createdAt: issues.createdAt, + updatedAt: issues.updatedAt, + startedAt: issues.startedAt, + }) + .from(issues) + .where(eq(issues.companyId, result.company.id)); + expect(row?.parentId).toBeNull(); + expect(row?.startedAt).toBeNull(); + // Without preserved timestamps the row keeps the old behavior: created and + // updated at import time, and the comment bump does not drag updatedAt + // back to the older comment createdAt. + expect(row!.createdAt.getTime()).toBeGreaterThanOrEqual(importStart); + expect(row!.updatedAt.getTime()).toBeGreaterThanOrEqual(importStart); + }); + it("rolls back the whole work-product batch when a later chunk fails", async () => { // Seed a real company + issue to hang work products off of. const bundle = buildSyntheticBundle({ issueCount: 1, commentsPerIssue: 0, documentsPerIssue: 0 }); diff --git a/server/src/__tests__/company-portability.test.ts b/server/src/__tests__/company-portability.test.ts index 9631ddd3b7..494087e704 100644 --- a/server/src/__tests__/company-portability.test.ts +++ b/server/src/__tests__/company-portability.test.ts @@ -3579,8 +3579,8 @@ describe("company portability", () => { expect(extension).not.toContain("labelIds"); expect(extension).not.toContain("label-a"); // Fresh exports declare the current bundle shape end-to-end. - expect(extension).toContain("schemaVersion: 6"); - expect(exported.manifest.schemaVersion).toBe(6); + expect(extension).toContain("schemaVersion: 7"); + expect(exported.manifest.schemaVersion).toBe(7); expect(exported.manifest.labels).toEqual([ { name: "bug", color: "#ff0000" }, { name: "urgent", color: "#00ff00" }, @@ -4030,6 +4030,207 @@ describe("company portability", () => { ); }); + function mockTaskHierarchyExportSources() { + projectSvc.list.mockResolvedValue([]); + projectSvc.listWorkspaces.mockResolvedValue([]); + const baseIssue = { + description: null, + projectId: null, + projectWorkspaceId: null, + assigneeAgentId: null, + priority: "medium", + labelIds: [], + billingCode: null, + executionWorkspaceSettings: null, + assigneeAdapterOverrides: null, + }; + issueSvc.list.mockResolvedValue([ + { + ...baseIssue, + id: "issue-1", + identifier: "PAP-1", + title: "Alpha task", + status: "done", + parentId: null, + createdAt: new Date("2026-01-01T00:00:00.000Z"), + updatedAt: new Date("2026-03-01T00:00:00.000Z"), + startedAt: new Date("2026-01-02T00:00:00.000Z"), + completedAt: new Date("2026-02-01T00:00:00.000Z"), + }, + { + ...baseIssue, + id: "issue-2", + identifier: "PAP-2", + title: "Beta task", + status: "todo", + parentId: "issue-1", + createdAt: new Date("2026-01-05T00:00:00.000Z"), + updatedAt: new Date("2026-01-06T00:00:00.000Z"), + }, + { + ...baseIssue, + id: "issue-3", + identifier: "PAP-3", + title: "Gamma task", + status: "todo", + parentId: "issue-2", + createdAt: new Date("2026-01-07T00:00:00.000Z"), + updatedAt: new Date("2026-01-08T00:00:00.000Z"), + }, + { + ...baseIssue, + id: "issue-4", + identifier: "PAP-4", + title: "Delta task", + status: "todo", + parentId: "issue-outside", + }, + ]); + } + + it("carries task timestamps and parent links through export and import", async () => { + const { db } = fakeImportDb(); + const portability = companyPortabilityService(db); + mockTaskHierarchyExportSources(); + + const exported = await portability.exportBundle("company-1", { + include: { company: true, agents: false, projects: false, issues: true }, + }); + + const extension = asTextFile(exported.files[".paperclip.yaml"]); + expect(extension).toContain("schemaVersion: 7"); + expect(extension).toContain('parent: "pap-1"'); + expect(extension).toContain('createdAt: "2026-01-01T00:00:00.000Z"'); + expect(exported.warnings).toContain( + "1 parent relation references a task outside this export and was not included.", + ); + + const alphaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-1"); + const betaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-2"); + const gammaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-3"); + const deltaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-4"); + expect(alphaEntry).toEqual(expect.objectContaining({ + parentSlug: null, + createdAt: "2026-01-01T00:00:00.000Z", + updatedAt: "2026-03-01T00:00:00.000Z", + startedAt: "2026-01-02T00:00:00.000Z", + completedAt: "2026-02-01T00:00:00.000Z", + cancelledAt: null, + })); + expect(betaEntry?.parentSlug).toBe("pap-1"); + expect(gammaEntry?.parentSlug).toBe("pap-2"); + // The unexported parent edge is dropped, not carried by raw id. + expect(deltaEntry?.parentSlug).toBeNull(); + expect(extension).not.toContain("issue-outside"); + + companySvc.create.mockResolvedValue({ id: "company-imported", name: "Imported" }); + accessSvc.ensureMembership.mockResolvedValue(undefined); + agentSvc.list.mockResolvedValue([]); + + await portability.importBundle({ + source: { type: "inline", rootPath: exported.rootPath, files: exported.files }, + include: { company: true, agents: false, projects: false, issues: true }, + target: { mode: "new_company", newCompanyName: "Imported" }, + agents: "all", + collisionStrategy: "rename", + }, "user-1"); + + const importedRows = issueSvc.importIssues.mock.calls[0]![1] as Array<{ + id: string; + title: string; + parentId: string | null; + createdAt: Date | null; + updatedAt: Date | null; + startedAt: Date | null; + completedAt: Date | null; + cancelledAt: Date | null; + }>; + const alphaRow = importedRows.find((row) => row.title === "Alpha task")!; + const betaRow = importedRows.find((row) => row.title === "Beta task")!; + const gammaRow = importedRows.find((row) => row.title === "Gamma task")!; + const deltaRow = importedRows.find((row) => row.title === "Delta task")!; + + // Timestamps survive the round trip as concrete dates. + expect(alphaRow.createdAt).toEqual(new Date("2026-01-01T00:00:00.000Z")); + expect(alphaRow.updatedAt).toEqual(new Date("2026-03-01T00:00:00.000Z")); + expect(alphaRow.startedAt).toEqual(new Date("2026-01-02T00:00:00.000Z")); + expect(alphaRow.completedAt).toEqual(new Date("2026-02-01T00:00:00.000Z")); + expect(alphaRow.cancelledAt).toBeNull(); + expect(betaRow.createdAt).toEqual(new Date("2026-01-05T00:00:00.000Z")); + + // The parent chain of three is intact against the pre-generated ids. + expect(alphaRow.parentId).toBeNull(); + expect(betaRow.parentId).toBe(alphaRow.id); + expect(gammaRow.parentId).toBe(betaRow.id); + expect(deltaRow.parentId).toBeNull(); + + // Rows arrive parents-first so chunked inserts satisfy the parent FK. + expect(importedRows.indexOf(alphaRow)).toBeLessThan(importedRows.indexOf(betaRow)); + expect(importedRows.indexOf(betaRow)).toBeLessThan(importedRows.indexOf(gammaRow)); + }); + + it("drops self-referencing and cyclic parent links from a hand-built bundle with warnings", async () => { + const { db } = fakeImportDb(); + const portability = companyPortabilityService(db); + + const taskFile = (name: string) => ["---", `name: "${name}"`, "kind: task", "---", "", `${name} body.`, ""].join("\n"); + const files = { + "COMPANY.md": ["---", 'schema: "agentcompanies/v1"', 'name: "Tampered Import"', "---", ""].join("\n"), + "tasks/task-a/TASK.md": taskFile("Task A"), + "tasks/task-b/TASK.md": taskFile("Task B"), + "tasks/task-c/TASK.md": taskFile("Task C"), + ".paperclip.yaml": [ + 'schema: "paperclip/v1"', + "schemaVersion: 7", + "tasks:", + " task-a:", + ' status: "todo"', + ' parent: "task-b"', + " task-b:", + ' status: "todo"', + ' parent: "task-a"', + " task-c:", + ' status: "todo"', + ' parent: "task-c"', + ' createdAt: "not-a-timestamp"', + "", + ].join("\n"), + }; + + companySvc.create.mockResolvedValue({ id: "company-imported", name: "Tampered Import" }); + accessSvc.ensureMembership.mockResolvedValue(undefined); + agentSvc.list.mockResolvedValue([]); + + const result = await portability.importBundle({ + source: { type: "inline", rootPath: "tampered-package", files }, + include: { company: true, agents: false, projects: false, issues: true }, + target: { mode: "new_company", newCompanyName: "Tampered Import" }, + agents: "all", + collisionStrategy: "rename", + }, "user-1"); + + const importedRows = issueSvc.importIssues.mock.calls[0]![1] as Array<{ + title: string; + parentId: string | null; + createdAt: Date | null; + }>; + const rowA = importedRows.find((row) => row.title === "Task A")!; + const rowB = importedRows.find((row) => row.title === "Task B")!; + const rowC = importedRows.find((row) => row.title === "Task C")!; + + // One direction of the two-task cycle survives; the closing edge drops. + expect([rowA.parentId, rowB.parentId].filter((parentId) => parentId !== null)).toHaveLength(1); + expect(rowC.parentId).toBeNull(); + expect(rowC.createdAt).toBeNull(); + expect(result.warnings.filter((warning) => warning.includes("would create a parent cycle"))).toHaveLength(2); + expect(result.warnings).toContain( + "Task task-c parent task-c was skipped because it would create a parent cycle.", + ); + expect(result.warnings).toContain( + "Task task-c createdAt was ignored because it is not a valid timestamp.", + ); + }); + const attachmentBytesByObjectKey: Record = { "issues/issue-1/notes.bin": "png-bytes", "issues/issue-1/screenshot.png": "png-bytes", @@ -4625,7 +4826,7 @@ describe("company portability", () => { it("imports unstamped v5 packages with an info warning about task data they predate", async () => { const portability = companyPortabilityService({} as any); const v5Warning = - "This package declares schemaVersion 5 and predates label, blocker, document, work product, monitor, attachment, and embedded image transfer; that task data imports only if the bundle carries it."; + "This package declares schemaVersion 5 and predates label, blocker, document, work product, monitor, attachment, embedded image, task timestamp, and parent link transfer; that task data imports only if the bundle carries it."; companySvc.create.mockResolvedValue({ id: "company-imported", name: "Legacy Import" }); accessSvc.ensureMembership.mockResolvedValue(undefined); @@ -4674,7 +4875,7 @@ describe("company portability", () => { const portability = companyPortabilityService({} as any); await expect(portability.importBundle({ - source: { type: "inline", rootPath: "future-package", files: legacyPackageFiles(["schemaVersion: 7"]) }, + source: { type: "inline", rootPath: "future-package", files: legacyPackageFiles(["schemaVersion: 8"]) }, include: { company: true, agents: false, projects: false, issues: true }, target: { mode: "new_company", newCompanyName: "Future Import" }, agents: "all", diff --git a/server/src/services/company-portability.ts b/server/src/services/company-portability.ts index cdf4cb5630..c86c2bbbef 100644 --- a/server/src/services/company-portability.ts +++ b/server/src/services/company-portability.ts @@ -161,8 +161,10 @@ const DEFAULT_INCLUDE: CompanyPortabilityInclude = { const DEFAULT_COLLISION_STRATEGY: CompanyPortabilityCollisionStrategy = "rename"; // The bundle shape this build reads and writes. Bundles began declaring // their schemaVersion in the .paperclip.yaml extension at 6; undeclared -// bundles are read as 5, the last unstamped shape. -const BUNDLE_SCHEMA_VERSION = 6; +// bundles are read as 5, the last unstamped shape. 7 adds preserved task +// timestamps and parent links; 5/6 bundles still import, with those fields +// falling back to import-time defaults. +const BUNDLE_SCHEMA_VERSION = 7; const UNSTAMPED_BUNDLE_SCHEMA_VERSION = 5; const DEFAULT_IMPORTED_LABEL_COLOR = "#6366f1"; // Blob entries are content-addressed by sha256; the store itself is @@ -899,6 +901,38 @@ function readPortableIssueLabelNames(value: unknown): string[] { return names; } +/** + * Read a preserved issue timestamp from the extension, validated the same way + * comment createdAt is (Date.parse). Invalid values are ignored with a + * warning — never a hard failure — so a hand-edited bundle still imports. + */ +function readPortableIssueTimestamp( + value: unknown, + warnings: string[], + sourceLabel: string, + fieldLabel: string, +): string | null { + if (value === undefined || value === null) return null; + const raw = asString(value); + if (raw && !Number.isNaN(Date.parse(raw))) return raw; + warnings.push(`${sourceLabel} ${fieldLabel} was ignored because it is not a valid timestamp.`); + return null; +} + +/** Render a stored timestamp as a portable ISO string, or omit it when unset/invalid. */ +function toPortableTimestamp(value: Date | string | null | undefined): string | undefined { + if (value == null) return undefined; + const date = value instanceof Date ? value : new Date(value); + return Number.isNaN(date.getTime()) ? undefined : date.toISOString(); +} + +/** Convert a manifest timestamp (already validated at parse time) to a Date, or null. */ +function portableManifestDate(value: string | null | undefined): Date | null { + if (!value) return null; + const date = new Date(value); + return Number.isNaN(date.getTime()) ? null : date; +} + function readPortableIssueBlockedBy(value: unknown): string[] { if (!Array.isArray(value)) return []; const slugs: string[] = []; @@ -3353,6 +3387,12 @@ function buildManifestFromPackageFiles( workProducts: normalizePortableIssueWorkProducts(extension.workProducts), monitor: normalizePortableIssueMonitor(extension.monitor), attachments: normalizePortableIssueAttachments(extension.attachments, warnings, `Task ${slug}`), + parentSlug: asString(extension.parent), + createdAt: readPortableIssueTimestamp(extension.createdAt, warnings, `Task ${slug}`, "createdAt"), + updatedAt: readPortableIssueTimestamp(extension.updatedAt, warnings, `Task ${slug}`, "updatedAt"), + startedAt: readPortableIssueTimestamp(extension.startedAt, warnings, `Task ${slug}`, "startedAt"), + completedAt: readPortableIssueTimestamp(extension.completedAt, warnings, `Task ${slug}`, "completedAt"), + cancelledAt: readPortableIssueTimestamp(extension.cancelledAt, warnings, `Task ${slug}`, "cancelledAt"), metadata: isPlainRecord(extension.metadata) ? extension.metadata : null, }); if (frontmatter.kind && frontmatter.kind !== "task") { @@ -3361,7 +3401,10 @@ function buildManifestFromPackageFiles( } if (bundleSchemaVersion < BUNDLE_SCHEMA_VERSION && manifest.issues.length > 0) { - warnings.push(`This package declares schemaVersion ${bundleSchemaVersion} and predates label, blocker, document, work product, monitor, attachment, and embedded image transfer; that task data imports only if the bundle carries it.`); + const predated = bundleSchemaVersion < 6 + ? "label, blocker, document, work product, monitor, attachment, embedded image, task timestamp, and parent link transfer" + : "task timestamp and parent link transfer"; + warnings.push(`This package declares schemaVersion ${bundleSchemaVersion} and predates ${predated}; that task data imports only if the bundle carries it.`); } manifest.envInputs = dedupeEnvInputs(manifest.envInputs); @@ -4216,6 +4259,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { } let unexportedBlockerEdgeCount = 0; + let unexportedParentEdgeCount = 0; let unportableWorkProductRefCount = 0; const exportedBlobs = new Map(); for (const issue of selectedIssueRows) { @@ -4256,6 +4300,13 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { unexportedBlockerEdgeCount += relationSummaries.blocks .filter((blocked) => !taskSlugByIssueId.has(blocked.id)) .length; + // Parent links travel by task slug like blockers; a parent outside the + // export selection drops the edge and is counted for one warning. + let parentTaskSlug: string | null = null; + if (issue.parentId) { + parentTaskSlug = taskSlugByIssueId.get(issue.parentId) ?? null; + if (!parentTaskSlug) unexportedParentEdgeCount += 1; + } const issueDocumentRows = await documentsSvc.listIssueDocuments(issue.id, { includeSystem: true }); const documentEntries = issueDocumentRows.map((document) => { const documentPath = `tasks/${taskSlug}/documents/${document.key}.md`; @@ -4339,6 +4390,12 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { identifier: issue.identifier, status: issue.status, priority: issue.priority, + parent: parentTaskSlug ?? undefined, + createdAt: toPortableTimestamp(issue.createdAt), + updatedAt: toPortableTimestamp(issue.updatedAt), + startedAt: toPortableTimestamp(issue.startedAt), + completedAt: toPortableTimestamp(issue.completedAt), + cancelledAt: toPortableTimestamp(issue.cancelledAt), // Labels travel by name (their natural key); the bundle-level labels // section carries the matching color definitions. labels: (issue.labelIds ?? []) @@ -4380,6 +4437,9 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { if (unexportedBlockerEdgeCount > 0) { warnings.push(`${unexportedBlockerEdgeCount} blocker relation${unexportedBlockerEdgeCount === 1 ? " references a task" : "s reference tasks"} outside this export and ${unexportedBlockerEdgeCount === 1 ? "was" : "were"} not included.`); } + if (unexportedParentEdgeCount > 0) { + warnings.push(`${unexportedParentEdgeCount} parent relation${unexportedParentEdgeCount === 1 ? " references a task" : "s reference tasks"} outside this export and ${unexportedParentEdgeCount === 1 ? "was" : "were"} not included.`); + } if (unportableWorkProductRefCount > 0) { warnings.push(`${unportableWorkProductRefCount} work product${unportableWorkProductRefCount === 1 ? " references" : "s reference"} execution workspaces or runs that are not portable; those references were omitted from the export.`); } @@ -5721,6 +5781,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { const importedIssueIdBySlug = new Map(); const blockedByBySlug = new Map(); + const parentSlugBySlug = new Map(); let unarmedMonitorCount = 0; let attachmentsSkippedNoStorage = 0; const attachmentMaxBytes = normalizeIssueAttachmentMaxBytes(targetCompany.attachmentMaxBytes ?? null); @@ -5924,6 +5985,9 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { if ((manifestIssue.blockedBy ?? []).length > 0) { blockedByBySlug.set(manifestIssue.slug, manifestIssue.blockedBy ?? []); } + if (manifestIssue.parentSlug) { + parentSlugBySlug.set(manifestIssue.slug, manifestIssue.parentSlug); + } for (const documentEntry of manifestIssue.documents ?? []) { const documentBody = readPortableTextFile(plan.source.files, documentEntry.path); if (documentBody === null) { @@ -6050,9 +6114,70 @@ export function companyPortabilityService(db: Db, storage?: StorageService) { labelIds: resolvedLabelIds, monitorNotes, monitorScheduledBy, + parentId: null, + createdAt: portableManifestDate(manifestIssue.createdAt), + updatedAt: portableManifestDate(manifestIssue.updatedAt), + startedAt: portableManifestDate(manifestIssue.startedAt), + completedAt: portableManifestDate(manifestIssue.completedAt), + cancelledAt: portableManifestDate(manifestIssue.cancelledAt), }); } + // Parent links resolve against the pre-generated ids before the flush + // so each issue row carries its parentId on insert. Edges whose parent + // was not imported, self-references, and cycles (possible only in a + // hand-edited bundle) drop with a warning, mirroring blocker handling. + if (parentSlugBySlug.size > 0) { + const rowBySlug = new Map(issueRows.map((row) => [row.ref, row] as const)); + const acceptedParentById = new Map(); + const wouldCreateParentCycle = (childId: string, parentId: string) => { + let current: string | undefined = parentId; + const visited = new Set(); + while (current) { + if (current === childId) return true; + if (visited.has(current)) return false; + visited.add(current); + current = acceptedParentById.get(current); + } + return false; + }; + for (const [slug, parentSlug] of parentSlugBySlug) { + const row = rowBySlug.get(slug); + if (!row) continue; + const parentId = importedIssueIdBySlug.get(parentSlug); + if (!parentId) { + warnings.push(`Task ${slug} parent ${parentSlug} was skipped because that task was not imported.`); + continue; + } + if (parentId === row.id || wouldCreateParentCycle(row.id, parentId)) { + warnings.push(`Task ${slug} parent ${parentSlug} was skipped because it would create a parent cycle.`); + continue; + } + acceptedParentById.set(row.id, parentId); + row.parentId = parentId; + } + // The parent foreign key is checked per insert statement, so order + // rows parents-first: a child must never land in an earlier chunk + // than its parent. + if (acceptedParentById.size > 0) { + const rowById = new Map(issueRows.map((row) => [row.id, row] as const)); + const orderedRows: typeof issueRows = []; + const emitted = new Set(); + for (const row of issueRows) { + const chain: typeof issueRows = []; + let current: (typeof issueRows)[number] | undefined = row; + while (current && !emitted.has(current.id)) { + emitted.add(current.id); + chain.push(current); + current = current.parentId ? rowById.get(current.parentId) : undefined; + } + for (const entry of chain.reverse()) orderedRows.push(entry); + } + issueRows.length = 0; + issueRows.push(...orderedRows); + } + } + // Flush the buffered rows in dependency order: issues first (parents of // every other row), then comments (attachments may reference them), then // the remaining children. Each writer inserts in chunked multi-row diff --git a/server/src/services/import-write-types.ts b/server/src/services/import-write-types.ts index 264f80f97a..f5d5744984 100644 --- a/server/src/services/import-write-types.ts +++ b/server/src/services/import-write-types.ts @@ -35,6 +35,18 @@ export interface ImportIssueRow { /** Imported monitors land un-armed; only notes/provenance are restored. */ monitorNotes: string | null; monitorScheduledBy: string | null; + /** Parent issue id resolved within the same import batch (pre-generated), or absent for roots. */ + parentId?: string | null; + /** + * Preserved source timestamps carried by schemaVersion >= 7 bundles. + * Absent/null createdAt and updatedAt fall back to the insert time; + * absent startedAt stays null — the writer never fabricates it. + */ + createdAt?: Date | null; + updatedAt?: Date | null; + startedAt?: Date | null; + completedAt?: Date | null; + cancelledAt?: Date | null; } /** A resolved comment row with a pre-generated id, ready for batch insert. */ diff --git a/server/src/services/issues.ts b/server/src/services/issues.ts index 16dbd66207..b87156d913 100644 --- a/server/src/services/issues.ts +++ b/server/src/services/issues.ts @@ -94,7 +94,7 @@ import { resolveIssueGoalId, resolveNextIssueGoalId } from "./issue-goal-fallbac import { getRunLogStore } from "./run-log-store.js"; import { getDefaultCompanyGoal } from "./goals.js"; import { assertAssignableAgent } from "./agent-assignability.js"; -import { insertRowsInChunks } from "./batch-insert.js"; +import { DEFAULT_INSERT_CHUNK_ROWS, insertRowsInChunks } from "./batch-insert.js"; import type { ImportIssueRow, ImportIssueCommentRow, @@ -7352,11 +7352,19 @@ export function issueService(db: Db) { responsibleUserId: null, requestDepth: clampIssueRequestDepth(undefined), originKind: "manual", + // The caller resolves parentId against ids in this same batch, so a + // parent always lands in the same insert (rows arrive parents-first). + parentId: row.parentId ?? null, + // Preserved bundle timestamps win; without them createdAt/updatedAt + // fall back to the insert time (the old defaultNow() behavior). + createdAt: row.createdAt ?? new Date(), + updatedAt: row.updatedAt ?? new Date(), // Imported in-progress work did not start at import time; fabricating - // startedAt here trips duration-based sweeps (e.g. productivity review). - startedAt: null, - completedAt: row.status === "done" ? new Date() : null, - cancelledAt: row.status === "cancelled" ? new Date() : null, + // startedAt here trips duration-based sweeps (e.g. productivity + // review). Only a bundle-carried startedAt is written. + startedAt: row.startedAt ?? null, + completedAt: row.completedAt ?? (row.status === "done" ? new Date() : null), + cancelledAt: row.cancelledAt ?? (row.status === "cancelled" ? new Date() : null), monitorNotes: row.monitorNotes ?? null, monitorScheduledBy: row.monitorScheduledBy ?? null, }); @@ -7396,10 +7404,28 @@ export function issueService(db: Db) { }; }); await insertRowsInChunks(tx, issueComments, commentRows); - // Mirror addComment's recency bump, once per affected issue. - const issueIds = [...new Set(rows.map((row) => row.issueId))]; - if (issueIds.length > 0) { - await tx.update(issues).set({ updatedAt: new Date() }).where(inArray(issues.id, issueIds)); + // Mirror addComment's recency bump, once per affected issue — but never + // backwards. An issue imported with a preserved updatedAt keeps it + // unless a newer imported comment outdates it; issues without preserved + // timestamps carry an insert-time updatedAt, so GREATEST reproduces the + // old "bump to now" behavior for them. + const bumpAtByIssueId = new Map(); + for (const row of commentRows) { + const existing = bumpAtByIssueId.get(row.issueId); + if (!existing || row.createdAt > existing) bumpAtByIssueId.set(row.issueId, row.createdAt); + } + const bumpEntries = [...bumpAtByIssueId.entries()]; + for (let start = 0; start < bumpEntries.length; start += DEFAULT_INSERT_CHUNK_ROWS) { + const chunk = bumpEntries.slice(start, start + DEFAULT_INSERT_CHUNK_ROWS); + await tx.execute(sql` + update ${issues} + set updated_at = greatest(${issues.updatedAt}, bumps.bump_at) + from (values ${sql.join( + chunk.map(([issueId, bumpAt]) => sql`(${issueId}::uuid, ${bumpAt.toISOString()}::timestamptz)`), + sql`, `, + )}) as bumps(issue_id, bump_at) + where ${issues.id} = bumps.issue_id + `); } }); },