feat(server): preserve task timestamps and hierarchy through company import/export (#11193)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Company export/import moves a whole company — agents, tasks, comments — between instances as a portable bundle > - The bundle never carried task timestamps or parent links: the export writes neither, the importer lets database defaults stamp "now", and sub-tasks arrive flattened > - Boards sort by recency, so every imported task showing "created just now" collapses the task list into import order, and the task hierarchy the user built is gone > - This pull request adds created/updated/started/completed/cancelled timestamps and a parent link to the bundle (schema v7), preserves them end to end on import, and keeps comment imports from clobbering a preserved updated time > - The benefit is that an imported company reads like the company the user left: same recency order, same task tree ## Linked Issues or Issue Description **What happened?** After a company import, every task showed as created at import time. Recency sorting collapsed to import order, and parent/child task nesting disappeared. The user called out losing "the meaningful task hierarchy and recency sorting". Cause: the export bundle has no fields for task timestamps or parent links, the importer lets `defaultNow()` win on insert, and the comment importer bumps every touched task's `updatedAt` to now. **Expected behavior** An imported company preserves each task's creation/update/start/completion times and its position in the task tree, so sorting and nesting on the destination match the source. **Steps to reproduce** 1. On a source instance, create tasks over several days, including sub-tasks nested under parents. 2. Export the company and import it into another instance. 3. Every task shows the import moment as its creation/update time and all tasks are top-level. ## What Changed - Export writes `createdAt`/`updatedAt`/`startedAt`/`completedAt`/`cancelledAt` (ISO, only when set) and `parent: <taskSlug>` into each task's bundle extension; a parent outside the export selection drops the edge with an aggregate warning, mirroring the existing blocker-edge warning (`server/src/services/company-portability.ts`). - Bundle schema version 6 → 7. All new fields are optional: v5/v6 bundles import unchanged with a version-aware downlevel warning; bundles newer than the board still fail closed. - Manifest parsing validates the new timestamps like comment timestamps (invalid → warn and ignore, never a hard failure); shared types and the zod validator carry the new optional fields. - Import resolves parent slugs to pre-generated destination ids, drops self-references and cycles from tampered bundles with warnings, and orders rows parents-first because the self-referencing FK is checked per insert chunk. - `importIssues` writes the preserved timestamps (falling back to insert time when absent; `startedAt` stays null unless bundle-carried, per #11191's semantics) and `parentId`. - `addImportedComments` no longer blanket-bumps `updatedAt = now()`; it takes `GREATEST(updated_at, newest imported comment createdAt)`, so a preserved update time never regresses while unpreserved rows keep the old behavior. ## Verification - `pnpm vitest run server/src/__tests__/company-portability.test.ts server/src/__tests__/company-portability-import-batching.test.ts server/src/__tests__/productivity-review-service.test.ts` — 102 passed, 1 pre-existing opt-in benchmark skip. Includes: full round-trip with exact timestamp equality and a 3-deep parent chain against embedded Postgres; v6 back-compat (defaults + warning); forward-compat rejection (v8); cycle/self-reference/invalid-timestamp tampered-bundle handling; comment-bump preserve-awareness in both directions. - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter @paperclipai/shared typecheck` — clean. ## Risks - **Rollout ordering**: a board on the previous build (max schema v6) refuses bundles exported by this build (stamped v7) — the existing newer-than-supported rejection, working as designed. Cross-instance moves need the importing board upgraded first. Called out here so operators aren't surprised during the transition window. - Parent edges from tampered bundles are dropped with warnings rather than failing the import; blocker relations already behave this way. - Timestamps are data-only; no destination schema migration. Stacked on #11191 (its commit is included here) — merge #11191 first; this PR then shows only the v7 changes. ## Model Used - Claude Fable 5 (`claude-fable-5`) via Claude Code CLI, with extended thinking and tool use (multi-agent implementation with independent verification). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
This commit is contained in:
Devin Foley
GitHub
parent
4c062a0eb2
commit
35aaaa0bd0
|
|
@ -200,6 +200,14 @@ export interface CompanyPortabilityIssueManifestEntry {
|
|||
workProducts?: CompanyPortabilityIssueWorkProductManifestEntry[];
|
||||
monitor?: CompanyPortabilityIssueMonitorManifestEntry | null;
|
||||
attachments?: CompanyPortabilityIssueAttachmentManifestEntry[];
|
||||
/** Slug of the parent task when it is part of the same bundle (schemaVersion >= 7). */
|
||||
parentSlug?: string | null;
|
||||
/** Preserved source timestamps as ISO strings (schemaVersion >= 7); absent in older bundles. */
|
||||
createdAt?: string | null;
|
||||
updatedAt?: string | null;
|
||||
startedAt?: string | null;
|
||||
completedAt?: string | null;
|
||||
cancelledAt?: string | null;
|
||||
metadata: Record<string, unknown> | null;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -222,6 +222,12 @@ export const portabilityIssueManifestEntrySchema = z.object({
|
|||
workProducts: z.array(portabilityIssueWorkProductManifestEntrySchema).default([]),
|
||||
monitor: portabilityIssueMonitorManifestEntrySchema.nullable().default(null),
|
||||
attachments: z.array(portabilityIssueAttachmentManifestEntrySchema).default([]),
|
||||
parentSlug: z.string().min(1).nullable().optional(),
|
||||
createdAt: z.string().datetime().nullable().optional(),
|
||||
updatedAt: z.string().datetime().nullable().optional(),
|
||||
startedAt: z.string().datetime().nullable().optional(),
|
||||
completedAt: z.string().datetime().nullable().optional(),
|
||||
cancelledAt: z.string().datetime().nullable().optional(),
|
||||
metadata: z.record(z.string(), z.unknown()).nullable(),
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -157,7 +157,7 @@ interface SyntheticBundleOptions {
|
|||
documentsPerIssue: number;
|
||||
}
|
||||
|
||||
/** Build an inline import bundle with the requested shape, valid at schemaVersion 6. */
|
||||
/** Build an inline import bundle with the requested shape, valid at schemaVersion 7. */
|
||||
function buildSyntheticBundle(options: SyntheticBundleOptions): {
|
||||
rootPath: string;
|
||||
files: Record<string, CompanyPortabilityFileEntry>;
|
||||
|
|
@ -193,7 +193,7 @@ function buildSyntheticBundle(options: SyntheticBundleOptions): {
|
|||
};
|
||||
}
|
||||
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 6, tasks });
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 7, tasks });
|
||||
|
||||
return { rootPath: "batching-bench", files };
|
||||
}
|
||||
|
|
@ -223,7 +223,7 @@ function buildTouchedIssuesBundle(issueCount: number): {
|
|||
};
|
||||
}
|
||||
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 6, tasks });
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({ schemaVersion: 7, tasks });
|
||||
return { rootPath: "inbox-flood", files };
|
||||
}
|
||||
|
||||
|
|
@ -423,6 +423,152 @@ describeEmbeddedPostgres("company import batches inserts", () => {
|
|||
expect(imported?.startedAt).toBeNull();
|
||||
});
|
||||
|
||||
it("preserves bundle timestamps and parent hierarchy through a real import", async () => {
|
||||
const files: Record<string, CompanyPortabilityFileEntry> = {};
|
||||
files["COMPANY.md"] = ['---', 'schema: "agentcompanies/v1"', 'name: "Timestamp Fidelity Co"', '---', '', 'Synthetic import bundle.', ''].join("\n");
|
||||
const taskFile = (name: string) => ['---', `name: "${name}"`, "kind: task", '---', '', `${name} body.`, ''].join("\n");
|
||||
files["tasks/a-child/TASK.md"] = taskFile("Child");
|
||||
files["tasks/m-grandchild/TASK.md"] = taskFile("Grandchild");
|
||||
files["tasks/z-parent/TASK.md"] = taskFile("Parent");
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({
|
||||
schemaVersion: 7,
|
||||
tasks: {
|
||||
// The child slug sorts before its parent so the importer's
|
||||
// parents-first row ordering (not manifest order) must satisfy the
|
||||
// parent foreign key.
|
||||
"a-child": {
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
parent: "z-parent",
|
||||
createdAt: "2026-01-05T00:00:00.000Z",
|
||||
updatedAt: "2026-03-05T00:00:00.000Z",
|
||||
comments: [{
|
||||
body: "Older than the preserved updatedAt.",
|
||||
authorType: "system",
|
||||
createdAt: "2026-02-01T00:00:00.000Z",
|
||||
}],
|
||||
},
|
||||
"m-grandchild": {
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
parent: "a-child",
|
||||
createdAt: "2026-01-06T00:00:00.000Z",
|
||||
updatedAt: "2026-01-07T00:00:00.000Z",
|
||||
},
|
||||
"z-parent": {
|
||||
status: "done",
|
||||
priority: "medium",
|
||||
createdAt: "2026-01-01T00:00:00.000Z",
|
||||
updatedAt: "2026-03-01T00:00:00.000Z",
|
||||
completedAt: "2026-02-20T00:00:00.000Z",
|
||||
comments: [{
|
||||
body: "Newer than the preserved updatedAt.",
|
||||
authorType: "system",
|
||||
createdAt: "2026-04-01T00:00:00.000Z",
|
||||
}],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const portability = companyPortabilityService(db);
|
||||
const result = await portability.importBundle(
|
||||
{
|
||||
source: { type: "inline", rootPath: "timestamp-fidelity", files },
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
target: { mode: "new_company", newCompanyName: "Timestamp Fidelity Co" },
|
||||
collisionStrategy: "rename",
|
||||
},
|
||||
"user-timestamp-fidelity",
|
||||
);
|
||||
expect(result.warnings).toEqual([]);
|
||||
|
||||
const rows = await db
|
||||
.select({
|
||||
id: issues.id,
|
||||
title: issues.title,
|
||||
parentId: issues.parentId,
|
||||
createdAt: issues.createdAt,
|
||||
updatedAt: issues.updatedAt,
|
||||
startedAt: issues.startedAt,
|
||||
completedAt: issues.completedAt,
|
||||
})
|
||||
.from(issues)
|
||||
.where(eq(issues.companyId, result.company.id));
|
||||
const parent = rows.find((row) => row.title === "Parent")!;
|
||||
const child = rows.find((row) => row.title === "Child")!;
|
||||
const grandchild = rows.find((row) => row.title === "Grandchild")!;
|
||||
|
||||
// The parent chain of three lands intact.
|
||||
expect(parent.parentId).toBeNull();
|
||||
expect(child.parentId).toBe(parent.id);
|
||||
expect(grandchild.parentId).toBe(child.id);
|
||||
|
||||
expect(parent.createdAt).toEqual(new Date("2026-01-01T00:00:00.000Z"));
|
||||
expect(parent.completedAt).toEqual(new Date("2026-02-20T00:00:00.000Z"));
|
||||
expect(parent.startedAt).toBeNull();
|
||||
expect(child.createdAt).toEqual(new Date("2026-01-05T00:00:00.000Z"));
|
||||
expect(grandchild.createdAt).toEqual(new Date("2026-01-06T00:00:00.000Z"));
|
||||
expect(grandchild.updatedAt).toEqual(new Date("2026-01-07T00:00:00.000Z"));
|
||||
|
||||
// A comment older than the preserved updatedAt must not regress it...
|
||||
expect(child.updatedAt).toEqual(new Date("2026-03-05T00:00:00.000Z"));
|
||||
// ...while a newer imported comment still bumps recency forward.
|
||||
expect(parent.updatedAt).toEqual(new Date("2026-04-01T00:00:00.000Z"));
|
||||
});
|
||||
|
||||
it("imports a v6 bundle without the timestamp fields using import-time defaults", async () => {
|
||||
// A little slack absorbs clock drift between this process and Postgres.
|
||||
const importStart = Date.now() - 1_000;
|
||||
const files: Record<string, CompanyPortabilityFileEntry> = {};
|
||||
files["COMPANY.md"] = ['---', 'schema: "agentcompanies/v1"', 'name: "Legacy Shape Co"', '---', '', 'Synthetic import bundle.', ''].join("\n");
|
||||
files["tasks/legacy-task/TASK.md"] = ['---', 'name: "Legacy task"', "kind: task", '---', '', 'Legacy body.', ''].join("\n");
|
||||
files[".paperclip.yaml"] = renderPaperclipYaml({
|
||||
schemaVersion: 6,
|
||||
tasks: {
|
||||
"legacy-task": {
|
||||
status: "todo",
|
||||
priority: "medium",
|
||||
comments: [{
|
||||
body: "Predates the import.",
|
||||
authorType: "system",
|
||||
createdAt: "2026-01-01T00:00:00.000Z",
|
||||
}],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
const portability = companyPortabilityService(db);
|
||||
const result = await portability.importBundle(
|
||||
{
|
||||
source: { type: "inline", rootPath: "legacy-shape", files },
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
target: { mode: "new_company", newCompanyName: "Legacy Shape Co" },
|
||||
collisionStrategy: "rename",
|
||||
},
|
||||
"user-legacy-shape",
|
||||
);
|
||||
expect(result.warnings).toContain(
|
||||
"This package declares schemaVersion 6 and predates task timestamp and parent link transfer; that task data imports only if the bundle carries it.",
|
||||
);
|
||||
|
||||
const [row] = await db
|
||||
.select({
|
||||
parentId: issues.parentId,
|
||||
createdAt: issues.createdAt,
|
||||
updatedAt: issues.updatedAt,
|
||||
startedAt: issues.startedAt,
|
||||
})
|
||||
.from(issues)
|
||||
.where(eq(issues.companyId, result.company.id));
|
||||
expect(row?.parentId).toBeNull();
|
||||
expect(row?.startedAt).toBeNull();
|
||||
// Without preserved timestamps the row keeps the old behavior: created and
|
||||
// updated at import time, and the comment bump does not drag updatedAt
|
||||
// back to the older comment createdAt.
|
||||
expect(row!.createdAt.getTime()).toBeGreaterThanOrEqual(importStart);
|
||||
expect(row!.updatedAt.getTime()).toBeGreaterThanOrEqual(importStart);
|
||||
});
|
||||
|
||||
it("rolls back the whole work-product batch when a later chunk fails", async () => {
|
||||
// Seed a real company + issue to hang work products off of.
|
||||
const bundle = buildSyntheticBundle({ issueCount: 1, commentsPerIssue: 0, documentsPerIssue: 0 });
|
||||
|
|
|
|||
|
|
@ -3579,8 +3579,8 @@ describe("company portability", () => {
|
|||
expect(extension).not.toContain("labelIds");
|
||||
expect(extension).not.toContain("label-a");
|
||||
// Fresh exports declare the current bundle shape end-to-end.
|
||||
expect(extension).toContain("schemaVersion: 6");
|
||||
expect(exported.manifest.schemaVersion).toBe(6);
|
||||
expect(extension).toContain("schemaVersion: 7");
|
||||
expect(exported.manifest.schemaVersion).toBe(7);
|
||||
expect(exported.manifest.labels).toEqual([
|
||||
{ name: "bug", color: "#ff0000" },
|
||||
{ name: "urgent", color: "#00ff00" },
|
||||
|
|
@ -4030,6 +4030,207 @@ describe("company portability", () => {
|
|||
);
|
||||
});
|
||||
|
||||
function mockTaskHierarchyExportSources() {
|
||||
projectSvc.list.mockResolvedValue([]);
|
||||
projectSvc.listWorkspaces.mockResolvedValue([]);
|
||||
const baseIssue = {
|
||||
description: null,
|
||||
projectId: null,
|
||||
projectWorkspaceId: null,
|
||||
assigneeAgentId: null,
|
||||
priority: "medium",
|
||||
labelIds: [],
|
||||
billingCode: null,
|
||||
executionWorkspaceSettings: null,
|
||||
assigneeAdapterOverrides: null,
|
||||
};
|
||||
issueSvc.list.mockResolvedValue([
|
||||
{
|
||||
...baseIssue,
|
||||
id: "issue-1",
|
||||
identifier: "PAP-1",
|
||||
title: "Alpha task",
|
||||
status: "done",
|
||||
parentId: null,
|
||||
createdAt: new Date("2026-01-01T00:00:00.000Z"),
|
||||
updatedAt: new Date("2026-03-01T00:00:00.000Z"),
|
||||
startedAt: new Date("2026-01-02T00:00:00.000Z"),
|
||||
completedAt: new Date("2026-02-01T00:00:00.000Z"),
|
||||
},
|
||||
{
|
||||
...baseIssue,
|
||||
id: "issue-2",
|
||||
identifier: "PAP-2",
|
||||
title: "Beta task",
|
||||
status: "todo",
|
||||
parentId: "issue-1",
|
||||
createdAt: new Date("2026-01-05T00:00:00.000Z"),
|
||||
updatedAt: new Date("2026-01-06T00:00:00.000Z"),
|
||||
},
|
||||
{
|
||||
...baseIssue,
|
||||
id: "issue-3",
|
||||
identifier: "PAP-3",
|
||||
title: "Gamma task",
|
||||
status: "todo",
|
||||
parentId: "issue-2",
|
||||
createdAt: new Date("2026-01-07T00:00:00.000Z"),
|
||||
updatedAt: new Date("2026-01-08T00:00:00.000Z"),
|
||||
},
|
||||
{
|
||||
...baseIssue,
|
||||
id: "issue-4",
|
||||
identifier: "PAP-4",
|
||||
title: "Delta task",
|
||||
status: "todo",
|
||||
parentId: "issue-outside",
|
||||
},
|
||||
]);
|
||||
}
|
||||
|
||||
it("carries task timestamps and parent links through export and import", async () => {
|
||||
const { db } = fakeImportDb();
|
||||
const portability = companyPortabilityService(db);
|
||||
mockTaskHierarchyExportSources();
|
||||
|
||||
const exported = await portability.exportBundle("company-1", {
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
});
|
||||
|
||||
const extension = asTextFile(exported.files[".paperclip.yaml"]);
|
||||
expect(extension).toContain("schemaVersion: 7");
|
||||
expect(extension).toContain('parent: "pap-1"');
|
||||
expect(extension).toContain('createdAt: "2026-01-01T00:00:00.000Z"');
|
||||
expect(exported.warnings).toContain(
|
||||
"1 parent relation references a task outside this export and was not included.",
|
||||
);
|
||||
|
||||
const alphaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-1");
|
||||
const betaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-2");
|
||||
const gammaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-3");
|
||||
const deltaEntry = exported.manifest.issues.find((issue) => issue.slug === "pap-4");
|
||||
expect(alphaEntry).toEqual(expect.objectContaining({
|
||||
parentSlug: null,
|
||||
createdAt: "2026-01-01T00:00:00.000Z",
|
||||
updatedAt: "2026-03-01T00:00:00.000Z",
|
||||
startedAt: "2026-01-02T00:00:00.000Z",
|
||||
completedAt: "2026-02-01T00:00:00.000Z",
|
||||
cancelledAt: null,
|
||||
}));
|
||||
expect(betaEntry?.parentSlug).toBe("pap-1");
|
||||
expect(gammaEntry?.parentSlug).toBe("pap-2");
|
||||
// The unexported parent edge is dropped, not carried by raw id.
|
||||
expect(deltaEntry?.parentSlug).toBeNull();
|
||||
expect(extension).not.toContain("issue-outside");
|
||||
|
||||
companySvc.create.mockResolvedValue({ id: "company-imported", name: "Imported" });
|
||||
accessSvc.ensureMembership.mockResolvedValue(undefined);
|
||||
agentSvc.list.mockResolvedValue([]);
|
||||
|
||||
await portability.importBundle({
|
||||
source: { type: "inline", rootPath: exported.rootPath, files: exported.files },
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
target: { mode: "new_company", newCompanyName: "Imported" },
|
||||
agents: "all",
|
||||
collisionStrategy: "rename",
|
||||
}, "user-1");
|
||||
|
||||
const importedRows = issueSvc.importIssues.mock.calls[0]![1] as Array<{
|
||||
id: string;
|
||||
title: string;
|
||||
parentId: string | null;
|
||||
createdAt: Date | null;
|
||||
updatedAt: Date | null;
|
||||
startedAt: Date | null;
|
||||
completedAt: Date | null;
|
||||
cancelledAt: Date | null;
|
||||
}>;
|
||||
const alphaRow = importedRows.find((row) => row.title === "Alpha task")!;
|
||||
const betaRow = importedRows.find((row) => row.title === "Beta task")!;
|
||||
const gammaRow = importedRows.find((row) => row.title === "Gamma task")!;
|
||||
const deltaRow = importedRows.find((row) => row.title === "Delta task")!;
|
||||
|
||||
// Timestamps survive the round trip as concrete dates.
|
||||
expect(alphaRow.createdAt).toEqual(new Date("2026-01-01T00:00:00.000Z"));
|
||||
expect(alphaRow.updatedAt).toEqual(new Date("2026-03-01T00:00:00.000Z"));
|
||||
expect(alphaRow.startedAt).toEqual(new Date("2026-01-02T00:00:00.000Z"));
|
||||
expect(alphaRow.completedAt).toEqual(new Date("2026-02-01T00:00:00.000Z"));
|
||||
expect(alphaRow.cancelledAt).toBeNull();
|
||||
expect(betaRow.createdAt).toEqual(new Date("2026-01-05T00:00:00.000Z"));
|
||||
|
||||
// The parent chain of three is intact against the pre-generated ids.
|
||||
expect(alphaRow.parentId).toBeNull();
|
||||
expect(betaRow.parentId).toBe(alphaRow.id);
|
||||
expect(gammaRow.parentId).toBe(betaRow.id);
|
||||
expect(deltaRow.parentId).toBeNull();
|
||||
|
||||
// Rows arrive parents-first so chunked inserts satisfy the parent FK.
|
||||
expect(importedRows.indexOf(alphaRow)).toBeLessThan(importedRows.indexOf(betaRow));
|
||||
expect(importedRows.indexOf(betaRow)).toBeLessThan(importedRows.indexOf(gammaRow));
|
||||
});
|
||||
|
||||
it("drops self-referencing and cyclic parent links from a hand-built bundle with warnings", async () => {
|
||||
const { db } = fakeImportDb();
|
||||
const portability = companyPortabilityService(db);
|
||||
|
||||
const taskFile = (name: string) => ["---", `name: "${name}"`, "kind: task", "---", "", `${name} body.`, ""].join("\n");
|
||||
const files = {
|
||||
"COMPANY.md": ["---", 'schema: "agentcompanies/v1"', 'name: "Tampered Import"', "---", ""].join("\n"),
|
||||
"tasks/task-a/TASK.md": taskFile("Task A"),
|
||||
"tasks/task-b/TASK.md": taskFile("Task B"),
|
||||
"tasks/task-c/TASK.md": taskFile("Task C"),
|
||||
".paperclip.yaml": [
|
||||
'schema: "paperclip/v1"',
|
||||
"schemaVersion: 7",
|
||||
"tasks:",
|
||||
" task-a:",
|
||||
' status: "todo"',
|
||||
' parent: "task-b"',
|
||||
" task-b:",
|
||||
' status: "todo"',
|
||||
' parent: "task-a"',
|
||||
" task-c:",
|
||||
' status: "todo"',
|
||||
' parent: "task-c"',
|
||||
' createdAt: "not-a-timestamp"',
|
||||
"",
|
||||
].join("\n"),
|
||||
};
|
||||
|
||||
companySvc.create.mockResolvedValue({ id: "company-imported", name: "Tampered Import" });
|
||||
accessSvc.ensureMembership.mockResolvedValue(undefined);
|
||||
agentSvc.list.mockResolvedValue([]);
|
||||
|
||||
const result = await portability.importBundle({
|
||||
source: { type: "inline", rootPath: "tampered-package", files },
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
target: { mode: "new_company", newCompanyName: "Tampered Import" },
|
||||
agents: "all",
|
||||
collisionStrategy: "rename",
|
||||
}, "user-1");
|
||||
|
||||
const importedRows = issueSvc.importIssues.mock.calls[0]![1] as Array<{
|
||||
title: string;
|
||||
parentId: string | null;
|
||||
createdAt: Date | null;
|
||||
}>;
|
||||
const rowA = importedRows.find((row) => row.title === "Task A")!;
|
||||
const rowB = importedRows.find((row) => row.title === "Task B")!;
|
||||
const rowC = importedRows.find((row) => row.title === "Task C")!;
|
||||
|
||||
// One direction of the two-task cycle survives; the closing edge drops.
|
||||
expect([rowA.parentId, rowB.parentId].filter((parentId) => parentId !== null)).toHaveLength(1);
|
||||
expect(rowC.parentId).toBeNull();
|
||||
expect(rowC.createdAt).toBeNull();
|
||||
expect(result.warnings.filter((warning) => warning.includes("would create a parent cycle"))).toHaveLength(2);
|
||||
expect(result.warnings).toContain(
|
||||
"Task task-c parent task-c was skipped because it would create a parent cycle.",
|
||||
);
|
||||
expect(result.warnings).toContain(
|
||||
"Task task-c createdAt was ignored because it is not a valid timestamp.",
|
||||
);
|
||||
});
|
||||
|
||||
const attachmentBytesByObjectKey: Record<string, string> = {
|
||||
"issues/issue-1/notes.bin": "png-bytes",
|
||||
"issues/issue-1/screenshot.png": "png-bytes",
|
||||
|
|
@ -4625,7 +4826,7 @@ describe("company portability", () => {
|
|||
it("imports unstamped v5 packages with an info warning about task data they predate", async () => {
|
||||
const portability = companyPortabilityService({} as any);
|
||||
const v5Warning =
|
||||
"This package declares schemaVersion 5 and predates label, blocker, document, work product, monitor, attachment, and embedded image transfer; that task data imports only if the bundle carries it.";
|
||||
"This package declares schemaVersion 5 and predates label, blocker, document, work product, monitor, attachment, embedded image, task timestamp, and parent link transfer; that task data imports only if the bundle carries it.";
|
||||
|
||||
companySvc.create.mockResolvedValue({ id: "company-imported", name: "Legacy Import" });
|
||||
accessSvc.ensureMembership.mockResolvedValue(undefined);
|
||||
|
|
@ -4674,7 +4875,7 @@ describe("company portability", () => {
|
|||
const portability = companyPortabilityService({} as any);
|
||||
|
||||
await expect(portability.importBundle({
|
||||
source: { type: "inline", rootPath: "future-package", files: legacyPackageFiles(["schemaVersion: 7"]) },
|
||||
source: { type: "inline", rootPath: "future-package", files: legacyPackageFiles(["schemaVersion: 8"]) },
|
||||
include: { company: true, agents: false, projects: false, issues: true },
|
||||
target: { mode: "new_company", newCompanyName: "Future Import" },
|
||||
agents: "all",
|
||||
|
|
|
|||
|
|
@ -161,8 +161,10 @@ const DEFAULT_INCLUDE: CompanyPortabilityInclude = {
|
|||
const DEFAULT_COLLISION_STRATEGY: CompanyPortabilityCollisionStrategy = "rename";
|
||||
// The bundle shape this build reads and writes. Bundles began declaring
|
||||
// their schemaVersion in the .paperclip.yaml extension at 6; undeclared
|
||||
// bundles are read as 5, the last unstamped shape.
|
||||
const BUNDLE_SCHEMA_VERSION = 6;
|
||||
// bundles are read as 5, the last unstamped shape. 7 adds preserved task
|
||||
// timestamps and parent links; 5/6 bundles still import, with those fields
|
||||
// falling back to import-time defaults.
|
||||
const BUNDLE_SCHEMA_VERSION = 7;
|
||||
const UNSTAMPED_BUNDLE_SCHEMA_VERSION = 5;
|
||||
const DEFAULT_IMPORTED_LABEL_COLOR = "#6366f1";
|
||||
// Blob entries are content-addressed by sha256; the store itself is
|
||||
|
|
@ -899,6 +901,38 @@ function readPortableIssueLabelNames(value: unknown): string[] {
|
|||
return names;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a preserved issue timestamp from the extension, validated the same way
|
||||
* comment createdAt is (Date.parse). Invalid values are ignored with a
|
||||
* warning — never a hard failure — so a hand-edited bundle still imports.
|
||||
*/
|
||||
function readPortableIssueTimestamp(
|
||||
value: unknown,
|
||||
warnings: string[],
|
||||
sourceLabel: string,
|
||||
fieldLabel: string,
|
||||
): string | null {
|
||||
if (value === undefined || value === null) return null;
|
||||
const raw = asString(value);
|
||||
if (raw && !Number.isNaN(Date.parse(raw))) return raw;
|
||||
warnings.push(`${sourceLabel} ${fieldLabel} was ignored because it is not a valid timestamp.`);
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Render a stored timestamp as a portable ISO string, or omit it when unset/invalid. */
|
||||
function toPortableTimestamp(value: Date | string | null | undefined): string | undefined {
|
||||
if (value == null) return undefined;
|
||||
const date = value instanceof Date ? value : new Date(value);
|
||||
return Number.isNaN(date.getTime()) ? undefined : date.toISOString();
|
||||
}
|
||||
|
||||
/** Convert a manifest timestamp (already validated at parse time) to a Date, or null. */
|
||||
function portableManifestDate(value: string | null | undefined): Date | null {
|
||||
if (!value) return null;
|
||||
const date = new Date(value);
|
||||
return Number.isNaN(date.getTime()) ? null : date;
|
||||
}
|
||||
|
||||
function readPortableIssueBlockedBy(value: unknown): string[] {
|
||||
if (!Array.isArray(value)) return [];
|
||||
const slugs: string[] = [];
|
||||
|
|
@ -3353,6 +3387,12 @@ function buildManifestFromPackageFiles(
|
|||
workProducts: normalizePortableIssueWorkProducts(extension.workProducts),
|
||||
monitor: normalizePortableIssueMonitor(extension.monitor),
|
||||
attachments: normalizePortableIssueAttachments(extension.attachments, warnings, `Task ${slug}`),
|
||||
parentSlug: asString(extension.parent),
|
||||
createdAt: readPortableIssueTimestamp(extension.createdAt, warnings, `Task ${slug}`, "createdAt"),
|
||||
updatedAt: readPortableIssueTimestamp(extension.updatedAt, warnings, `Task ${slug}`, "updatedAt"),
|
||||
startedAt: readPortableIssueTimestamp(extension.startedAt, warnings, `Task ${slug}`, "startedAt"),
|
||||
completedAt: readPortableIssueTimestamp(extension.completedAt, warnings, `Task ${slug}`, "completedAt"),
|
||||
cancelledAt: readPortableIssueTimestamp(extension.cancelledAt, warnings, `Task ${slug}`, "cancelledAt"),
|
||||
metadata: isPlainRecord(extension.metadata) ? extension.metadata : null,
|
||||
});
|
||||
if (frontmatter.kind && frontmatter.kind !== "task") {
|
||||
|
|
@ -3361,7 +3401,10 @@ function buildManifestFromPackageFiles(
|
|||
}
|
||||
|
||||
if (bundleSchemaVersion < BUNDLE_SCHEMA_VERSION && manifest.issues.length > 0) {
|
||||
warnings.push(`This package declares schemaVersion ${bundleSchemaVersion} and predates label, blocker, document, work product, monitor, attachment, and embedded image transfer; that task data imports only if the bundle carries it.`);
|
||||
const predated = bundleSchemaVersion < 6
|
||||
? "label, blocker, document, work product, monitor, attachment, embedded image, task timestamp, and parent link transfer"
|
||||
: "task timestamp and parent link transfer";
|
||||
warnings.push(`This package declares schemaVersion ${bundleSchemaVersion} and predates ${predated}; that task data imports only if the bundle carries it.`);
|
||||
}
|
||||
|
||||
manifest.envInputs = dedupeEnvInputs(manifest.envInputs);
|
||||
|
|
@ -4216,6 +4259,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
}
|
||||
|
||||
let unexportedBlockerEdgeCount = 0;
|
||||
let unexportedParentEdgeCount = 0;
|
||||
let unportableWorkProductRefCount = 0;
|
||||
const exportedBlobs = new Map<string, CompanyPortabilityBlobManifestEntry>();
|
||||
for (const issue of selectedIssueRows) {
|
||||
|
|
@ -4256,6 +4300,13 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
unexportedBlockerEdgeCount += relationSummaries.blocks
|
||||
.filter((blocked) => !taskSlugByIssueId.has(blocked.id))
|
||||
.length;
|
||||
// Parent links travel by task slug like blockers; a parent outside the
|
||||
// export selection drops the edge and is counted for one warning.
|
||||
let parentTaskSlug: string | null = null;
|
||||
if (issue.parentId) {
|
||||
parentTaskSlug = taskSlugByIssueId.get(issue.parentId) ?? null;
|
||||
if (!parentTaskSlug) unexportedParentEdgeCount += 1;
|
||||
}
|
||||
const issueDocumentRows = await documentsSvc.listIssueDocuments(issue.id, { includeSystem: true });
|
||||
const documentEntries = issueDocumentRows.map((document) => {
|
||||
const documentPath = `tasks/${taskSlug}/documents/${document.key}.md`;
|
||||
|
|
@ -4339,6 +4390,12 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
identifier: issue.identifier,
|
||||
status: issue.status,
|
||||
priority: issue.priority,
|
||||
parent: parentTaskSlug ?? undefined,
|
||||
createdAt: toPortableTimestamp(issue.createdAt),
|
||||
updatedAt: toPortableTimestamp(issue.updatedAt),
|
||||
startedAt: toPortableTimestamp(issue.startedAt),
|
||||
completedAt: toPortableTimestamp(issue.completedAt),
|
||||
cancelledAt: toPortableTimestamp(issue.cancelledAt),
|
||||
// Labels travel by name (their natural key); the bundle-level labels
|
||||
// section carries the matching color definitions.
|
||||
labels: (issue.labelIds ?? [])
|
||||
|
|
@ -4380,6 +4437,9 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
if (unexportedBlockerEdgeCount > 0) {
|
||||
warnings.push(`${unexportedBlockerEdgeCount} blocker relation${unexportedBlockerEdgeCount === 1 ? " references a task" : "s reference tasks"} outside this export and ${unexportedBlockerEdgeCount === 1 ? "was" : "were"} not included.`);
|
||||
}
|
||||
if (unexportedParentEdgeCount > 0) {
|
||||
warnings.push(`${unexportedParentEdgeCount} parent relation${unexportedParentEdgeCount === 1 ? " references a task" : "s reference tasks"} outside this export and ${unexportedParentEdgeCount === 1 ? "was" : "were"} not included.`);
|
||||
}
|
||||
if (unportableWorkProductRefCount > 0) {
|
||||
warnings.push(`${unportableWorkProductRefCount} work product${unportableWorkProductRefCount === 1 ? " references" : "s reference"} execution workspaces or runs that are not portable; those references were omitted from the export.`);
|
||||
}
|
||||
|
|
@ -5721,6 +5781,7 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
|
||||
const importedIssueIdBySlug = new Map<string, string>();
|
||||
const blockedByBySlug = new Map<string, string[]>();
|
||||
const parentSlugBySlug = new Map<string, string>();
|
||||
let unarmedMonitorCount = 0;
|
||||
let attachmentsSkippedNoStorage = 0;
|
||||
const attachmentMaxBytes = normalizeIssueAttachmentMaxBytes(targetCompany.attachmentMaxBytes ?? null);
|
||||
|
|
@ -5924,6 +5985,9 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
if ((manifestIssue.blockedBy ?? []).length > 0) {
|
||||
blockedByBySlug.set(manifestIssue.slug, manifestIssue.blockedBy ?? []);
|
||||
}
|
||||
if (manifestIssue.parentSlug) {
|
||||
parentSlugBySlug.set(manifestIssue.slug, manifestIssue.parentSlug);
|
||||
}
|
||||
for (const documentEntry of manifestIssue.documents ?? []) {
|
||||
const documentBody = readPortableTextFile(plan.source.files, documentEntry.path);
|
||||
if (documentBody === null) {
|
||||
|
|
@ -6050,9 +6114,70 @@ export function companyPortabilityService(db: Db, storage?: StorageService) {
|
|||
labelIds: resolvedLabelIds,
|
||||
monitorNotes,
|
||||
monitorScheduledBy,
|
||||
parentId: null,
|
||||
createdAt: portableManifestDate(manifestIssue.createdAt),
|
||||
updatedAt: portableManifestDate(manifestIssue.updatedAt),
|
||||
startedAt: portableManifestDate(manifestIssue.startedAt),
|
||||
completedAt: portableManifestDate(manifestIssue.completedAt),
|
||||
cancelledAt: portableManifestDate(manifestIssue.cancelledAt),
|
||||
});
|
||||
}
|
||||
|
||||
// Parent links resolve against the pre-generated ids before the flush
|
||||
// so each issue row carries its parentId on insert. Edges whose parent
|
||||
// was not imported, self-references, and cycles (possible only in a
|
||||
// hand-edited bundle) drop with a warning, mirroring blocker handling.
|
||||
if (parentSlugBySlug.size > 0) {
|
||||
const rowBySlug = new Map(issueRows.map((row) => [row.ref, row] as const));
|
||||
const acceptedParentById = new Map<string, string>();
|
||||
const wouldCreateParentCycle = (childId: string, parentId: string) => {
|
||||
let current: string | undefined = parentId;
|
||||
const visited = new Set<string>();
|
||||
while (current) {
|
||||
if (current === childId) return true;
|
||||
if (visited.has(current)) return false;
|
||||
visited.add(current);
|
||||
current = acceptedParentById.get(current);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
for (const [slug, parentSlug] of parentSlugBySlug) {
|
||||
const row = rowBySlug.get(slug);
|
||||
if (!row) continue;
|
||||
const parentId = importedIssueIdBySlug.get(parentSlug);
|
||||
if (!parentId) {
|
||||
warnings.push(`Task ${slug} parent ${parentSlug} was skipped because that task was not imported.`);
|
||||
continue;
|
||||
}
|
||||
if (parentId === row.id || wouldCreateParentCycle(row.id, parentId)) {
|
||||
warnings.push(`Task ${slug} parent ${parentSlug} was skipped because it would create a parent cycle.`);
|
||||
continue;
|
||||
}
|
||||
acceptedParentById.set(row.id, parentId);
|
||||
row.parentId = parentId;
|
||||
}
|
||||
// The parent foreign key is checked per insert statement, so order
|
||||
// rows parents-first: a child must never land in an earlier chunk
|
||||
// than its parent.
|
||||
if (acceptedParentById.size > 0) {
|
||||
const rowById = new Map(issueRows.map((row) => [row.id, row] as const));
|
||||
const orderedRows: typeof issueRows = [];
|
||||
const emitted = new Set<string>();
|
||||
for (const row of issueRows) {
|
||||
const chain: typeof issueRows = [];
|
||||
let current: (typeof issueRows)[number] | undefined = row;
|
||||
while (current && !emitted.has(current.id)) {
|
||||
emitted.add(current.id);
|
||||
chain.push(current);
|
||||
current = current.parentId ? rowById.get(current.parentId) : undefined;
|
||||
}
|
||||
for (const entry of chain.reverse()) orderedRows.push(entry);
|
||||
}
|
||||
issueRows.length = 0;
|
||||
issueRows.push(...orderedRows);
|
||||
}
|
||||
}
|
||||
|
||||
// Flush the buffered rows in dependency order: issues first (parents of
|
||||
// every other row), then comments (attachments may reference them), then
|
||||
// the remaining children. Each writer inserts in chunked multi-row
|
||||
|
|
|
|||
|
|
@ -35,6 +35,18 @@ export interface ImportIssueRow {
|
|||
/** Imported monitors land un-armed; only notes/provenance are restored. */
|
||||
monitorNotes: string | null;
|
||||
monitorScheduledBy: string | null;
|
||||
/** Parent issue id resolved within the same import batch (pre-generated), or absent for roots. */
|
||||
parentId?: string | null;
|
||||
/**
|
||||
* Preserved source timestamps carried by schemaVersion >= 7 bundles.
|
||||
* Absent/null createdAt and updatedAt fall back to the insert time;
|
||||
* absent startedAt stays null — the writer never fabricates it.
|
||||
*/
|
||||
createdAt?: Date | null;
|
||||
updatedAt?: Date | null;
|
||||
startedAt?: Date | null;
|
||||
completedAt?: Date | null;
|
||||
cancelledAt?: Date | null;
|
||||
}
|
||||
|
||||
/** A resolved comment row with a pre-generated id, ready for batch insert. */
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ import { resolveIssueGoalId, resolveNextIssueGoalId } from "./issue-goal-fallbac
|
|||
import { getRunLogStore } from "./run-log-store.js";
|
||||
import { getDefaultCompanyGoal } from "./goals.js";
|
||||
import { assertAssignableAgent } from "./agent-assignability.js";
|
||||
import { insertRowsInChunks } from "./batch-insert.js";
|
||||
import { DEFAULT_INSERT_CHUNK_ROWS, insertRowsInChunks } from "./batch-insert.js";
|
||||
import type {
|
||||
ImportIssueRow,
|
||||
ImportIssueCommentRow,
|
||||
|
|
@ -7352,11 +7352,19 @@ export function issueService(db: Db) {
|
|||
responsibleUserId: null,
|
||||
requestDepth: clampIssueRequestDepth(undefined),
|
||||
originKind: "manual",
|
||||
// The caller resolves parentId against ids in this same batch, so a
|
||||
// parent always lands in the same insert (rows arrive parents-first).
|
||||
parentId: row.parentId ?? null,
|
||||
// Preserved bundle timestamps win; without them createdAt/updatedAt
|
||||
// fall back to the insert time (the old defaultNow() behavior).
|
||||
createdAt: row.createdAt ?? new Date(),
|
||||
updatedAt: row.updatedAt ?? new Date(),
|
||||
// Imported in-progress work did not start at import time; fabricating
|
||||
// startedAt here trips duration-based sweeps (e.g. productivity review).
|
||||
startedAt: null,
|
||||
completedAt: row.status === "done" ? new Date() : null,
|
||||
cancelledAt: row.status === "cancelled" ? new Date() : null,
|
||||
// startedAt here trips duration-based sweeps (e.g. productivity
|
||||
// review). Only a bundle-carried startedAt is written.
|
||||
startedAt: row.startedAt ?? null,
|
||||
completedAt: row.completedAt ?? (row.status === "done" ? new Date() : null),
|
||||
cancelledAt: row.cancelledAt ?? (row.status === "cancelled" ? new Date() : null),
|
||||
monitorNotes: row.monitorNotes ?? null,
|
||||
monitorScheduledBy: row.monitorScheduledBy ?? null,
|
||||
});
|
||||
|
|
@ -7396,10 +7404,28 @@ export function issueService(db: Db) {
|
|||
};
|
||||
});
|
||||
await insertRowsInChunks(tx, issueComments, commentRows);
|
||||
// Mirror addComment's recency bump, once per affected issue.
|
||||
const issueIds = [...new Set(rows.map((row) => row.issueId))];
|
||||
if (issueIds.length > 0) {
|
||||
await tx.update(issues).set({ updatedAt: new Date() }).where(inArray(issues.id, issueIds));
|
||||
// Mirror addComment's recency bump, once per affected issue — but never
|
||||
// backwards. An issue imported with a preserved updatedAt keeps it
|
||||
// unless a newer imported comment outdates it; issues without preserved
|
||||
// timestamps carry an insert-time updatedAt, so GREATEST reproduces the
|
||||
// old "bump to now" behavior for them.
|
||||
const bumpAtByIssueId = new Map<string, Date>();
|
||||
for (const row of commentRows) {
|
||||
const existing = bumpAtByIssueId.get(row.issueId);
|
||||
if (!existing || row.createdAt > existing) bumpAtByIssueId.set(row.issueId, row.createdAt);
|
||||
}
|
||||
const bumpEntries = [...bumpAtByIssueId.entries()];
|
||||
for (let start = 0; start < bumpEntries.length; start += DEFAULT_INSERT_CHUNK_ROWS) {
|
||||
const chunk = bumpEntries.slice(start, start + DEFAULT_INSERT_CHUNK_ROWS);
|
||||
await tx.execute(sql`
|
||||
update ${issues}
|
||||
set updated_at = greatest(${issues.updatedAt}, bumps.bump_at)
|
||||
from (values ${sql.join(
|
||||
chunk.map(([issueId, bumpAt]) => sql`(${issueId}::uuid, ${bumpAt.toISOString()}::timestamptz)`),
|
||||
sql`, `,
|
||||
)}) as bumps(issue_id, bump_at)
|
||||
where ${issues.id} = bumps.issue_id
|
||||
`);
|
||||
}
|
||||
});
|
||||
},
|
||||
|
|
|
|||
Loading…
Reference in New Issue