From 1e81bd188ba82c2f965cea49004e019fcb189d83 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Mon, 6 Jul 2026 13:14:18 -0500 Subject: [PATCH] Fix heartbeat run responsible user migration for identifier refs (#9107) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip stores heartbeat run ownership context so operators can audit which user was responsible for agent work > - A migration backfills missing heartbeat run `responsible_user_id` values from issue references in each run's context snapshot > - Some context snapshots can store issue identifiers as public ticket strings rather than UUIDs > - The migration needs to resolve both UUID issue ids and issue identifiers without trying to cast identifier strings to UUID > - This pull request tightens the migration query so UUID matching only casts validated UUID-shaped values and identifier matching remains a separate fallback > - A companion repair migration is needed so installations that already recorded `0130` still get the corrected heartbeat-run backfill > - The benefit is that existing installations can apply the responsible-user invariant migrations without failing on non-UUID issue references and without leaving already-migrated databases unrepaired ## Linked Issues or Issue Description No public GitHub issue was found in a quick search for this migration failure. Bug report context following `.github/ISSUE_TEMPLATE/bug_report.yml`: **Pre-submission checklist** - I searched existing open and closed issues and did not find a duplicate. - I can reproduce against current `master` plus the responsible-user invariant migration path. - The error originates in Paperclip's database migration, not in an adapter, API provider, or local configuration. **What happened?** Applying the heartbeat run responsible-user backfill migration could fail when `heartbeat_runs.context_snapshot->>'issueId'` or `taskId` contained an issue identifier such as `PAP-123` instead of a UUID. The migration attempted to use issue refs for UUID matching and identifier fallback, but the UUID path needed to avoid casting non-UUID identifier strings. Because `0130` may already have been applied in some installations, a follow-up repair migration is needed as well. **Expected behavior** The migration should backfill from UUID issue ids when present, from issue identifiers when present, and fall back to the company default responsible user without unsafe UUID casts. Already-migrated installations should receive the repaired heartbeat-run context-ref backfill through a new migration. **Steps to reproduce** 1. Use a migrated database with a company, issue, agent, and heartbeat run. 2. Store a null `heartbeat_runs.responsible_user_id` and a `context_snapshot` like `{"issueId":"PAP-123"}`. 3. Replay/apply the run responsible-user repair migration. 4. Observe that the migration must not cast `PAP-123` to UUID and should backfill from the matching issue identifier. **Paperclip version or commit** Current `master` plus this migration fix branch. **Deployment mode** Database migration during server startup or explicit migration command. **Installation method** Built from source / self-hosted migration path. **Agent adapter(s) involved** Not adapter-specific; core database migration bug. **Database mode** Postgres migration path, including embedded Postgres in development. **Access context** Not applicable; migration-time data backfill. **Relevant logs or output** Unsafe UUID casts can surface as Postgres invalid input syntax errors when a context snapshot issue ref is an identifier rather than a UUID. **Privacy checklist** No private logs, paths, API keys, tokens, company names, or internal Paperclip issue links are included. ## What Changed - Split heartbeat run context issue reference extraction into reusable CTEs. - Only cast `issueId` / `taskId` values to UUID after a UUID-shape regex check. - Preserve fallback matching by issue identifier within the same company. - Keep deterministic candidate priority with `issueId` before `taskId` and UUID matches before identifier matches. - Added `0131_repair_run_responsible_user_context_refs.sql` so installations that already applied `0130` still receive the corrected heartbeat-run backfill. - Added a DB migration regression test that replays the repair migration with an identifier-style heartbeat run issue ref. ## Verification - `pnpm --filter @paperclipai/db typecheck` - `pnpm --filter @paperclipai/db exec vitest run src/client.test.ts` - Isolated embedded Postgres migration run with a temporary `PAPERCLIP_CONFIG`: `pnpm --filter @paperclipai/db migrate` applied all pending migrations successfully. - GitHub Actions PR workflow is green on commit `1c2655bc457cef3716a43e66463e1e5bc2fdcfab`. - Greptile is 5/5 with no unresolved threads on commit `1c2655bc457cef3716a43e66463e1e5bc2fdcfab`. - Searched for duplicate public issues/PRs with GitHub search; no direct duplicate found. - Checked `ROADMAP.md` for overlap; no related roadmap item found. ## Risks - Low-to-medium migration risk because this modifies an existing data backfill migration and adds a companion repair migration. - The query still relies on `context_snapshot` containing either issue UUIDs or identifiers that match issues in the same company. - Installations with unusual malformed context snapshots now skip unsafe UUID casts and fall through to identifier/default backfill behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex coding agent with tool use and local command execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes, or confirmed no docs update is needed for this migration-only fix - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- packages/db/src/client.test.ts | 127 ++++++++++++++++++ .../0130_run_responsible_user_invariant.sql | 77 +++++++++-- ...pair_run_responsible_user_context_refs.sql | 67 +++++++++ packages/db/src/migrations/meta/_journal.json | 7 + 4 files changed, 267 insertions(+), 11 deletions(-) create mode 100644 packages/db/src/migrations/0131_repair_run_responsible_user_context_refs.sql diff --git a/packages/db/src/client.test.ts b/packages/db/src/client.test.ts index 552db3b0d7..75e7effcc5 100644 --- a/packages/db/src/client.test.ts +++ b/packages/db/src/client.test.ts @@ -541,4 +541,131 @@ describeEmbeddedPostgres("applyPendingMigrations", () => { }, 20_000, ); + + it( + "replays the run responsible user repair migration when heartbeat run issue refs are identifiers", + async () => { + const connectionString = await createTempDatabase(); + + await applyPendingMigrations(connectionString); + + const sql = postgres(connectionString, { max: 1, onnotice: () => {} }); + try { + const runResponsibleUserRepairHash = await migrationHash( + "0131_repair_run_responsible_user_context_refs.sql", + ); + + await sql.unsafe(` + INSERT INTO "companies" ("id", "name", "issue_prefix", "created_at", "updated_at") + VALUES ('00000000-0000-0000-0000-000000000130', 'Migration Test Co', 'TST130', now(), now()) + `); + await sql.unsafe(` + INSERT INTO "company_memberships" ( + "id", + "company_id", + "principal_type", + "principal_id", + "status", + "membership_role", + "created_at", + "updated_at" + ) + VALUES ( + '00000000-0000-0000-0000-000000000131', + '00000000-0000-0000-0000-000000000130', + 'user', + 'owner-user', + 'active', + 'owner', + now(), + now() + ) + `); + await sql.unsafe(` + INSERT INTO "agents" ("id", "company_id", "name", "role", "adapter_type", "created_at", "updated_at") + VALUES ( + '00000000-0000-0000-0000-000000000132', + '00000000-0000-0000-0000-000000000130', + 'Migration Agent', + 'general', + 'process', + now(), + now() + ) + `); + await sql.unsafe(` + INSERT INTO "issues" ( + "id", + "company_id", + "title", + "status", + "responsible_user_id", + "identifier", + "created_at", + "updated_at" + ) + VALUES ( + '00000000-0000-0000-0000-000000000133', + '00000000-0000-0000-0000-000000000130', + 'Identifier referenced issue', + 'todo', + 'issue-user', + 'TST130-1', + now(), + now() + ) + `); + await sql.unsafe(` + INSERT INTO "heartbeat_runs" ( + "id", + "company_id", + "agent_id", + "status", + "responsible_user_id", + "context_snapshot", + "created_at", + "updated_at" + ) + VALUES ( + '00000000-0000-0000-0000-000000000134', + '00000000-0000-0000-0000-000000000130', + '00000000-0000-0000-0000-000000000132', + 'completed', + NULL, + '{"issueId":"TST130-1"}'::jsonb, + now(), + now() + ) + `); + await sql.unsafe( + `DELETE FROM "drizzle"."__drizzle_migrations" WHERE hash = '${runResponsibleUserRepairHash}'`, + ); + } finally { + await sql.end(); + } + + const pendingState = await inspectMigrations(connectionString); + expect(pendingState).toMatchObject({ + status: "needsMigrations", + pendingMigrations: ["0131_repair_run_responsible_user_context_refs.sql"], + reason: "pending-migrations", + }); + + await applyPendingMigrations(connectionString); + + const verifySql = postgres(connectionString, { max: 1, onnotice: () => {} }); + try { + const runs = await verifySql.unsafe<{ responsible_user_id: string | null }[]>(` + SELECT "responsible_user_id" + FROM "heartbeat_runs" + WHERE "id" = '00000000-0000-0000-0000-000000000134' + `); + expect(runs).toEqual([{ responsible_user_id: "issue-user" }]); + + } finally { + await verifySql.end(); + } + }, + 20_000, + ); }); diff --git a/packages/db/src/migrations/0130_run_responsible_user_invariant.sql b/packages/db/src/migrations/0130_run_responsible_user_invariant.sql index 87bc74d771..1ae59fa5a9 100644 --- a/packages/db/src/migrations/0130_run_responsible_user_invariant.sql +++ b/packages/db/src/migrations/0130_run_responsible_user_invariant.sql @@ -129,19 +129,74 @@ WHERE h."retry_of_run_id" = original."id" AND h."responsible_user_id" IS NULL AND original."responsible_user_id" IS NOT NULL; --> statement-breakpoint +WITH extracted_run_refs AS ( + SELECT + h."id" AS "run_id", + h."company_id", + NULLIF(h."context_snapshot" ->> 'issueId', '') AS "issue_ref", + 1 AS "ref_priority" + FROM "heartbeat_runs" AS h + WHERE h."responsible_user_id" IS NULL + AND NULLIF(h."context_snapshot" ->> 'issueId', '') IS NOT NULL + + UNION ALL + + SELECT + h."id" AS "run_id", + h."company_id", + NULLIF(h."context_snapshot" ->> 'taskId', '') AS "issue_ref", + 2 AS "ref_priority" + FROM "heartbeat_runs" AS h + WHERE h."responsible_user_id" IS NULL + AND NULLIF(h."context_snapshot" ->> 'taskId', '') IS NOT NULL +), +uuid_run_refs AS ( + SELECT + "run_id", + "company_id", + "issue_ref"::uuid AS "issue_id", + "ref_priority" + FROM extracted_run_refs + WHERE "issue_ref" ~* '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$' +), +resolved_run_users AS ( + SELECT DISTINCT ON ("run_id") + "run_id", + "responsible_user_id" + FROM ( + SELECT + refs."run_id", + i."responsible_user_id", + refs."ref_priority", + 1 AS "match_priority" + FROM uuid_run_refs AS refs + JOIN "issues" AS i + ON i."id" = refs."issue_id" + AND i."company_id" = refs."company_id" + WHERE i."responsible_user_id" IS NOT NULL + + UNION ALL + + SELECT + refs."run_id", + i."responsible_user_id", + refs."ref_priority", + 2 AS "match_priority" + FROM extracted_run_refs AS refs + JOIN "issues" AS i + ON i."identifier" = refs."issue_ref" + AND i."company_id" = refs."company_id" + WHERE refs."issue_ref" IS NOT NULL + AND i."responsible_user_id" IS NOT NULL + ) AS candidates + ORDER BY "run_id", "ref_priority" ASC, "match_priority" ASC +) UPDATE "heartbeat_runs" AS h -SET "responsible_user_id" = i."responsible_user_id", +SET "responsible_user_id" = resolved_run_users."responsible_user_id", "updated_at" = now() -FROM "issues" AS i -WHERE h."company_id" = i."company_id" - AND h."responsible_user_id" IS NULL - AND i."responsible_user_id" IS NOT NULL - AND ( - h."context_snapshot" ->> 'issueId' = i."id"::text - OR h."context_snapshot" ->> 'taskId' = i."id"::text - OR h."context_snapshot" ->> 'issueId' = i."identifier" - OR h."context_snapshot" ->> 'taskId' = i."identifier" - ); +FROM resolved_run_users +WHERE h."id" = resolved_run_users."run_id" + AND h."responsible_user_id" IS NULL; --> statement-breakpoint UPDATE "heartbeat_runs" AS h SET "responsible_user_id" = awr."requested_by_actor_id", diff --git a/packages/db/src/migrations/0131_repair_run_responsible_user_context_refs.sql b/packages/db/src/migrations/0131_repair_run_responsible_user_context_refs.sql new file mode 100644 index 0000000000..6edaa302c6 --- /dev/null +++ b/packages/db/src/migrations/0131_repair_run_responsible_user_context_refs.sql @@ -0,0 +1,67 @@ +WITH extracted_run_refs AS ( + SELECT + h."id" AS "run_id", + h."company_id", + NULLIF(h."context_snapshot" ->> 'issueId', '') AS "issue_ref", + 1 AS "ref_priority" + FROM "heartbeat_runs" AS h + WHERE h."responsible_user_id" IS NULL + AND NULLIF(h."context_snapshot" ->> 'issueId', '') IS NOT NULL + + UNION ALL + + SELECT + h."id" AS "run_id", + h."company_id", + NULLIF(h."context_snapshot" ->> 'taskId', '') AS "issue_ref", + 2 AS "ref_priority" + FROM "heartbeat_runs" AS h + WHERE h."responsible_user_id" IS NULL + AND NULLIF(h."context_snapshot" ->> 'taskId', '') IS NOT NULL +), +uuid_run_refs AS ( + SELECT + "run_id", + "company_id", + "issue_ref"::uuid AS "issue_id", + "ref_priority" + FROM extracted_run_refs + WHERE "issue_ref" ~* '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$' +), +resolved_run_users AS ( + SELECT DISTINCT ON ("run_id") + "run_id", + "responsible_user_id" + FROM ( + SELECT + refs."run_id", + i."responsible_user_id", + refs."ref_priority", + 1 AS "match_priority" + FROM uuid_run_refs AS refs + JOIN "issues" AS i + ON i."id" = refs."issue_id" + AND i."company_id" = refs."company_id" + WHERE i."responsible_user_id" IS NOT NULL + + UNION ALL + + SELECT + refs."run_id", + i."responsible_user_id", + refs."ref_priority", + 2 AS "match_priority" + FROM extracted_run_refs AS refs + JOIN "issues" AS i + ON i."identifier" = refs."issue_ref" + AND i."company_id" = refs."company_id" + WHERE i."responsible_user_id" IS NOT NULL + ) AS candidates + ORDER BY "run_id", "ref_priority" ASC, "match_priority" ASC +) +UPDATE "heartbeat_runs" AS h +SET "responsible_user_id" = resolved_run_users."responsible_user_id", + "updated_at" = now() +FROM resolved_run_users +WHERE h."id" = resolved_run_users."run_id" + AND h."responsible_user_id" IS NULL; diff --git a/packages/db/src/migrations/meta/_journal.json b/packages/db/src/migrations/meta/_journal.json index e3e0473eb2..81ba9a4972 100644 --- a/packages/db/src/migrations/meta/_journal.json +++ b/packages/db/src/migrations/meta/_journal.json @@ -918,6 +918,13 @@ "when": 1783025224120, "tag": "0130_run_responsible_user_invariant", "breakpoints": true + }, + { + "idx": 131, + "version": "7", + "when": 1783025324120, + "tag": "0131_repair_run_responsible_user_context_refs", + "breakpoints": true } ] }