fix(external-objects): refresh PR status labels (#10704)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The issue properties panel can show external objects such as GitHub pull requests. > - Those objects are resolved by external-object providers and then displayed as compact status labels. > - A GitHub pull request could remain in the fallback `unknown` state and appear as `Not yet resolved`. > - That label is confusing when the object is known but has not been refreshed yet. > - This pull request refreshes due external objects from the heartbeat scheduler and improves the unknown-status copy. > - The benefit is a properties panel that moves from pending refresh to the real pull request state without a manual refresh. ## Linked Issues or Issue Description No public GitHub issue exists for this bug. I searched for related public issues and pull requests using the terms `Not yet refreshed`, `external objects refresh`, and `external PR status`, and did not find a duplicate implementation. **What happened?** The issue properties panel could show a GitHub pull request as `Not yet resolved` even when the referenced pull request was valid. The object stayed stale unless a manual refresh path ran. **Expected behavior** A known external object should show pending-refresh copy while it waits for provider data. When the scheduler refreshes it, the properties panel should show the provider status such as open, merged, or closed. **Steps to reproduce** 1. Create or view an issue that references a GitHub pull request. 2. Open the issue properties panel. 3. Observe the external object row before a manual refresh has run. **Paperclip version or commit** Current `master` before this pull request. **Deployment mode** Local dev and self-hosted server. **Installation method** Built from source. **Agent adapter(s) involved** Not adapter-specific. **Database mode** Not database-related. **Access context** Board view. **Privacy checklist** I reviewed this description and did not include logs, credentials, private URLs, internal issue IDs, or PII. ## What Changed - Added a heartbeat scheduler tick that refreshes due external objects for active companies. - Kept manual external-object refresh behavior on the same service path. - Changed display copy so known provider objects use liveness labels such as `Not yet refreshed`, while fresh unknown provider statuses show `Status unavailable`. - Added server and UI tests for scheduled refresh and label behavior. ## Verification - `corepack pnpm install --frozen-lockfile` - `pnpm check:token-gates` - `pnpm exec vitest run server/src/__tests__/external-objects-service.test.ts server/src/__tests__/server-startup-feedback-export.test.ts ui/src/components/ExternalObjectPill.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/lib/external-objects.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/server build` - `pnpm --filter @paperclipai/ui build` - `pnpm run typecheck:build-gaps` - GitHub PR checks passed on head `0e7fcd30` - Greptile reported 5/5 on head `0e7fcd30` with no unresolved review threads Notes: - I ran recursive typecheck and build first. Both hit container resource limits with exit 137 during concurrent package work, so I reran the affected server and UI targets separately. - An unrelated workspace-runtime auto-port test fails in this container with a PID ownership mismatch. It is outside the files changed here. ## Risks Low to medium risk. The scheduler does more periodic external-object work, so the main risk is extra provider refresh load. The implementation bounds the work to active companies, due non-terminal objects, and 50 objects per company per tick. The path also stays behind the external-objects experimental setting. ## Model Used OpenAI GPT-5 Codex in the Codex execution environment, with shell and GitHub CLI tool use. The runtime did not expose a more specific internal model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
GitHub
parent
7a3815eb9a
commit
185515c97b
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE "external_objects" ADD COLUMN "refresh_started_at" timestamp with time zone;
|
||||
|
|
@ -0,0 +1 @@
|
|||
ALTER TABLE "external_objects" ADD COLUMN "refresh_token" uuid;
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
|
|
@ -1394,6 +1394,20 @@
|
|||
"when": 1785635155420,
|
||||
"tag": "0200_yellow_maria_hill",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 201,
|
||||
"version": "7",
|
||||
"when": 1785701828695,
|
||||
"tag": "0201_concerned_captain_midlands",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 202,
|
||||
"version": "7",
|
||||
"when": 1785702264747,
|
||||
"tag": "0202_eminent_marvel_zombies",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -44,6 +44,8 @@ export const externalObjects = pgTable(
|
|||
lastChangedAt: timestamp("last_changed_at", { withTimezone: true }),
|
||||
lastErrorAt: timestamp("last_error_at", { withTimezone: true }),
|
||||
nextRefreshAt: timestamp("next_refresh_at", { withTimezone: true }),
|
||||
refreshStartedAt: timestamp("refresh_started_at", { withTimezone: true }),
|
||||
refreshToken: uuid("refresh_token"),
|
||||
lastErrorCode: text("last_error_code"),
|
||||
lastErrorMessage: text("last_error_message"),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ export interface ExternalObject {
|
|||
lastChangedAt: string | null;
|
||||
lastErrorAt: string | null;
|
||||
nextRefreshAt: string | null;
|
||||
refreshStartedAt: string | null;
|
||||
lastErrorCode: string | null;
|
||||
lastErrorMessage: string | null;
|
||||
createdAt: string;
|
||||
|
|
|
|||
|
|
@ -529,6 +529,207 @@ describeEmbeddedPostgres("externalObjectService", () => {
|
|||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("coalesces automatic and manual refreshes for the same object", async () => {
|
||||
const { companyId, issueId } = await createIssue();
|
||||
let markResolverStarted!: () => void;
|
||||
const resolverStarted = new Promise<void>((resolve) => {
|
||||
markResolverStarted = resolve;
|
||||
});
|
||||
let finishResolver!: () => void;
|
||||
const resolverCanFinish = new Promise<void>((resolve) => {
|
||||
finishResolver = resolve;
|
||||
});
|
||||
const resolve = vi.fn(async () => {
|
||||
markResolverStarted();
|
||||
await resolverCanFinish;
|
||||
return {
|
||||
ok: true as const,
|
||||
snapshot: {
|
||||
statusCategory: "open" as const,
|
||||
statusTone: "info" as const,
|
||||
statusKey: "open",
|
||||
statusLabel: "Open",
|
||||
ttlSeconds: 300,
|
||||
},
|
||||
};
|
||||
});
|
||||
const resolver: ExternalObjectResolver = {
|
||||
providerKey: "url",
|
||||
objectType: "link",
|
||||
resolve,
|
||||
};
|
||||
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
||||
await svc.syncIssue(issueId);
|
||||
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
||||
|
||||
const dueRefresh = svc.refreshDueObjects(companyId, 50, new Date(Date.now() + 1_000));
|
||||
await resolverStarted;
|
||||
const manualRefresh = svc.refreshObject(object.id, { companyId, force: true });
|
||||
finishResolver();
|
||||
|
||||
const [dueResults, manualResult] = await Promise.all([dueRefresh, manualRefresh]);
|
||||
|
||||
expect(dueResults).toHaveLength(1);
|
||||
expect(dueResults[0]?.refreshed).toBe(true);
|
||||
expect(manualResult.refreshed).toBe(true);
|
||||
expect(manualResult.object.statusLabel).toBe("Open");
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("prevents duplicate refreshes across service instances", async () => {
|
||||
const { companyId, issueId } = await createIssue();
|
||||
let markResolverStarted!: () => void;
|
||||
const resolverStarted = new Promise<void>((resolve) => {
|
||||
markResolverStarted = resolve;
|
||||
});
|
||||
let finishResolver!: () => void;
|
||||
const resolverCanFinish = new Promise<void>((resolve) => {
|
||||
finishResolver = resolve;
|
||||
});
|
||||
const resolve = vi.fn(async () => {
|
||||
markResolverStarted();
|
||||
await resolverCanFinish;
|
||||
return {
|
||||
ok: true as const,
|
||||
snapshot: {
|
||||
statusCategory: "open" as const,
|
||||
statusTone: "info" as const,
|
||||
statusKey: "open",
|
||||
statusLabel: "Open",
|
||||
ttlSeconds: 300,
|
||||
},
|
||||
};
|
||||
});
|
||||
const resolver: ExternalObjectResolver = {
|
||||
providerKey: "url",
|
||||
objectType: "link",
|
||||
resolve,
|
||||
};
|
||||
const scheduledService = externalObjectService(db, { resolvers: [resolver], github: false });
|
||||
const manualService = externalObjectService(db, { resolvers: [resolver], github: false });
|
||||
await scheduledService.syncIssue(issueId);
|
||||
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
||||
|
||||
const dueRefresh = scheduledService.refreshDueObjects(companyId, 50, new Date(Date.now() + 1_000));
|
||||
await resolverStarted;
|
||||
const manualRefresh = await manualService.refreshObject(object.id, { companyId, force: true });
|
||||
finishResolver();
|
||||
const dueResults = await dueRefresh;
|
||||
|
||||
expect(dueResults).toHaveLength(1);
|
||||
expect(dueResults[0]?.refreshed).toBe(true);
|
||||
expect(manualRefresh.refreshed).toBe(false);
|
||||
expect(manualRefresh.reason).toBe("refresh_in_progress");
|
||||
expect(manualRefresh.object).not.toHaveProperty("refreshToken");
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
await expect(
|
||||
db.select().from(externalObjects).then((rows) => rows[0]?.refreshStartedAt ?? null),
|
||||
).resolves.toBeNull();
|
||||
});
|
||||
|
||||
it("does not let a stale refresh overwrite a replacement claimant", async () => {
|
||||
const { companyId, issueId } = await createIssue();
|
||||
let markResolverStarted!: () => void;
|
||||
const resolverStarted = new Promise<void>((resolve) => {
|
||||
markResolverStarted = resolve;
|
||||
});
|
||||
let finishResolver!: () => void;
|
||||
const resolverCanFinish = new Promise<void>((resolve) => {
|
||||
finishResolver = resolve;
|
||||
});
|
||||
const slowResolve = vi.fn(async () => {
|
||||
markResolverStarted();
|
||||
await resolverCanFinish;
|
||||
return {
|
||||
ok: true as const,
|
||||
snapshot: {
|
||||
statusCategory: "open" as const,
|
||||
statusTone: "info" as const,
|
||||
statusKey: "open",
|
||||
statusLabel: "Open",
|
||||
ttlSeconds: 300,
|
||||
},
|
||||
};
|
||||
});
|
||||
const replacementResolve = vi.fn(async () => ({
|
||||
ok: true as const,
|
||||
snapshot: {
|
||||
statusCategory: "closed" as const,
|
||||
statusTone: "muted" as const,
|
||||
statusKey: "closed",
|
||||
statusLabel: "Closed",
|
||||
ttlSeconds: 300,
|
||||
},
|
||||
}));
|
||||
const scheduledService = externalObjectService(db, {
|
||||
resolvers: [{ providerKey: "url", objectType: "link", resolve: slowResolve }],
|
||||
github: false,
|
||||
});
|
||||
const replacementService = externalObjectService(db, {
|
||||
resolvers: [{ providerKey: "url", objectType: "link", resolve: replacementResolve }],
|
||||
github: false,
|
||||
});
|
||||
await scheduledService.syncIssue(issueId);
|
||||
const object = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
||||
const claimedAt = new Date(Date.now() + 1_000);
|
||||
|
||||
const dueRefresh = scheduledService.refreshDueObjects(companyId, 50, claimedAt);
|
||||
await resolverStarted;
|
||||
const replacementResult = await replacementService.refreshObject(object.id, {
|
||||
companyId,
|
||||
force: true,
|
||||
now: new Date(claimedAt.getTime() + 301_000),
|
||||
});
|
||||
finishResolver();
|
||||
const dueResults = await dueRefresh;
|
||||
const finalObject = await db.select().from(externalObjects).then((rows) => rows[0]!);
|
||||
|
||||
expect(replacementResult.refreshed).toBe(true);
|
||||
expect(dueResults[0]).toMatchObject({ refreshed: false, reason: "refresh_superseded" });
|
||||
expect(finalObject.statusLabel).toBe("Closed");
|
||||
expect(finalObject.refreshStartedAt).toBeNull();
|
||||
expect(finalObject.refreshToken).toBeNull();
|
||||
expect(slowResolve).toHaveBeenCalledTimes(1);
|
||||
expect(replacementResolve).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("refreshes due objects for active companies only", async () => {
|
||||
const active = await createIssue();
|
||||
const paused = await createIssue();
|
||||
await db
|
||||
.update(companies)
|
||||
.set({ status: "paused" })
|
||||
.where(eq(companies.id, paused.companyId));
|
||||
const resolve = vi.fn(async () => ({
|
||||
ok: true as const,
|
||||
snapshot: {
|
||||
statusCategory: "open" as const,
|
||||
statusTone: "info" as const,
|
||||
statusKey: "open",
|
||||
statusLabel: "Open",
|
||||
ttlSeconds: 300,
|
||||
},
|
||||
}));
|
||||
const resolver: ExternalObjectResolver = {
|
||||
providerKey: "url",
|
||||
objectType: "link",
|
||||
resolve,
|
||||
};
|
||||
const svc = externalObjectService(db, { resolvers: [resolver], github: false });
|
||||
await svc.syncIssue(active.issueId);
|
||||
await svc.syncIssue(paused.issueId);
|
||||
|
||||
const result = await svc.refreshDueObjectsForActiveCompanies(50, new Date(Date.now() + 1_000));
|
||||
|
||||
expect(result).toEqual({ companies: 1, checked: 1, refreshed: 1 });
|
||||
expect(resolve).toHaveBeenCalledTimes(1);
|
||||
const rows = await db.select().from(externalObjects);
|
||||
const activeObject = rows.find((row) => row.companyId === active.companyId);
|
||||
const pausedObject = rows.find((row) => row.companyId === paused.companyId);
|
||||
expect(activeObject?.statusLabel).toBe("Open");
|
||||
expect(pausedObject?.statusLabel).toBeNull();
|
||||
});
|
||||
|
||||
it("removes comment mentions when a synced comment is hard-deleted", async () => {
|
||||
const { companyId, issueId } = await createIssue();
|
||||
const commentId = randomUUID();
|
||||
|
|
|
|||
|
|
@ -17,6 +17,8 @@ const {
|
|||
deriveAuthTrustedOriginsMock,
|
||||
environmentCustomImagesServiceMock,
|
||||
environmentCustomImagesServiceFactoryMock,
|
||||
externalObjectsServiceMock,
|
||||
externalObjectsServiceFactoryMock,
|
||||
feedbackExportServiceMock,
|
||||
feedbackServiceFactoryMock,
|
||||
fakeServer,
|
||||
|
|
@ -71,6 +73,10 @@ const {
|
|||
cleanupExpiredSetupSessions: vi.fn(async () => ({ scanned: 0, timedOut: 0, failed: 0 })),
|
||||
};
|
||||
const environmentCustomImagesServiceFactoryMock = vi.fn(() => environmentCustomImagesServiceMock);
|
||||
const externalObjectsServiceMock = {
|
||||
refreshDueObjectsForActiveCompanies: vi.fn(async () => ({ companies: 0, checked: 0, refreshed: 0 })),
|
||||
};
|
||||
const externalObjectsServiceFactoryMock = vi.fn(() => externalObjectsServiceMock);
|
||||
const routineServiceMock = {
|
||||
tickScheduledTriggers: vi.fn(async () => ({ triggered: 0 })),
|
||||
};
|
||||
|
|
@ -98,6 +104,8 @@ const {
|
|||
deriveAuthTrustedOriginsMock,
|
||||
environmentCustomImagesServiceMock,
|
||||
environmentCustomImagesServiceFactoryMock,
|
||||
externalObjectsServiceMock,
|
||||
externalObjectsServiceFactoryMock,
|
||||
feedbackExportServiceMock,
|
||||
feedbackServiceFactoryMock,
|
||||
fakeServer,
|
||||
|
|
@ -225,9 +233,14 @@ vi.mock("../services/index.js", () => ({
|
|||
bootstrapExecutionPolicyFromEnv: vi.fn(async () => null),
|
||||
applyManagedEnvironments: vi.fn(async () => null),
|
||||
environmentCustomImageService: environmentCustomImagesServiceFactoryMock,
|
||||
externalObjectService: externalObjectsServiceFactoryMock,
|
||||
heartbeatService: heartbeatServiceFactoryMock,
|
||||
issueService: vi.fn(() => ({ update: vi.fn(async () => null) })),
|
||||
instanceSettingsService: vi.fn(() => ({
|
||||
getExperimental: vi.fn(async () => ({
|
||||
enableExternalObjects: true,
|
||||
enableStatusCards: false,
|
||||
})),
|
||||
getGeneral: vi.fn(async () => ({
|
||||
backupRetention: {
|
||||
dailyDays: 7,
|
||||
|
|
@ -444,6 +457,7 @@ describe("startServer feedback export wiring", () => {
|
|||
await Promise.resolve();
|
||||
|
||||
expect(heartbeatServiceMock.tickTimers).not.toHaveBeenCalled();
|
||||
expect(externalObjectsServiceMock.refreshDueObjectsForActiveCompanies).toHaveBeenCalledTimes(1);
|
||||
expect(routineServiceMock.tickScheduledTriggers).toHaveBeenCalledTimes(1);
|
||||
expect(environmentCustomImagesServiceMock.cleanupExpiredSetupSessions).toHaveBeenCalledTimes(2);
|
||||
} finally {
|
||||
|
|
@ -451,6 +465,36 @@ describe("startServer feedback export wiring", () => {
|
|||
}
|
||||
});
|
||||
|
||||
it("keeps external object refresh active when heartbeat scheduling is disabled", async () => {
|
||||
loadConfigMock.mockReturnValue(buildTestConfig({
|
||||
heartbeatSchedulerEnabled: false,
|
||||
heartbeatSchedulerIntervalMs: 30000,
|
||||
}));
|
||||
let intervalCallback: (() => void) | null = null;
|
||||
const setIntervalSpy = vi
|
||||
.spyOn(globalThis, "setInterval")
|
||||
.mockImplementation(((callback: () => void) => {
|
||||
intervalCallback = callback;
|
||||
return 1 as unknown as ReturnType<typeof setInterval>;
|
||||
}) as typeof setInterval);
|
||||
|
||||
try {
|
||||
await startServer();
|
||||
|
||||
expect(heartbeatServiceFactoryMock).not.toHaveBeenCalled();
|
||||
expect(intervalCallback).not.toBeNull();
|
||||
intervalCallback?.();
|
||||
await Promise.resolve();
|
||||
await Promise.resolve();
|
||||
|
||||
expect(externalObjectsServiceMock.refreshDueObjectsForActiveCompanies).toHaveBeenCalledTimes(1);
|
||||
expect(routineServiceMock.tickScheduledTriggers).not.toHaveBeenCalled();
|
||||
expect(environmentCustomImagesServiceMock.cleanupExpiredSetupSessions).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
setIntervalSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("does not replay hot-restart adoption when the orphan reaper retries", async () => {
|
||||
loadConfigMock.mockReturnValue(buildTestConfig({
|
||||
heartbeatSchedulerEnabled: true,
|
||||
|
|
|
|||
|
|
@ -49,6 +49,7 @@ import {
|
|||
environmentCustomImageService,
|
||||
decisionService,
|
||||
decisionRetentionService,
|
||||
externalObjectService,
|
||||
heartbeatService,
|
||||
issueService,
|
||||
instanceSettingsService,
|
||||
|
|
@ -912,6 +913,27 @@ export async function startServer(): Promise<StartedServer> {
|
|||
await Promise.allSettled([...heartbeatSchedulerInFlight]);
|
||||
}
|
||||
};
|
||||
const startHeartbeatSchedulerInterval = (callback: () => void) => {
|
||||
heartbeatSchedulerInterval = setInterval(callback, config.heartbeatSchedulerIntervalMs);
|
||||
heartbeatSchedulerInterval?.unref?.();
|
||||
};
|
||||
const externalObjects = externalObjectService(db as any, {
|
||||
pluginWorkerManager,
|
||||
enabled: async () => (await instanceSettingsService(db).getExperimental()).enableExternalObjects === true,
|
||||
});
|
||||
const scheduleExternalObjectRefreshSweep = (now = new Date()) => {
|
||||
if (heartbeatSchedulerStopped) return;
|
||||
trackHeartbeatSchedulerWork(externalObjects
|
||||
.refreshDueObjectsForActiveCompanies(50, now)
|
||||
.then((result) => {
|
||||
if (result.checked > 0 || result.refreshed > 0) {
|
||||
logger.info({ ...result }, "external-object scheduler tick refreshed due objects");
|
||||
}
|
||||
})
|
||||
.catch((err) => {
|
||||
logger.error({ err }, "external-object scheduler tick failed");
|
||||
}));
|
||||
};
|
||||
|
||||
if (heartbeat) {
|
||||
const decisionExecutor = decisionService(db as any, decisionServiceOptions);
|
||||
|
|
@ -1073,7 +1095,7 @@ export async function startServer(): Promise<StartedServer> {
|
|||
};
|
||||
await runRetentionSweep();
|
||||
|
||||
heartbeatSchedulerInterval = setInterval(() => {
|
||||
startHeartbeatSchedulerInterval(() => {
|
||||
// Async so the suppression checks below can honor the override-aware
|
||||
// resolver (e.g. worktree run-execution opt-in). The gated work is still
|
||||
// wrapped in trackHeartbeatSchedulerWork with its own error handling.
|
||||
|
|
@ -1106,6 +1128,9 @@ export async function startServer(): Promise<StartedServer> {
|
|||
}));
|
||||
}
|
||||
|
||||
if (heartbeatSchedulerStopped) return;
|
||||
scheduleExternalObjectRefreshSweep(new Date());
|
||||
|
||||
if (heartbeatSchedulerStopped) return;
|
||||
trackHeartbeatSchedulerWork(routines
|
||||
.tickScheduledTriggers(new Date())
|
||||
|
|
@ -1230,7 +1255,11 @@ export async function startServer(): Promise<StartedServer> {
|
|||
}));
|
||||
}
|
||||
})();
|
||||
}, config.heartbeatSchedulerIntervalMs);
|
||||
});
|
||||
} else {
|
||||
startHeartbeatSchedulerInterval(() => {
|
||||
scheduleExternalObjectRefreshSweep(new Date());
|
||||
});
|
||||
}
|
||||
|
||||
if (config.databaseBackupEnabled) {
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { and, asc, eq, inArray, isNull, lte, sql } from "drizzle-orm";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { and, asc, eq, inArray, isNull, lte, or, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { documents, externalObjectMentions, externalObjects, issueComments, issueDocuments, issues, plugins } from "@paperclipai/db";
|
||||
import { companies, documents, externalObjectMentions, externalObjects, issueComments, issueDocuments, issues, plugins } from "@paperclipai/db";
|
||||
import {
|
||||
formatExternalObjectMentionSourceLabel,
|
||||
type ExternalObjectCanonicalUrl,
|
||||
|
|
@ -91,6 +92,8 @@ type ExternalObjectMentionRecord = typeof externalObjectMentions.$inferSelect;
|
|||
|
||||
const DEFAULT_REFRESH_TTL_SECONDS = 300;
|
||||
const DEFAULT_RETRY_AFTER_SECONDS = 300;
|
||||
const DEFAULT_REFRESH_LEASE_SECONDS = 300;
|
||||
const REFRESH_LEASE_RENEW_INTERVAL_MS = 60_000;
|
||||
|
||||
function sourceWhere(input: ExternalObjectSourceContext) {
|
||||
const conditions = [
|
||||
|
|
@ -601,8 +604,10 @@ export function externalObjectService(
|
|||
}
|
||||
|
||||
function toObjectPayload(object: ExternalObjectRecord, now = new Date()) {
|
||||
const { refreshToken, ...payload } = object;
|
||||
void refreshToken;
|
||||
return {
|
||||
...object,
|
||||
...payload,
|
||||
liveness: visibleLiveness(object, now),
|
||||
};
|
||||
}
|
||||
|
|
@ -768,26 +773,37 @@ export function externalObjectService(
|
|||
return summarizeObjectPayloads(objects, 25);
|
||||
}
|
||||
|
||||
async function refreshObject(
|
||||
objectId: string,
|
||||
input: {
|
||||
companyId: string;
|
||||
actor?: Pick<LogActivityInput, "actorType" | "actorId" | "agentId" | "runId">;
|
||||
force?: boolean;
|
||||
now?: Date;
|
||||
},
|
||||
) {
|
||||
const now = input.now ?? new Date();
|
||||
const object = await db
|
||||
type RefreshObjectInput = {
|
||||
companyId: string;
|
||||
actor?: Pick<LogActivityInput, "actorType" | "actorId" | "agentId" | "runId">;
|
||||
force?: boolean;
|
||||
now?: Date;
|
||||
};
|
||||
|
||||
function refreshOwnerWhere(object: ExternalObjectRecord, refreshToken: string) {
|
||||
return and(
|
||||
eq(externalObjects.id, object.id),
|
||||
eq(externalObjects.companyId, object.companyId),
|
||||
eq(externalObjects.refreshToken, refreshToken),
|
||||
);
|
||||
}
|
||||
|
||||
async function refreshSupersededResult(object: ExternalObjectRecord, now: Date) {
|
||||
const latest = await db
|
||||
.select()
|
||||
.from(externalObjects)
|
||||
.where(and(eq(externalObjects.id, objectId), eq(externalObjects.companyId, input.companyId)))
|
||||
.where(and(eq(externalObjects.id, object.id), eq(externalObjects.companyId, object.companyId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!object) throw notFound("External object not found");
|
||||
if (!input.force && object.nextRefreshAt && object.nextRefreshAt > now) {
|
||||
return { object: toObjectPayload(object, now), refreshed: false, reason: "backoff" as const };
|
||||
}
|
||||
if (!latest) throw notFound("External object not found");
|
||||
return { object: toObjectPayload(latest, now), refreshed: false, reason: "refresh_superseded" as const };
|
||||
}
|
||||
|
||||
async function resolveObjectRefresh(
|
||||
object: ExternalObjectRecord,
|
||||
input: RefreshObjectInput,
|
||||
now: Date,
|
||||
refreshToken: string,
|
||||
) {
|
||||
const pluginResult = await resolveViaPluginProvider(db, opts.pluginWorkerManager, object);
|
||||
const resolver = pluginResult ? null : resolverRegistry.find(object);
|
||||
if (!pluginResult && !resolver) {
|
||||
|
|
@ -796,10 +812,13 @@ export function externalObjectService(
|
|||
.set({
|
||||
liveness: visibleLiveness(object, now) === "fresh" ? "stale" : object.liveness,
|
||||
nextRefreshAt: addSeconds(now, DEFAULT_RETRY_AFTER_SECONDS),
|
||||
refreshStartedAt: null,
|
||||
refreshToken: null,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(and(eq(externalObjects.id, object.id), eq(externalObjects.companyId, object.companyId)))
|
||||
.where(refreshOwnerWhere(object, refreshToken))
|
||||
.returning();
|
||||
if (!updated) return refreshSupersededResult(object, now);
|
||||
return { object: toObjectPayload(updated ?? object, now), refreshed: false, reason: "no_resolver" as const };
|
||||
}
|
||||
|
||||
|
|
@ -813,10 +832,13 @@ export function externalObjectService(
|
|||
lastErrorCode: result.errorCode,
|
||||
lastErrorMessage: sanitizeErrorMessage(result.errorMessage),
|
||||
nextRefreshAt: addSeconds(now, result.retryAfterSeconds ?? DEFAULT_RETRY_AFTER_SECONDS),
|
||||
refreshStartedAt: null,
|
||||
refreshToken: null,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(and(eq(externalObjects.id, object.id), eq(externalObjects.companyId, object.companyId)))
|
||||
.where(refreshOwnerWhere(object, refreshToken))
|
||||
.returning();
|
||||
if (!updated) return refreshSupersededResult(object, now);
|
||||
publishLiveEvent({
|
||||
companyId: object.companyId,
|
||||
type: "external_object.updated",
|
||||
|
|
@ -845,16 +867,19 @@ export function externalObjectService(
|
|||
lastErrorCode: null,
|
||||
lastErrorMessage: null,
|
||||
nextRefreshAt: addSeconds(now, snapshot.ttlSeconds ?? DEFAULT_REFRESH_TTL_SECONDS),
|
||||
refreshStartedAt: null,
|
||||
refreshToken: null,
|
||||
updatedAt: now,
|
||||
};
|
||||
const [updated] = await db
|
||||
.update(externalObjects)
|
||||
.set({
|
||||
...patch,
|
||||
lastChangedAt: objectChanged(object, { ...object, ...patch }) ? now : object.lastChangedAt,
|
||||
})
|
||||
.where(and(eq(externalObjects.id, object.id), eq(externalObjects.companyId, object.companyId)))
|
||||
...patch,
|
||||
lastChangedAt: objectChanged(object, { ...object, ...patch }) ? now : object.lastChangedAt,
|
||||
})
|
||||
.where(refreshOwnerWhere(object, refreshToken))
|
||||
.returning();
|
||||
if (!updated) return refreshSupersededResult(object, now);
|
||||
const next = updated ?? object;
|
||||
if (objectChanged(object, next) && input.actor) {
|
||||
await logActivity(db, {
|
||||
|
|
@ -886,6 +911,100 @@ export function externalObjectService(
|
|||
return { object: toObjectPayload(next, now), refreshed: true, reason: "resolved" as const };
|
||||
}
|
||||
|
||||
async function claimObjectRefresh(
|
||||
object: ExternalObjectRecord,
|
||||
input: RefreshObjectInput,
|
||||
now: Date,
|
||||
) {
|
||||
const staleRefreshStartedBefore = new Date(now.getTime() - DEFAULT_REFRESH_LEASE_SECONDS * 1000);
|
||||
const leaseAvailable = or(
|
||||
isNull(externalObjects.refreshStartedAt),
|
||||
lte(externalObjects.refreshStartedAt, staleRefreshStartedBefore),
|
||||
)!;
|
||||
const refreshToken = randomUUID();
|
||||
const dueOrForced = input.force
|
||||
? leaseAvailable
|
||||
: and(
|
||||
leaseAvailable,
|
||||
or(isNull(externalObjects.nextRefreshAt), lte(externalObjects.nextRefreshAt, now))!,
|
||||
)!;
|
||||
const [claimed] = await db
|
||||
.update(externalObjects)
|
||||
.set({
|
||||
refreshStartedAt: now,
|
||||
refreshToken,
|
||||
updatedAt: now,
|
||||
})
|
||||
.where(and(
|
||||
eq(externalObjects.id, object.id),
|
||||
eq(externalObjects.companyId, object.companyId),
|
||||
dueOrForced,
|
||||
))
|
||||
.returning();
|
||||
return claimed ?? null;
|
||||
}
|
||||
|
||||
function startRefreshLeaseRenewal(object: ExternalObjectRecord, refreshToken: string) {
|
||||
const interval = setInterval(() => {
|
||||
const renewedAt = new Date();
|
||||
void db
|
||||
.update(externalObjects)
|
||||
.set({ refreshStartedAt: renewedAt, updatedAt: renewedAt })
|
||||
.where(refreshOwnerWhere(object, refreshToken))
|
||||
.catch((err: unknown) => {
|
||||
logger.warn({ err, objectId: object.id }, "external object refresh lease renewal failed");
|
||||
});
|
||||
}, REFRESH_LEASE_RENEW_INTERVAL_MS);
|
||||
interval.unref?.();
|
||||
return () => {
|
||||
clearInterval(interval);
|
||||
};
|
||||
}
|
||||
|
||||
const objectRefreshesInFlight = new Map<string, Promise<Awaited<ReturnType<typeof resolveObjectRefresh>>>>();
|
||||
|
||||
async function refreshObject(
|
||||
objectId: string,
|
||||
input: RefreshObjectInput,
|
||||
) {
|
||||
const now = input.now ?? new Date();
|
||||
const object = await db
|
||||
.select()
|
||||
.from(externalObjects)
|
||||
.where(and(eq(externalObjects.id, objectId), eq(externalObjects.companyId, input.companyId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!object) throw notFound("External object not found");
|
||||
if (!input.force && object.nextRefreshAt && object.nextRefreshAt > now) {
|
||||
return { object: toObjectPayload(object, now), refreshed: false, reason: "backoff" as const };
|
||||
}
|
||||
|
||||
const refreshKey = `${object.companyId}:${object.id}`;
|
||||
const existingRefresh = objectRefreshesInFlight.get(refreshKey);
|
||||
if (existingRefresh) return existingRefresh;
|
||||
|
||||
const claimed = await claimObjectRefresh(object, input, now);
|
||||
if (!claimed) {
|
||||
const latest = await db
|
||||
.select()
|
||||
.from(externalObjects)
|
||||
.where(and(eq(externalObjects.id, object.id), eq(externalObjects.companyId, object.companyId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!latest) throw notFound("External object not found");
|
||||
return { object: toObjectPayload(latest, now), refreshed: false, reason: "refresh_in_progress" as const };
|
||||
}
|
||||
if (!claimed.refreshToken) {
|
||||
throw new Error("External object refresh claim did not return a refresh token");
|
||||
}
|
||||
|
||||
const stopRenewingRefreshLease = startRefreshLeaseRenewal(object, claimed.refreshToken);
|
||||
const refresh = resolveObjectRefresh(object, input, now, claimed.refreshToken).finally(() => {
|
||||
stopRenewingRefreshLease();
|
||||
objectRefreshesInFlight.delete(refreshKey);
|
||||
});
|
||||
objectRefreshesInFlight.set(refreshKey, refresh);
|
||||
return refresh;
|
||||
}
|
||||
|
||||
async function refreshIssueObjects(issueId: string, input: {
|
||||
companyId: string;
|
||||
objectIds?: string[];
|
||||
|
|
@ -903,8 +1022,8 @@ export function externalObjectService(
|
|||
return results;
|
||||
}
|
||||
|
||||
async function refreshDueObjects(companyId: string, limit = 50, now = new Date()) {
|
||||
if (!(await isEnabled())) return [];
|
||||
async function refreshDueObjectsUnchecked(companyId: string, limit = 50, now = new Date()) {
|
||||
const staleRefreshStartedBefore = new Date(now.getTime() - DEFAULT_REFRESH_LEASE_SECONDS * 1000);
|
||||
const due = await db
|
||||
.select({ id: externalObjects.id })
|
||||
.from(externalObjects)
|
||||
|
|
@ -913,6 +1032,10 @@ export function externalObjectService(
|
|||
eq(externalObjects.companyId, companyId),
|
||||
eq(externalObjects.isTerminal, false),
|
||||
lte(externalObjects.nextRefreshAt, now),
|
||||
or(
|
||||
isNull(externalObjects.refreshStartedAt),
|
||||
lte(externalObjects.refreshStartedAt, staleRefreshStartedBefore),
|
||||
),
|
||||
),
|
||||
)
|
||||
.limit(limit);
|
||||
|
|
@ -927,6 +1050,27 @@ export function externalObjectService(
|
|||
return results;
|
||||
}
|
||||
|
||||
async function refreshDueObjects(companyId: string, limit = 50, now = new Date()) {
|
||||
if (!(await isEnabled())) return [];
|
||||
return refreshDueObjectsUnchecked(companyId, limit, now);
|
||||
}
|
||||
|
||||
async function refreshDueObjectsForActiveCompanies(limitPerCompany = 50, now = new Date()) {
|
||||
if (!(await isEnabled())) return { companies: 0, checked: 0, refreshed: 0 };
|
||||
const activeCompanies = await db
|
||||
.select({ id: companies.id })
|
||||
.from(companies)
|
||||
.where(eq(companies.status, "active"));
|
||||
let checked = 0;
|
||||
let refreshed = 0;
|
||||
for (const company of activeCompanies) {
|
||||
const results = await refreshDueObjectsUnchecked(company.id, limitPerCompany, now);
|
||||
checked += results.length;
|
||||
refreshed += results.filter((result) => result.refreshed).length;
|
||||
}
|
||||
return { companies: activeCompanies.length, checked, refreshed };
|
||||
}
|
||||
|
||||
return {
|
||||
syncIssue,
|
||||
syncComment,
|
||||
|
|
@ -941,5 +1085,6 @@ export function externalObjectService(
|
|||
refreshObject,
|
||||
refreshIssueObjects,
|
||||
refreshDueObjects,
|
||||
refreshDueObjectsForActiveCompanies,
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -91,6 +91,25 @@ describe("ExternalObjectPill", () => {
|
|||
expect(html).not.toContain("×");
|
||||
});
|
||||
|
||||
it("labels detected GitHub pull requests without fetched status as not refreshed", () => {
|
||||
const html = renderToStaticMarkup(
|
||||
<ExternalObjectPill
|
||||
object={{
|
||||
providerKey: "github",
|
||||
objectType: "pull_request",
|
||||
statusCategory: "unknown",
|
||||
liveness: "unknown",
|
||||
displayTitle: "acme/web#241",
|
||||
url: "https://github.com/acme/web/pull/241",
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(html).toContain("PR 241 - Not yet refreshed");
|
||||
expect(html).toContain('aria-label="GitHub pull request — Not yet refreshed: acme/web#241"');
|
||||
expect(html).not.toContain("Not yet resolved");
|
||||
});
|
||||
|
||||
it("uses the object link label, provider icon, and visible status when supplied", () => {
|
||||
const html = renderToStaticMarkup(
|
||||
<ExternalObjectPill
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ import {
|
|||
externalObjectLivenessOverlay,
|
||||
} from "../lib/status-colors";
|
||||
import {
|
||||
externalObjectCategoryLabel,
|
||||
externalObjectDisplayStatusLabel,
|
||||
externalObjectDisplayLabel,
|
||||
externalObjectLivenessLabel,
|
||||
externalObjectIconForKey,
|
||||
|
|
@ -114,7 +114,7 @@ export function ExternalObjectPill({
|
|||
const providerLabel = externalObjectProviderLabel(object.providerKey);
|
||||
const typeLabel = externalObjectTypeLabel(object.objectType);
|
||||
const displayKey = externalObjectDisplayLabel(object.providerKey, object.objectType, object.displayKey);
|
||||
const statusLabel = object.statusLabel ?? externalObjectCategoryLabel(object.statusCategory);
|
||||
const statusLabel = externalObjectDisplayStatusLabel(object);
|
||||
const tone = externalObjectPillTone(object, statusLabel);
|
||||
const valueLabel = externalObjectValueLabel(object, displayKey, statusLabel);
|
||||
const statusIconKey = externalObjectStatusIconKey(object, statusLabel);
|
||||
|
|
|
|||
|
|
@ -2484,6 +2484,28 @@ describe("IssueProperties", () => {
|
|||
sourceLabels: ["Comment"],
|
||||
},
|
||||
},
|
||||
{
|
||||
mentionCount: 1,
|
||||
sourceLabels: ["Comment"],
|
||||
pill: {
|
||||
providerKey: "github",
|
||||
objectType: "pull_request",
|
||||
displayKey: "Github PR",
|
||||
iconKey: "github",
|
||||
statusCategory: "unknown",
|
||||
statusIconKey: null,
|
||||
statusLabel: null,
|
||||
liveness: "unknown",
|
||||
displayTitle: "acme/web#242",
|
||||
url: "https://github.com/acme/web/pull/242",
|
||||
},
|
||||
group: {
|
||||
object: null,
|
||||
mentions: [],
|
||||
mentionCount: 1,
|
||||
sourceLabels: ["Comment"],
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
await flush();
|
||||
|
|
@ -2510,6 +2532,10 @@ describe("IssueProperties", () => {
|
|||
expect(pullRequestLink?.className).not.toContain("paperclip-mention-chip");
|
||||
expect(pullRequestLink?.className).not.toContain("rounded-full");
|
||||
expect(pullRequestLink?.className).not.toContain("border");
|
||||
const unrefreshedPullRequestLink = Array.from(container.querySelectorAll("a"))
|
||||
.find((anchor) => anchor.getAttribute("href") === "https://github.com/acme/web/pull/242");
|
||||
expect(unrefreshedPullRequestLink?.textContent).toContain("PR 242 - Not yet refreshed");
|
||||
expect(unrefreshedPullRequestLink?.textContent).not.toContain("Not yet resolved");
|
||||
|
||||
act(() => root.unmount());
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { useEffect, useState, type ReactNode } from "react";
|
||||
import type { IssueExternalObjectGroup } from "../../hooks/useIssueExternalObjects";
|
||||
import {
|
||||
externalObjectCategoryLabel,
|
||||
externalObjectDisplayStatusLabel,
|
||||
externalObjectDisplayLabel,
|
||||
externalObjectIconForKey,
|
||||
externalObjectProviderLabel,
|
||||
|
|
@ -66,7 +66,7 @@ function githubObjectPropertyValue(url: string | null | undefined): string | nul
|
|||
}
|
||||
|
||||
function externalObjectPropertyStatusLabel(group: IssueExternalObjectGroup): string {
|
||||
return group.pill.statusLabel ?? externalObjectCategoryLabel(group.pill.statusCategory);
|
||||
return externalObjectDisplayStatusLabel(group.pill);
|
||||
}
|
||||
|
||||
function externalObjectPropertyValue(group: IssueExternalObjectGroup): string {
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import {
|
|||
import {
|
||||
dominantExternalObjectTone,
|
||||
externalObjectCategoryLabel,
|
||||
externalObjectDisplayStatusLabel,
|
||||
externalObjectDisplayLabel,
|
||||
externalObjectDominantCount,
|
||||
externalObjectFallbackTone,
|
||||
|
|
@ -114,6 +115,42 @@ describe("external-objects helpers", () => {
|
|||
expect(externalObjectDisplayLabel("github", "pull_request")).toBe("GitHub pull request");
|
||||
});
|
||||
|
||||
it("labels unresolved known objects as not refreshed while preserving generic URL copy", () => {
|
||||
expect(
|
||||
externalObjectDisplayStatusLabel({
|
||||
providerKey: "github",
|
||||
objectType: "pull_request",
|
||||
statusCategory: "unknown",
|
||||
liveness: "unknown",
|
||||
}),
|
||||
).toBe("Not yet refreshed");
|
||||
expect(
|
||||
externalObjectDisplayStatusLabel({
|
||||
providerKey: "url",
|
||||
objectType: "link",
|
||||
statusCategory: "unknown",
|
||||
liveness: "unknown",
|
||||
}),
|
||||
).toBe("Not yet resolved");
|
||||
expect(
|
||||
externalObjectDisplayStatusLabel({
|
||||
providerKey: "github",
|
||||
objectType: "pull_request",
|
||||
statusCategory: "unknown",
|
||||
liveness: "fresh",
|
||||
}),
|
||||
).toBe("Status unavailable");
|
||||
expect(
|
||||
externalObjectDisplayStatusLabel({
|
||||
providerKey: "github",
|
||||
objectType: "pull_request",
|
||||
statusCategory: "open",
|
||||
liveness: "fresh",
|
||||
statusLabel: "Open",
|
||||
}),
|
||||
).toBe("Open");
|
||||
});
|
||||
|
||||
it("orders tones from danger down to muted", () => {
|
||||
expect(externalObjectToneSeverity("danger")).toBeGreaterThan(externalObjectToneSeverity("warning"));
|
||||
expect(externalObjectToneSeverity("warning")).toBeGreaterThan(externalObjectToneSeverity("info"));
|
||||
|
|
|
|||
|
|
@ -104,6 +104,24 @@ export function externalObjectLivenessLabel(liveness: string): string {
|
|||
return LIVENESS_LABELS[liveness] ?? liveness.replace(/_/g, " ");
|
||||
}
|
||||
|
||||
export function externalObjectDisplayStatusLabel(input: {
|
||||
providerKey: string | null | undefined;
|
||||
objectType: string | null | undefined;
|
||||
statusCategory: string;
|
||||
liveness: string;
|
||||
statusLabel?: string | null;
|
||||
}): string {
|
||||
const trimmedStatusLabel = input.statusLabel?.trim();
|
||||
if (trimmedStatusLabel) return trimmedStatusLabel;
|
||||
const isGenericUrl = input.providerKey === "url" && input.objectType === "link";
|
||||
const hasKnownObjectType = Boolean(input.providerKey && input.objectType);
|
||||
if (input.statusCategory === "unknown" && hasKnownObjectType && !isGenericUrl) {
|
||||
if (input.liveness === "fresh") return "Status unavailable";
|
||||
return externalObjectLivenessLabel(input.liveness);
|
||||
}
|
||||
return externalObjectCategoryLabel(input.statusCategory);
|
||||
}
|
||||
|
||||
/**
|
||||
* Higher number = more attention-worthy. The rollups in §5 sort by tone first.
|
||||
* Mirrors `externalObjectStatusToneSeverity` in `status-colors.ts`.
|
||||
|
|
|
|||
|
|
@ -71,6 +71,7 @@ function makeObject(args: {
|
|||
lastChangedAt: "2026-04-24T22:45:00.000Z",
|
||||
lastErrorAt: args.liveness === "unreachable" ? "2026-04-24T22:50:00.000Z" : null,
|
||||
nextRefreshAt: null,
|
||||
refreshStartedAt: null,
|
||||
lastErrorCode: null,
|
||||
lastErrorMessage: null,
|
||||
createdAt: "2026-04-24T20:00:00.000Z",
|
||||
|
|
|
|||
Loading…
Reference in New Issue