RomanNum3ral
/
paperclip
mirror of https://github.com/paperclipai/paperclip.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge current master into PR 4682

This commit is contained in:
Michel Tomas 2026-09-13 12:28:16 +02:00
parent 9bbfef3016 c9e3bb7ca4
commit ba42f0d36f
3769 changed files with 3346250 additions and 93746 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.agents/skills/prepare-paperclip-pr/SKILL.md
View File

@ -79,7 +79,6 @@ each one).
## Hard rules
* **YOU DO NOT MERGE THE PR YOURSELF. NEVER MERGE THE PR YOURSELF.**
* Never lose work: no orphaned stashes, no dropped files, no force-pushes
that discard commits.
* Always post the URLs to every pull request you created.

5
.agents/skills/release-changelog/SKILL.md
View File

@ -168,6 +168,11 @@ Guidelines:
- write from the user perspective
- keep highlights short and concrete
- spell out upgrade actions for breaking changes
- **write at full stable depth from the first pass**: the beta-keyed
draft ships verbatim as the stable's notes, so the previous stable's
file is the density bar the moment the draft is first written — never
leave it at generated-skeleton density for the soak. The skeleton's
nested PR summaries are raw material to rewrite, not a format to keep.
- **describe deltas, not repeats**: read the previous stable's notes
(`releases/v<last-stable>.md`) before writing. When they already
introduced a feature, this release's entry covers only what changed —

13
.codex/agents/codemod-runner.toml Normal file
View File

@ -0,0 +1,13 @@
name = "codemod-runner"
description = "Writes and runs codemod scripts that replace hardcoded visual values with token references in ui/src/index.css. Use for Phase 2 of the design simplification run — mechanical refactors only."
developer_instructions = """
You perform mechanical refactors via scripts, never hand-edits. Follow DESIGN.md at the repo root.
Rules:
- The token destination is ui/src/index.css (Tailwind v4), optionally a tokens.css imported by it. NEVER create a parallel token source. Tokens that must be runtime-tunable go in a NON-inline block — `@theme inline` bakes literals at build time.
- Where a hardcoded value EXACTLY matches an existing token, replace it with that token reference. Otherwise extract the value into a new token VERBATIM — no normalizing, rounding, or inventing a scale. Ugly values stay ugly.
- Every rewrite happens through a codemod script committed to scripts/ before it is run. Scripts must be idempotent and reviewable.
- Third-party style overrides that cannot use tokens go on a documented allowlist in the token source, each with an inline comment saying why.
- Verify after every script run: rg gates (zero hardcoded hex, zero arbitrary px/bracket values in ui/src/components/** and ui/src/pages/** outside the allowlist), pnpm typecheck, and the Storybook snapshot suite. Snapshots must match the Phase 0 baseline exactly.
- If a replacement cannot be made without visual change, skip it and record it in doc/design/TOKEN-AUDIT.md under "Needs human decision"."""

12
.codex/agents/token-auditor.toml Normal file
View File

@ -0,0 +1,12 @@
name = "token-auditor"
description = "Scans ui/src/ for hardcoded visual values, duplicate components, and shadcn replacement candidates; produces doc/design/TOKEN-AUDIT.md and doc/design/COMPONENT-INVENTORY.md. Read-only on source — never modifies component files. Use for Phase 1 of the design simplification run."
developer_instructions = """
You inventory design-system debt in this repository. Follow DESIGN.md at the repo root; read doc/design/PRIOR-ART.md first — a previous audit found only 6 of ~220 drift sites were exact-value-mappable to existing tokens, so expect most hardcoded values to need new verbatim tokens.
Your outputs (written to the repo root):
1. TOKEN-AUDIT.md — every hardcoded color/spacing/radius/type/shadow value in ui/src/, with frequency, file locations, and near-duplicate clusters (e.g. 13/14/15px used interchangeably). For each value, note whether it EXACTLY matches one of the ~80 existing tokens in ui/src/index.css (semantic / brand / domain tiers — see DESIGN.md). Flag clusters for human review; never merge or normalize them. Include a "Needs human decision" section.
2. COMPONENT-INVENTORY.md — all components under ui/src/components/ (24 primitives in ui/, ~277 feature components), their variants, and suspected duplicates with evidence (similar props, similar rendered output, copy-pasted origins). Include a "shadcn candidates" section: (a) custom components duplicating an available shadcn primitive, (b) installed shadcn components that drifted from the registry (npx shadcn@latest diff where available), (c) raw Radix/plain elements where an installed shadcn wrapper exists. For each, state the recommended replacement and expected visual impact. Recommendations only — merges and swaps happen in later human-approved runs, never this one.
Never modify source files. Bash access is for read-only commands (rg, find, npx shadcn diff) and writing the two report files only."""

34
.dockerignore
View File

@ -8,3 +8,37 @@ coverage
data
tmp
*.log
packages/paperclip-runner/dist
packages/paperclip-runner/runner/target
packages/paperclip-runner/devtools
packages/paperclip-runner/docs
packages/paperclip-runner/examples
packages/paperclip-runner/test
packages/paperclip-runner/test-fixtures
packages/paperclip-runner/test-support
packages/paperclip-runner/**/*.md
packages/paperclip-runner/**/*.spec.ts
packages/paperclip-runner/**/*.spec.tsx
packages/paperclip-runner/**/*.test.cjs
packages/paperclip-runner/**/*.test.cts
packages/paperclip-runner/**/*.test.js
packages/paperclip-runner/**/*.test.jsx
packages/paperclip-runner/**/*.test.mjs
packages/paperclip-runner/**/*.test.mts
packages/paperclip-runner/**/*.test.ts
packages/paperclip-runner/**/*.test.tsx
packages/paperclip-runner/runner/crates/*/tests
packages/paperclip-runner/scripts/*-smoke.mjs
# Exceptions (last match wins): the image build re-runs the runner's
# generated-file drift checks, so their committed outputs and inputs must
# survive the slimming above. 2026-09-04: the *.md rule stripped the
# committed capability contract out of the context and every image build
# on master failed its drift check — .github/docker-context-checks.Dockerfile
# now guards this in PR CI.
!packages/paperclip-runner/generated/**
!packages/paperclip-runner/docs/capability-contract.md
# check:runner-workflow-traceability access()es every regression test the
# stress-traceability spec names — those are src/**/*.test.ts files, so
# they must survive the *.test.ts rule above.
!packages/paperclip-runner/src/**/*.test.ts
!packages/paperclip-runner/src/**/*.test.tsx

12
.env.example
View File

@ -4,11 +4,23 @@ SERVE_UI=false
BETTER_AUTH_SECRET=paperclip-dev-secret
PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=paperclip-dev-tool-action-signing-secret-change-me
# Optional Paperclip ID Gmail OAuth broker. Enroll the instance first; keep both
# private keys in the deployment secret manager. HTTP base URLs must be loopback.
# PAPERCLIP_ID_CONNECTOR_BASE_URL=http://localhost:3000
# PAPERCLIP_ID_CONNECTOR_ENVIRONMENT=development
# PAPERCLIP_ID_CONNECTOR_INSTANCE_ID=inst_example
# PAPERCLIP_ID_CONNECTOR_SIGN_PRIVATE_KEY=
# PAPERCLIP_ID_CONNECTOR_SEAL_PRIVATE_KEY=
# Process-wide protection for expensive full-tree workspace Git scans.
# PAPERCLIP_WORKSPACE_GIT_SCAN_CONCURRENCY=2
# PAPERCLIP_WORKSPACE_GIT_SCAN_QUEUE_CAPACITY=32
# PAPERCLIP_WORKSPACE_GIT_SCAN_TIMEOUT_MS=8000
# PAPERCLIP_WORKSPACE_GIT_SCAN_CACHE_TTL_MS=10000
# HTTP adapters may call public HTTP(S) origins by default. Opt trusted private
# origins in explicitly; entries are exact origins (scheme, host, and port).
# PAPERCLIP_HTTP_ADAPTER_PRIVATE_ENDPOINT_ALLOWLIST=http://hooks.internal.example:8080
# Discord webhook for daily merge digest (scripts/discord-daily-digest.sh)
# DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...

10
.env.runner-e2e.example Normal file
View File

@ -0,0 +1,10 @@
# Copy to .env.runner-e2e.local. Existing shell variables take precedence.
# Never commit the local file or put these values in fixture source.
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
OPENROUTER_API_KEY=
DAYTONA_API_KEY=
# Required only for Daytona cells. Use an immutable, anonymously pullable
# digest from the Runner Full-Stack E2E image job or a locally published image.
PAPERCLIP_E2E_DAYTONA_IMAGE=ghcr.io/paperclipai/paperclip-daytona-runner@sha256:REPLACE_ME

1
.gitattributes vendored Normal file
View File

@ -0,0 +1 @@
patches/*.patch whitespace=-space-before-tab

26
.github/CODEOWNERS vendored
View File

@ -1,18 +1,18 @@
# Replace @cryppadotta if a different maintainer or team should own release infrastructure.
.github/** @cryppadotta @devinfoley @nickyleach
scripts/release*.sh @cryppadotta @devinfoley @nickyleach
scripts/release-*.mjs @cryppadotta @devinfoley @nickyleach
scripts/create-github-release.sh @cryppadotta @devinfoley @nickyleach
scripts/rollback-latest.sh @cryppadotta @devinfoley @nickyleach
doc/RELEASING.md @cryppadotta @devinfoley @nickyleach
doc/PUBLISHING.md @cryppadotta @devinfoley @nickyleach
doc/RELEASE-AUTOMATION-SETUP.md @cryppadotta @devinfoley @nickyleach
skills/** @cryppadotta @devinfoley @nickyleach
.github/** @cryppadotta @devinfoley @nickyleach @forgottendev
scripts/release*.sh @cryppadotta @devinfoley @nickyleach @forgottendev
scripts/release-*.mjs @cryppadotta @devinfoley @nickyleach @forgottendev
scripts/create-github-release.sh @cryppadotta @devinfoley @nickyleach @forgottendev
scripts/rollback-latest.sh @cryppadotta @devinfoley @nickyleach @forgottendev
doc/RELEASING.md @cryppadotta @devinfoley @nickyleach @forgottendev
doc/PUBLISHING.md @cryppadotta @devinfoley @nickyleach @forgottendev
doc/RELEASE-AUTOMATION-SETUP.md @cryppadotta @devinfoley @nickyleach @forgottendev
skills/** @cryppadotta @devinfoley @nickyleach @forgottendev
# Package files — dependency changes require review
# package.json matches recursively at all depths (covers root + all workspaces)
package.json @cryppadotta @devinfoley @nickyleach
pnpm-lock.yaml @cryppadotta @devinfoley @nickyleach
pnpm-workspace.yaml @cryppadotta @devinfoley @nickyleach
.npmrc @cryppadotta @devinfoley @nickyleach
package.json @cryppadotta @devinfoley @nickyleach @forgottendev
pnpm-lock.yaml @cryppadotta @devinfoley @nickyleach @forgottendev
pnpm-workspace.yaml @cryppadotta @devinfoley @nickyleach @forgottendev
.npmrc @cryppadotta @devinfoley @nickyleach @forgottendev

8
.github/dependabot.yml vendored
View File

@ -14,6 +14,14 @@ updates:
open-pull-requests-limit: 20
labels:
- "dependencies"
# Dependabot's npm parser reads only dependencies, devDependencies, and
# optionalDependencies — never peerDependencies. It cannot see the
# optional OpenTelemetry peer dependencies in server/package.json, so it
# never bumps their declared versions. The same limit applies to the
# optional @sentry/node peer dependency in server/package.json: Dependabot
# cannot bump it either, for the same reason. @sentry/browser stays a
# normal devDependency of ui/package.json, so Dependabot does track that
# one.
ignore:
# @types/node describes the APIs available in the supported Node runtime.
# Runtime major upgrades are deliberate compatibility changes, so keep

60
.github/docker-context-checks.Dockerfile vendored Normal file
View File

@ -0,0 +1,60 @@
# Runs the runner's generated-file drift checks against the EXACT build
# context the image builds see — same .dockerignore semantics — so a
# context-slimming change that strips a committed build input fails the
# pull request instead of every post-merge image build. (2026-09-04: a new
# `packages/paperclip-runner/**/*.md` ignore rule stripped the committed
# capability contract out of the context; every Docker build on master then
# failed its drift check, and no cloud image published for eight hours
# while PR CI stayed green.)
#
# Only checks whose compared output is independent of dependency versions
# run here: ajv is installed for schema VALIDATION only (pinned to the
# runner's declared range), while codegen checks like
# generate-protocol-schema-module stay out — their emitted bytes vary with
# the ajv release, so running them against a fresh install would raise
# false drift alarms. Those still run inside the real image build, which
# installs the locked dependency tree; the existence assertions below keep
# their committed inputs and outputs covered by this probe regardless.
#
# node:24-slim — the runner requires Node >= 24.11 and the production
# image builds on Node 24; the digest pin keeps the security gate's own
# runtime immutable.
FROM node:24-slim@sha256:ba849c60be29959425b8734d57b8b4b7d56f98edd9504c9af091d5281095a71e
WORKDIR /context
COPY . .
# Committed artifacts the image build reads whose drift checks cannot run
# here (they need the locked dependency tree or compiled dist/). Existence
# in the context is the property this probe guards; content correctness is
# the real build's job. If a path is intentionally removed from the repo,
# update this list in the same PR.
RUN test -f packages/paperclip-runner/generated/capability/semantic-tool-contracts.json \
&& test -f packages/paperclip-runner/generated/semantic-action-catalog.json \
&& test -f packages/paperclip-runner/spec/evals/stress-workflow-traceability.json \
&& test -d packages/paperclip-runner/protocol/fixtures/replay
# check:runner-workflow-traceability access()es every regression test its
# spec names (it needs dist/ to RUN, so it cannot run here) — replicate
# exactly its existence walk, driven by the spec itself so this never
# needs a hand-maintained path list. (2026-09-04, second unmasking: the
# *.test.ts ignore rule stripped src/contracts/native-execution.test.ts
# and the image build failed there once the capability checks were fixed.)
RUN node -e ' \
const manifest = require("/context/packages/paperclip-runner/spec/evals/stress-workflow-traceability.json"); \
const { accessSync } = require("node:fs"); \
const { resolve } = require("node:path"); \
let count = 0; \
for (const finding of manifest.findings) \
for (const path of finding.regressionTests) { \
accessSync(resolve("/context/packages/paperclip-runner", path)); \
count += 1; \
} \
console.log(`traceability regression-test paths present: ${count}`);'
# ajv is installed in an isolated directory (the runner's own package.json
# uses workspace: ranges npm cannot install from) and symlinked in so ESM
# resolution finds it from the scripts' location.
RUN AJV_RANGE="$(node -p "require('/context/packages/paperclip-runner/package.json').dependencies.ajv")" \
&& mkdir /probe-deps && cd /probe-deps && npm init -y >/dev/null \
&& npm install --ignore-scripts --no-audit --no-fund "ajv@${AJV_RANGE}" \
&& ln -s /probe-deps/node_modules /context/packages/paperclip-runner/node_modules \
&& cd /context/packages/paperclip-runner \
&& node scripts/generate-capability-contract.mjs --check \
&& node scripts/check-capability-inventory.mjs

50
.github/scripts/authorize-storybook-deploy.cjs vendored Normal file
View File

@ -0,0 +1,50 @@
// Run before building, and again inside the protected deployment job on reruns.
module.exports = async function authorizeStorybookDeploy({ github, context }) {
const fail = (message) => { throw new Error(message); };
if (context.repo.owner !== "paperclipai" || context.repo.repo !== "paperclip") {
fail("Storybook publishing is restricted to paperclipai/paperclip.");
}
if (context.eventName !== "workflow_dispatch" || !context.ref.startsWith("refs/heads/")) {
fail("Storybook publishing requires a manual run from a repository branch.");
}
// The selected branch must never be able to add itself to the allowlist.
const { data: repository } = await github.rest.repos.get(context.repo);
const { data: file } = await github.rest.repos.getContent({
...context.repo,
path: ".github/CODEOWNERS",
ref: repository.default_branch,
});
if (file.encoding !== "base64" || typeof file.content !== "string") {
fail("Cannot read the default branch CODEOWNERS file.");
}
const owners = new Set();
for (const line of Buffer.from(file.content, "base64").toString("utf8").split(/\r?\n/)) {
const fields = line.split("#", 1)[0].trim().split(/\s+/);
for (const owner of fields.slice(1)) {
// Individual GitHub accounts only. Teams/email entries do not grant access.
if (/^@[a-z\d](?:[a-z\d-]*[a-z\d])?$/i.test(owner)) {
owners.add(owner.slice(1).toLowerCase());
}
}
}
if (owners.size === 0) fail("CODEOWNERS has no individual GitHub accounts.");
for (const actor of [context.actor, process.env.GITHUB_TRIGGERING_ACTOR]) {
if (!actor || !owners.has(actor.toLowerCase())) {
fail(`Only default-branch CODEOWNERS may publish Storybook (${actor || "missing actor"}).`);
}
}
// A branch can edit its workflow. Require a GitHub-enforced CODEOWNER review
// as well, so editing this check cannot grant an outsider deployment access.
const { data: environment } = await github.rest.repos.getEnvironment({
...context.repo,
environment_name: "storybook-deploy",
});
const reviewers = environment.protection_rules
?.find((rule) => rule.type === "required_reviewers")?.reviewers;
if (environment.can_admins_bypass !== false || !reviewers?.length ||
reviewers.some(({ type, reviewer }) => type !== "User" || !owners.has(reviewer.login.toLowerCase()))) {
fail("storybook-deploy must require CODEOWNER reviewers and disable administrator bypass.");
}
};

112
.github/scripts/check-pr-migration-order.mjs vendored Normal file
View File

@ -0,0 +1,112 @@
#!/usr/bin/env node
import { execFileSync } from 'node:child_process';
import { fileURLToPath } from 'node:url';
const MIGRATIONS_DIRECTORY = 'packages/db/src/migrations/';
const MIGRATION_FILE_PATTERN = /^packages\/db\/src\/migrations\/(\d{4})_[^/]+\.sql$/;
function parseMigration(file) {
const match = file.match(MIGRATION_FILE_PATTERN);
return match ? { file, number: Number.parseInt(match[1], 10) } : null;
}
function formatMigrationNumber(number) {
return String(number).padStart(4, '0');
}
export function checkMigrationOrder(baseMigrationFiles, prMigrationFiles) {
const invalidFiles = [...baseMigrationFiles, ...prMigrationFiles]
.filter((file) => !parseMigration(file));
if (invalidFiles.length > 0) {
return {
passed: false,
message: [
'Migration SQL files must start with a 4-digit number:',
...invalidFiles.map((file) => `- ${file}`),
].join('\n'),
};
}
if (prMigrationFiles.length === 0) {
return { passed: true, message: 'No new migrations in this PR.' };
}
const baseMigrations = baseMigrationFiles.map(parseMigration);
const prMigrations = prMigrationFiles.map(parseMigration);
const latestBaseMigration = baseMigrations.reduce(
(latest, migration) => migration.number > latest.number ? migration : latest,
{ file: '(none)', number: -1 },
);
const outOfOrder = prMigrations.filter(
(migration) => migration.number <= latestBaseMigration.number,
);
if (outOfOrder.length === 0) {
return {
passed: true,
message: `All new migrations follow ${latestBaseMigration.file}.`,
};
}
const nextNumber = formatMigrationNumber(latestBaseMigration.number + 1);
return {
passed: false,
message: [
`The target branch already contains migrations through ${latestBaseMigration.file}.`,
'This PR adds migration numbers that would be inserted into or collide with that history:',
...outOfOrder.map((migration) => `- ${migration.file}`),
'',
`Update from the target branch, then renumber this PR's migrations starting at ${nextNumber}`,
'in their intended order. Keep each SQL filename, matching meta snapshot, and',
'packages/db/src/migrations/meta/_journal.json entry aligned, then push again.',
'Migration numbers are append-only and cannot reuse a number already present on the target branch.',
].join('\n'),
};
}
function gitPaths(args) {
return execFileSync('git', args, { encoding: 'utf8' })
.split('\0')
.filter(Boolean);
}
function escapeWorkflowCommand(message) {
return message
.replaceAll('%', '%25')
.replaceAll('\r', '%0D')
.replaceAll('\n', '%0A');
}
function main() {
const [baseSha, headSha] = process.argv.slice(2);
const shaPattern = /^[0-9a-f]{40}$/i;
if (!shaPattern.test(baseSha ?? '') || !shaPattern.test(headSha ?? '')) {
console.error('Usage: check-pr-migration-order.mjs <40-character base SHA> <40-character head SHA>');
process.exit(2);
}
const baseMigrationFiles = gitPaths([
'ls-tree', '-r', '--name-only', '-z', baseSha, '--', MIGRATIONS_DIRECTORY,
]).filter((file) => file.endsWith('.sql'));
const prMigrationFiles = gitPaths([
'diff', '--name-only', '--diff-filter=A', '-z', `${baseSha}...${headSha}`, '--',
MIGRATIONS_DIRECTORY,
]).filter((file) => file.endsWith('.sql'));
const result = checkMigrationOrder(baseMigrationFiles, prMigrationFiles);
if (result.passed) {
console.log(result.message);
return;
}
console.error(
`::error title=Migration numbers must follow the target branch::${escapeWorkflowCommand(result.message)}`,
);
console.error(result.message);
process.exit(1);
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
main();
}

46
.github/scripts/publish-storybook.cjs vendored Normal file
View File

@ -0,0 +1,46 @@
const { execFileSync } = require('node:child_process');
const fs = require('node:fs');
const path = require('node:path');
const { storybookDestination, branchIndex } = require('./storybook-destination.cjs');
const destination = storybookDestination({
branch: process.env.SOURCE_BRANCH, sha: process.env.SOURCE_SHA,
runId: process.env.GITHUB_RUN_ID, runAttempt: process.env.GITHUB_RUN_ATTEMPT,
bucket: process.env.STORYBOOK_S3_BUCKET, baseUrl: process.env.STORYBOOK_PUBLIC_BASE_URL,
});
const source = path.resolve('storybook-static');
// Treat the artifact as public files, never as executable publisher code.
function validateTree(dir) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
if (entry.isSymbolicLink() || entry.name.startsWith('.') || (!entry.isDirectory() && !entry.isFile())) {
throw new Error(`Unsupported artifact entry: ${path.join(dir, entry.name)}`);
}
if (entry.isDirectory()) validateTree(path.join(dir, entry.name));
}
}
validateTree(source);
for (const name of ['index.html', 'iframe.html', 'index.json']) {
if (!fs.statSync(path.join(source, name)).isFile() || fs.statSync(path.join(source, name)).size === 0) {
throw new Error(`Missing Storybook output: ${name}`);
}
}
fs.writeFileSync(path.join(source, 'deployment.json'), JSON.stringify(destination, null, 2) + '\n');
const aws = (args) => execFileSync('aws', args, { stdio: 'inherit' });
// Complete a unique build before changing the branch's entry point. No deletion
// permissions, shared root writes or mixed-version branch assets are needed.
aws(['s3', 'cp', source, `s3://${destination.bucket}/${destination.buildPrefix}/`,
'--recursive', '--no-follow-symlinks', '--only-show-errors',
'--cache-control', 'public,max-age=31536000,immutable']);
const indexFile = path.join(process.env.RUNNER_TEMP, 'storybook-branch-index.html');
fs.writeFileSync(indexFile, branchIndex(destination.buildUrl));
aws(['s3', 'cp', indexFile, `s3://${destination.bucket}/${destination.prefix}/index.html`,
'--content-type', 'text/html; charset=utf-8', '--cache-control', 'no-cache,max-age=0,must-revalidate', '--only-show-errors']);
aws(['s3', 'cp', indexFile, `s3://${destination.bucket}/${destination.bookmarkPrefix}/index.html`,
'--content-type', 'text/html; charset=utf-8', '--cache-control', 'no-cache,max-age=0,must-revalidate', '--only-show-errors']);
const report = `[Branch Storybook](${destination.url})\n\n[This build](${destination.buildUrl})\n\nCommit: \`${destination.sha}\`\n`;
const reportPath = path.join(process.env.RUNNER_TEMP, 'storybook-deployment.md');
fs.writeFileSync(reportPath, report);
if (process.env.GITHUB_OUTPUT) fs.appendFileSync(process.env.GITHUB_OUTPUT,
`url=${destination.url}\nbuild_url=${destination.buildUrl}\nreport_path=${reportPath}\n`);
if (process.env.GITHUB_STEP_SUMMARY) fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, report);
console.log(JSON.stringify(destination));

47
.github/scripts/storybook-destination.cjs vendored Normal file
View File

@ -0,0 +1,47 @@
const { createHash } = require('node:crypto');
function storybookDestination({ branch, sha, runId, runAttempt, bucket, baseUrl }) {
if (typeof branch !== 'string' || !branch || /[\x00-\x20\x7f]/.test(branch)) {
throw new Error('A non-empty repository branch name is required.');
}
if (!/^[a-f0-9]{40}$/.test(sha)) throw new Error('A full source commit SHA is required.');
if (![runId, runAttempt].every((value) => /^[1-9]\d*$/.test(String(value)))) {
throw new Error('A valid workflow run and attempt are required.');
}
if (!/^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/.test(bucket)) throw new Error('Invalid Storybook S3 bucket.');
const base = new URL(baseUrl);
if (base.protocol !== 'https:' || base.username || base.password || base.search || base.hash || base.pathname !== '/') {
throw new Error('Storybook base URL must be a credential-free HTTPS origin.');
}
const label = branch.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').slice(0,60) || 'branch';
const digest = createHash('sha256').update(branch).digest('hex').slice(0,16);
const branchKey = `${label}-${digest}`;
const prefix = `storybook/branches/${branchKey}`;
const buildPrefix = `${prefix}/builds/${runId}-${runAttempt}`;
// Use one reversible path segment: slashes and special characters become
// ~HH UTF-8 bytes, so feature/foo and feature-foo never share a bookmark.
let bookmarkKey = [...Buffer.from(branch)].map((byte) =>
/[A-Za-z0-9_-]/.test(String.fromCharCode(byte))
? String.fromCharCode(byte) : `~${byte.toString(16).toUpperCase().padStart(2, '0')}`).join('');
// Reserve the existing hashed directories, including all immutable builds.
bookmarkKey = bookmarkKey.replace(/-([a-f0-9]{16})$/, '~2D$1');
// Keep arbitrarily long ref names within S3's object-key limit. ~long cannot
// occur in the reversible encoding, whose escapes contain only hex digits.
if (bookmarkKey.length > 900) bookmarkKey = `${bookmarkKey.slice(0, 800)}~long-${digest}`;
const bookmarkPrefix = `storybook/branches/${bookmarkKey}`;
return {
branch, sha, bucket, branchKey, prefix, buildPrefix, bookmarkPrefix,
url: `${base.origin}/${bookmarkPrefix}/`,
legacyUrl: `${base.origin}/${prefix}/index.html`,
buildUrl: `${base.origin}/${buildPrefix}/index.html`,
};
}
function branchIndex(buildUrl) {
// The target is generated from a validated origin and ASCII path segments.
const target = JSON.stringify(buildUrl).replace(/</g, '\\u003c');
return `<!doctype html><html lang="en"><meta charset="utf-8"><title>Storybook preview</title>
<script>const target = new URL(${target}); target.search = location.search; target.hash = location.hash; location.replace(target.href);</script>
<noscript><a href="${buildUrl}">Open this branch's Storybook</a></noscript></html>\n`;
}
module.exports = { storybookDestination, branchIndex };

43
.github/scripts/tests/check-pr-migration-order.test.mjs vendored Normal file
View File

@ -0,0 +1,43 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { checkMigrationOrder } from '../check-pr-migration-order.mjs';
const migration = (name) => `packages/db/src/migrations/${name}.sql`;
test('passes when a PR has no new migrations', () => {
const result = checkMigrationOrder([migration('0230_on_master')], []);
assert.equal(result.passed, true);
});
test('passes when every PR migration follows the target branch', () => {
const result = checkMigrationOrder(
[migration('0230_on_master')],
[migration('0231_first_in_pr'), migration('0232_second_in_pr')],
);
assert.equal(result.passed, true);
});
test('fails with renumbering guidance when a PR reuses the target branch number', () => {
const result = checkMigrationOrder(
[migration('0230_on_master')],
[migration('0230_from_stale_branch')],
);
assert.equal(result.passed, false);
assert.match(result.message, /already contains migrations through .*0230_on_master\.sql/);
assert.match(result.message, /renumber this PR's migrations starting at 0231/);
assert.match(result.message, /meta\/_journal\.json/);
});
test('fails when a PR inserts a migration before the target branch tip', () => {
const result = checkMigrationOrder(
[migration('0230_on_master')],
[migration('0229_from_stale_branch'), migration('0231_valid_but_after_stale')],
);
assert.equal(result.passed, false);
assert.match(result.message, /0229_from_stale_branch\.sql/);
assert.doesNotMatch(result.message, /- packages\/db\/src\/migrations\/0231_valid_but_after_stale\.sql/);
});

91
.github/scripts/tests/cloud-readiness.test.mjs vendored Normal file
View File

@ -0,0 +1,91 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { waitForCloudArtifacts } from "../../../scripts/cloud-readiness.mjs";
import { previewManifest } from "../../../scripts/preview-artifacts.mjs";
const sha = "a".repeat(40);
const digest = `sha256:${"b".repeat(64)}`;
const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
function registry({ missing = new Set(), failure, wrongImage = false, wrongPackage = false } = {}) {
return async (url) => {
if (failure) return json({}, failure);
if (url.startsWith("https://registry.npmjs.org/")) {
const name = decodeURIComponent(new URL(url).pathname.split("/")[1]);
if (missing.has(name.split("/")[1])) return json({}, 404);
const pkg = previewManifest({ name, version: "0.0.0" }, sha);
return json({ ...pkg, ...(wrongPackage ? { gitHead: "c".repeat(40) } : {}), dist: { integrity: "sha512-fixture", tarball: "https://registry.npmjs.org/fixture.tgz" } });
}
if (url.includes("/token?")) return json({ token: "fixture" });
if (url.includes("/manifests/")) return missing.has("image") ? json({}, 404) : json({ config: { digest } });
if (url.includes("/blobs/")) return json({ config: { Labels: { "org.opencontainers.image.revision": wrongImage ? "c".repeat(40) : sha } } });
throw new Error(`Unexpected request: ${url}`);
};
}
test("readiness requires the image and both exact-source packages on the successful poll", async () => {
const missing = new Set(["image", "shared", "db"]);
let clock = 0;
const states = [];
const result = await waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing }), now: () => clock, intervalMs: 10, timeoutMs: 100, log: (message) => states.push(message),
sleep: async (ms) => {
clock += ms;
if (clock === 10) missing.delete("image");
if (clock === 20) missing.delete("shared");
if (clock === 30) { missing.delete("db"); missing.add("image"); }
if (clock === 40) missing.delete("image");
},
});
assert.equal(clock, 40, "an artifact disappearing before the final poll must prevent readiness");
assert.deepEqual(result, { version: 1, sha, packageVersion: `0.0.0-preview.g${sha}` });
assert.match(states.at(-1), /Cloud artifacts available/);
});
test("missing artifacts time out with a precise inventory and bounded sleep", async () => {
let clock = 0;
const sleeps = [];
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry({ missing: new Set(["db"]) }), now: () => clock, timeoutMs: 25, intervalMs: 20, log: () => {},
sleep: async (ms) => { sleeps.push(ms); clock += ms; },
}), /timed out.*missing: db/);
assert.deepEqual(sleeps, [20, 5]);
});
for (const fixture of [{ failure: 403 }, { failure: 503 }, { wrongImage: true }, { wrongPackage: true }]) {
test(`registry errors and identity mismatches fail without waiting: ${JSON.stringify(fixture)}`, async () => {
await assert.rejects(waitForCloudArtifacts(sha, {
fetchImpl: registry(fixture), sleep: async () => assert.fail("must not retry an invalid artifact or upstream error"), log: () => {},
}));
});
}
test("invalid source and timing configuration are rejected before registry access", async () => {
const fetchImpl = async () => assert.fail("invalid inputs must not reach a registry");
await assert.rejects(waitForCloudArtifacts("master", { fetchImpl }), /full immutable commit SHA/);
for (const options of [{ timeoutMs: 0 }, { intervalMs: -1 }, { timeoutMs: Infinity }]) {
await assert.rejects(waitForCloudArtifacts(sha, { ...options, fetchImpl }), /positive finite/);
}
});
test("the versioned readiness job requires successful source, image and artifact jobs", () => {
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
assert.match(workflow, /push:\s*\n\s*branches: \[master\]/);
assert.match(workflow, /group: cloud-readiness-\$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/release-verify.yml\s+with:\s+ref: \$\{\{ github.sha \}\}/);
assert.match(workflow, /uses: \.\/\.github\/workflows\/docker-cloud.yml/);
const ready = workflow.split(" ready:")[1];
assert.match(ready, /name: Cloud deployable v1/);
assert.match(ready, /needs: \[verify, image, artifacts\]/);
assert.match(ready, /if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'/);
assert.doesNotMatch(ready, /^\s*(?:if:.*always\(|continue-on-error:)/m);
assert.doesNotMatch(workflow, /secrets: inherit|id-token: write|actions: write|checks: write|uses: .*@v\d\b/);
const cloud = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
assert.doesNotMatch(cloud, /^ push:/m, "the master image must build only once");
const migrator = readFileSync(new URL("../../workflows/cloud-artifacts.yml", import.meta.url), "utf8");
assert.match(migrator, /push:\s*\n\s*branches: \[master\]/);
assert.match(migrator, /SOURCE_SHA: \$\{\{ github.sha \}\}/);
assert.match(migrator, /gh workflow run release.yml .*--ref master/);
assert.match(migrator, /--field channel=cloud-migrator/);
assert.match(migrator, /--field source_ref="\$SOURCE_SHA"/);
});

35
.github/scripts/tests/cloud-runner-routing.test.mjs vendored Normal file
View File

@ -0,0 +1,35 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { runInNewContext } from "node:vm";
const workflow = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
// Exercise the workflow's actual boolean expression. Its string comparisons and
// boolean operators have the same results in JS for these canonical contexts.
const expression = workflow.match(/^ runs-on: \$\{\{ (.+) \}\}$/m)?.[1];
assert.ok(expression, "cloud routing must remain an explicit job expression");
const timeoutExpression = workflow.match(/^ timeout-minutes: \$\{\{ (.+) \}\}$/m)?.[1];
assert.ok(timeoutExpression, "AWS jobs must finish before the Fleet instance lifetime");
const fleet = "runs-on/fleet=paperclip-cloud-build-x64/env=public-ci";
const base = { repository: "paperclipai/paperclip", repository_id: "1170821064", ref: "refs/heads/master", event_name: "push" };
for (const { name, github = {}, enabled = "true", expected = "ubuntu-latest" } of [
{ name: "canonical master push", expected: fleet },
{ name: "manual master build", github: { event_name: "workflow_dispatch" }, expected: fleet },
{ name: "disabled switch", enabled: "false" },
{ name: "missing switch", enabled: "" },
{ name: "invalid switch", enabled: "yes" },
{ name: "fork", github: { repository: "someone/paperclip", repository_id: "123" } },
{ name: "wrong repository identity", github: { repository_id: "123" } },
{ name: "pull request", github: { event_name: "pull_request", ref: "refs/pull/123/merge" } },
{ name: "privileged PR event", github: { event_name: "pull_request_target" } },
{ name: "release tag", github: { ref: "refs/tags/v2026.911.0" } },
{ name: "branch push", github: { ref: "refs/heads/feature" } },
{ name: "manual branch build", github: { event_name: "workflow_dispatch", ref: "refs/heads/feature" } },
{ name: "workflow completion event", github: { event_name: "workflow_run" } },
]) {
test(`cloud runner routing: ${name}`, () => {
const context = { github: { ...base, ...github }, vars: { AWS_CLOUD_BUILDS_ENABLED: enabled } };
assert.equal(runInNewContext(expression, context), expected);
assert.equal(runInNewContext(timeoutExpression, context), expected === fleet ? 40 : 60);
});
}

65
.github/scripts/tests/docker-disk-workflow.test.mjs vendored Normal file
View File

@ -0,0 +1,65 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { spawnSync } from "node:child_process";
const workflow = readFileSync(new URL("../../workflows/docker-cloud.yml", import.meta.url), "utf8");
const step = workflow.split(" - name: Free runner disk")[1].split(" - name: Login to GitHub Container Registry")[0];
const script = step.split(" run: |\n")[1].split("\n").map((line) => line.replace(/^ {10}/, "")).join("\n");
const threshold = 64 * 1024 * 1024;
for (const { name, dockerFree, workspaceFree, dfStatus = "0", infoStatus = "0", cleanup } of [
{ name: "ample free space", dockerFree: threshold + 1, workspaceFree: threshold + 1, cleanup: false },
{ name: "exactly the headroom threshold", dockerFree: threshold, workspaceFree: threshold, cleanup: false },
{ name: "Docker filesystem below threshold", dockerFree: threshold - 1, workspaceFree: threshold + 1, cleanup: true },
{ name: "workspace filesystem below threshold", dockerFree: threshold + 1, workspaceFree: threshold - 1, cleanup: true },
{ name: "invalid Docker measurement", dockerFree: "unknown", workspaceFree: threshold + 1, cleanup: true },
{ name: "invalid workspace measurement", dockerFree: threshold + 1, workspaceFree: "unknown", cleanup: true },
{ name: "failed df command", dockerFree: threshold + 1, workspaceFree: threshold + 1, dfStatus: "1", cleanup: true },
{ name: "failed Docker inspection", dockerFree: threshold + 1, workspaceFree: threshold + 1, infoStatus: "1", cleanup: true },
]) {
test(`cloud disk cleanup: ${name}`, () => {
const dir = mkdtempSync(path.join(tmpdir(), "cloud-disk-test-"));
const log = path.join(dir, "commands.log");
// Every mutating command is a recording fixture; no real SDKs, caches,
// images, or directories are deleted when the workflow shell executes.
const fixture = `#!/bin/bash
printf '%s %s\\n' "\${0##*/}" "$*" >> "$COMMAND_LOG"
case "\${0##*/}" in
df)
printf 'Filesystem 1024-blocks Used Available Capacity Mounted on\\n'
if [ "$1" = '-Pk' ]; then
printf '/dev/docker 200000000 1 %s 1%% /docker\\n' "$DOCKER_FREE"
printf '/dev/workspace 200000000 1 %s 1%% /workspace\\n' "$WORKSPACE_FREE"
exit "$DF_STATUS"
fi
;;
docker)
if [ "$1" = 'info' ]; then
printf '/docker-data\\n'
exit "$INFO_STATUS"
fi
;;
esac
`;
try {
for (const command of ["df", "docker", "pnpm", "sudo"]) {
writeFileSync(path.join(dir, command), fixture, { mode: 0o755 });
}
const result = spawnSync("bash", ["-c", script], {
encoding: "utf8",
env: { ...process.env, PATH: `${dir}:${process.env.PATH}`, GITHUB_WORKSPACE: "/workspace", COMMAND_LOG: log,
DOCKER_FREE: String(dockerFree), WORKSPACE_FREE: String(workspaceFree), DF_STATUS: dfStatus, INFO_STATUS: infoStatus },
});
assert.equal(result.status, 0, result.stderr);
const commands = readFileSync(log, "utf8");
if (infoStatus === "0") assert.match(commands, /df -Pk \/docker-data \/workspace/);
assert.equal(commands.includes("pnpm store prune"), cleanup);
assert.equal(commands.includes("sudo rm -rf /usr/share/dotnet"), cleanup);
assert.equal(commands.includes("docker system prune -af"), cleanup);
assert.equal(result.stdout.includes("skipping cleanup"), !cleanup);
} finally { rmSync(dir, { recursive: true, force: true }); }
});
}

29
.github/scripts/tests/lockfile-refresh-cache.test.mjs vendored Normal file
View File

@ -0,0 +1,29 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const workflow = readFileSync(new URL("../../workflows/refresh-lockfile.yml", import.meta.url), "utf8");
const nodeStep = workflow.split(" - name: Setup Node.js\n")[1]?.split(" - name:")[0];
assert.ok(nodeStep, "the refresh workflow must set up Node");
const input = (name) => nodeStep.match(new RegExp(`^ ${name}: (.+)$`, "m"))?.[1].trim();
// setup-node's explicit cache input enables a store cache independently of its
// automatic npm detection. Disabling only automatic detection is insufficient.
function cacheProvider(explicitCache, automaticCache, packageManager) {
if (explicitCache) return explicitCache;
if (automaticCache !== "false" && packageManager.startsWith("npm@")) return "npm";
return undefined;
}
for (const packageManager of ["pnpm@9.15.4", "npm@11.0.0"]) {
test(`resolution-only refresh cannot write a package-store cache (${packageManager})`, () => {
assert.match(workflow, /run: pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile/);
assert.equal(
cacheProvider(input("cache"), input("package-manager-cache"), packageManager),
undefined,
"a metadata-only job must not claim the shared cache key with an empty store",
);
// This is the original failure mode, even with automatic caching disabled.
assert.equal(cacheProvider("pnpm", "false", packageManager), "pnpm");
});
}

26
.github/scripts/tests/lockfile-refresh-workflows.test.mjs vendored Normal file
View File

@ -0,0 +1,26 @@
import { readFile } from 'node:fs/promises';
import { test } from 'node:test';
import assert from 'node:assert/strict';
const workflows = [
'.github/workflows/refresh-lockfile.yml',
'.github/workflows/pr-trusted.yml',
'.github/workflows/docker.yml',
'.github/workflows/docker-cloud.yml',
];
test('lockfile repair workflows resolve dependencies instead of updating metadata only', async () => {
for (const workflow of workflows) {
const contents = await readFile(workflow, 'utf8');
const repairCommands = contents
.split('\n')
.filter((line) => line.includes('pnpm install') && line.includes('--no-frozen-lockfile'));
assert.ok(repairCommands.length > 0, `${workflow} must contain a lockfile repair command`);
for (const command of repairCommands) {
assert.match(command, /--resolution-only/);
assert.match(command, /--ignore-scripts/);
assert.doesNotMatch(command, /--lockfile-only/);
}
}
});

107
.github/scripts/tests/post-merge-runner-routing.test.mjs vendored Normal file
View File

@ -0,0 +1,107 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { runInNewContext } from "node:vm";
const fleet = "runs-on/fleet=paperclip-post-merge-x64/env=public-ci";
const sha = "a".repeat(40);
const base = {
repository: "paperclipai/paperclip", repository_id: "1170821064",
ref: "refs/heads/master", event_name: "push", sha,
};
const expectedJobs = {
"cloud-readiness.yml": [],
"cloud-artifacts.yml": ["dispatch_migrator"],
"release-verify.yml": ["typecheck", "general_tests", "serialized_tests", "runner_workflow_evals", "verify_paperclip_runner", "build"],
"runner-chaos-evals.yml": ["chaos_and_recovery"],
"release.yml": ["plan_preview", "package_preview"],
};
for (const [file, expectedNames] of Object.entries(expectedJobs)) {
const workflow = readFileSync(new URL(`../../workflows/${file}`, import.meta.url), "utf8");
const jobs = [...workflow.matchAll(/^ ([a-z_]+):\n([\s\S]*?)(?=^ [a-z_]+:\n|(?![\s\S]))/gm)];
const routed = jobs.filter(([, , body]) => body.includes(fleet));
test(`${file}: all intended jobs carry the post-merge guard`, () => {
assert.deepEqual(routed.map(([, name]) => name).sort(), [...expectedNames].sort());
});
for (const [, job, body] of routed) {
const expression = body.match(/^ runs-on: \$\{\{ (.+) \}\}$/m)?.[1];
assert.ok(expression, `${file}/${job} must use an explicit runner expression`);
const release = file === "release.yml";
const checkRef = release || file === "release-verify.yml" || file === "runner-chaos-evals.yml";
const inputs = { ref: sha, source_ref: sha, channel: "cloud-migrator" };
const defaultContext = { ...base, event_name: release ? "workflow_dispatch" : "push" };
const cases = [
{ name: "exact master source", expected: fleet },
{ name: "manual exact master source", github: { event_name: "workflow_dispatch" }, expected: fleet },
{ name: "switch disabled", enabled: "false" },
{ name: "switch absent", enabled: "" },
{ name: "malformed switch", enabled: "yes" },
{ name: "fork", github: { repository: "someone/paperclip", repository_id: "123" } },
{ name: "repository renamed or transferred", github: { repository_id: "123" } },
{ name: "unapproved PR", github: { event_name: "pull_request", ref: "refs/pull/1/merge" } },
{ name: "PR event even with master ref", github: { event_name: "pull_request" } },
{ name: "privileged PR event", github: { event_name: "pull_request_target" } },
{ name: "workflow completion event", github: { event_name: "workflow_run" } },
{ name: "repository dispatch", github: { event_name: "repository_dispatch" } },
{ name: "scheduled caller", github: { event_name: "schedule" } },
{ name: "branch workflow", github: { ref: "refs/heads/feature" } },
{ name: "release tag", github: { ref: "refs/tags/v2026.911.0" } },
];
if (checkRef) {
const key = release ? "source_ref" : "ref";
for (const value of ["b".repeat(40), "refs/pull/1/head", "master", "feature", "v1.0.0", ""]) {
cases.push({ name: `unverified source ${value || "(empty)"}`, inputs: { [key]: value } });
}
cases.push({ name: "missing source identity", github: { sha: "" }, inputs: { [key]: "" } });
}
if (release) {
cases.push({ name: "preview of master", inputs: { channel: "preview" } });
cases.push({ name: "stable release", inputs: { channel: "stable" } });
}
for (const { name, github = {}, inputs: overrides = {}, enabled = "true", expected = "ubuntu-latest" } of cases) {
test(`${file}/${job}: ${name}`, () => {
const context = { github: { ...defaultContext, ...github }, inputs: { ...inputs, ...overrides }, vars: { AWS_POST_MERGE_CI_ENABLED: enabled } };
// These canonical contexts use boolean operators and string comparisons
// whose results match GitHub's expression evaluation.
assert.equal(runInNewContext(expression, context), expected);
const timeout = body.match(/^ timeout-minutes: (.+)$/m)?.[1];
assert.ok(timeout, "AWS jobs need a timeout below the 45-minute instance lifetime");
const minutes = timeout.startsWith("${{") ? runInNewContext(timeout.slice(3, -2), context) : Number(timeout);
if (expected === fleet) assert.ok(minutes > 0 && minutes < 45);
if (release && job === "plan_preview") assert.equal(minutes, expected === fleet ? 10 : 360);
});
}
}
if (file === "release.yml") {
test("npm publisher always uses a GitHub-hosted runner", () => {
const publisher = jobs.find(([ , job]) => job === "publish_preview")?.[2];
assert.match(publisher, /^ runs-on: ubuntu-latest$/m);
assert.match(publisher, /^ environment: npm-canary$/m);
assert.match(publisher, /^ id-token: write$/m);
});
}
}
test("Cloud readiness bookkeeping never waits for the AWS verification fleet", () => {
const workflow = readFileSync(new URL("../../workflows/cloud-readiness.yml", import.meta.url), "utf8");
const bodies = new Map();
for (const [name, needs] of [
["artifacts", null],
["source_verified", "[verify]"],
["ready", "[verify, image, artifacts]"],
]) {
const body = workflow.match(new RegExp(`^ ${name}:\\n([\\s\\S]*?)(?=^ [a-z_]+:|(?![\\s\\S]))`, "m"))?.[1];
assert.ok(body, `missing ${name} job`);
bodies.set(name, body);
assert.match(body, /^ runs-on: ubuntu-latest$/m);
assert.doesNotMatch(body, /^ +continue-on-error:|^ +if:.*always\(\)/m);
assert.match(body, /^ if: github.repository == 'paperclipai\/paperclip' && github.ref == 'refs\/heads\/master'$/m);
assert.match(body, /^ +SOURCE_SHA: \$\{\{ github.sha \}\}$/m);
assert.equal(body.match(/^ needs: (.+)$/m)?.[1] ?? null, needs, `${name} prerequisites`);
}
assert.match(bodies.get("artifacts"), /^ run: node scripts\/cloud-readiness.mjs "\$SOURCE_SHA"$/m);
assert.match(bodies.get("source_verified"), /^ run: node --test scripts\/cloud-source-verification.test.mjs$/m);
assert.match(bodies.get("source_verified"), /echo "Cloud source verified v1: \$SOURCE_SHA"/);
assert.match(bodies.get("ready"), /echo "Cloud deployable v1: \$SOURCE_SHA"/);
});

39
.github/scripts/tests/pr-dependency-cache.test.mjs vendored Normal file
View File

@ -0,0 +1,39 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const workflow = readFileSync(new URL("../../workflows/pr-trusted.yml", import.meta.url), "utf8");
const jobs = [...workflow.matchAll(/^ ([a-z_][a-z_0-9]*):\n([\s\S]*?)(?=^ [a-z_][a-z_0-9]*:\n|$(?![\s\S]))/gm)];
const installers = jobs.filter(([, , body]) => body.includes("run: pnpm install --frozen-lockfile"));
test("PR workflows restore dependency stores without creating branch copies", () => {
assert.equal(installers.length, 7);
assert.doesNotMatch(workflow, /^ +cache: pnpm$/m);
assert.doesNotMatch(workflow, /uses: actions\/cache(?:@|\/save@)/);
for (const [, job, body] of jobs) {
for (const step of body.split(" - name:").filter((step) => step.includes("uses: actions/setup-node@"))) {
assert.match(step, /package-manager-cache: false/, job);
}
}
const policy = jobs.find(([, name]) => name === "policy")[2];
assert.doesNotMatch(policy, /uses: actions\/cache|cache: pnpm/);
});
for (const [, job, body] of installers) {
test(`${job}: reuse master keys before restoring the resolved PR lockfile`, () => {
const locate = body.indexOf(" - name: Locate pnpm store");
const restore = body.indexOf(" - name: Restore pnpm store (read only)");
const artifact = body.indexOf(" - name: Restore regenerated PR lockfile");
const install = body.indexOf("run: pnpm install --frozen-lockfile");
assert.ok(locate >= 0 && locate < restore && restore < artifact && artifact < install);
const cache = body.slice(restore, artifact);
assert.match(body.slice(locate, restore), /pnpm store path --silent/);
assert.match(body.slice(locate, restore), /node -p 'process.arch'/);
assert.match(cache, /uses: actions\/cache\/restore@[a-f0-9]{40}/);
assert.ok(cache.includes("key: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}"));
assert.ok(cache.includes("restore-keys: node-cache-${{ runner.os }}-${{ steps.pnpm_store.outputs.arch }}-pnpm-"));
assert.match(body.slice(artifact, install), /if: needs.policy.outputs.lockfile_regenerated == '1'/);
assert.match(body.slice(artifact, install), /name: pr-lockfile/);
assert.doesNotMatch(body.slice(artifact, install), /continue-on-error/);
});
}

70
.github/scripts/tests/release-runner-cache.test.mjs vendored Normal file
View File

@ -0,0 +1,70 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { runInNewContext } from "node:vm";
const workflow = readFileSync(new URL("../../workflows/release-verify.yml", import.meta.url), "utf8");
const runner = workflow.split(" verify_paperclip_runner:")[1].split(" build:")[0];
test("Runner dependency caching selects the package's pinned compiler before computing its key", () => {
const select = runner.indexOf(" - name: Select the pinned Runner Rust toolchain");
const cache = runner.indexOf(" - name: Cache Runner Rust dependencies");
assert.ok(select >= 0 && cache > select);
const setup = runner.slice(select, cache);
assert.match(setup, /working-directory: packages\/paperclip-runner/);
assert.match(setup, /rustup show active-toolchain/);
assert.match(setup, /echo "RUSTUP_TOOLCHAIN=\$toolchain" >> "\$GITHUB_ENV"/);
assert.match(runner, /uses: Swatinem\/rust-cache@[0-9a-f]{40} # v[0-9.]+/);
assert.match(runner, /workspaces: packages\/paperclip-runner\/runner -> target/);
assert.match(runner, /shared-key: release-runner-v1/);
});
test("the shared cache excludes workspace artifacts and only restores or saves the exact master-push source", () => {
assert.match(runner, /cache-workspace-crates: false/);
assert.match(runner, /cache-bin: false/);
const saveIf = runner.match(/^\s*save-if: (.+)$/m)?.[1];
assert.equal(saveIf, "${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}");
const cacheStep = runner.split(" - name: Cache Runner Rust dependencies")[1].split(" - name: Install dependencies")[0];
assert.equal(cacheStep.match(/^\s*if: (.+)$/m)?.[1], saveIf.replace("matrix.lane == 'rust' && ", ""));
assert.doesNotMatch(runner, /cache-on-failure: true|cache-all-crates: true/);
});
test("parallel lanes cover check:all exactly once and never bypass verification", () => {
const scripts = JSON.parse(readFileSync(new URL("../../../packages/paperclip-runner/package.json", import.meta.url))).scripts;
const checks = [...runner.matchAll(/^ checks: (.+)$/gm)].flatMap(([, value]) => value.split(" "));
assert.deepEqual(checks, scripts["check:all"].split(" && ").map((command) => command.replace(/^pnpm run /, "")));
assert.deepEqual([...runner.matchAll(/^ - lane: (.+)$/gm)].map(([, value]) => value), ["protocol", "rust"]);
assert.match(runner, /fail-fast: false/);
assert.doesNotMatch(runner, /max-parallel: 1|^ needs:|continue-on-error:/m);
const verify = runner.split(" - name: Verify Paperclip Runner\n")[1].split(" - name: Warm debug")[0];
assert.match(verify, /RUNNER_CHECKS: \$\{\{ matrix.checks \}\}/);
assert.match(verify, /set -euo pipefail/);
assert.match(verify, /for check in \$RUNNER_CHECKS; do\s+pnpm --filter @paperclipai\/paperclip-runner "\$check"\s+done/);
assert.doesNotMatch(verify, /if:|cache-hit/);
assert.doesNotMatch(runner, /id-token: write|packages: write|secrets: inherit/);
});
test("only the trusted Rust lane writes, and warms both build profiles before saving", () => {
const cache = runner.split(" - name: Cache Runner Rust dependencies")[1].split(" - name: Install dependencies")[0];
const warm = runner.split(" - name: Warm debug dependencies for the shared Runner cache")[1];
const expr = (body, field) => body.match(new RegExp(`^ +${field}: \\$\\{\\{ (.+) \\}\\}$`, "m"))[1];
assert.equal(expr(cache, "save-if"), expr(warm, "if"));
assert.match(warm, /run: pnpm --filter @paperclipai\/paperclip-runner build:rust/);
const sha = "a".repeat(40);
const base = { repository: "paperclipai/paperclip", event_name: "push", ref: "refs/heads/master", sha };
for (const lane of ["protocol", "rust"]) {
for (const [overrides, ref, trusted] of [
[{}, sha, true],
[{ event_name: "pull_request", ref: "refs/pull/1/merge" }, sha, false],
[{ event_name: "pull_request_target" }, sha, false],
[{ event_name: "workflow_dispatch" }, sha, false],
[{ repository: "someone/paperclip" }, sha, false],
[{ ref: "refs/heads/feature" }, sha, false],
[{}, "b".repeat(40), false],
]) {
const context = { matrix: { lane }, github: { ...base, ...overrides }, inputs: { ref } };
assert.equal(runInNewContext(expr(cache, "if"), context), trusted);
assert.equal(runInNewContext(expr(cache, "save-if"), context), trusted && lane === "rust");
}
}
});

38
.github/scripts/tests/typecheck-rust-cache.test.mjs vendored Normal file
View File

@ -0,0 +1,38 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { runInNewContext } from "node:vm";
const workflow = readFileSync(new URL("../../workflows/release-verify.yml", import.meta.url), "utf8");
const typecheck = workflow.split(" typecheck:\n")[1].split(" general_tests:\n")[0];
const cache = typecheck.split(" - name: Cache typecheck Rust dependencies\n")[1].split(" - name: Validate release package manifest")[0];
const sha = "a".repeat(40);
const github = { repository: "paperclipai/paperclip", event_name: "push", ref: "refs/heads/master", sha };
for (const [name, overrides, ref, allowed] of [
["exact master push", {}, sha, true],
["PR", { event_name: "pull_request", ref: "refs/pull/1/merge" }, sha, false],
["privileged PR", { event_name: "pull_request_target" }, sha, false],
["fork", { repository: "someone/paperclip" }, sha, false],
["branch", { ref: "refs/heads/feature" }, sha, false],
["manual source", { event_name: "workflow_dispatch" }, sha, false],
["unmerged source", {}, "b".repeat(40), false],
["moving ref", {}, "master", false],
]) {
test(`typecheck cache restore and save: ${name}`, () => {
for (const field of ["if", "save-if"]) {
const expr = cache.match(new RegExp(`^ +${field}: \\$\\{\\{ (.+) \\}\\}$`, "m"))?.[1];
assert.ok(expr);
assert.equal(runInNewContext(expr, { github: { ...github, ...overrides }, inputs: { ref } }), allowed);
}
});
}
test("cache excludes workspace code and executable installs, and preserves full checks", () => {
assert.match(cache, /uses: Swatinem\/rust-cache@[a-f0-9]{40}/);
assert.match(cache, /workspaces: packages\/paperclip-runner\/runner -> target/);
assert.match(cache, /shared-key: release-typecheck-v1/);
assert.match(cache, /cache-workspace-crates: false/);
assert.match(cache, /cache-bin: false/);
assert.ok(typecheck.indexOf('echo "RUSTUP_TOOLCHAIN=$toolchain"') < typecheck.indexOf("uses: Swatinem/rust-cache"));
assert.match(typecheck, /run: pnpm -r typecheck/);
assert.match(workflow, /shared-key: release-runner-v1/);
});

26
.github/scripts/verify-storybook.cjs vendored Normal file
View File

@ -0,0 +1,26 @@
const { branchIndex } = require('./storybook-destination.cjs');
async function verifyStorybook({ branchUrl, buildUrl, sha, fetch = globalThis.fetch,
sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)), attempts = 6 }) {
for (let attempt = 1; attempt <= attempts; attempt++) {
try {
const [metadata, index] = await Promise.all([
fetch(new URL('deployment.json', buildUrl), { signal: AbortSignal.timeout(15000) }),
fetch(branchUrl, { signal: AbortSignal.timeout(15000) }),
]);
if (!metadata.ok || !index.ok) throw new Error(`Public deployment returned HTTP ${metadata.status}/${index.status}.`);
if ((await metadata.json()).sha !== sha) throw new Error('Public build has the wrong source commit.');
if ((await index.text()) !== branchIndex(buildUrl)) throw new Error('Public branch URL does not point to this build.');
return;
} catch (error) {
if (attempt === attempts) throw error;
await sleep(10000);
}
}
}
module.exports = { verifyStorybook };
if (require.main === module) {
verifyStorybook({ branchUrl: process.env.BRANCH_URL, buildUrl: process.env.BUILD_URL,
sha: process.env.SOURCE_SHA }).catch((error) => { console.error(error); process.exitCode = 1; });
}

10
.github/storybook-deploy/cloudfront-read-statement.json vendored Normal file
View File

@ -0,0 +1,10 @@
{
"Sid": "AllowCloudFrontReadStorybook",
"Effect": "Allow",
"Principal": {"Service": "cloudfront.amazonaws.com"},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::paperclipai-runner-e2e-history-078455283791-us-east-1/storybook/branches/*",
"Condition": {"StringEquals": {
"AWS:SourceArn": "arn:aws:cloudfront::078455283791:distribution/E3GTU28BBO2SFR"
}}
}

12
.github/storybook-deploy/trust-policy.json vendored Normal file
View File

@ -0,0 +1,12 @@
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Federated": "arn:aws:iam::078455283791:oidc-provider/token.actions.githubusercontent.com"},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
"token.actions.githubusercontent.com:sub": "repo:paperclipai/paperclip:environment:storybook-deploy"
}}
}]
}

8
.github/storybook-deploy/upload-policy.json vendored Normal file
View File

@ -0,0 +1,8 @@
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::paperclipai-runner-e2e-history-078455283791-us-east-1/storybook/branches/*"
}]
}

34
.github/workflows/cloud-artifacts.yml vendored Normal file
View File

@ -0,0 +1,34 @@
name: Cloud artifacts
on:
push:
branches: [master]
workflow_dispatch:
permissions: {}
jobs:
dispatch_migrator:
name: Start exact-source cloud migrator publication
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 5
permissions:
actions: write
steps:
# This separate workflow starts at merge, outside the full npm release's
# concurrency group. Publication stays in release.yml so npm recognizes
# the established trusted-publisher identity and npm-canary environment.
# No source checkout or package code runs with the dispatch credential.
- name: Dispatch the migrator-only release
env:
GH_TOKEN: ${{ github.token }}
SOURCE_SHA: ${{ github.sha }}
run: |
set -euo pipefail
request_id="$(cat /proc/sys/kernel/random/uuid)"
gh workflow run release.yml --repo "$GITHUB_REPOSITORY" --ref master \
--field channel=cloud-migrator \
--field source_ref="$SOURCE_SHA" \
--field request_id="$request_id"
echo "Started Cloud migrator $SOURCE_SHA in release.yml (request $request_id)." >> "$GITHUB_STEP_SUMMARY"

96
.github/workflows/cloud-readiness.yml vendored Normal file
View File

@ -0,0 +1,96 @@
name: Cloud readiness
run-name: Cloud readiness ${{ github.sha }}
on:
push:
branches: [master]
workflow_dispatch:
permissions: {}
# Source verification must start outside the full npm release's queue.
concurrency:
group: cloud-readiness-${{ github.sha }}
cancel-in-progress: false
jobs:
image:
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
permissions:
contents: read
packages: write
uses: ./.github/workflows/docker-cloud.yml
verify:
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
permissions:
contents: read
uses: ./.github/workflows/release-verify.yml
with:
ref: ${{ github.sha }}
artifacts:
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
name: Wait for exact-source cloud artifacts
# Bookkeeping must not wait for the AWS builders it observes.
runs-on: ubuntu-latest
timeout-minutes: 35
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Wait for verified image and exact-source migrator
env:
SOURCE_SHA: ${{ github.sha }}
run: node scripts/cloud-readiness.mjs "$SOURCE_SHA"
source_verified:
# npm canary publication reuses this exact-source verification proof.
# Keep it independent of image/migrator availability, and fail closed when
# any source check fails, is cancelled, or is skipped.
name: Cloud source verified v1
needs: [verify]
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
# Bookkeeping must not wait for the AWS builders it observes.
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Check the source verification consumer
run: node --test scripts/cloud-source-verification.test.mjs
- name: Record source verification
env:
SOURCE_SHA: ${{ github.sha }}
run: |
echo "Cloud source verified v1: $SOURCE_SHA" >> "$GITHUB_STEP_SUMMARY"
ready:
# Versioned consumer contract. Never add always() or continue-on-error:
# failed, cancelled, or skipped prerequisites must not report readiness.
name: Cloud deployable v1
needs: [verify, image, artifacts]
if: github.repository == 'paperclipai/paperclip' && github.ref == 'refs/heads/master'
# Bookkeeping must not wait for the AWS builders it observes.
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Record cloud readiness
env:
SOURCE_SHA: ${{ github.sha }}
run: |
echo "Cloud deployable v1: $SOURCE_SHA" >> "$GITHUB_STEP_SUMMARY"
echo "Source verification passed; the full-SHA image and exact-source migrator are available." >> "$GITHUB_STEP_SUMMARY"
echo "Deployment tooling must still resolve and pin the image and migrator and validate migration compatibility." >> "$GITHUB_STEP_SUMMARY"

290
.github/workflows/docker-cloud.yml vendored Normal file
View File

@ -0,0 +1,290 @@
name: Docker cloud
on:
workflow_dispatch:
workflow_call:
permissions: {}
# Independent SHAs can build immediately on separate runners.
# Repeated requests for the same source serialize without cancelling a build.
# No mutable canary channel is promoted here; docker.yml owns that operation.
concurrency:
group: docker-cloud-${{ github.sha }}
cancel-in-progress: false
jobs:
build-and-push-cloud:
# Only canonical master builds can consume the release Fleet. The runner
# group must also allow this workflow only at refs/heads/master.
# Keep an operator switch for a full-run retry on GitHub-hosted runners.
runs-on: ${{ vars.AWS_CLOUD_BUILDS_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && 'runs-on/fleet=paperclip-cloud-build-x64/env=public-ci' || 'ubuntu-latest' }}
# Fleet instances expire after 45 minutes, including bootstrap and cleanup.
timeout-minutes: ${{ vars.AWS_CLOUD_BUILDS_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && 40 || 60 }}
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
# Full history and tags so `git describe` below can compute the
# release version to stamp into the image.
fetch-depth: 0
# `.git` is dockerignored, so a running image cannot derive its own
# version and otherwise reports the source package.json placeholder in
# analytics and the debug panel. Compute it here from the pristine
# checkout (real CalVer drift from the nearest release tag) and pass it
# into the build. Empty when no release tag is reachable — the server
# then keeps its existing fallbacks.
- name: Compute build version
id: build-version
run: |
set -euo pipefail
case "${GITHUB_REF}" in
refs/tags/nightly/v*)
# Lane tags carry the exact published version; stamp it verbatim
# instead of describing drift from the nearest stable tag.
version="${GITHUB_REF#refs/tags/nightly/v}"
;;
refs/tags/beta/v*)
version="${GITHUB_REF#refs/tags/beta/v}"
;;
*)
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
;;
esac
echo "version=${version}" >> "$GITHUB_OUTPUT"
echo "Stamping build version: ${version:-<none>}"
# ISO week stamp for the Dockerfile's tool layer: the layer caches
# across commits and re-pulls the @latest CLI tools when the week rolls
# over, instead of on every build.
- name: Compute tool cache epoch
id: tools-epoch
run: echo "epoch=$(date -u +%G-W%V)" >> "$GITHUB_OUTPUT"
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
run_install: false
# No dependency cache here: this workflow publishes release images, and
# restoring a shared Actions cache into the build inputs would let a
# poisoned cache entry reach the published artifact.
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Refresh lockfile for Docker build context
run: |
set -euo pipefail
pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
changed="$(git status --porcelain)"
if [ -z "$changed" ]; then
echo "Lockfile already matches package metadata."
exit 0
fi
if printf '%s\n' "$changed" | grep -Fvq ' pnpm-lock.yaml'; then
echo "Unexpected files changed during lockfile refresh:"
echo "$changed"
exit 1
fi
echo "Using refreshed pnpm-lock.yaml in the Docker build context."
- name: Free runner disk
run: |
set -euo pipefail
echo "Disk before cleanup:"
df -h
# A measured hosted cloud build started with 86 GB available.
# Keep ample headroom for BuildKit and image verification, but
# avoid minutes deleting SDKs when neither filesystem needs space.
minimum_free_kib=$((64 * 1024 * 1024))
if docker_root="$(docker info --format '{{.DockerRootDir}}')" \
&& available_kib="$(df -Pk "$docker_root" "$GITHUB_WORKSPACE" | awk 'NR > 1 { rows++; if ($4 !~ /^[0-9]+$/) invalid = 1; if (min == "" || $4 < min) min = $4 } END { if (invalid || rows != 2) exit 1; print min }')" \
&& [[ "$available_kib" =~ ^[0-9]+$ ]] \
&& (( available_kib >= minimum_free_kib )); then
echo "At least 64 GiB is available for Docker and the workspace; skipping cleanup."
exit 0
fi
pnpm store prune || true
sudo apt-get clean || true
sudo rm -rf \
/usr/share/dotnet \
/usr/share/swift \
/usr/local/lib/android \
/usr/local/share/boost \
/usr/local/share/powershell \
/opt/ghc \
/opt/hostedtoolcache/CodeQL \
/opt/hostedtoolcache/PyPy \
/opt/hostedtoolcache/Ruby || true
docker system prune -af || true
echo "Disk after cleanup:"
df -h
- name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Mixing several historical manifests missed otherwise reusable native
# layers on fresh builders. Import the nearest available complete cache.
- name: Select cloud cache ancestry
id: cloud-cache
env:
CACHE_IMAGE: ghcr.io/${{ github.repository }}
run: node scripts/select-cloud-cache.mjs
# Deployment tooling reads these labels from the registry to verify an
# image's schema expectations against a migrator before deploying it,
# without pulling the image. The server refuses to start when the
# database is missing bundled migrations, so orchestrators need a cheap
# way to check image/migrator compatibility up front.
- name: Compute schema migration labels
id: schema
run: |
set -euo pipefail
last=$(ls packages/db/src/migrations/*.sql | sed 's|.*/||' | LC_ALL=C sort | tail -1)
count=$(ls packages/db/src/migrations/*.sql | wc -l | tr -d ' ')
echo "last=${last}" >> "$GITHUB_OUTPUT"
echo "count=${count}" >> "$GITHUB_OUTPUT"
# Published under the same lane tag set as the self-hosted image, with a
# `-cloud` suffix (nightly-cloud, latest-cloud, <version>-cloud,
# sha-<short>-cloud). `:canary-cloud` follows the same retag-step
# ownership rule as `:canary` above.
- name: Docker meta (cloud)
id: meta-cloud
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6
with:
images: ghcr.io/${{ github.repository }}
flavor: |
suffix=-cloud,onlatest=true
tags: |
type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }}
type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=semver,pattern={{major}}.{{minor}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=sha
labels: |
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
- name: Build and push (cloud)
id: build-cloud
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
with:
context: .
target: cloud
# Space-separated sandbox-provider directory names to build into
# the variant; add here when managed deployments need another.
# CLOUD_BUNDLED_SERVER_DEPS names the optional peer packages the
# variant installs from server/package.json's declared version;
# add another name there when a managed tenant needs it.
build-args: |
USER_UID=1001
USER_GID=1001
CLOUD_BUNDLED_PLUGINS=daytona
CLOUD_BUNDLED_SERVER_DEPS=@sentry/node
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
PAPERCLIP_BUILD_COMMIT=${{ github.sha }}
CLI_TOOLS_CACHE_EPOCH=${{ steps.tools-epoch.outputs.epoch }}
# amd64 only, unlike the self-hosted image above: the cloud variant
# is consumed exclusively by managed-deployment hosts, which run
# amd64. The QEMU-emulated arm64 half dominated this job's wall
# clock, and dropping it roughly halves time-to-deployable-image.
platforms: linux/amd64
push: true
# Same-SHA builds serialize above; different SHAs never share a
# writable cache ref. Registry layers are content-addressed and
# shared even when cache manifests have separate tags.
cache-from: ${{ steps.cloud-cache.outputs.source }}
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-cloud-${{ github.sha }},mode=max
tags: ${{ steps.meta-cloud.outputs.tags }}
labels: ${{ steps.meta-cloud.outputs.labels }}
# The cloud target installs @sentry/node at the version
# server/package.json declares, into a directory the server's own
# module resolution walks. Verify the image this job just pushed, not
# a local build, so a build-cache or layer-ordering regression is
# caught before any tenant runs the image.
- name: Verify the pushed image resolves the declared Sentry version
env:
IMAGE: ghcr.io/${{ github.repository }}@${{ steps.build-cloud.outputs.digest }}
run: |
set -euo pipefail
expected="$(node -e "process.stdout.write(require('./server/package.json').peerDependencies['@sentry/node'])")"
test -n "$expected"
installed="$(docker run --rm --pull always \
-v "$PWD/scripts/assert-cloud-image-sentry.mjs:/app/server/.ci-sentry-probe.mjs:ro" \
--entrypoint node "$IMAGE" /app/server/.ci-sentry-probe.mjs)"
echo "Declared optional peer version: $expected"
echo "Installed in the pushed image: $installed"
if [ "$installed" != "$expected" ]; then
echo "ERROR: the pushed image resolves @sentry/node@$installed, expected @sentry/node@$expected" >&2
exit 1
fi
echo "The pushed image resolves the declared @sentry/node version."
# Managed hosts run node as 1001:1001. Bake that identity into the image
# so usermod does not walk the mounted home on every container start.
# Check before the entrypoint can repair a wrongly built identity.
- name: Verify cloud runtime user
env:
IMAGE: ghcr.io/${{ github.repository }}@${{ steps.build-cloud.outputs.digest }}
run: |
set -euo pipefail
docker run --rm --entrypoint sh "$IMAGE" -ec '
test "$(id -u node)" = 1001
test "$(id -g node)" = 1001
test "$USER_UID" = 1001
test "$USER_GID" = 1001
'
docker run --rm -e USER_UID=1001 -e USER_GID=1001 "$IMAGE" sh -ec '
test "$(id -u)" = 1001
test "$(id -g)" = 1001
test -w "$PAPERCLIP_HOME"
'
# Verify the independently published cloud image without waiting for
# the self-hosted manifest job. The Sentry check already pulled it.
- name: Verify cloud PID 1 reaps orphaned processes
env:
IMAGE: ghcr.io/${{ github.repository }}@${{ steps.build-cloud.outputs.digest }}
run: docker run --rm -i "$IMAGE" sh -s < scripts/assert-orphan-reaping.sh
# Cloud's commit resolver and preview-artifact planner use the full SHA.
# Publish that address only after checking this build's exact digest.
# Retagging reuses the registry manifest and does not rebuild the image.
- name: Publish verified full-SHA cloud tag
env:
IMAGE: ghcr.io/${{ github.repository }}@${{ steps.build-cloud.outputs.digest }}
FULL_SHA_TAG: ghcr.io/${{ github.repository }}:sha-${{ github.sha }}-cloud
run: |
set -euo pipefail
revision="$(docker image inspect "$IMAGE" --format '{{ index .Config.Labels "org.opencontainers.image.revision" }}')"
platform="$(docker image inspect "$IMAGE" --format '{{ .Os }}/{{ .Architecture }}')"
test "$revision" = "$GITHUB_SHA"
test "$platform" = linux/amd64
docker buildx imagetools create --prefer-index=false --tag "$FULL_SHA_TAG" "$IMAGE"

38
.github/workflows/docker-runner-check.yml vendored Normal file
View File

@ -0,0 +1,38 @@
name: Docker Runner check
on:
pull_request:
paths:
- .github/workflows/docker-runner-check.yml
- Dockerfile
- .dockerignore
- scripts/check-docker-runner-cache.sh
- packages/paperclip-runner/rust-toolchain.toml
- packages/paperclip-runner/runner/**
- packages/paperclip-runner/protocol/**
permissions: {}
concurrency:
group: docker-runner-check-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
runner:
name: Compile isolated native Runner
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Compile the real target, then change source in a disposable context.
# A fresh builder must import dependencies and produce changed binary metadata.
# No registry credentials, external cache, or image publication.
- name: Verify native build and dependency cache reuse
run: bash scripts/check-docker-runner-cache.sh

375
.github/workflows/docker.yml vendored
View File

@ -14,21 +14,70 @@ on:
# way.
workflow_dispatch:
permissions:
contents: read
packages: write
# Least privilege: nothing at the workflow level; each job declares exactly
# the token scopes it uses (checkout needs contents:read, GHCR pushes need
# packages:write).
permissions: {}
# Serialise builds per ref without killing an in-flight one: a newer push
# supersedes only the pending slot, so the image build that is already
# running always finishes and publishes.
# running always finishes and publishes. Canary TAG refs each get their
# own group on purpose: their builds run in parallel so every published
# canary gets its sha images regardless of merge cadence. The mutable
# `:canary` channel tags are NOT written by the build matrix (which
# would race across parallel runs) — each canary-tag run retags the
# channel afterwards, only if it still matches the npm `canary`
# dist-tag, so the channel moves monotonically and always mirrors npm.
concurrency:
group: docker-${{ github.ref }}
cancel-in-progress: false
jobs:
# Multi-arch by native runner, not QEMU.
#
# This was one job building linux/amd64,linux/arm64 together on an x86
# runner. The arm64 half is emulated there, and it did not merely run slow:
# it wedged, every time, in `RUN pnpm --filter @paperclipai/server build`,
# emitting nothing for 38-45 minutes until `timeout-minutes: 60` killed the
# job. Verified across three consecutive runs on 2026-09-04; the amd64 half
# reached `production 5/5` minutes earlier in every one.
#
# A timed-out job is reported as *cancelled*, not failed, so the run read
# "cancelled" and the production image simply stopped publishing without
# anything going red in an obvious way.
#
# It also starved the queue. A run that burns the full hour holds the
# top-level concurrency slot for that hour, and `cancel-in-progress: false`
# keeps exactly one pending slot — so with merges arriving faster than one
# an hour, most runs were superseded before they ever started a job. Five of
# ten master commits sampled that day never produced an image at all.
#
# Each platform now builds on a runner of its own architecture and pushes by
# digest; `merge` assembles the manifest list. arm64 is kept rather than
# dropped (the cloud variant below dropped it and is amd64-only) because
# this is the self-hosted image, and ARM hosts consume it.
build-and-push:
runs-on: ubuntu-latest
strategy:
# Independent legs: one architecture failing should still publish
# nothing, but it must not also hide the other's logs behind a
# cancellation.
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
runs-on: ${{ matrix.runner }}
# Native builds land well inside this; it is a backstop, not a budget.
# (The interim fix while this PR landed raised the single QEMU job's cap
# to 120 minutes; native per-arch legs make that headroom unnecessary.)
timeout-minutes: 60
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v7
@ -87,7 +136,7 @@ jobs:
- name: Refresh lockfile for Docker build context
run: |
set -euo pipefail
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
changed="$(git status --porcelain)"
if [ -z "$changed" ]; then
@ -150,16 +199,18 @@ jobs:
echo "last=${last}" >> "$GITHUB_OUTPUT"
echo "count=${count}" >> "$GITHUB_OUTPUT"
# Lane tag mapping: master pushes publish `:canary`, nightly/v* tags
# publish `:nightly`, and only stable v* tags move `:latest` and the
# versioned tags. `:sha-<short>` is published on every build.
# Lane tag mapping: nightly/v* tags publish `:nightly`, and only
# stable v* tags move `:latest` and the versioned tags.
# `:sha-<short>` is published on every build. `:canary` is
# deliberately absent here — the channel tag is moved by the
# dist-tag-checked retag step below, never by the build matrix,
# so parallel canary builds cannot race it backwards.
- name: Docker meta
id: meta
uses: docker/metadata-action@v6
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }}
type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }}
type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
@ -169,8 +220,8 @@ jobs:
labels: |
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
- name: Build and push
- name: Build and push by digest
id: build
uses: docker/build-push-action@v7
with:
context: .
@ -182,140 +233,79 @@ jobs:
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
PAPERCLIP_BUILD_COMMIT=${{ github.sha }}
CLI_TOOLS_CACHE_EPOCH=${{ steps.tools-epoch.outputs.epoch }}
platforms: linux/amd64,linux/arm64
push: true
platforms: ${{ matrix.platform }}
# By digest, not by tag: two runners cannot each push the same tag
# and end up with a manifest list. Each leg publishes an untagged
# image and `merge` names them together.
outputs: type=image,name=ghcr.io/${{ github.repository }},push-by-digest=true,name-canonical=true,push=true
# Registry-backed BuildKit cache instead of type=gha: the Actions
# cache is capped at 10GB per repo, and two multi-arch mode=max jobs
# evict each other, so most builds ran effectively cold. The cache
# ref lives in ghcr next to the image and is written only by this
# workflow (docker.yml runs on master/tag pushes, never on PRs).
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache,mode=max
tags: ${{ steps.meta.outputs.tags }}
#
# Per-arch refs now the legs are separate runners: a shared ref would
# have each leg overwrite the other's cache on every build.
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-${{ matrix.arch }}
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-${{ matrix.arch }},mode=max
labels: ${{ steps.meta.outputs.labels }}
# The cloud variant carries built bundled plugins for managed deployments
# (see the `cloud` stage in the Dockerfile). It runs as its own job with no
# `needs:` on the stock publish above, so the two builds run in parallel and
# a failure or slow build in one never gates, delays, or skips the other.
# Both jobs share only the single top-level concurrency slot. Each job is a
# separate runner, so this one carries its own copy of the prep steps
# (checkout through schema labels) — the accepted cost of that isolation.
build-and-push-cloud:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v7
# The digest is the only thing `merge` needs from this job. Carried as an
# empty file named for it, which is the upstream pattern — the name is
# the payload, so several legs can upload without colliding on content.
- name: Export digest
run: |
set -euo pipefail
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
test -n "$digest"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: digests-production-${{ matrix.arch }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
# Names the per-architecture digests as one manifest list under the real
# tags. Nothing is publicly tagged until this runs, so a half-published
# multi-arch image is not a state anything can pull.
merge-and-push:
needs: build-and-push
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
packages: write
steps:
# Checked out for `packages/db` (schema labels) and the orphan-reaping
# script the verification step pipes in.
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
# Full history and tags so `git describe` below can compute the
# release version to stamp into the image.
fetch-depth: 0
# `.git` is dockerignored, so a running image cannot derive its own
# version and otherwise reports the source package.json placeholder in
# analytics and the debug panel. Compute it here from the pristine
# checkout (real CalVer drift from the nearest release tag) and pass it
# into the build. Empty when no release tag is reachable — the server
# then keeps its existing fallbacks.
- name: Compute build version
id: build-version
run: |
set -euo pipefail
case "${GITHUB_REF}" in
refs/tags/nightly/v*)
# Lane tags carry the exact published version; stamp it verbatim
# instead of describing drift from the nearest stable tag.
version="${GITHUB_REF#refs/tags/nightly/v}"
;;
refs/tags/beta/v*)
version="${GITHUB_REF#refs/tags/beta/v}"
;;
*)
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
;;
esac
echo "version=${version}" >> "$GITHUB_OUTPUT"
echo "Stamping build version: ${version:-<none>}"
# ISO week stamp for the Dockerfile's tool layer: the layer caches
# across commits and re-pulls the @latest CLI tools when the week rolls
# over, instead of on every build.
- name: Compute tool cache epoch
id: tools-epoch
run: echo "epoch=$(date -u +%G-W%V)" >> "$GITHUB_OUTPUT"
- name: Setup pnpm
uses: pnpm/action-setup@v6
- name: Download digests
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
version: 9.15.4
run_install: false
# No dependency cache here: this workflow publishes release images, and
# restoring a shared Actions cache into the build inputs would let a
# poisoned cache entry reach the published artifact.
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
- name: Refresh lockfile for Docker build context
run: |
set -euo pipefail
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
changed="$(git status --porcelain)"
if [ -z "$changed" ]; then
echo "Lockfile already matches package metadata."
exit 0
fi
if printf '%s\n' "$changed" | grep -Fvq ' pnpm-lock.yaml'; then
echo "Unexpected files changed during lockfile refresh:"
echo "$changed"
exit 1
fi
echo "Using refreshed pnpm-lock.yaml in the Docker build context."
- name: Free runner disk
run: |
set -euo pipefail
echo "Disk before cleanup:"
df -h
pnpm store prune || true
sudo apt-get clean || true
sudo rm -rf \
/usr/share/dotnet \
/usr/share/swift \
/usr/local/lib/android \
/usr/local/share/boost \
/usr/local/share/powershell \
/opt/ghc \
/opt/hostedtoolcache/CodeQL \
/opt/hostedtoolcache/PyPy \
/opt/hostedtoolcache/Ruby || true
docker system prune -af || true
echo "Disk after cleanup:"
df -h
path: /tmp/digests
pattern: digests-production-*
merge-multiple: true
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
# Deployment tooling reads these labels from the registry to verify an
# image's schema expectations against a migrator before deploying it,
# without pulling the image. The server refuses to start when the
# database is missing bundled migrations, so orchestrators need a cheap
# way to check image/migrator compatibility up front.
# Repeated from the build job rather than passed between them: job
# outputs would have to survive a matrix, and this is two `ls` calls.
- name: Compute schema migration labels
id: schema
run: |
@ -325,18 +315,17 @@ jobs:
echo "last=${last}" >> "$GITHUB_OUTPUT"
echo "count=${count}" >> "$GITHUB_OUTPUT"
# Published under the same lane tag set as the self-hosted image, with a
# `-cloud` suffix (canary-cloud, nightly-cloud, latest-cloud,
# <version>-cloud, sha-<short>-cloud).
- name: Docker meta (cloud)
id: meta-cloud
uses: docker/metadata-action@v6
# Same lane mapping as the build job; this is where it is actually
# applied, since the legs push untagged. `:canary` is deliberately
# absent, exactly as in the build job's mapping: the channel tag is
# moved only by the dist-tag-checked promote_canary_channel job below,
# so parallel canary-tag builds can never race the channel backwards.
- name: Docker meta
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6
with:
images: ghcr.io/${{ github.repository }}
flavor: |
suffix=-cloud,onlatest=true
tags: |
type=raw,value=canary,enable=${{ github.ref == 'refs/heads/master' }}
type=raw,value=nightly,enable=${{ startsWith(github.ref, 'refs/tags/nightly/v') }}
type=raw,value=beta,enable=${{ startsWith(github.ref, 'refs/tags/beta/v') }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
@ -347,28 +336,96 @@ jobs:
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
- name: Build and push (cloud)
uses: docker/build-push-action@v7
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
set -euo pipefail
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf 'ghcr.io/${{ github.repository }}@sha256:%s ' *)
# PID 1 must be an init that reaps adopted orphans. With node there, the
# orphans agent runs leave behind are never wait()ed and pin as zombies
# until the cgroup pid limit is exhausted and every fork() in the
# container fails. Run against the pushed manifest rather than a local
# build: the legs push by digest, so nothing is loaded into this
# runner's daemon. The independent cloud workflow checks its own image.
- name: Verify PID 1 reaps orphaned processes
env:
# Through the environment, not interpolated into the script body, so
# the tag text is data rather than shell.
IMAGE_TAGS: ${{ steps.meta.outputs.tags }}
run: |
set -euo pipefail
image="$(printf '%s\n' "$IMAGE_TAGS" | head -n 1)"
test -n "$image"
echo "Verifying orphan reaping in $image"
docker run --rm -i --pull always "$image" sh -s < scripts/assert-orphan-reaping.sh
# Master cloud builds start independently in docker-cloud.yml. Tag builds
# and manual Docker dispatches call the same implementation, preserving the
# release tags and the canary promotion dependency below.
build-and-push-cloud:
if: github.event_name != 'push' || github.ref != 'refs/heads/master'
uses: ./.github/workflows/docker-cloud.yml
permissions:
contents: read
packages: write
# Moves the mutable `:canary` / `:canary-cloud` channel tags. Kept OUT
# of the build jobs and serialized in its own lane, and — the load-
# bearing property — CONVERGENT rather than self-interested: a
# promotion does not promote "its own" canary, it retags the channel
# to whatever the npm `canary` dist-tag names at execution time,
# provided that version's sha images are published. GitHub's shared
# concurrency lane keeps one running and one pending promotion and
# REPLACES the pending slot with the latest enqueued — an older build
# finishing late can therefore evict the newest canary's pending
# promotion. With convergent promotion that eviction is harmless:
# whichever promotion survives resolves the current dist-tag fresh
# and lands the channel there (the current canary's images always
# exist by the time any later promotion runs, because per-tag build
# groups mean canary builds are never superseded and each run's
# promotion is gated on its own completed pushes). Every interleaving
# converges the Docker channel onto the npm channel.
promote_canary_channel:
if: startsWith(github.ref, 'refs/tags/canary/v')
# merge-and-push, not build-and-push: the per-arch legs push untagged
# digests, and the production `sha-*` tags this promotion retags only
# exist once the manifest merge has named them.
needs: [merge-and-push, build-and-push-cloud]
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
packages: write
concurrency:
group: docker-canary-channel-promotion
cancel-in-progress: false
steps:
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
with:
context: .
target: cloud
# Space-separated sandbox-provider directory names to build into
# the variant; add here when managed deployments need another.
build-args: |
CLOUD_BUNDLED_PLUGINS=daytona
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
PAPERCLIP_BUILD_COMMIT=${{ github.sha }}
CLI_TOOLS_CACHE_EPOCH=${{ steps.tools-epoch.outputs.epoch }}
# amd64 only, unlike the self-hosted image above: the cloud variant
# is consumed exclusively by managed-deployment hosts, which run
# amd64. The QEMU-emulated arm64 half dominated this job's wall
# clock, and dropping it roughly halves time-to-deployable-image.
platforms: linux/amd64
push: true
# Registry-backed BuildKit cache, separate ref from the self-hosted
# job so the two parallel builds never clobber each other's cache
# manifest (see the rationale on the job above).
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-cloud
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache-cloud,mode=max
tags: ${{ steps.meta-cloud.outputs.tags }}
labels: ${{ steps.meta-cloud.outputs.labels }}
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Converge the channel tags onto the current npm canary
env:
IMAGE: ghcr.io/${{ github.repository }}
GH_TOKEN: ${{ github.token }}
run: |
current="$(curl -fsS "https://registry.npmjs.org/-/package/@paperclipai%2Fdb/dist-tags" | jq -er .canary)"
sha="$(gh api "repos/${GITHUB_REPOSITORY}/commits/$(printf 'canary/v%s' "$current" | jq -sRr @uri)" --jq .sha 2>/dev/null || true)"
if [ -z "$sha" ]; then
echo "canary/v${current} does not resolve yet; a later promotion converges the channel"
exit 0
fi
short="$(printf '%s' "$sha" | cut -c1-7)"
if ! docker buildx imagetools inspect "$IMAGE:sha-${short}-cloud" >/dev/null 2>&1; then
echo "images for ${current} (sha-${short}) not published yet; its own promotion converges the channel"
exit 0
fi
docker buildx imagetools create -t "$IMAGE:canary" "$IMAGE:sha-${short}"
docker buildx imagetools create -t "$IMAGE:canary-cloud" "$IMAGE:sha-${short}-cloud"
echo "channel tags moved to canary ${current} (sha-${short})"

55
.github/workflows/e2e.yml vendored
View File

@ -9,23 +9,65 @@ on:
default: true
jobs:
authorize:
name: Authorize optional paid E2E
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Require default branch and allowlisted numeric actor IDs
env:
GH_TOKEN: ${{ github.token }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
test "$REF" = "refs/heads/$DEFAULT_BRANCH"
jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
e2e:
needs: authorize
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
environment:
name: runner-e2e-paid
env:
PAPERCLIP_E2E_SKIP_LLM: ${{ inputs.skip_llm && 'true' || 'false' }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
steps:
- uses: actions/checkout@v7
- name: Reauthorize execution before optional provider access
env:
GH_TOKEN: ${{ github.token }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
test "$REF" = "refs/heads/$DEFAULT_BRANCH"
jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
- uses: pnpm/action-setup@v6
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9
- uses: actions/setup-node@v7
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm build
@ -34,9 +76,10 @@ jobs:
- name: Run e2e tests
env:
PAPERCLIP_PLAYWRIGHT_CHANNEL: "chrome"
ANTHROPIC_API_KEY: ${{ !inputs.skip_llm && secrets.ANTHROPIC_API_KEY || '' }}
run: pnpm run test:e2e
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
if: always()
with:
name: playwright-report

1024
.github/workflows/pr-trusted.yml vendored Normal file
View File

File diff suppressed because it is too large Load Diff

508
.github/workflows/pr.yml vendored
View File

@ -2,507 +2,13 @@ name: PR
on:
pull_request:
branches:
- master
concurrency:
group: pr-${{ github.event.pull_request.number }}
cancel-in-progress: true
permissions:
actions: read
contents: read
pull-requests: read
jobs:
policy:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
lockfile_regenerated: ${{ steps.regen_lockfile.outputs.regenerated }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Block manual lockfile edits
if: >-
github.head_ref != 'chore/refresh-lockfile' &&
github.event.pull_request.user.login != 'dependabot[bot]'
run: |
# Diff the PR branch against its merge base so recent base-branch commits
# do not masquerade as changes made by the PR itself.
changed="$(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")"
if printf '%s\n' "$changed" | grep -qx 'pnpm-lock.yaml'; then
echo "Do not commit pnpm-lock.yaml in pull requests. CI owns lockfile updates."
exit 1
fi
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
- name: Validate Dockerfile deps stage
run: node ./scripts/check-docker-deps-stage.mjs
- name: Validate Node version policy
run: pnpm check:node-version
- name: Reject git push in adapter/runtime code
run: node ./scripts/check-no-git-push.mjs
- name: Test no-git-push check
run: node --test ./scripts/check-no-git-push.test.mjs
- name: Test PR quality-gate scripts
run: node --test '.github/scripts/tests/*.test.mjs'
- name: Test general-server shard partition
run: node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs
- name: Test e2e shard partition
run: node --test ./scripts/__tests__/e2e-shard.test.mjs
- name: Test release verify workflow wiring
run: node --test ./scripts/__tests__/release-verify-workflow.test.mjs
- name: Test standalone package build concurrency
run: node --test ./scripts/__tests__/build-standalone-concurrency.test.mjs
- name: Validate release package manifest
run: node ./scripts/release-package-map.mjs check
- name: Verify release package bootstrap for changed manifests
run: |
mapfile -t changed_paths < <(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")
PAPERCLIP_RELEASE_BOOTSTRAP_BASE_SHA="${{ github.event.pull_request.base.sha }}" \
node ./scripts/check-release-package-bootstrap.mjs "${changed_paths[@]}"
- name: Validate dependency resolution when manifests change
id: regen_lockfile
run: |
changed="$(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")"
manifest_pattern='(^|/)package\.json$|^pnpm-workspace\.yaml$|^\.npmrc$|^pnpmfile\.(cjs|js|mjs)$|^patches/'
if printf '%s\n' "$changed" | grep -Eq "$manifest_pattern"; then
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
echo "regenerated=1" >> "$GITHUB_OUTPUT"
else
echo "regenerated=0" >> "$GITHUB_OUTPUT"
fi
# Manifest-only PRs (where pnpm-lock.yaml stays at base because the policy
# job above blocks committing it) need the regenerated lockfile for the
# downstream `pnpm install --frozen-lockfile` steps. Upload it here so
# every job consumes the same hash without recomputing.
- name: Upload regenerated lockfile for downstream jobs
if: steps.regen_lockfile.outputs.regenerated == '1'
uses: actions/upload-artifact@v7
with:
name: pr-lockfile
path: pnpm-lock.yaml
retention-days: 1
if-no-files-found: error
typecheck_release_registry:
name: Typecheck + Release Registry
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Typecheck workspaces whose build scripts skip TypeScript
run: pnpm run typecheck:build-gaps
- name: Verify release registry test coverage
run: pnpm run test:release-registry
general_tests:
name: General tests (${{ matrix.group_label }})
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
# The server suite is pinned to maxWorkers=1 (server/vitest.config.ts),
# so it can only be parallelized across runners. Shard it to keep this
# lane off the PR critical path. Five shards because the suite has
# grown to ~946s of serial vitest wall time (run 30930345729,
# 2026-08-04): at four shards the worst shard ran 311s and was the
# slowest check in the whole PR run; five brings each shard to ~196s
# of suite time (~240s job), level with the other ~250-300s lanes.
- group: general-server
group_label: server (1/5)
shard_index: 0
shard_count: 5
- group: general-server
group_label: server (2/5)
shard_index: 1
shard_count: 5
- group: general-server
group_label: server (3/5)
shard_index: 2
shard_count: 5
- group: general-server
group_label: server (4/5)
shard_index: 3
shard_count: 5
- group: general-server
group_label: server (5/5)
shard_index: 4
shard_count: 5
# workspaces-a was the slowest check in the fully-green PR run
# 31371439296 (2026-08-10) at 319s, with the ui project's single
# vitest invocation accounting for ~224s and the paperclipai CLI
# ~37s. Two shards use Vitest's native --shard on each project's
# file list (ui: 439 files, cli: 54), bringing each job to roughly
# half the suite time (~130s + setup) without a duration manifest.
- group: general-workspaces-a
group_label: workspaces-a (1/2)
shard_index: 0
shard_count: 2
- group: general-workspaces-a
group_label: workspaces-a (2/2)
shard_index: 1
shard_count: 2
- group: general-workspaces-b
group_label: workspaces-b
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run grouped general test suites
run: |
if [ -n "${{ matrix.shard_count }}" ]; then
pnpm test:run:general -- --group '${{ matrix.group }}' \
--shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}
else
pnpm test:run:general -- --group '${{ matrix.group }}'
fi
verify:
# Preserve the legacy required-check name while the underlying work runs in parallel.
name: verify
if: ${{ always() }}
needs: [typecheck_release_registry, general_tests, build]
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Fail if any split verify lane failed
env:
TYPECHECK_RELEASE_REGISTRY_RESULT: ${{ needs.typecheck_release_registry.result }}
GENERAL_TESTS_RESULT: ${{ needs.general_tests.result }}
BUILD_RESULT: ${{ needs.build.result }}
run: |
test "$TYPECHECK_RELEASE_REGISTRY_RESULT" = "success"
test "$GENERAL_TESTS_RESULT" = "success"
test "$BUILD_RESULT" = "success"
build:
name: Build
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build
run: pnpm build
verify_serialized_server:
name: Verify serialized server suites (${{ matrix.shard_label }})
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
# A successful PR run on 2026-08-17 (32012408876) spent 291s in
# serialized shard 1/5 while its siblings ran 170-201s: round-robin
# clustered the heavy suites on one runner. Shards are now balanced
# by recorded duration (scripts/serialized-shard-durations.json),
# which levels the measured 968s suite total to about 194s per
# runner before setup overhead.
- shard_index: 0
shard_count: 5
shard_label: 1/5
- shard_index: 1
shard_count: 5
shard_label: 2/5
- shard_index: 2
shard_count: 5
shard_label: 3/5
- shard_index: 3
shard_count: 5
shard_label: 4/5
- shard_index: 4
shard_count: 5
shard_label: 5/5
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run serialized server test shard
run: pnpm test:run:serialized -- --shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}
canary_dry_run:
name: Canary Dry Run
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
# `release.sh` always executes its Step 2/7 workspace build, even when
# `--skip-verify` bypasses the initial verification gate. release.sh
# also requires a clean working tree, so any in-place lockfile churn
# from `pnpm install --frozen-lockfile` must be reverted first — unless
# the policy job uploaded a regenerated lockfile (manifest-changing
# PRs), in which case we stage the artifact-restored copy into an
# ephemeral local commit so release.sh sees a clean tree and its
# workspace build sees a lockfile that matches the manifest.
- name: Release canary dry run via release.sh internal build
env:
USED_ARTIFACT_LOCKFILE: ${{ needs.policy.outputs.lockfile_regenerated || '0' }}
run: |
git checkout -B master HEAD
if [ "$USED_ARTIFACT_LOCKFILE" = "1" ]; then
git add pnpm-lock.yaml
if ! git diff --cached --quiet; then
git -c user.email=ci@paperclip.local -c user.name=CI \
commit --no-verify -m "ci(canary): stage regenerated lockfile"
fi
else
git checkout -- pnpm-lock.yaml
fi
./scripts/release.sh canary --skip-verify --dry-run
e2e_shards:
name: e2e shard (${{ matrix.shard_label }})
needs: [policy]
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
# The Playwright lane is pinned to workers=1 (tests/e2e/playwright.config.ts)
# because every spec shares one throwaway server and some toggle
# instance-level flags, so it can only be parallelized across runners.
# Each shard boots its own server, which keeps that isolation intact.
# Three shards let the ~3min smoke-lab spec ride alone while the rest
# of the catalog splits evenly, pulling this lane off the PR critical
# path (it was the slowest check at ~8min20s with two shards).
- shard_index: 0
shard_count: 3
shard_label: 1/3
- shard_index: 1
shard_count: 3
shard_label: 2/3
- shard_index: 2
shard_count: 3
shard_label: 3/3
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup pnpm
uses: pnpm/action-setup@v6
with:
version: 9.15.4
- name: Restore regenerated PR lockfile (if policy uploaded one)
uses: actions/download-artifact@v8
continue-on-error: true
with:
name: pr-lockfile
path: .
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Verify runner Chrome
# GitHub's Ubuntu runner image already ships Google Chrome, so use that
# directly for the headless e2e lane instead of downloading Playwright
# browser bundles inside the 30 minute job budget.
run: google-chrome --version
- name: Generate Paperclip config
run: |
mkdir -p ~/.paperclip/instances/default
cat > ~/.paperclip/instances/default/config.json << 'CONF'
{
"$meta": { "version": 1, "updatedAt": "2026-01-01T00:00:00.000Z", "source": "onboard" },
"database": { "mode": "embedded-postgres" },
"logging": { "mode": "file" },
"server": { "deploymentMode": "local_trusted", "host": "127.0.0.1", "port": 3100 },
"auth": { "baseUrlMode": "auto" },
"storage": { "provider": "local_disk" },
"secrets": { "provider": "local_encrypted", "strictMode": false }
}
CONF
- name: Run e2e tests
env:
PAPERCLIP_E2E_SKIP_LLM: "true"
PAPERCLIP_PLAYWRIGHT_CHANNEL: "chrome"
run: |
# Playwright's own --shard balances by test count, and one spec
# (smoke-lab) is ~40% of the lane's wall clock. Partition by recorded
# spec duration instead so both runners finish together.
specs="$(node ./scripts/e2e-shard.mjs \
--shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }})"
echo "shard ${{ matrix.shard_label }} specs: $specs"
pnpm run test:e2e $specs
- name: Upload Playwright report
uses: actions/upload-artifact@v7
if: always()
with:
name: playwright-report-${{ matrix.shard_index }}
path: |
tests/e2e/playwright-report/
tests/e2e/test-results/
retention-days: 14
e2e:
# Preserve the legacy required-check name while the specs run sharded
# across the matrix above (same pattern as the `verify` aggregate).
name: e2e
if: ${{ always() }}
needs: [e2e_shards]
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Fail if any e2e shard failed
env:
E2E_SHARDS_RESULT: ${{ needs.e2e_shards.result }}
run: test "$E2E_SHARDS_RESULT" = "success"
ci:
# Pin: #13300 merge — restore-only dependency caches and parallel native verification.
uses: paperclipai/paperclip/.github/workflows/pr-trusted.yml@44dde2dec42a22746a2f36b595acacc9ccfa1df6

6
.github/workflows/refresh-lockfile.yml vendored
View File

@ -32,10 +32,12 @@ jobs:
uses: actions/setup-node@v7
with:
node-version: 24
cache: pnpm
# Resolution-only installs do not populate the package store. Do not
# claim the shared cache key with an empty archive before full installs.
package-manager-cache: false
- name: Refresh pnpm lockfile
run: pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
run: pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
- name: Fail on unexpected file changes
run: |

100
.github/workflows/release-smoke.yml vendored
View File

@ -38,10 +38,76 @@ on:
type: string
jobs:
# The Docker smoke below can never exercise the background-service leg of
# onboarding: containers have no service manager, so v2026.824.0 shipped a
# service install that crash-looped on a missing shim while every
# golden-path check stayed green. Run the same published artifact directly
# on the runner VM's systemd and require the installed service to end up
# serving.
smoke_service:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 24
- name: Start a user systemd session
# The hosted runner has no login session for the runner user, so
# `systemctl --user` cannot reach a user manager until lingering
# starts one. Export the session address for the steps below.
run: |
sudo loginctl enable-linger "$(id -un)"
uid="$(id -u)"
for _ in $(seq 1 30); do
[[ -S "/run/user/$uid/bus" ]] && break
sleep 1
done
[[ -S "/run/user/$uid/bus" ]]
{
echo "XDG_RUNTIME_DIR=/run/user/$uid"
echo "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$uid/bus"
} >> "$GITHUB_ENV"
- name: Onboard with the background service
env:
PAPERCLIPAI_VERSION: ${{ inputs.paperclip_version }}
DATA_DIR: ${{ runner.temp }}/service-smoke-data
SMOKE_CLEANUP: "false"
run: ./scripts/service-onboard-smoke.sh
- name: Capture service diagnostics
if: always()
run: |
{
systemctl --user --no-pager status paperclipai.service || true
journalctl --user -u paperclipai.service --no-pager || true
} > "$RUNNER_TEMP/paperclipai-service.log" 2>&1
- name: Upload service diagnostics
if: always()
uses: actions/upload-artifact@v7
with:
name: ${{ inputs.artifact_name }}-service
path: ${{ runner.temp }}/paperclipai-service.log
retention-days: 14
smoke:
runs-on: ubuntu-latest
timeout-minutes: 45
# Fixed here rather than read back out of the harness, so the `always()`
# diagnostics steps below still know the container's name when the launch
# step is the thing that failed. Reading it back was why a failing smoke
# uploaded no Docker logs at all.
env:
SMOKE_CONTAINER_NAME: release-smoke-onboard
steps:
- name: Checkout repository
uses: actions/checkout@v7
@ -67,26 +133,24 @@ jobs:
- name: Launch Docker smoke harness
run: |
metadata_file="$RUNNER_TEMP/release-smoke.env"
HOST_PORT="${{ inputs.host_port }}" \
DATA_DIR="$RUNNER_TEMP/release-smoke-data" \
PAPERCLIPAI_VERSION="${{ inputs.paperclip_version }}" \
SMOKE_READY_TIMEOUT_SECONDS=420 \
SMOKE_DETACH=true \
SMOKE_METADATA_FILE="$metadata_file" \
SMOKE_METADATA_FILE="${{ runner.temp }}/release-smoke.env" \
SMOKE_LOG_FILE="${{ runner.temp }}/docker-onboard-smoke.log" \
./scripts/docker-onboard-smoke.sh
set -a
source "$metadata_file"
source "${{ runner.temp }}/release-smoke.env"
set +a
{
echo "SMOKE_BASE_URL=$SMOKE_BASE_URL"
echo "SMOKE_ADMIN_EMAIL=$SMOKE_ADMIN_EMAIL"
echo "SMOKE_ADMIN_PASSWORD=$SMOKE_ADMIN_PASSWORD"
echo "SMOKE_CONTAINER_NAME=$SMOKE_CONTAINER_NAME"
echo "SMOKE_DATA_DIR=$SMOKE_DATA_DIR"
echo "SMOKE_IMAGE_NAME=$SMOKE_IMAGE_NAME"
echo "SMOKE_PAPERCLIPAI_VERSION=$SMOKE_PAPERCLIPAI_VERSION"
echo "SMOKE_METADATA_FILE=$metadata_file"
} >> "$GITHUB_ENV"
- name: Run release smoke Playwright suite
@ -100,9 +164,20 @@ jobs:
- name: Capture Docker logs
if: always()
run: |
if [[ -n "${SMOKE_CONTAINER_NAME:-}" ]]; then
docker logs "$SMOKE_CONTAINER_NAME" >"$RUNNER_TEMP/docker-onboard-smoke.log" 2>&1 || true
log_file="${{ runner.temp }}/docker-onboard-smoke.log"
# A live container has the fuller story, so prefer it. When the
# harness already tore the container down it wrote this file on its
# way out, and that copy is kept rather than clobbered.
if docker inspect "$SMOKE_CONTAINER_NAME" >/dev/null 2>&1; then
docker logs "$SMOKE_CONTAINER_NAME" >"$log_file" 2>&1 || true
fi
# Never leave the upload with nothing to say. An absent log reads as
# a missing artifact; a file saying the container was gone reads as
# the diagnosis it is.
if [[ ! -s "$log_file" ]]; then
echo "No Docker logs captured: container '$SMOKE_CONTAINER_NAME' left no log dump and is no longer present." >"$log_file"
fi
echo "Captured $(wc -l <"$log_file") log lines to $log_file"
- name: Upload diagnostics
if: always()
@ -111,14 +186,15 @@ jobs:
name: ${{ inputs.artifact_name }}
path: |
${{ runner.temp }}/docker-onboard-smoke.log
${{ env.SMOKE_METADATA_FILE }}
${{ runner.temp }}/release-smoke.env
tests/release-smoke/playwright-report/
tests/release-smoke/test-results/
# The capture step above guarantees the log file, so an empty upload
# means the diagnostics wiring itself broke — which is worth failing
# over rather than burying in a warning nobody reads.
if-no-files-found: error
retention-days: 14
- name: Stop Docker smoke container
if: always()
run: |
if [[ -n "${SMOKE_CONTAINER_NAME:-}" ]]; then
docker rm -f "$SMOKE_CONTAINER_NAME" >/dev/null 2>&1 || true
fi
run: docker rm -f "$SMOKE_CONTAINER_NAME" >/dev/null 2>&1 || true

231
.github/workflows/release-verify.yml vendored
View File

@ -8,31 +8,64 @@ on:
required: true
type: string
# Caller-provided refs may name unmerged PR code. AWS is eligible only when
# the caller runs on canonical master and verifies that event's exact SHA.
# The organization group also restricts these workflow files to master.
jobs:
runner_chaos_evals:
name: Pre-release Runner chaos evals
uses: ./.github/workflows/runner-chaos-evals.yml
with:
ref: ${{ inputs.ref }}
typecheck:
name: Typecheck
runs-on: ubuntu-latest
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 20
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Select the pinned Runner Rust toolchain
working-directory: packages/paperclip-runner
run: |
set -euo pipefail
rustup show
toolchain="$(rustup show active-toolchain | awk '{print $1}')"
echo "RUSTUP_TOOLCHAIN=$toolchain" >> "$GITHUB_ENV"
- name: Cache typecheck Rust dependencies
# Restore and save only within trusted master-push verification. GitHub
# isolates branch/PR caches from master; other callers compile afresh.
if: ${{ github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: packages/paperclip-runner/runner -> target
shared-key: release-typecheck-v1
# Rebuild workspace code and rerun every check. Cache only compiled
# dependencies; never restore installed executables from cargo/bin.
cache-workspace-crates: false
cache-bin: false
# The step guard also restricts restores. Save only after a successful
# master-push verification of that push's exact commit.
save-if: ${{ github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
- name: Validate release package manifest
run: node ./scripts/release-package-map.mjs check
@ -44,7 +77,7 @@ jobs:
general_tests:
name: General tests (${{ matrix.group_label }})
runs-on: ubuntu-latest
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 20
permissions:
contents: read
@ -52,16 +85,59 @@ jobs:
fail-fast: false
matrix:
include:
- group: general-server
group_label: server (1/3)
# Split the long chat file by collected test locations, and balance
# the remaining server files across ten runners. Normal PR/local
# invocations retain their complete general-server group.
- group: general-server-without-chat
group_label: server (1/10)
shard_index: 0
shard_count: 10
- group: general-server-without-chat
group_label: server (2/10)
shard_index: 1
shard_count: 10
- group: general-server-without-chat
group_label: server (3/10)
shard_index: 2
shard_count: 10
- group: general-server-without-chat
group_label: server (4/10)
shard_index: 3
shard_count: 10
- group: general-server-without-chat
group_label: server (5/10)
shard_index: 4
shard_count: 10
- group: general-server-without-chat
group_label: server (6/10)
shard_index: 5
shard_count: 10
- group: general-server-without-chat
group_label: server (7/10)
shard_index: 6
shard_count: 10
- group: general-server-without-chat
group_label: server (8/10)
shard_index: 7
shard_count: 10
- group: general-server-without-chat
group_label: server (9/10)
shard_index: 8
shard_count: 10
- group: general-server-without-chat
group_label: server (10/10)
shard_index: 9
shard_count: 10
- group: general-chat
group_label: chat (1/3)
shard_index: 0
shard_count: 3
- group: general-server
group_label: server (2/3)
- group: general-chat
group_label: chat (2/3)
shard_index: 1
shard_count: 3
- group: general-server
group_label: server (3/3)
- group: general-chat
group_label: chat (3/3)
shard_index: 2
shard_count: 3
# Keep parity with pr.yml: workspaces-a is split with Vitest's
@ -79,17 +155,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
@ -108,7 +184,7 @@ jobs:
serialized_tests:
name: Serialized tests (${{ matrix.shard_label }})
runs-on: ubuntu-latest
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 20
permissions:
contents: read
@ -134,17 +210,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
@ -155,26 +231,26 @@ jobs:
- name: Run serialized server test shard
run: pnpm test:run:serialized -- --shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}
build:
name: Build
runs-on: ubuntu-latest
timeout-minutes: 20
runner_workflow_evals:
name: Runner workflow eval scorer contract
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@v6
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@v7
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
@ -182,5 +258,110 @@ jobs:
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Run deterministic Runner workflow scorer tests
run: pnpm test:runner-workflow-evals
verify_paperclip_runner:
name: Verify Paperclip Runner (${{ matrix.lane }})
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 20
permissions:
contents: read
strategy:
fail-fast: false
matrix:
include:
- lane: protocol
checks: check:eval-kernel check:protocol
- lane: rust
checks: check:runner check:api-authority
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Select the pinned Runner Rust toolchain
working-directory: packages/paperclip-runner
run: |
set -euo pipefail
rustup show
toolchain="$(rustup show active-toolchain | awk '{print $1}')"
echo "RUSTUP_TOOLCHAIN=$toolchain" >> "$GITHUB_ENV"
- name: Cache Runner Rust dependencies
# Restore and save only within trusted master-push verification. GitHub
# isolates branch/PR caches from master; other callers compile afresh.
if: ${{ github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: packages/paperclip-runner/runner -> target
shared-key: release-runner-v1
# Rebuild workspace code and rerun every check. Cache only compiled
# dependencies; never restore installed executables from cargo/bin.
cache-workspace-crates: false
cache-bin: false
# Both lanes restore the existing dependency cache. Only the Rust
# lane saves it, after warming both release and debug dependencies.
save-if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Verify Paperclip Runner
env:
RUNNER_CHECKS: ${{ matrix.checks }}
run: |
set -euo pipefail
for check in $RUNNER_CHECKS; do
pnpm --filter @paperclipai/paperclip-runner "$check"
done
- name: Warm debug dependencies for the shared Runner cache
# Protocol tests need debug binaries. Populate their dependencies in
# the sole cache writer, so a cold save also serves the protocol lane.
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
run: pnpm --filter @paperclipai/paperclip-runner build:rust
build:
name: Build
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 20
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref }}
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Build
run: pnpm build

350
.github/workflows/release.yml vendored
View File

@ -1,4 +1,5 @@
name: Release
run-name: ${{ inputs.channel == 'preview' && format('Stack deploy {0} build', inputs.request_id) || inputs.channel == 'cloud-migrator' && format('Cloud migrator {0}', inputs.source_ref) || 'Release' }}
on:
push:
@ -17,12 +18,22 @@ on:
- stable
- beta
- nightly
- preview
- cloud-migrator
default: stable
source_ref:
description: (stable) Commit SHA, branch, or tag to publish as stable
description: Stable source ref, or full immutable SHA for a preview or cloud migrator build
required: true
type: string
default: master
request_id:
description: (preview/cloud-migrator) Correlation UUID
type: string
default: ""
preview_migrator:
description: (preview) Publish isolated shared and database packages if missing
type: boolean
default: false
stable_date:
description: Enter a UTC date in YYYY-MM-DD format, for example 2026-03-18. Do not enter a version string. The workflow will resolve that date to a stable version such as 2026.318.0, then 2026.318.1 for the next same-day stable.
required: false
@ -46,7 +57,7 @@ on:
default: false
concurrency:
group: release-${{ github.event_name }}-${{ github.ref }}
group: ${{ (inputs.channel == 'preview' || inputs.channel == 'cloud-migrator') && format('{0}-{1}', inputs.channel, inputs.source_ref) || format('release-{0}-{1}', github.event_name, github.ref) }}
cancel-in-progress: false
env:
@ -64,11 +75,252 @@ env:
NPM_PUBLISH_VERIFY_DELAY_SECONDS: "10"
jobs:
plan_preview:
# Only the current master commit can use AWS. A preview or older source
# falls back to GitHub-hosted runners, including raced merge dispatches.
name: Check preview artifacts
if: github.ref == 'refs/heads/master' && github.event_name == 'workflow_dispatch' && (inputs.channel == 'preview' || inputs.channel == 'cloud-migrator') && !inputs.dry_run
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.event_name == 'workflow_dispatch' && inputs.channel == 'cloud-migrator' && github.sha != '' && inputs.source_ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
permissions:
contents: read
# Preserve the previous hosted default; only AWS needs the Fleet limit.
timeout-minutes: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.event_name == 'workflow_dispatch' && inputs.channel == 'cloud-migrator' && github.sha != '' && inputs.source_ref == github.sha && 10 || 360 }}
outputs:
image: ${{ steps.plan.outputs.image }}
packages: ${{ steps.plan.outputs.packages }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Validate immutable source and inspect existing artifacts
id: plan
env:
SOURCE_SHA: ${{ inputs.source_ref }}
REQUEST_ID: ${{ inputs.request_id }}
PREVIEW_MIGRATOR: ${{ inputs.preview_migrator }}
PLAN_COMMAND: ${{ inputs.channel == 'cloud-migrator' && 'plan-migrator' || 'plan' }}
run: node scripts/preview-artifacts.mjs "$PLAN_COMMAND" "$SOURCE_SHA" "$REQUEST_ID" "$PREVIEW_MIGRATOR"
package_preview:
name: Build preview migrator
needs: plan_preview
if: needs.plan_preview.outputs.packages == 'true'
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.event_name == 'workflow_dispatch' && inputs.channel == 'cloud-migrator' && github.sha != '' && inputs.source_ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
path: trusted
persist-credentials: false
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.source_ref }}
path: source
persist-credentials: false
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
run_install: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Install build dependencies without lifecycle scripts
working-directory: source
run: pnpm install --ignore-scripts --no-frozen-lockfile
- name: Build and pack exact-source preview packages
env:
SOURCE_SHA: ${{ inputs.source_ref }}
run: node trusted/scripts/preview-artifacts.mjs pack source packages "$SOURCE_SHA"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: preview-packages
overwrite: true
path: packages/*.tgz
if-no-files-found: error
retention-days: 7
publish_preview:
# npm trusted publishing supports GitHub-hosted runners only.
name: Publish preview migrator
needs: [plan_preview, package_preview]
if: github.ref == 'refs/heads/master' && needs.plan_preview.outputs.packages == 'true' && needs.package_preview.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 30
# A manual preview and a merge-triggered migrator may compile in parallel.
# Serialize only publication so they cannot race an immutable npm version,
# without making the migrator wait for a preview's separate image build.
concurrency:
group: preview-package-publish-${{ inputs.source_ref }}
cancel-in-progress: false
# Reuse release.yml's established npm trusted-publisher identity. This job
# publishes only isolated preview versions; it cannot advance lane tags.
environment: npm-canary
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Install npm with trusted publishing support
run: npm install --global npm@11.18.0 --ignore-scripts
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: preview-packages
path: preview-packages
- name: Publish immutable preview packages without running package code
env:
SOURCE_SHA: ${{ inputs.source_ref }}
run: node scripts/preview-artifacts.mjs publish preview-packages "$SOURCE_SHA"
image_preview:
name: Build preview cloud image
needs: plan_preview
if: needs.plan_preview.outputs.image == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.source_ref }}
persist-credentials: false
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
run_install: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Prepare locked image context
env:
SOURCE_SHA: ${{ inputs.source_ref }}
run: |
set -euo pipefail
test "$(git rev-parse HEAD)" = "$SOURCE_SHA"
pnpm install --resolution-only --ignore-scripts --ignore-pnpmfile --no-frozen-lockfile
echo "TOOLS_EPOCH=$(date -u +%G-W%V)" >> "$GITHUB_ENV"
- uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
- name: Build the immutable cloud image without registry credentials
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
with:
context: .
target: cloud
platforms: linux/amd64
push: false
provenance: false # Docker archives cannot carry registry attestations.
outputs: type=docker,dest=${{ runner.temp }}/preview-image.tar
tags: ghcr.io/paperclipai/paperclip:sha-${{ inputs.source_ref }}-cloud
build-args: |
CLOUD_BUNDLED_PLUGINS=daytona
CLOUD_BUNDLED_SERVER_DEPS=@sentry/node
PAPERCLIP_BUILD_COMMIT=${{ inputs.source_ref }}
PAPERCLIP_BUILD_VERSION=0.0.0-preview.g${{ inputs.source_ref }}
CLI_TOOLS_CACHE_EPOCH=${{ env.TOOLS_EPOCH }}
labels: |
org.opencontainers.image.revision=${{ inputs.source_ref }}
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: preview-image
overwrite: true
path: ${{ runner.temp }}/preview-image.tar
compression-level: 0
if-no-files-found: error
retention-days: 1
publish_image_preview:
name: Publish preview cloud image
needs: [plan_preview, image_preview]
if: github.ref == 'refs/heads/master' && needs.plan_preview.outputs.image == 'true' && needs.image_preview.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 30
# This existing environment has an external master-only branch policy.
environment: npm-canary
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: preview-image
path: preview-image
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Verify image identity and publish without executing image code
env:
SOURCE_SHA: ${{ inputs.source_ref }}
run: node scripts/preview-artifacts.mjs publish-image preview-image/preview-image.tar "$SOURCE_SHA"
result_preview:
name: Verify preview artifacts
needs: [plan_preview, image_preview, publish_image_preview, package_preview, publish_preview]
if: >-
always() && inputs.channel == 'preview' && needs.plan_preview.result == 'success' &&
(needs.publish_image_preview.result == 'success' || needs.plan_preview.outputs.image == 'false') &&
(needs.publish_preview.result == 'success' || needs.plan_preview.outputs.packages == 'false')
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Confirm exact artifacts are visible
env:
SOURCE_SHA: ${{ inputs.source_ref }}
REQUEST_ID: ${{ inputs.request_id }}
PREVIEW_MIGRATOR: ${{ inputs.preview_migrator }}
run: node scripts/preview-artifacts.mjs result "$SOURCE_SHA" "$REQUEST_ID"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: stack-deploy-result
overwrite: true
path: stack-deploy-result/result.json
if-no-files-found: error
retention-days: 30
verify_canary:
if: github.event_name == 'push'
uses: ./.github/workflows/release-verify.yml
with:
ref: ${{ github.sha }}
name: Reuse exact-source verification
if: github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
timeout-minutes: 50
permissions:
contents: read
actions: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Require successful source checks for this exact master push
env:
GITHUB_TOKEN: ${{ github.token }}
SOURCE_SHA: ${{ github.sha }}
run: node scripts/cloud-source-verification.mjs "$SOURCE_SHA"
publish_canary:
if: github.event_name == 'push'
@ -76,9 +328,13 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 90
environment: npm-canary
outputs:
canary_version: ${{ steps.canary_tag.outputs.version }}
permissions:
contents: write
id-token: write
# For the explicit docker.yml dispatch below.
actions: write
steps:
- name: Checkout repository
@ -127,6 +383,7 @@ jobs:
done
- name: Push canary tag
id: canary_tag
run: |
tag="$(git tag --points-at HEAD | grep '^canary/v' | head -1)"
if [ -z "$tag" ]; then
@ -134,6 +391,81 @@ jobs:
exit 1
fi
git push origin "refs/tags/${tag}"
echo "version=${tag#canary/v}" >> "$GITHUB_OUTPUT"
# Canary images previously relied on the master-push docker.yml run,
# whose single pending concurrency slot gets superseded by every
# newer push — on a busy day no canary image publishes at all (five
# consecutive canaries shipped npm packages with no cloud image on
# 2026-09-06, starving downstream managed deploys for ~18 hours).
# Tag pushes made with GITHUB_TOKEN do not fire docker.yml's
# triggers, so dispatch the image build at the canary tag
# explicitly, exactly like the nightly and beta lanes: the run keys
# its concurrency off the tag ref, so no master push can supersede
# it, and docker.yml's `type=sha` mapping publishes the
# sha-<short> and sha-<short>-cloud images either way.
- name: Build Docker images for the canary tag
env:
GH_TOKEN: ${{ github.token }}
run: |
{
echo "## Canary published"
echo ""
echo "- Published canary: \`${{ steps.canary_tag.outputs.version }}\`"
echo "- Docker build dispatched at \`canary/v${{ steps.canary_tag.outputs.version }}\`"
} >> "$GITHUB_STEP_SUMMARY"
gh workflow run docker.yml --ref "refs/tags/canary/v${{ steps.canary_tag.outputs.version }}" --repo "$GITHUB_REPOSITORY"
# The package is already public when this gate runs. A red result leaves the
# immutable canary in npm, but makes the release workflow visibly fail before
# anyone mistakes an installable package for an onboardable one.
smoke_canary_onboarding:
needs: publish_canary
if: needs.publish_canary.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
env:
PAPERCLIPAI_VERSION: ${{ needs.publish_canary.outputs.canary_version }}
PAPERCLIP_PLAYWRIGHT_CHANNEL: chrome
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Install test dependencies
run: pnpm install --frozen-lockfile
- name: Show browser version
run: google-chrome --version
- name: Smoke exact published canary through onboarding
env:
PAPERCLIP_CANARY_SMOKE_SERVER_LOG: ${{ runner.temp }}/canary-onboarding-server.log
run: pnpm run test:canary-onboarding-smoke
- name: Upload failed canary onboarding diagnostics
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: canary-onboarding-smoke-${{ needs.publish_canary.outputs.canary_version }}
if-no-files-found: warn
retention-days: 14
path: |
${{ runner.temp }}/canary-onboarding-server.log
tests/canary-onboarding/playwright-report/
tests/canary-onboarding/test-results/
# ----- Nightly lane -----------------------------------------------------
# Once a night (or on a forced nightly dispatch), promote the newest master
@ -647,6 +979,9 @@ jobs:
timeout-minutes: 10
permissions:
contents: write
# gh pr view needs PR read for the skeleton's nested summaries;
# without it the enrichment silently degrades to bare subjects.
pull-requests: read
steps:
- name: Checkout repository
uses: actions/checkout@v7
@ -661,6 +996,9 @@ jobs:
- name: Draft stable notes from the published beta
env:
# gh needs a token so the generator can nest each PR's summary
# under its subject line (best-effort thoroughness).
GH_TOKEN: ${{ github.token }}
BETA_VERSION: ${{ needs.publish_beta.outputs.beta_version }}
SOURCE_SHA: ${{ needs.select_beta.outputs.sha }}
run: |

85
.github/workflows/runner-chaos-evals.yml vendored Normal file
View File

@ -0,0 +1,85 @@
name: Runner Chaos Evals
on:
schedule:
- cron: "43 7 * * 0"
workflow_dispatch:
workflow_call:
inputs:
ref:
description: Commit SHA, branch, or tag to verify before release
required: false
type: string
concurrency:
# Reusable calls inherit the caller's workflow name. Cloud readiness and
# Release verify the same SHA independently and must not cancel each other.
group: runner-chaos-evals-${{ github.workflow }}-${{ inputs.ref || github.ref }}
cancel-in-progress: true
jobs:
chaos_and_recovery:
name: Restart, replay, trace, and recovery faults
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
timeout-minutes: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 40 || 45 }}
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ inputs.ref || github.sha }}
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- name: Install dependencies
run: pnpm install --no-frozen-lockfile
- name: Build eval and Runner contracts
run: |
pnpm --filter @paperclipai/paperclip-eval-kernel build
pnpm --filter @paperclipai/paperclip-runner report:runner-chaos-evals
- name: Run Runner fault and replay suites
run: |
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
src/eval/workflow-evals.test.ts \
src/native-session-runtime.test.ts \
src/live/live-session.test.ts \
src/live/turn-stream.test.ts \
src/protocol/replay-contract.test.ts \
src/drivers/opencode/mcp-bridge.test.ts \
src/drivers/acpx/runtime-host.test.ts
- name: Build server test dependencies
run: pnpm --filter @paperclipai/plugin-sdk ensure-build-deps
- name: Run server finalization and recovery suites
run: |
pnpm --filter @paperclipai/server exec vitest run \
src/__tests__/native-finalization-recovery.test.ts \
src/__tests__/heartbeat-process-recovery.test.ts \
src/__tests__/heartbeat-comment-wake-batching.test.ts \
src/__tests__/heartbeat-dependency-scheduling.test.ts \
src/__tests__/provider-trace-store.test.ts \
src/services/issue-thread-interaction-resolution.test.ts \
src/services/recovery/successful-run-handoff.test.ts
- name: Upload chaos eval bundle
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-chaos-evals-${{ github.run_id }}
path: packages/paperclip-runner/.paperclip-local/evals/workflows/
retention-days: 30
if-no-files-found: error

1260
.github/workflows/runner-full-stack-e2e.yml vendored Normal file
View File

File diff suppressed because it is too large Load Diff

192
.github/workflows/runner-live-evals.yml vendored Normal file
View File

@ -0,0 +1,192 @@
name: Runner Live Evals
on:
schedule:
- cron: "17 6 * * 0"
workflow_dispatch:
inputs:
candidate:
description: "Comma-separated live candidate IDs (for example codex-luna)"
type: string
required: false
case:
description: "Comma-separated workflow case IDs"
type: string
required: false
limit:
description: "Maximum executions after candidate/case filtering"
type: string
required: false
concurrency:
group: runner-live-evals-${{ github.ref }}
cancel-in-progress: true
jobs:
authorize:
name: Authorize paid campaign
if: github.event_name != 'schedule' || vars.RUNNER_LIVE_EVALS_NIGHTLY_ENABLED == 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
eval_runner: ${{ steps.runner.outputs.runner }}
steps:
- name: Require default branch and allowlisted numeric actor IDs
env:
GH_TOKEN: ${{ github.token }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR: ${{ github.actor }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
if [ "$REF" != "refs/heads/$DEFAULT_BRANCH" ]; then
echo "Paid runner live evals may run only from the default branch." >&2
exit 1
fi
if ! jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null; then
echo "RUNNER_E2E_ALLOWED_ACTOR_IDS must be a non-empty JSON array of numeric GitHub user IDs." >&2
exit 1
fi
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
if [ "$triggering_actor_id" != "$ACTOR_ID" ] && [ "$TRIGGERING_ACTOR" = "$ACTOR" ]; then
echo "GitHub actor identity contexts disagree; refusing the paid run." >&2
exit 1
fi
candidates=("$triggering_actor_id" "$ACTOR_ID")
for candidate in "${candidates[@]}"; do
if ! jq -e --argjson candidate "$candidate" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null; then
echo "The initiating GitHub account is not authorized to run paid runner live evals." >&2
exit 1
fi
done
- name: Select paid eval runner
id: runner
env:
AWS_PAID_RUNNER_ENABLED: ${{ vars.RUNNER_E2E_AWS_ENABLED }}
run: |
set -euo pipefail
github_runner='ubuntu-latest'
aws_runner='runs-on/fleet=paperclip-public-pr-x64/env=public-ci'
if [ "$AWS_PAID_RUNNER_ENABLED" = true ]; then
echo "runner=$aws_runner" >> "$GITHUB_OUTPUT"
echo '::notice title=Paid eval routing::Using an ephemeral RunsOn Fleet runner'
else
echo "runner=$github_runner" >> "$GITHUB_OUTPUT"
echo '::notice title=Paid eval routing::RUNNER_E2E_AWS_ENABLED is not true; using the proven GitHub-hosted runner'
fi
live_matrix:
name: Balanced provider/model matrix
needs: authorize
if: github.event_name != 'schedule' || vars.RUNNER_LIVE_EVALS_NIGHTLY_ENABLED == 'true'
# The authorize job selects one of two literal, reviewed runner labels;
# dispatch inputs and repository variables cannot inject an arbitrary label.
runs-on: ${{ needs.authorize.outputs.eval_runner }}
timeout-minutes: 180
permissions:
contents: read
environment:
name: runner-e2e-paid
steps:
- name: Reauthorize paid execution before provider access
env:
GH_TOKEN: ${{ github.token }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
test "$REF" = "refs/heads/$DEFAULT_BRANCH"
jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: Generate private eval-repository token
id: evals_token
env:
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
GH_REPO: ${{ github.repository }}
run: |
set -euo pipefail
token="$(node .github/scripts/get-bot-token.mjs)"
echo "::add-mask::$token"
echo "value=$token" >> "$GITHUB_OUTPUT"
- name: Checkout canonical Evalbook reporter
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
repository: paperclipai/paperclip-evals
ref: 0c8dfea0ef71a73e909b59a5c0484554cbee199b
path: .paperclip-evals
token: ${{ steps.evals_token.outputs.value }}
persist-credentials: false
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Setup Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Restore compatible weekly baseline
uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: packages/paperclip-runner/.paperclip-local/evals/workflows/history
key: runner-live-eval-history-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
runner-live-eval-history-${{ github.ref_name }}-
- name: Build provider-neutral eval kernel
run: pnpm --filter @paperclipai/paperclip-eval-kernel build
- name: Run trend-only live matrix
env:
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PAPERCLIP_EVAL_BASELINE_READY: "true"
PAPERCLIP_EVAL_RUNNER_BUILD: ${{ github.sha }}
PAPERCLIP_EVAL_MAX_CAMPAIGN_COST_USD: "12"
PAPERCLIP_EVAL_SCHEDULE_SEED: runner-live-seven-week-v1
PAPERCLIP_EVAL_CANDIDATE: ${{ inputs.candidate }}
PAPERCLIP_EVAL_CASE: ${{ inputs.case }}
PAPERCLIP_EVAL_LIMIT: ${{ inputs.limit }}
PAPERCLIP_EVALBOOK_PROGRAM: ${{ github.workspace }}/.paperclip-evals/evals/paperclip-runner/tools/eval_program.py
run: pnpm --filter @paperclipai/paperclip-runner report:runner-live-evals
- name: Publish job summary
if: always()
run: |
summary=packages/paperclip-runner/.paperclip-local/evals/workflows/github-live-summary.md
if [ -f "$summary" ]; then
cat "$summary" >> "$GITHUB_STEP_SUMMARY"
fi
- name: Upload safe live eval bundle
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-live-evals-${{ github.run_id }}
path: packages/paperclip-runner/.paperclip-local/evals/workflows/
retention-days: 30
if-no-files-found: error

720
.github/workflows/runner-protocol-live-evals.yml vendored Normal file
View File

@ -0,0 +1,720 @@
name: Runner Direct Live Protocol Evals
on:
schedule:
- cron: "23 9 * * 0"
workflow_dispatch:
inputs:
target_branch:
description: "Branch in paperclipai/paperclip to evaluate; trusted orchestration still runs from master"
type: string
required: false
evals_sha:
description: "Exact 40-character paperclipai/paperclip-evals commit to execute"
type: string
required: false
rosters:
description: "Comma-separated live roster IDs/files, or all for the maintained enabled direct suite"
type: string
default: "all"
required: false
max_infrastructure_retries:
description: "Automatic retries only for explicitly retryable infrastructure failures (0-3)"
type: number
default: 1
permissions:
contents: read
concurrency:
group: runner-protocol-live-evals-${{ github.event_name == 'workflow_dispatch' && inputs.target_branch != '' && inputs.target_branch != github.event.repository.default_branch && format('development-{0}', inputs.target_branch) || format('protected-{0}', github.run_id) }}
cancel-in-progress: ${{ github.event_name == 'workflow_dispatch' && inputs.target_branch != '' && inputs.target_branch != github.event.repository.default_branch }}
jobs:
authorize:
name: Authorize paid direct eval campaign
if: github.event_name != 'schedule' || vars.RUNNER_PROTOCOL_EVAL_NIGHTLY_ENABLED == 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
test_runner: ${{ steps.runner.outputs.runner }}
max_parallel_default: ${{ steps.runner.outputs.max_parallel_default }}
max_parallel_limit: ${{ steps.runner.outputs.max_parallel_limit }}
target_sha: ${{ steps.target.outputs.sha }}
target_ref: ${{ steps.target.outputs.ref }}
evals_sha: ${{ steps.evals.outputs.sha }}
steps:
- name: Require default branch and allowlisted numeric actor IDs
env:
GH_TOKEN: ${{ github.token }}
REPOSITORY: ${{ github.repository }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR: ${{ github.actor }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
if [ "$REF" != "refs/heads/$DEFAULT_BRANCH" ]; then
echo "Paid direct Runner eval campaigns may run only from the default branch." >&2
exit 1
fi
if ! jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null; then
echo "RUNNER_E2E_ALLOWED_ACTOR_IDS must be a non-empty JSON array of numeric GitHub user IDs." >&2
exit 1
fi
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
if [ "$triggering_actor_id" != "$ACTOR_ID" ] && [ "$TRIGGERING_ACTOR" = "$ACTOR" ]; then
echo "GitHub actor identity contexts disagree; refusing the paid run." >&2
exit 1
fi
for candidate in "$triggering_actor_id" "$ACTOR_ID"; do
if ! jq -e --argjson candidate "$candidate" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null; then
echo "The initiating GitHub account is not authorized to run paid Runner eval campaigns." >&2
exit 1
fi
done
- name: Resolve requested Paperclip branch to an immutable commit
id: target
env:
GH_TOKEN: ${{ github.token }}
REPOSITORY: ${{ github.repository }}
TARGET_BRANCH: ${{ inputs.target_branch || github.event.repository.default_branch }}
run: |
set -euo pipefail
if [ -z "$TARGET_BRANCH" ] || [[ "$TARGET_BRANCH" == refs/* ]]; then
echo "target_branch must name a branch in this repository without a refs/ prefix." >&2
exit 1
fi
encoded_branch="$(jq -rn --arg branch "$TARGET_BRANCH" '$branch | @uri')"
target_sha="$(gh api -X GET "repos/$REPOSITORY/branches/$encoded_branch" --jq .commit.sha)"
[[ "$target_sha" =~ ^[0-9a-f]{40}$ ]]
echo "sha=$target_sha" >> "$GITHUB_OUTPUT"
echo "ref=refs/heads/$TARGET_BRANCH" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Generate private eval-repository token
id: evals_token
env:
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
GH_REPO: paperclipai/paperclip-evals
run: |
set -euo pipefail
token="$(node .github/scripts/get-bot-token.mjs)"
echo "::add-mask::$token"
echo "value=$token" >> "$GITHUB_OUTPUT"
- name: Verify the private eval program is pinned to an exact commit
id: evals
env:
GH_TOKEN: ${{ steps.evals_token.outputs.value }}
EVALS_SHA: ${{ inputs.evals_sha || vars.RUNNER_PROTOCOL_EVALS_SHA }}
run: |
set -euo pipefail
if ! [[ "$EVALS_SHA" =~ ^[0-9a-f]{40}$ ]]; then
echo "evals_sha (or RUNNER_PROTOCOL_EVALS_SHA for schedules) must be an exact 40-character commit." >&2
exit 1
fi
resolved="$(gh api -X GET "repos/paperclipai/paperclip-evals/commits/$EVALS_SHA" --jq .sha)"
test "$resolved" = "$EVALS_SHA"
echo "sha=$resolved" >> "$GITHUB_OUTPUT"
- name: Validate retry envelope
env:
RETRIES: ${{ github.event_name == 'schedule' && 1 || inputs.max_infrastructure_retries }}
run: |
set -euo pipefail
[[ "$RETRIES" =~ ^[0-3]$ ]]
- name: Select paid test runner
id: runner
env:
AWS_PAID_RUNNER_ENABLED: ${{ vars.RUNNER_E2E_AWS_ENABLED }}
run: |
set -euo pipefail
if [ "$AWS_PAID_RUNNER_ENABLED" = true ]; then
{
echo 'runner=runs-on/fleet=paperclip-public-pr-x64/env=public-ci'
echo 'max_parallel_default=100'
echo 'max_parallel_limit=100'
} >> "$GITHUB_OUTPUT"
else
{
echo 'runner=ubuntu-latest'
echo 'max_parallel_default=32'
echo 'max_parallel_limit=57'
} >> "$GITHUB_OUTPUT"
fi
catalog:
name: Pin and fan out the direct Evalbook roster
needs: authorize
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
outputs:
matrix_0: ${{ steps.catalog.outputs.matrix_0 }}
matrix_1: ${{ steps.catalog.outputs.matrix_1 }}
matrix_1_present: ${{ steps.catalog.outputs.matrix_1_present }}
max_parallel_per_shard: ${{ steps.catalog.outputs.max_parallel_per_shard }}
selected: ${{ steps.catalog.outputs.selected }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Generate private eval-repository token
id: evals_token
env:
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
GH_REPO: paperclipai/paperclip-evals
run: |
set -euo pipefail
token="$(node .github/scripts/get-bot-token.mjs)"
echo "::add-mask::$token"
echo "value=$token" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
repository: paperclipai/paperclip-evals
ref: ${{ needs.authorize.outputs.evals_sha }}
path: .paperclip-evals
token: ${{ steps.evals_token.outputs.value }}
persist-credentials: false
- name: Build the two bounded roster-plus-case matrices
id: catalog
env:
PAPERCLIP_PROTOCOL_EVAL_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
PAPERCLIP_PROTOCOL_EVALS_SHA: ${{ needs.authorize.outputs.evals_sha }}
MAX_PARALLEL: ${{ vars.RUNNER_E2E_MAX_PARALLEL || needs.authorize.outputs.max_parallel_default }}
MAX_PARALLEL_LIMIT: ${{ needs.authorize.outputs.max_parallel_limit }}
ROSTERS: ${{ inputs.rosters || 'all' }}
run: |
set -euo pipefail
if ! [[ "$MAX_PARALLEL" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL" -lt 2 ] || [ "$MAX_PARALLEL" -gt "$MAX_PARALLEL_LIMIT" ]; then
echo "RUNNER_E2E_MAX_PARALLEL must be an integer from 2 through $MAX_PARALLEL_LIMIT for the two-shard direct suite." >&2
exit 1
fi
node packages/paperclip-runner/scripts/runner-protocol-eval-campaign.mjs catalog \
--evals-root .paperclip-evals \
--rosters "$ROSTERS" \
--campaign-id "gha-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" \
--max-parallel "$MAX_PARALLEL" \
--output runner-protocol-eval-catalog.json
- name: Require the chat-report renderer before paid execution
run: |
set -euo pipefail
python3 .paperclip-evals/evals/paperclip-runner/tools/eval_program.py report --help | grep -q -- --public-viewer
- name: Upload immutable campaign catalog
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-eval-catalog-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-eval-catalog.json
retention-days: 30
if-no-files-found: error
build_runner:
name: Build portable direct-eval runner once
needs: [authorize, catalog]
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ needs.authorize.outputs.target_sha }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
env:
NPM_CONFIG_AUDIT: "false"
NPM_CONFIG_FUND: "false"
NPM_CONFIG_UPDATE_NOTIFIER: "false"
with:
version: 9.15.4
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
cache: pnpm
- run: pnpm install --frozen-lockfile --ignore-scripts
- name: Build runner CLI, daemon, and canonical attempt viewer
run: |
set -euo pipefail
pnpm --filter @paperclipai/paperclip-runner build:typescript
pnpm --filter @paperclipai/paperclip-runner build:runner-binaries
pnpm --filter @paperclipai/paperclip-runner build:issue-thread
# Older target refs must fail before paid cells, not publish an empty viewer.
grep -q 'paperclip-eval-report' packages/paperclip-runner/dist-issue-thread/assets/*.js
grep -q 'evalbook-site' packages/paperclip-runner/dist-issue-thread/assets/*.css
- name: Package a portable provider runtime
run: |
set -euo pipefail
mkdir -p "$RUNNER_TEMP/runner-protocol-build/package" "$RUNNER_TEMP/runner-protocol-build/portable"
pnpm --dir packages/paperclip-runner pack \
--pack-destination "$RUNNER_TEMP/runner-protocol-build/package"
package="$(find "$RUNNER_TEMP/runner-protocol-build/package" -maxdepth 1 -type f -name '*.tgz' -print -quit)"
test -f "$package"
pnpm --filter @paperclipai/paperclip-runner deploy --prod \
"$RUNNER_TEMP/runner-protocol-build/portable"
cp "$package" "$RUNNER_TEMP/runner-protocol-build/paperclip-runner.tgz"
cp packages/paperclip-runner/runner/target/debug/paperclip-runnerd "$RUNNER_TEMP/runner-protocol-build/paperclip-runnerd"
cp -R packages/paperclip-runner/dist-issue-thread "$RUNNER_TEMP/runner-protocol-build/dist-issue-thread"
test -f "$RUNNER_TEMP/runner-protocol-build/portable/dist/cli/eval-session.js"
test -d "$RUNNER_TEMP/runner-protocol-build/portable/node_modules/.pnpm"
test -x "$RUNNER_TEMP/runner-protocol-build/paperclip-runnerd"
tar --create --gzip --file runner-protocol-build.tar.gz -C "$RUNNER_TEMP/runner-protocol-build" .
sha256sum runner-protocol-build.tar.gz > runner-protocol-build.tar.gz.sha256
- name: Upload immutable portable runner
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-build-${{ needs.authorize.outputs.target_sha }}-${{ github.run_id }}-${{ github.run_attempt }}
path: |
runner-protocol-build.tar.gz
runner-protocol-build.tar.gz.sha256
retention-days: 1
compression-level: 0
if-no-files-found: error
- name: Upload canonical viewer for publisher byte verification
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-viewer-${{ github.run_id }}-${{ github.run_attempt }}
path: packages/paperclip-runner/dist-issue-thread/
retention-days: 30
if-no-files-found: error
eval_shard_0:
name: Direct eval ${{ matrix.rosterId }} / ${{ matrix.caseId }}
needs: [authorize, catalog, build_runner]
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: 18
permissions:
contents: read
id-token: write
environment:
name: runner-e2e-paid
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel_per_shard) }}
matrix: ${{ fromJSON(needs.catalog.outputs.matrix_0) }}
steps: &direct_eval_steps
- name: Reauthorize paid execution before provider access
env:
GH_TOKEN: ${{ github.token }}
REF: ${{ github.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
ACTOR: ${{ github.actor }}
ACTOR_ID: ${{ github.actor_id }}
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
run: |
set -euo pipefail
test "$REF" = "refs/heads/$DEFAULT_BRANCH"
triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
test "$triggering_actor_id" = "$ACTOR_ID" || test "$TRIGGERING_ACTOR" != "$ACTOR"
for candidate in "$triggering_actor_id" "$ACTOR_ID"; do
jq -e --argjson candidate "$candidate" 'type == "array" and index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
done
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Generate private eval-repository token
id: evals_token
env:
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
GH_REPO: paperclipai/paperclip-evals
run: |
set -euo pipefail
token="$(node .github/scripts/get-bot-token.mjs)"
echo "::add-mask::$token"
echo "value=$token" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
repository: paperclipai/paperclip-evals
ref: ${{ needs.authorize.outputs.evals_sha }}
path: .paperclip-evals
token: ${{ steps.evals_token.outputs.value }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Download portable runner
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runner-protocol-build-${{ needs.authorize.outputs.target_sha }}-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-build
- name: Verify and extract portable runner
run: |
set -euo pipefail
cd runner-protocol-build
sha256sum --check runner-protocol-build.tar.gz.sha256
mkdir extracted
tar --extract --gzip --file runner-protocol-build.tar.gz --directory extracted
test -x extracted/paperclip-runnerd
- name: Prepare short-lived AgentCore web identity
if: matrix.credentialName == 'AWS_AGENTCORE_OIDC'
env:
AGENTCORE_ROLE_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_EXECUTION_ROLE_ARN }}
run: |
set -euo pipefail
test -n "$AGENTCORE_ROLE_ARN"
token="$(curl --fail --silent --show-error \
-H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=sts.amazonaws.com" | jq -r .value)"
test -n "$token"
echo "::add-mask::$token"
token_file="$RUNNER_TEMP/runner-protocol-agentcore-token"
printf '%s' "$token" > "$token_file"
chmod 600 "$token_file"
{
echo "AWS_WEB_IDENTITY_TOKEN_FILE=$token_file"
echo "AWS_ROLE_ARN=$AGENTCORE_ROLE_ARN"
echo "AWS_ROLE_SESSION_NAME=runner-protocol-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
} >> "$GITHUB_ENV"
- name: Run one immutable direct protocol cell
id: direct_eval
env:
CELL_ID: ${{ matrix.cellId }}
ROSTER_FILE: ${{ matrix.rosterFile }}
CASE_ID: ${{ matrix.caseId }}
CREDENTIAL_NAME: ${{ matrix.credentialName }}
PROVIDER: ${{ matrix.provider }}
MAX_INFRASTRUCTURE_RETRIES: ${{ github.event_name == 'schedule' && 1 || inputs.max_infrastructure_retries }}
OPENAI_API_KEY: ${{ matrix.credentialName == 'OPENAI_API_KEY' && secrets.OPENAI_API_KEY || '' }}
ANTHROPIC_API_KEY: ${{ matrix.credentialName == 'ANTHROPIC_API_KEY' && secrets.ANTHROPIC_API_KEY || '' }}
OPENROUTER_API_KEY: ${{ matrix.credentialName == 'OPENROUTER_API_KEY' && secrets.OPENROUTER_API_KEY || '' }}
PAPERCLIP_CLAUDE_MANAGED_PROFILE_ID: ${{ vars.PAPERCLIP_CLAUDE_MANAGED_PROFILE_ID }}
PAPERCLIP_CLAUDE_MANAGED_AGENT_ID: ${{ vars.PAPERCLIP_CLAUDE_MANAGED_AGENT_ID }}
PAPERCLIP_CLAUDE_MANAGED_AGENT_VERSION: ${{ vars.PAPERCLIP_CLAUDE_MANAGED_AGENT_VERSION }}
PAPERCLIP_CLAUDE_MANAGED_ENVIRONMENT_ID: ${{ vars.PAPERCLIP_CLAUDE_MANAGED_ENVIRONMENT_ID }}
AWS_REGION: ${{ vars.PAPERCLIP_AWS_AGENTCORE_REGION }}
AWS_DEFAULT_REGION: ${{ vars.PAPERCLIP_AWS_AGENTCORE_REGION }}
PAPERCLIP_AWS_AGENTCORE_PROFILE_ID: ${{ vars.PAPERCLIP_AWS_AGENTCORE_PROFILE_ID }}
PAPERCLIP_AWS_AGENTCORE_ACCOUNT_ID: ${{ vars.PAPERCLIP_AWS_AGENTCORE_ACCOUNT_ID }}
PAPERCLIP_AWS_AGENTCORE_HARNESS_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_HARNESS_ARN }}
PAPERCLIP_AWS_AGENTCORE_HARNESS_VERSION: ${{ vars.PAPERCLIP_AWS_AGENTCORE_HARNESS_VERSION }}
PAPERCLIP_AWS_AGENTCORE_ENDPOINT_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_ENDPOINT_ARN }}
PAPERCLIP_AWS_AGENTCORE_ENDPOINT_QUALIFIER: ${{ vars.PAPERCLIP_AWS_AGENTCORE_ENDPOINT_QUALIFIER }}
PAPERCLIP_AWS_AGENTCORE_RUNTIME_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_RUNTIME_ARN }}
PAPERCLIP_AWS_AGENTCORE_MEMORY_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_MEMORY_ARN }}
PAPERCLIP_AWS_AGENTCORE_MEMORY_ID: ${{ vars.PAPERCLIP_AWS_AGENTCORE_MEMORY_ID }}
PAPERCLIP_AWS_AGENTCORE_INVOCATION_ROLE_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_INVOCATION_ROLE_ARN }}
PAPERCLIP_AWS_AGENTCORE_CONTEXT_BUCKET: ${{ vars.PAPERCLIP_AWS_AGENTCORE_CONTEXT_BUCKET }}
PAPERCLIP_AWS_AGENTCORE_CONTEXT_PREFIX: ${{ vars.PAPERCLIP_AWS_AGENTCORE_CONTEXT_PREFIX }}
PAPERCLIP_AWS_AGENTCORE_CONTEXT_KMS_KEY_ARN: ${{ vars.PAPERCLIP_AWS_AGENTCORE_CONTEXT_KMS_KEY_ARN }}
PAPERCLIP_AWS_AGENTCORE_QUALIFICATION_REVISION: ${{ vars.PAPERCLIP_AWS_AGENTCORE_QUALIFICATION_REVISION }}
run: |
set -euo pipefail
mkdir -p cell-output/runs
if [ "$CREDENTIAL_NAME" != "AWS_AGENTCORE_OIDC" ]; then
test -n "${!CREDENTIAL_NAME:-}"
fi
if [ "$PROVIDER" = "claude_managed" ]; then
test -n "$PAPERCLIP_CLAUDE_MANAGED_PROFILE_ID"
test -n "$PAPERCLIP_CLAUDE_MANAGED_AGENT_ID"
test -n "$PAPERCLIP_CLAUDE_MANAGED_AGENT_VERSION"
test -n "$PAPERCLIP_CLAUDE_MANAGED_ENVIRONMENT_ID"
fi
set +e
python3 .paperclip-evals/evals/paperclip-runner/tools/run_live_roster.py run \
--roster ".paperclip-evals/evals/paperclip-runner/rosters/$ROSTER_FILE" \
--case "$CASE_ID" \
--runner-cli runner-protocol-build/extracted/portable/dist/cli/eval-session.js \
--runner-package runner-protocol-build/extracted/paperclip-runner.tgz \
--runnerd runner-protocol-build/extracted/paperclip-runnerd \
--runs-root cell-output/runs \
--max-infrastructure-retries "$MAX_INFRASTRUCTURE_RETRIES" \
--run-id "gha-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${CELL_ID}"
status=$?
set -e
CELL_EXIT_CODE="$status" node --input-type=module <<'NODE'
import { writeFileSync } from "node:fs";
writeFileSync("cell-output/cell.json", `${JSON.stringify({
schema: "paperclip.runner-protocol-eval.cell/v1",
cellId: process.env.CELL_ID,
rosterFile: process.env.ROSTER_FILE,
caseId: process.env.CASE_ID,
exitCode: Number(process.env.CELL_EXIT_CODE),
}, null, 2)}\n`, { mode: 0o600 });
NODE
exit "$status"
- name: Upload access-controlled cell attempt
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-eval-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.cellId }}
path: cell-output/
retention-days: 30
if-no-files-found: error
eval_shard_1:
name: Direct eval ${{ matrix.rosterId }} / ${{ matrix.caseId }}
if: needs.catalog.outputs.matrix_1_present == 'true'
needs: [authorize, catalog, build_runner]
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: 18
permissions:
contents: read
id-token: write
environment:
name: runner-e2e-paid
strategy:
fail-fast: false
max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel_per_shard) }}
matrix: ${{ fromJSON(needs.catalog.outputs.matrix_1) }}
steps: *direct_eval_steps
report:
name: Merge attempts and render canonical Evalbook
if: always() && !cancelled() && needs.catalog.result == 'success' && needs.build_runner.result == 'success'
needs: [authorize, catalog, build_runner, eval_shard_0, eval_shard_1]
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
actions: read
contents: read
outputs:
public_report_ready: ${{ steps.public_report.outputs.ready }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- name: Generate private eval-repository token
id: evals_token
env:
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
GH_REPO: paperclipai/paperclip-evals
run: |
set -euo pipefail
token="$(node .github/scripts/get-bot-token.mjs)"
echo "::add-mask::$token"
echo "value=$token" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
repository: paperclipai/paperclip-evals
ref: ${{ needs.authorize.outputs.evals_sha }}
path: .paperclip-evals
token: ${{ steps.evals_token.outputs.value }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Download immutable campaign catalog
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runner-protocol-eval-catalog-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-catalog
- name: Download portable runner and viewer
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runner-protocol-build-${{ needs.authorize.outputs.target_sha }}-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-build
- name: Download every access-controlled cell
id: download_cells
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
pattern: runner-protocol-eval-${{ github.run_id }}-${{ github.run_attempt }}-*
path: downloaded-runner-protocol-evals
merge-multiple: false
- name: Retry cell download after artifact transport failure
if: steps.download_cells.outcome == 'failure'
continue-on-error: true
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
pattern: runner-protocol-eval-${{ github.run_id }}-${{ github.run_attempt }}-*
path: downloaded-runner-protocol-evals
merge-multiple: false
- name: Materialize an empty download root when every cell failed early
run: mkdir -p downloaded-runner-protocol-evals
- name: Verify portable viewer
run: |
set -euo pipefail
cd runner-protocol-build
sha256sum --check runner-protocol-build.tar.gz.sha256
mkdir extracted
tar --extract --gzip --file runner-protocol-build.tar.gz --directory extracted
test -f extracted/dist-issue-thread/index.html
- name: Aggregate every expected cell, including missing infrastructure cells
env:
PAPERCLIP_PROTOCOL_EVAL_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
PAPERCLIP_PROTOCOL_EVAL_SOURCE_REF: ${{ needs.authorize.outputs.target_ref }}
PAPERCLIP_PROTOCOL_EVALS_SHA: ${{ needs.authorize.outputs.evals_sha }}
PAPERCLIP_PROTOCOL_EVAL_WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
node packages/paperclip-runner/scripts/runner-protocol-eval-campaign.mjs aggregate \
--catalog runner-protocol-catalog/runner-protocol-eval-catalog.json \
--downloads downloaded-runner-protocol-evals \
--evals-root .paperclip-evals \
--runs-out runner-protocol-merged/runs \
--campaign-out runner-protocol-merged/campaign.json
- name: Render the access-controlled canonical Evalbook report
run: |
python3 .paperclip-evals/evals/paperclip-runner/tools/eval_program.py report \
--runs-root runner-protocol-merged/runs \
--output runner-protocol-merged/report \
--viewer-root runner-protocol-build/extracted/dist-issue-thread \
--inventory .paperclip-evals/evals/paperclip-runner/inventory.json \
--coverage-matrix .paperclip-evals/evals/paperclip-runner/coverage-matrix.json
cp runner-protocol-merged/campaign.json runner-protocol-merged/report/campaign.json
- name: Render the same canonical grid from a public-safe evidence projection
run: |
node packages/paperclip-runner/scripts/runner-protocol-eval-campaign.mjs sanitize \
--runs-root runner-protocol-merged/runs \
--output runner-protocol-merged/public-runs
python3 .paperclip-evals/evals/paperclip-runner/tools/eval_program.py report \
--runs-root runner-protocol-merged/public-runs \
--output runner-protocol-merged/public-report \
--viewer-root runner-protocol-build/extracted/dist-issue-thread \
--public-viewer \
--inventory .paperclip-evals/evals/paperclip-runner/inventory.json \
--coverage-matrix .paperclip-evals/evals/paperclip-runner/coverage-matrix.json
cp runner-protocol-merged/campaign.json runner-protocol-merged/public-report/campaign.json
- name: Set up report browser verification
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- name: Verify the actual chat viewer before publication
run: |
pnpm install --frozen-lockfile --ignore-scripts
pnpm --filter @paperclipai/paperclip-runner exec playwright install --with-deps chromium
node packages/paperclip-runner/scripts/verify-runner-evalbook-viewer.mjs --report-root runner-protocol-merged/public-report --screenshots runner-protocol-merged/viewer-proof
node packages/paperclip-runner/scripts/verify-runner-evalbook-viewer.mjs --report-root runner-protocol-merged/report
- name: Enforce the static public allowlist
id: public_report
run: |
node --input-type=module -e 'import { validatePublicProtocolEvalReport } from "./packages/paperclip-runner/scripts/publish-runner-protocol-eval-history.mjs"; await validatePublicProtocolEvalReport("runner-protocol-merged/public-report", { viewerRoot: "runner-protocol-build/extracted/dist-issue-thread" });'
echo "ready=true" >> "$GITHUB_OUTPUT"
- name: Add campaign result to the workflow summary
run: |
{
echo '## Runner direct live protocol evals'
echo
jq -r '"- Cells: \(.totals.passed)/\(.totals.selected) passed\n- Behavior failures: \(.totals.behaviorFailures)\n- Infrastructure failures: \(.totals.infrastructureFailures)\n- Paperclip: `\(.source.paperclip.sha)`\n- Evals: `\(.source.evals.sha)`"' runner-protocol-merged/campaign.json
} >> "$GITHUB_STEP_SUMMARY"
- name: Upload access-controlled canonical Evalbook and raw attempts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-eval-report-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-merged/
retention-days: 30
if-no-files-found: error
- name: Upload publisher-only sanitized Evalbook
if: steps.public_report.outputs.ready == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runner-protocol-eval-public-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-merged/public-report/
retention-days: 1
if-no-files-found: error
- name: Enforce complete green campaign
if: always()
run: jq -e '.complete == true and .allPassed == true' runner-protocol-merged/campaign.json >/dev/null
publish_history:
name: Publish immutable Evalbook and mutable campaign index
needs: [authorize, catalog, report]
if: always() && needs.report.outputs.public_report_ready == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: runner-protocol-eval-history-publish
cancel-in-progress: false
permissions:
contents: read
id-token: write
environment:
name: runner-e2e-history
url: ${{ steps.publish.outputs.report_url }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
# AWS credentials can execute only the publisher from the trusted workflow revision.
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Download only the sanitized canonical Evalbook
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runner-protocol-eval-public-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-public-report
- name: Download the same-run canonical viewer for byte verification
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runner-protocol-viewer-${{ github.run_id }}-${{ github.run_attempt }}
path: runner-protocol-trusted-viewer
- name: Exchange GitHub OIDC identity for scoped AWS credentials
uses: aws-actions/configure-aws-credentials@e6de054238d6b7531b4efff3b6587d9aade6a06c # v6
with:
role-to-assume: ${{ vars.RUNNER_PROTOCOL_EVAL_HISTORY_AWS_ROLE_ARN || vars.RUNNER_E2E_HISTORY_AWS_ROLE_ARN }}
aws-region: ${{ vars.RUNNER_PROTOCOL_EVAL_HISTORY_AWS_REGION || vars.RUNNER_E2E_HISTORY_AWS_REGION }}
- name: Publish versioned report and refresh the root index
id: publish
env:
PAPERCLIP_RUNNER_PROTOCOL_EVAL_PUBLIC_REPORT_DIR: ${{ github.workspace }}/runner-protocol-public-report
PAPERCLIP_RUNNER_PROTOCOL_EVAL_VIEWER_DIR: ${{ github.workspace }}/runner-protocol-trusted-viewer
RUNNER_PROTOCOL_EVAL_HISTORY_S3_BUCKET: ${{ vars.RUNNER_PROTOCOL_EVAL_HISTORY_S3_BUCKET || vars.RUNNER_E2E_HISTORY_S3_BUCKET }}
RUNNER_PROTOCOL_EVAL_HISTORY_PREFIX: ${{ vars.RUNNER_PROTOCOL_EVAL_HISTORY_PREFIX || 'runner-protocol-evals' }}
RUNNER_PROTOCOL_EVAL_HISTORY_PUBLIC_BASE_URL: ${{ vars.RUNNER_PROTOCOL_EVAL_HISTORY_PUBLIC_BASE_URL || vars.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL }}
run: node packages/paperclip-runner/scripts/publish-runner-protocol-eval-history.mjs

169
.github/workflows/storybook-deploy.yml vendored Normal file
View File

@ -0,0 +1,169 @@
name: Storybook Deploy
on:
workflow_dispatch:
inputs:
branch:
description: "Repository branch to publish (empty uses the selected workflow branch)"
type: string
default: ""
# Also exposed by Storybook Visual, which is already available on master.
workflow_call:
inputs:
branch:
type: string
default: ""
permissions:
contents: read
jobs:
authorize:
name: Authorize Storybook publisher
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
sha: ${{ steps.source.outputs.sha }}
branch: ${{ steps.source.outputs.branch }}
branch_key: ${{ steps.source.outputs.branch_key }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- name: Require CODEOWNER initiator and rerunner
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const authorize = require('./.github/scripts/authorize-storybook-deploy.cjs');
await authorize({ github, context });
- name: Pin requested repository branch
id: source
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
env:
SOURCE_BRANCH: ${{ inputs.branch }}
STORYBOOK_S3_BUCKET: ${{ vars.STORYBOOK_S3_BUCKET }}
STORYBOOK_PUBLIC_BASE_URL: ${{ vars.STORYBOOK_PUBLIC_BASE_URL }}
with:
script: |
const branch = process.env.SOURCE_BRANCH || context.ref.slice('refs/heads/'.length);
const { data } = await github.rest.git.getRef({ ...context.repo, ref: `heads/${branch}` });
if (data.ref !== `refs/heads/${branch}` || data.object.type !== 'commit') {
throw new Error('Select an existing branch in this repository.');
}
// With no source override, preserve the exact dispatched commit.
const sha = process.env.SOURCE_BRANCH ? data.object.sha : context.sha;
const { storybookDestination } = require('./.github/scripts/storybook-destination.cjs');
const destination = storybookDestination({ branch, sha,
runId: context.runId, runAttempt: process.env.GITHUB_RUN_ATTEMPT,
bucket: process.env.STORYBOOK_S3_BUCKET, baseUrl: process.env.STORYBOOK_PUBLIC_BASE_URL });
core.setOutput('sha', sha);
core.setOutput('branch_key', destination.branchKey);
core.setOutput('branch', branch);
build:
name: Build selected branch Storybook
permissions: {}
needs: authorize
runs-on: ubuntu-latest
timeout-minutes: 25
outputs:
artifact_name: ${{ steps.artifact.outputs.name }}
env:
STORYBOOK_DISABLE_TELEMETRY: "1"
steps:
- name: Download public source without repository credentials
env:
SOURCE_SHA: ${{ needs.authorize.outputs.sha }}
run: |
[[ "$SOURCE_SHA" =~ ^[a-f0-9]{40}$ ]]
curl --fail --silent --show-error --location --retry 3 \
"https://codeload.github.com/paperclipai/paperclip/tar.gz/$SOURCE_SHA" \
--output "$RUNNER_TEMP/source.tar.gz"
tar -xzf "$RUNNER_TEMP/source.tar.gz" --strip-components=1
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with:
version: 9.15.4
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
package-manager-cache: false
- run: pnpm install --frozen-lockfile --ignore-scripts
- run: pnpm build-storybook
- name: Record source and validate output
id: artifact
env:
SOURCE_SHA: ${{ needs.authorize.outputs.sha }}
SOURCE_BRANCH: ${{ needs.authorize.outputs.branch }}
run: |
test -s ui/storybook-static/index.html
test -s ui/storybook-static/iframe.html
test -s ui/storybook-static/index.json
jq -n --arg sha "$SOURCE_SHA" --arg branch "$SOURCE_BRANCH" \
'{sha: $sha, branch: $branch}' > ui/storybook-static/deployment.json
echo "name=storybook-deploy-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" >> "$GITHUB_OUTPUT"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: ${{ steps.artifact.outputs.name }}
path: ui/storybook-static
if-no-files-found: error
retention-days: 7
deploy:
name: Publish branch Storybook to S3
needs: [authorize, build]
runs-on: ubuntu-latest
timeout-minutes: 15
concurrency:
group: storybook-deploy-${{ needs.authorize.outputs.branch_key }}
cancel-in-progress: false
permissions:
contents: read
id-token: write
environment:
name: storybook-deploy
url: ${{ steps.deployment.outputs.url }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
sparse-checkout: .github/scripts
- name: Recheck CODEOWNER access before publishing
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const authorize = require('./.github/scripts/authorize-storybook-deploy.cjs');
await authorize({ github, context });
- name: Download the successful build artifact
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: ${{ needs.build.outputs.artifact_name }}
path: storybook-static
- name: Assume the Storybook-only uploader role
uses: aws-actions/configure-aws-credentials@e6de054238d6b7531b4efff3b6587d9aade6a06c # v6
with:
role-to-assume: ${{ vars.STORYBOOK_AWS_ROLE_ARN }}
aws-region: ${{ vars.STORYBOOK_AWS_REGION }}
role-duration-seconds: 900
- name: Publish this branch preview
id: deployment
env:
SOURCE_SHA: ${{ needs.authorize.outputs.sha }}
SOURCE_BRANCH: ${{ needs.authorize.outputs.branch }}
STORYBOOK_S3_BUCKET: ${{ vars.STORYBOOK_S3_BUCKET }}
STORYBOOK_PUBLIC_BASE_URL: ${{ vars.STORYBOOK_PUBLIC_BASE_URL }}
run: node .github/scripts/publish-storybook.cjs
- name: Verify public build and stable branch URL
env:
BUILD_URL: ${{ steps.deployment.outputs.build_url }}
BRANCH_URL: ${{ steps.deployment.outputs.url }}
SOURCE_SHA: ${{ needs.authorize.outputs.sha }}
run: node .github/scripts/verify-storybook.cjs
- name: Upload deployment links
if: ${{ !cancelled() && steps.deployment.outcome == 'success' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: storybook-deployment-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ steps.deployment.outputs.report_path }}
if-no-files-found: error
retention-days: 30

24
.github/workflows/storybook-visual.yml vendored
View File

@ -3,6 +3,16 @@ name: Storybook Visual
on:
workflow_dispatch:
inputs:
branch:
description: "Repository branch to publish (deployment only; empty uses the selected workflow branch)"
required: false
type: string
default: ""
deploy_preview:
description: "Publish this branch to S3/CloudFront instead of running visual tests (CODEOWNERS only)"
required: false
type: boolean
default: false
update_snapshots:
description: "Generate updated snapshots and a baseline review bundle"
required: false
@ -18,7 +28,7 @@ on:
- labeled
concurrency:
group: storybook-visual-${{ github.event.pull_request.number || github.ref }}
group: storybook-visual-${{ inputs.deploy_preview && github.run_id || 'visual' }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
@ -28,7 +38,7 @@ jobs:
visual:
name: Storybook visual regression
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'workflow_dispatch' && !inputs.deploy_preview) ||
(github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'storybook-visual'))
runs-on: ubuntu-latest
timeout-minutes: 35
@ -100,3 +110,13 @@ jobs:
path: tests/storybook-visual/baseline-review/snapshots.tgz
retention-days: 30
if-no-files-found: error
preview:
name: Deploy selected branch Storybook
if: github.event_name == 'workflow_dispatch' && inputs.deploy_preview
permissions:
contents: read
id-token: write
uses: ./.github/workflows/storybook-deploy.yml
with:
branch: ${{ inputs.branch }}

8
.gitignore vendored
View File

@ -3,6 +3,9 @@ node_modules/
**/node_modules
**/node_modules/
dist/
dist-preview/
dist-flow-preview/
packages/paperclip-runner/runner/target/
ui/storybook-static/
.env
*.tsbuildinfo
@ -58,6 +61,8 @@ diagnostics/
# Playwright
tests/e2e/test-results/
tests/e2e/playwright-report/
tests/runner-e2e/results/
.env.runner-e2e.local
tests/release-smoke/test-results/
tests/release-smoke/playwright-report/
test-results/issue-detail-perf/
@ -70,3 +75,6 @@ tests/storybook-visual/playwright-report/
.superpowers/
.claude/worktrees/
.herenow
.vercel/
.env.local
test-results/

2
.npmrc
View File

@ -1 +1 @@
auto-install-peers=true
auto-install-peers=false

32
AGENTS.md
View File

@ -20,6 +20,12 @@ Before making changes, read in this order:
`doc/SPEC.md` is long-horizon product context.
`doc/SPEC-implementation.md` is the concrete V1 build contract.
When adding or changing an Apps catalog connection, also follow
`doc/connections/CONNECTOR-PLAYBOOK.md`. It is the canonical connection
authoring runbook for provider research, supported transport/auth patterns,
credential handling, branding, implementation, testing, live proof, and PR
submission.
## 3. Repo Map
- `server/`: Express REST API and orchestration services
@ -91,6 +97,32 @@ When you are creating a plan file in the repository itself, new plan documents b
6. Attach inspectable generated artifacts.
When your task produces a user-inspectable deliverable file, follow the Paperclip skill's "Generated Artifacts and Work Products" workflow before final disposition. In this repo, prefer the self-contained skill helper at `skills/paperclip/scripts/paperclip-upload-artifact.sh` so the file is available through the Paperclip API, create/update an artifact work product when the file is the deliverable, link the uploaded artifact in the final issue comment, and then set status. Do not rely on local filesystem paths as the only access path. If an important file intentionally remains workspace-only, create/update a work product with `metadata.resourceRef.kind: "workspace_file"` and a workspace-relative path, then name that work product and path in the final comment. Treat browse/search as a fallback for recovering workspace files, not the preferred deliverable path. See `doc/AGENT-ARTIFACTS.md` for details and `.mp4`/`.webm` examples.
7. Name the three data paths correctly.
This repo has three separate data paths. Do not confuse them. Match a change to a path by its file path, not by the word "observability" or "telemetry" alone.
- **Telemetry** is the Paperclip first-party event system. It is opt-out and it sends data to a Paperclip endpoint by default. Its paths are:
- `packages/shared/src/telemetry/`
- the generated contract `packages/shared/src/telemetry/generated/paperclip-telemetry.ts`
- each caller of `packages/shared/src/telemetry/events.ts` or `packages/shared/src/telemetry/client.ts`
- **Observability** is the OpenTelemetry trace path. An operator must set an OTLP endpoint. Until an operator sets the endpoint, the tracer is a no-operation. Its paths are:
- `server/src/instrumentation.ts`
- `doc/observability.md`
- `packages/adapter-utils/src/duplex-observability.ts`
- `server/src/services/duplex-observability-recorder.ts`
- the span attributes in `packages/adapter-utils/src/acpx-engine/startup-timing.ts`
- **The run log** holds rows in the local `heartbeat_run_events` table. The data stays in the instance database. Its paths are:
- `doc/run-log-events.md`
- `packages/db/src/schema/heartbeat_run_events.ts`
- the append path `appendRunEvent` in `server/src/services/heartbeat.ts`
Apply a review level that matches the path:
- **Telemetry change (strict review).** The author updates the generated contract first. The author updates `packages/shared/src/telemetry/README.md` in the same pull request. The author requests a privacy review. Reason: a Telemetry event goes to a Paperclip endpoint by default, so a mistake sends data immediately.
- **Observability change (lighter review).** The operator endpoint gate stays in place. The no-operation behaviour stays when no endpoint is set. A privacy review is not necessary while the change stays inside the closed span-attribute allowlist.
- **Run-log change (no extra review).** A run-log change needs neither review level above, because the data stays in the instance database.
**Exclusion.** The word "observability" in a file such as `server/src/services/recovery-observability.ts` names a different concept. Apply this rule by path, not by word match.
## 6. Database Change Workflow
When changing data model:

4
CONTRIBUTING.md
View File

@ -104,7 +104,9 @@ All tests must pass before a PR can be merged. Run them locally first and verify
### Telemetry Changes
If your change adds, removes, or modifies emitted telemetry events, update the [Telemetry Data Contract](packages/shared/src/telemetry/README.md) in the same PR. Keep clients emitting raw dimension values and avoid documenting or relying on private delivery details.
This repo has three separate data paths: Telemetry, Observability, and the run log. See rule 7 in `AGENTS.md` for the full definitions and the review level each path needs.
If your change adds, removes, or modifies emitted telemetry events, update the [Telemetry Data Contract](packages/shared/src/telemetry/README.md) in the same PR. Keep clients emitting raw dimension values and avoid documenting or relying on private delivery details. If your change adds, removes, or modifies an OpenTelemetry span or span attribute, keep the change inside the closed span-attribute allowlist in `packages/adapter-utils/src/acpx-engine/startup-timing.ts`. If your change adds or modifies a run-log event, update `doc/run-log-events.md` in the same PR.
### Paperclip Gates Must Pass

10
DESIGN.md
View File

@ -35,6 +35,16 @@ Existing tiers already in index.css (~80+ tokens) — extraction maps to these o
7. **Words are part of the system.** One name per concept across the entire UI — the canonical term is *task* (never *issue* or *ticket* in copy, labels, or empty states). Buttons name the action ("Approve hire," not "Submit"). Errors say what happened and what to do. Empty states say what to do first. **Note:** enforcing the task rename is a visible change and is explicitly OUT of the zero-visual-change extraction run; it happens in its own follow-up run.
8. **Agent-modifiable by design.** The system must be changeable via instructions: single token source, lint rules that enforce it, and this document kept current. A correct change should be expressible as "edit tokens + run checks," not "visit 40 files."
## Contextual feedback
Do not show a toast for task or run state already visible on the current screen.
This includes descendant runs represented by the open subtree. Show local action
results in place; keep failures actionable inline. Notifications for other work
remain useful. Expected cancellation is neutral gray, not an error. The composer's Stop action stops the current response and leaves the composer available for a new message. Pause work is a separate explicit task or subtree action. A paused task replaces the composer with an amber takeover. It says “Task is
paused.” and “Resume this task to send a message.” with a “Resume task” action.
Subtrees use “Subtree is paused.” and “Resume subtree.” The takeover cannot be
dismissed, retains drafts, and hides message inputs until the pause is released.
## Enforcement (what "compliant" means for the extraction run)
- **Zero visual change is proven, not promised:** Storybook visual snapshots are baselined before any refactor, and all snapshots match baseline after it. A change that alters rendered output must be intentional and human-approved.

165
Dockerfile
View File

@ -3,7 +3,7 @@ FROM node:24-trixie-slim AS base
ARG USER_UID=1000
ARG USER_GID=1000
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates gosu curl gh git wget ripgrep python3 \
&& apt-get install -y --no-install-recommends ca-certificates gosu curl gh git wget ripgrep python3 tini \
&& rm -rf /var/lib/apt/lists/* \
&& corepack enable
@ -24,6 +24,7 @@ COPY packages/adapter-utils/package.json packages/adapter-utils/
COPY packages/google-sheets-mcp-server/package.json packages/google-sheets-mcp-server/
COPY packages/kv-demo-mcp-server/package.json packages/kv-demo-mcp-server/
COPY packages/mcp-server/package.json packages/mcp-server/
COPY packages/paperclip-eval-kernel/package.json packages/paperclip-eval-kernel/
COPY packages/paperclip-runner/package.json packages/paperclip-runner/
COPY packages/skills-catalog/package.json packages/skills-catalog/
COPY packages/tailscale-https-broker/package.json packages/tailscale-https-broker/
@ -50,10 +51,83 @@ COPY scripts/link-plugin-dev-sdk.mjs scripts/
RUN pnpm install --frozen-lockfile
FROM base AS build
FROM base AS rust-toolchain
WORKDIR /app
# Debian's packaged rust lags the ecosystem (trixie ships 1.85) and the
# runner's dependency tree now requires a newer rustc. Install rustup from a
# version-pinned, checksum-verified installer and let the runner's own
# rust-toolchain.toml choose the compiler — one pin, owned by the runner
# package, shared by CI and image builds alike.
#
# The C toolchain is explicit: apt's cargo used to pull gcc in as a
# dependency, and rustup does not — without it every build script dies on
# "linker `cc` not found".
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc libc6-dev pkg-config \
&& rm -rf /var/lib/apt/lists/*
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH
ARG RUSTUP_VERSION=1.29.0
ARG RUSTUP_SHA256_AMD64=4acc9acc76d5079515b46346a485974457b5a79893cfb01112423c89aeb5aa10
ARG RUSTUP_SHA256_ARM64=9732d6c5e2a098d3521fca8145d826ae0aaa067ef2385ead08e6feac88fa5792
RUN set -eux; \
arch="$(dpkg --print-architecture)"; \
case "$arch" in \
amd64) rustTarget="x86_64-unknown-linux-gnu"; sha256="$RUSTUP_SHA256_AMD64" ;; \
arm64) rustTarget="aarch64-unknown-linux-gnu"; sha256="$RUSTUP_SHA256_ARM64" ;; \
*) echo "unsupported architecture: $arch" >&2; exit 1 ;; \
esac; \
curl -fsSLo /tmp/rustup-init "https://static.rust-lang.org/rustup/archive/${RUSTUP_VERSION}/${rustTarget}/rustup-init"; \
echo "${sha256} /tmp/rustup-init" | sha256sum -c -; \
chmod +x /tmp/rustup-init; \
/tmp/rustup-init -y --no-modify-path --profile minimal --default-toolchain none; \
rm /tmp/rustup-init
# Install the package-owned compiler before any application source enters the
# stage. rustup-init above installs rustup itself, not the selected compiler.
COPY packages/paperclip-runner/rust-toolchain.toml /tmp/runner-toolchain/rust-toolchain.toml
RUN cd /tmp/runner-toolchain && rustup show
# Pin the recipe generator and its dependency lockfile. It is a build-only tool
# and uses the same package-owned compiler as both native build stages.
FROM rust-toolchain AS rust-chef
RUN cd /tmp/runner-toolchain && cargo install cargo-chef --version 0.1.73 --locked
FROM rust-chef AS runner-plan
WORKDIR /app/packages/paperclip-runner
COPY packages/paperclip-runner/rust-toolchain.toml ./
COPY packages/paperclip-runner/runner ./runner
RUN cd runner && cargo chef prepare --recipe-path /tmp/runner-recipe.json
FROM rust-chef AS runner-deps
WORKDIR /app/packages/paperclip-runner/runner
COPY packages/paperclip-runner/rust-toolchain.toml ../
# The recipe changes only when dependency manifests, the lockfile, or target
# metadata change. Source edits can reuse this compiled dependency layer.
COPY --from=runner-plan /tmp/runner-recipe.json /tmp/runner-recipe.json
RUN cargo chef cook --release --locked --package paperclip-runner-core --bin paperclip-runnerd --recipe-path /tmp/runner-recipe.json \
&& find . -mindepth 1 -maxdepth 1 ! -name target -exec rm -rf {} +
FROM runner-deps AS runner-build
WORKDIR /app/packages/paperclip-runner
# Rust embeds protocol schemas and fixtures with include_str!. Keep those
# alongside the complete Cargo workspace so every compile-time input keys
# this layer. Ordinary server/UI edits can then reuse the native build.
COPY packages/paperclip-runner/rust-toolchain.toml ./
COPY packages/paperclip-runner/runner ./runner
COPY packages/paperclip-runner/protocol ./protocol
# Cargo fingerprints source mtimes. Normalize them here and after the full
# source copy below so a fresh checkout cannot invalidate unchanged inputs.
RUN find runner protocol -type f -exec touch -d @0 {} + \
&& touch -d @0 rust-toolchain.toml \
&& cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd
FROM runner-build AS build
WORKDIR /app
COPY --from=deps /app /app
COPY . .
RUN find packages/paperclip-runner/runner packages/paperclip-runner/protocol -type f -exec touch -d @0 {} + \
&& touch -d @0 packages/paperclip-runner/rust-toolchain.toml
RUN pnpm --filter @paperclipai/ui build
RUN pnpm --filter @paperclipai/plugin-sdk build
# The server build runs scripts/write-build-stamp.mjs, which stamps the built
@ -64,20 +138,14 @@ RUN pnpm --filter @paperclipai/plugin-sdk build
# same ARG again for the runtime fallback; an ARG goes out of scope at the
# end of its stage. Empty for local `docker build`, which then writes no stamp.
ARG PAPERCLIP_BUILD_COMMIT=""
ENV NODE_OPTIONS=--max-old-space-size=4096
RUN pnpm --filter @paperclipai/server build
RUN test -f server/dist/index.js || (echo "ERROR: server build output missing" && exit 1)
RUN rm -rf packages/paperclip-runner/runner/target
FROM base AS production
ARG USER_UID=1000
ARG USER_GID=1000
# Real version for this build, computed from `git describe` on the CI runner
# (the image has no .git, so the server cannot derive it at runtime). Empty for
# local `docker build`, which just leaves the server on its normal fallbacks.
ARG PAPERCLIP_BUILD_VERSION=""
# The exact commit this image was built from, for the same reason: server-info
# falls back to PAPERCLIP_BUILD_COMMIT when git is unavailable, which feeds the
# /api/health `commit` field that deploy tooling verifies. Empty locally.
ARG PAPERCLIP_BUILD_COMMIT=""
# Refreshes the tool layer below when it changes (CI stamps an ISO week, so
# the @latest CLI tools advance weekly). Without it the cached layer would
# freeze the tools until an unrelated cache bust.
@ -100,6 +168,13 @@ RUN chmod +x /usr/local/bin/docker-entrypoint.sh
COPY --chown=node:node --from=build /app /app
# Declare per-build metadata after the stable RUN layers. Docker includes
# in-scope ARG values in a RUN's environment even when its command does not
# mention them; declaring these earlier invalidates the weekly tool cache.
# The build stage still receives the commit before writing dist/build-info.json.
# Empty for local builds, preserving the server's normal version fallbacks.
ARG PAPERCLIP_BUILD_VERSION=""
ARG PAPERCLIP_BUILD_COMMIT=""
ENV NODE_ENV=production \
HOME=/paperclip \
HOST=0.0.0.0 \
@ -119,7 +194,14 @@ ENV NODE_ENV=production \
EXPOSE 3100
ENTRYPOINT ["docker-entrypoint.sh"]
# tini, not node, is PID 1. The entrypoint ends in `exec`, so without an init
# node inherits PID 1 and never wait()s the orphans the kernel re-parents onto
# it -- agent runs spawn git/claude/esbuild/sh descendants that outlive their
# leader, so they pile up as permanent zombies (~79/h measured) until the
# cgroup pid limit is exhausted and *every* fork() in the container fails.
# tini reaps adopted orphans and forwards signals, so the exec chain below and
# graceful shutdown are unchanged. Mirrors docker/agent-runtime/Dockerfile.base.
ENTRYPOINT ["/usr/bin/tini", "--", "docker-entrypoint.sh"]
CMD ["node", "--import", "./server/node_modules/tsx/dist/loader.mjs", "server/dist/index.js"]
# Cloud image variant (build with `--target cloud`): the production image
@ -154,5 +236,66 @@ RUN set -eu; \
test -f "$dir/dist/manifest.js" || { echo "ERROR: $dir is missing dist/manifest.js after build" >&2; exit 1; }; \
done
# The hosted image variant ships selected optional peer packages
# pre-installed. A managed tenant then needs no separate install step.
# The self-hosted image stays on the opt-in contract: it never runs this
# stage, so a package like `@sentry/node` stays a true optional peer
# dependency. A self-hosted operator installs it by hand (see
# doc/observability.md).
#
# CLOUD_BUNDLED_SERVER_DEPS names the optional peer packages to install.
# The value is a space-separated list, the same shape as
# CLOUD_BUNDLED_PLUGINS above. The stage reads each package's version
# from the `peerDependencies` block of `server/package.json` at build
# time, so the version has one committed home.
#
# The stage fails the build in three cases:
# - the argument is empty
# - server/package.json declares no version for a named package
# - the named package is not an optional peer
#
# This check keeps the argument limited to packages the server already
# treats as optional.
#
# The install happens in its own isolated directory, not inside
# `server`'s own workspace install. The self-hosted target above never
# gains these packages this way. The directory sits under `/app`, not
# `server/`, and `--ignore-workspace` below excludes it from the pnpm
# workspace. From that directory, pnpm still finds the `packageManager`
# pin in the repo's own `package.json` by walking up — the same pnpm
# version the rest of the build uses.
#
# The install writes no lock file (`--no-lockfile`, the same flag the
# `cloud-plugins` stage above uses). Two builds of the same commit can
# therefore install different transitive versions of a named package.
# Three facts make this an accepted trade-off:
# - the `cloud-plugins` stage above already has the same property, with
# the same flag
# - the direct version of each named package comes from one exact,
# single-sourced place: the `peerDependencies` block of
# `server/package.json`
# - an automated check asserts the installed direct version after every
# build, so a transitive drift that breaks the package still fails the
# build
FROM build AS cloud-server-deps
WORKDIR /app/.cloud-server-deps
ARG CLOUD_BUNDLED_SERVER_DEPS="@sentry/node"
RUN set -eu; \
test -n "$CLOUD_BUNDLED_SERVER_DEPS" || { echo "ERROR: CLOUD_BUNDLED_SERVER_DEPS is empty; name at least one optional peer package to install" >&2; exit 1; }; \
echo '{"name":"paperclip-cloud-server-deps","private":true}' > package.json; \
specifiers=""; \
for name in $CLOUD_BUNDLED_SERVER_DEPS; do \
version="$(node -e "const pkg=require('/app/server/package.json'); const name=process.argv[1]; const version=(pkg.peerDependencies||{})[name]; if(!version){console.error('ERROR: server/package.json declares no peerDependencies version for '+JSON.stringify(name));process.exit(1);} const meta=(pkg.peerDependenciesMeta||{})[name]; if(!meta||meta.optional!==true){console.error('ERROR: '+JSON.stringify(name)+' is not declared as an optional peer dependency in server/package.json; CLOUD_BUNDLED_SERVER_DEPS may name only optional peer packages');process.exit(1);} process.stdout.write(version);" "$name")"; \
test -n "$version" || { echo "ERROR: could not resolve a version for '$name'" >&2; exit 1; }; \
specifiers="$specifiers ${name}@${version}"; \
done; \
test -n "$specifiers" || { echo "ERROR: CLOUD_BUNDLED_SERVER_DEPS names no package" >&2; exit 1; }; \
pnpm add --ignore-workspace --no-lockfile $specifiers
FROM production AS cloud
COPY --chown=node:node --from=cloud-plugins /app/packages/plugins/sandbox-providers /app/packages/plugins/sandbox-providers
# Land the isolated install inside the server's own `node_modules`, the
# directory Node's module resolution walks up to from `/app/server` for
# both a CommonJS `require.resolve` and an ECMAScript `import` — an entry
# on `NODE_PATH` would satisfy only the first and silently fail the second.
COPY --chown=node:node --from=cloud-server-deps /app/.cloud-server-deps/node_modules /app/server/node_modules

33
README.md
View File

@ -67,7 +67,7 @@ It looks like a task manager. Under the hood: org charts, budgets, governance, g
## Paperclip is right for you if
- ✅ You want to build **autonomous AI companies**
- ✅ You want to build **autonomous AI organizations**
- ✅ You **coordinate many different agents** (OpenClaw, Codex, Claude, Cursor) toward a common goal
- ✅ You have **20 simultaneous Claude Code terminals** open and lose track of what everyone is doing
- ✅ You want agents running **autonomously 24/7**, but still want to audit work and chime in when needed
@ -106,7 +106,7 @@ Any agent, any runtime, one org chart. If it can receive a heartbeat, it's hired
</td>
<td align="center" width="33%">
<h3>🎯 Goal Alignment</h3>
Every task traces back to the company mission. Agents know <em>what</em> to do and <em>why</em>.
Every task traces back to the organization mission. Agents know <em>what</em> to do and <em>why</em>.
</td>
<td align="center" width="33%">
<h3>💓 Heartbeats</h3>
@ -119,8 +119,8 @@ Agents wake on a schedule, check work, and act. Delegation flows up and down the
Monthly budgets per agent. When they hit the limit, they stop. No runaway costs.
</td>
<td align="center">
<h3>🏢 Multi-Company</h3>
One deployment, many companies. Complete data isolation. One control plane for your portfolio.
<h3>🏢 Multi-Organization</h3>
One deployment, many organizations. Complete data isolation. One control plane for your portfolio.
</td>
<td align="center">
<h3>🎫 Ticket System</h3>
@ -170,7 +170,7 @@ Paperclip handles the hard orchestration details correctly.
| **Governance with rollback.** | Approval gates are enforced, config changes are revisioned, and bad changes can be rolled back safely. |
| **Goal-aware execution.** | Tasks carry full goal ancestry so agents consistently see the "why," not just a title. |
| **Portable company templates.** | Export/import orgs, agents, and skills with secret scrubbing and collision handling. |
| **True multi-company isolation.** | Every entity is company-scoped, so one deployment can run many companies with separate data and audit trails. |
| **True multi-organization isolation.** | Every entity is company-scoped, so one deployment can run many companies with separate data and audit trails. |
<br/>
@ -336,6 +336,25 @@ To try Paperclip without installing anything permanently:
npx --registry https://registry.npmjs.org paperclipai onboard --yes
```
For an isolated manual test instance that is already initialized with a CEO
agent, use `test-drive`. It stays in the foreground, never installs a service
or creates a first task, and opens the browser only after setup succeeds:
```bash
ANTHROPIC_API_KEY=... npx paperclipai test-drive
OPENAI_API_KEY=... npx paperclipai test-drive --harness codex
OPENROUTER_API_KEY=... npx paperclipai test-drive \
--harness opencode \
--model openrouter/anthropic/claude-sonnet-4.5
```
Each run without `--data-dir` gets a unique, retained temporary directory; its
absolute path is printed at startup. Pass `--data-dir` to reuse one, or
`--no-browser` to leave the initialized instance unopened. When invoked from a
linked Git worktree, `test-drive` also enables task execution in that worktree.
See [`doc/CLI.md`](doc/CLI.md#isolated-manual-test-drives) for credential and
reuse behavior.
> **Troubleshooting: private npm registry `.npmrc`**
>
> If this fails with an `E404` for `paperclipai` (or similar) and you use a private npm registry (for example GitHub Packages) via a global `~/.npmrc`, `npx` may be resolving `paperclipai` against that private registry instead of the public npm registry.
@ -467,7 +486,9 @@ Find Plugins and more at [awesome-paperclip](https://github.com/gsxdsm/awesome-p
## Observability
Paperclip ships with opt-in OpenTelemetry auto-instrumentation for the server (traces only). It activates when `OTEL_EXPORTER_OTLP_ENDPOINT` is set and supports `grpc`, `http/protobuf`, and `http/json` via the standard `OTEL_EXPORTER_OTLP_PROTOCOL` env var. The `@opentelemetry/*` packages are optional peer dependencies — install them only if you want tracing. See [doc/observability.md](doc/observability.md) for install commands and the full env-var reference.
Paperclip ships with opt-in OpenTelemetry auto-instrumentation for the server (traces only). It activates when `OTEL_EXPORTER_OTLP_ENDPOINT` is set and supports `grpc`, `http/protobuf`, and `http/json` via the standard `OTEL_EXPORTER_OTLP_PROTOCOL` env var. `@opentelemetry/api` is a normal server dependency; the SDK, auto-instrumentation, and exporter packages are optional peer dependencies — install them only if you want tracing. See [doc/observability.md](doc/observability.md) for install commands and the full env-var reference.
Paperclip also ships with opt-in Sentry error monitoring for the server and the browser. Set `SENTRY_DSN_FRONTEND` to activate it for the browser and `SENTRY_DSN_BACKEND` to activate it for the server — each variable is optional, and the legacy `SENTRY_DSN` variable still works as a fallback for either component. The supported server SDK version is `@sentry/node@10.71.0`; it is an optional peer dependency for the server, so install it only if you want error monitoring. The browser SDK, `@sentry/browser`, is pinned to the same exact version. See [doc/observability.md](doc/observability.md#sentry-error-monitoring) for the install command, the privacy settings, and the full default capture set.
## Telemetry

4
cli/package.json
View File

@ -54,14 +54,14 @@
"@paperclipai/hermes-paperclip-adapter": "workspace:*",
"drizzle-orm": "0.45.2",
"dotenv": "^17.4.2",
"commander": "^13.1.0",
"commander": "^15.0.0",
"embedded-postgres": "^18.1.0-beta.16",
"picocolors": "^1.1.1"
},
"devDependencies": {
"@types/node": "^24.0.0",
"tsx": "^4.23.12",
"typescript": "^5.7.3"
"typescript": "^7.0.2"
},
"engines": {
"node": ">=24.11.0"

2
cli/src/__tests__/admin-asset-skill-parity.test.ts
View File

@ -90,7 +90,6 @@ describe("admin, asset, and skill parity commands", () => {
await run(["adapter", "config-schema", "codex_local"]);
await run(["adapter", "ui-parser", "codex_local"]);
await run(["adapter", "models", "codex_local", "--company-id", COMPANY_ID, "--refresh", "--environment-id", "env-1"]);
await run(["adapter", "model-profiles", "codex_local", "--company-id", COMPANY_ID]);
await run(["adapter", "detect-model", "codex_local", "--company-id", COMPANY_ID]);
await run(["adapter", "test-environment", "codex_local", "--company-id", COMPANY_ID, "--payload-json", "{}"]);
await run(["adapter", "delete", "codex_local"]);
@ -107,7 +106,6 @@ describe("admin, asset, and skill parity commands", () => {
["GET", "http://localhost:3100/api/adapters/codex_local/config-schema"],
["GET", "http://localhost:3100/api/adapters/codex_local/ui-parser.js"],
["GET", `http://localhost:3100/api/companies/${COMPANY_ID}/adapters/codex_local/models?refresh=true&environmentId=env-1`],
["GET", `http://localhost:3100/api/companies/${COMPANY_ID}/adapters/codex_local/model-profiles`],
["GET", `http://localhost:3100/api/companies/${COMPANY_ID}/adapters/codex_local/detect-model`],
["POST", `http://localhost:3100/api/companies/${COMPANY_ID}/adapters/codex_local/test-environment`],
["DELETE", "http://localhost:3100/api/adapters/codex_local"],

13
cli/src/__tests__/agent-jwt-env.test.ts
View File

@ -4,6 +4,7 @@ import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import {
ensureAgentJwtSecret,
ensureToolActionSigningSecret,
mergePaperclipEnvEntries,
readAgentJwtSecretFromEnv,
readPaperclipEnvEntries,
@ -24,6 +25,7 @@ describe("agent jwt env helpers", () => {
beforeEach(() => {
process.env = { ...ORIGINAL_ENV };
delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
delete process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET;
});
afterEach(() => {
@ -42,6 +44,17 @@ describe("agent jwt env helpers", () => {
expect(contents).toContain("PAPERCLIP_AGENT_JWT_SECRET=");
});
it("creates an independent tool-action signing secret next to the config", () => {
const configPath = tempConfigPath();
const result = ensureToolActionSigningSecret(configPath);
expect(result.created).toBe(true);
expect(result.secret).toHaveLength(64);
const entries = readPaperclipEnvEntries(resolveAgentJwtEnvFile(configPath));
expect(entries.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET).toBe(result.secret);
expect(entries.PAPERCLIP_AGENT_JWT_SECRET).toBeUndefined();
});
it("loads secret from .env next to explicit config path", () => {
const configPath = tempConfigPath();
const envPath = resolveAgentJwtEnvFile(configPath);

2
cli/src/__tests__/company-delete.test.ts
View File

@ -14,13 +14,11 @@ function makeCompany(overrides: Partial<Company>): Company {
issueCounter: 1,
budgetMonthlyCents: 0,
spentMonthlyCents: 0,
attachmentMaxBytes: 10 * 1024 * 1024,
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,
feedbackDataSharingConsentAt: null,
feedbackDataSharingConsentByUserId: null,
feedbackDataSharingTermsVersion: null,
brandColor: null,
logoAssetId: null,
logoUrl: null,
defaultResponsibleUserId: null,

18
cli/src/__tests__/company-import-transfer.test.ts
View File

@ -320,6 +320,24 @@ describe("uploadCompanyImportTransfer", () => {
await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow(/already imported/);
expect(putRaw).not.toHaveBeenCalled();
});
it("names the company the completed transfer created", async () => {
const { api, putRaw } = fakeApi({
post: vi.fn().mockResolvedValue({
transferId: "transfer-1",
status: "completed",
alreadyCompleted: true,
totalParts: 2,
missingParts: [],
company: { id: "company-2", name: "Paperclip", issuePrefix: "PAPA" },
}),
});
await expect(uploadCompanyImportTransfer(api, zipBytes)).rejects.toThrow(
/landed in the company "Paperclip" \(PAPA\)/,
);
expect(putRaw).not.toHaveBeenCalled();
});
});
describe("company import command over the chunked transfer path", () => {

8
cli/src/__tests__/company.test.ts
View File

@ -49,13 +49,11 @@ function company(overrides: Record<string, unknown> = {}) {
issueCounter: 1,
budgetMonthlyCents: 0,
spentMonthlyCents: 0,
attachmentMaxBytes: 1073741824,
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,
feedbackDataSharingConsentAt: null,
feedbackDataSharingConsentByUserId: null,
feedbackDataSharingTermsVersion: null,
brandColor: "#5c5fff",
logoAssetId: null,
createdAt: "2026-06-04T00:00:00.000Z",
updatedAt: "2026-06-04T00:00:00.000Z",
@ -347,8 +345,6 @@ describe("renderCompanyImportPreview", () => {
path: "COMPANY.md",
name: "Source Co",
description: null,
attachmentMaxBytes: null,
brandColor: null,
logoPath: null,
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,
@ -584,8 +580,6 @@ describe("import selection catalog", () => {
path: "COMPANY.md",
name: "Source Co",
description: null,
attachmentMaxBytes: null,
brandColor: null,
logoPath: "images/company-logo.png",
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,
@ -761,8 +755,6 @@ describe("import selection catalog", () => {
path: "COMPANY.md",
name: "Source Co",
description: null,
attachmentMaxBytes: null,
brandColor: null,
logoPath: null,
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,

18
cli/src/__tests__/doctor.test.ts
View File

@ -1,4 +1,5 @@
import fs from "node:fs";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
@ -8,7 +9,18 @@ import type { PaperclipConfig } from "../config/schema.js";
const ORIGINAL_ENV = { ...process.env };
function createTempConfig(): string {
async function availablePort(): Promise<number> {
const server = net.createServer();
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", resolve);
});
const address = server.address() as net.AddressInfo;
await new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
return address.port;
}
function createTempConfig(serverPort: number): string {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-doctor-"));
const configPath = path.join(root, ".paperclip", "config.json");
const runtimeRoot = path.join(root, "runtime");
@ -38,7 +50,7 @@ function createTempConfig(): string {
deploymentMode: "local_trusted",
exposure: "private",
host: "127.0.0.1",
port: 3199,
port: serverPort,
allowedHostnames: [],
serveUi: true,
},
@ -87,7 +99,7 @@ describe("doctor", () => {
});
it("re-runs repairable checks so repaired failures do not remain blocking", async () => {
const configPath = createTempConfig();
const configPath = createTempConfig(await availablePort());
const summary = await doctor({
config: configPath,

43
cli/src/__tests__/git-workspace.test.ts Normal file
View File

@ -0,0 +1,43 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { execFileSync } from "node:child_process";
import { afterEach, describe, expect, it } from "vitest";
import { detectGitWorkspaceInfo, isLinkedGitWorktree } from "../commands/git-workspace.js";
const cleanupDirectories: string[] = [];
afterEach(() => {
while (cleanupDirectories.length > 0) {
fs.rmSync(cleanupDirectories.pop()!, { recursive: true, force: true });
}
});
describe("Git worktree detection", () => {
it("distinguishes a linked worktree from the primary checkout and non-Git paths", () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-git-workspace-"));
cleanupDirectories.push(root);
const primary = path.join(root, "primary");
const linked = path.join(root, "linked");
fs.mkdirSync(primary);
execFileSync("git", ["init"], { cwd: primary, stdio: "ignore" });
execFileSync("git", ["config", "user.email", "test@example.com"], { cwd: primary });
execFileSync("git", ["config", "user.name", "Paperclip Test"], { cwd: primary });
fs.writeFileSync(path.join(primary, "README.md"), "test\n");
execFileSync("git", ["add", "README.md"], { cwd: primary });
execFileSync("git", ["commit", "-m", "initial"], { cwd: primary, stdio: "ignore" });
execFileSync("git", ["worktree", "add", "-b", "linked-test", linked], {
cwd: primary,
stdio: "ignore",
});
const primaryInfo = detectGitWorkspaceInfo(primary);
const linkedInfo = detectGitWorkspaceInfo(linked);
expect(primaryInfo?.gitDir).toBe(primaryInfo?.commonDir);
expect(linkedInfo?.gitDir).not.toBe(linkedInfo?.commonDir);
expect(isLinkedGitWorktree(primary)).toBe(false);
expect(isLinkedGitWorktree(linked)).toBe(true);
expect(detectGitWorkspaceInfo(root)).toBeNull();
expect(isLinkedGitWorktree(root)).toBe(false);
});
});

1
cli/src/__tests__/helpers/embedded-postgres.ts
View File

@ -1,6 +1,7 @@
export {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
EMBEDDED_POSTGRES_TEST_TIMEOUT_MS,
type EmbeddedPostgresTestDatabase,
type EmbeddedPostgresTestSupport,
} from "@paperclipai/db";

24
cli/src/__tests__/install-store.test.ts
View File

@ -1,4 +1,5 @@
import fs from "node:fs";
import { execFileSync } from "node:child_process";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
@ -121,6 +122,29 @@ describe("managed install store", () => {
expect(fs.statSync(rcPath).mode & 0o777).toBe(0o640);
});
it("uses the pinned Node for child tools even with an older node first on the service PATH", () => {
const entrypoint = path.join(paths.currentPath, "node_modules", "paperclipai", "dist", "index.js");
fs.mkdirSync(path.dirname(entrypoint), { recursive: true });
fs.writeFileSync(entrypoint, `console.log(require("node:child_process").execFileSync("node", ["-p", "process.execPath"], {encoding: "utf8"}).trim())`);
const oldBin = path.join(root, "old-bin");
fs.mkdirSync(oldBin);
fs.writeFileSync(path.join(oldBin, "node"), "#!/bin/sh\nexit 42\n", { mode: 0o755 });
writeManagedShim(paths);
const output = execFileSync(paths.shimPath, [], { env: { ...process.env, PATH: oldBin }, encoding: "utf8" });
expect(fs.realpathSync(output.trim())).toBe(fs.realpathSync(process.execPath));
expect(removeManagedShim(paths)).toBe(true);
});
it("upgrades and removes the original managed shim format", () => {
writeManagedShim(paths);
const original = fs.readFileSync(paths.shimPath, "utf8").split("\n").filter((line) => !line.startsWith("export PATH=")).join("\n");
fs.writeFileSync(paths.shimPath, original);
writeManagedShim(paths);
expect(fs.readFileSync(paths.shimPath, "utf8")).toContain("export PATH=");
fs.writeFileSync(paths.shimPath, original);
expect(removeManagedShim(paths)).toBe(true);
});
it("rejects marker substrings that are not the exact managed shim format", () => {
fs.mkdirSync(path.dirname(paths.shimPath), { recursive: true });
fs.writeFileSync(paths.shimPath, `#!/bin/sh\necho '${MANAGED_SHIM_MARKER}'\n`);

19
cli/src/__tests__/issue-subresources.test.ts
View File

@ -13,6 +13,7 @@ const PRODUCT_ID = "77777777-7777-4777-8777-777777777777";
const INTERACTION_ID = "88888888-8888-4888-8888-888888888888";
const HOLD_ID = "99999999-9999-4999-8999-999999999999";
const ATTACHMENT_ID = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa";
const SECOND_ATTACHMENT_ID = "abababab-abab-4bab-8bab-abababababab";
const LABEL_ID = "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb";
function createProgram(): Command {
@ -61,6 +62,24 @@ describe("issue subresource commands", () => {
]);
});
it("binds explicit uploaded attachments when adding a comment", async () => {
const fetchMock = vi.fn().mockImplementation(() => Promise.resolve(jsonResponse()));
vi.stubGlobal("fetch", fetchMock);
await run([
"issue", "comment", ISSUE_ID,
"--body", "The requested files are ready.",
"--attachment-id", ATTACHMENT_ID, SECOND_ATTACHMENT_ID,
]);
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(fetchMock.mock.calls[0]?.[0]).toBe(`http://localhost:3100/api/issues/${ISSUE_ID}/comments`);
expect(JSON.parse(String(fetchMock.mock.calls[0]?.[1]?.body))).toEqual({
body: "The requested files are ready.",
attachmentIds: [ATTACHMENT_ID, SECOND_ATTACHMENT_ID],
});
});
it("wraps comments, approvals, markers, and recovery action endpoints", async () => {
const fetchMock = vi
.fn()

332
cli/src/__tests__/managed-agent.test.ts Normal file
View File

@ -0,0 +1,332 @@
import { Command } from "commander";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
CLAUDE_MANAGED_BETA_VERSION,
CLAUDE_MANAGED_SYSTEM_PROMPT,
assertSafeManagedAgent,
assertSafeManagedEnvironment,
registerManagedAgentCommands,
setupManagedAgent,
validateManagedAgentSetup,
type ManagedAgentSetupOptions,
} from "../commands/managed-agent.js";
const ORIGINAL_ENV = { ...process.env };
function setupOptions(
overrides: Partial<ManagedAgentSetupOptions> = {},
): ManagedAgentSetupOptions {
return {
profileKey: "primary",
displayName: "Primary Claude",
apiKeySecretId: "11111111-1111-4111-8111-111111111111",
model: "claude-sonnet-5",
maxSessionListCostUsd: "1.25",
acknowledgeRetention: true,
companyId: "company-1",
apiBase: "http://localhost:3100",
apiKey: "paperclip-board-token",
json: true,
...overrides,
};
}
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "content-type": "application/json" },
});
}
function safeEnvironment(id = "env-1") {
return {
id,
archived_at: null,
config: {
type: "cloud",
networking: {
type: "limited",
allow_mcp_servers: false,
allow_package_managers: false,
allowed_hosts: [],
},
packages: {
type: "packages",
apt: [],
cargo: [],
gem: [],
go: [],
npm: [],
pip: [],
},
},
};
}
function safeAgent(id = "agent-1") {
return {
id,
archived_at: null,
version: "7",
model: { id: "claude-sonnet-5" },
system: CLAUDE_MANAGED_SYSTEM_PROMPT,
tools: [],
mcp_servers: [],
skills: [],
multiagent: null,
};
}
describe("managed-agent CLI registration", () => {
it("registers setup with an explicit retention gate and no Anthropic key option", () => {
const program = new Command();
registerManagedAgentCommands(program);
const managedAgent = program.commands.find((command) => command.name() === "managed-agent");
const setup = managedAgent?.commands.find((command) => command.name() === "setup");
expect(setup).toBeDefined();
expect(setup?.options.some((option) => option.long === "--acknowledge-retention")).toBe(true);
expect(setup?.options.some((option) => option.long === "--api-key-secret-id")).toBe(true);
expect(setup?.options.some((option) => option.long === "--anthropic-api-key")).toBe(false);
});
});
describe("managed-agent CLI validation", () => {
it("requires the Anthropic key only through the CLI environment", () => {
expect(() => validateManagedAgentSetup(setupOptions(), {})).toThrow(
"ANTHROPIC_API_KEY is required in the CLI process environment",
);
});
it("requires retention acknowledgement before provisioning", () => {
expect(() =>
validateManagedAgentSetup(setupOptions({ acknowledgeRetention: false }), {
ANTHROPIC_API_KEY: "sk-ant-test",
}),
).toThrow("--acknowledge-retention");
});
it("rejects a model outside the qualified Managed Agents profile", () => {
expect(() =>
validateManagedAgentSetup(setupOptions({ model: "claude-opus-5" }), {
ANTHROPIC_API_KEY: "sk-ant-test",
}),
).toThrow("qualified Managed Agents model claude-sonnet-5");
});
it("requires a positive spend ceiling that rounds to at least one cent", () => {
expect(() =>
validateManagedAgentSetup(setupOptions({ maxSessionListCostUsd: "0.001" }), {
ANTHROPIC_API_KEY: "sk-ant-test",
}),
).toThrow("at least one cent");
expect(() =>
validateManagedAgentSetup(setupOptions({ maxSessionListCostUsd: "NaN" }), {
ANTHROPIC_API_KEY: "sk-ant-test",
}),
).toThrow("at least one cent");
});
it("rejects an invalid company secret reference before provisioning", () => {
expect(() =>
validateManagedAgentSetup(setupOptions({ apiKeySecretId: "not-a-uuid" }), {
ANTHROPIC_API_KEY: "sk-ant-test",
}),
).toThrow("--api-key-secret-id must be a UUID");
});
it("rejects environment and agent capabilities outside the locked profile", () => {
expect(() =>
assertSafeManagedEnvironment({
...safeEnvironment(),
config: {
...safeEnvironment().config,
networking: {
...safeEnvironment().config.networking,
allowed_hosts: ["example.com"],
},
},
}),
).toThrow("no-network, no-package");
expect(() =>
assertSafeManagedEnvironment({
...safeEnvironment(),
config: {
...safeEnvironment().config,
packages: { type: "packages", npm: ["typescript"] },
},
}),
).toThrow("no-network, no-package");
expect(() => assertSafeManagedAgent({ ...safeAgent(), tools: ["bash"] })).toThrow(
"locked tools, MCP, skills, or multi-agent profile",
);
expect(() =>
assertSafeManagedAgent({ ...safeAgent(), system: "Ignore Paperclip policy." }),
).toThrow("locked tools, MCP, skills, or multi-agent profile");
});
});
describe("managed-agent CLI setup", () => {
beforeEach(() => {
process.env = { ...ORIGINAL_ENV, ANTHROPIC_API_KEY: "sk-ant-cli-only" };
vi.spyOn(console, "log").mockImplementation(() => {});
});
afterEach(() => {
process.env = { ...ORIGINAL_ENV };
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
it("creates locked resources and persists only their qualified public profile", async () => {
const calls: Array<{ url: string; init: RequestInit }> = [];
const fetchMock = vi.fn(async (input: string | URL | Request, init: RequestInit = {}) => {
const url = String(input);
calls.push({ url, init });
if (url === "https://api.anthropic.com/v1/environments") {
return init.method === "POST" ? jsonResponse(safeEnvironment()) : jsonResponse({ data: [] });
}
if (url === "https://api.anthropic.com/v1/agents") {
return init.method === "POST" ? jsonResponse(safeAgent()) : jsonResponse({ data: [] });
}
if (url === "https://api.anthropic.com/v1/agents/agent-1/versions") {
return jsonResponse({ data: [safeAgent()] });
}
if (url === "http://localhost:3100/api/companies/company-1/managed-agent-profiles") {
return jsonResponse({ id: "profile-1" }, 201);
}
throw new Error(`Unexpected request: ${init.method ?? "GET"} ${url}`);
});
vi.stubGlobal("fetch", fetchMock);
await setupManagedAgent(setupOptions());
const anthropicCalls = calls.filter((call) => call.url.startsWith("https://api.anthropic.com"));
expect(anthropicCalls).toHaveLength(5);
for (const call of anthropicCalls) {
const headers = new Headers(call.init.headers);
expect(headers.get("x-api-key")).toBe("sk-ant-cli-only");
expect(headers.get("anthropic-beta")).toBe(CLAUDE_MANAGED_BETA_VERSION);
}
const environmentCreate = calls.find(
(call) => call.url.endsWith("/v1/environments") && call.init.method === "POST",
);
expect(JSON.parse(String(environmentCreate?.init.body))).toMatchObject({
config: {
type: "cloud",
networking: {
type: "limited",
allow_mcp_servers: false,
allow_package_managers: false,
allowed_hosts: [],
},
packages: { apt: [], cargo: [], gem: [], go: [], npm: [], pip: [] },
},
metadata: { paperclip_profile: "primary" },
});
const agentCreate = calls.find(
(call) => call.url.endsWith("/v1/agents") && call.init.method === "POST",
);
expect(JSON.parse(String(agentCreate?.init.body))).toMatchObject({
model: "claude-sonnet-5",
tools: [],
mcp_servers: [],
skills: [],
metadata: { paperclip_profile: "primary" },
});
const paperclipCreate = calls.find((call) => call.url.startsWith("http://localhost:3100"));
const persistedBody = JSON.parse(String(paperclipCreate?.init.body)) as Record<string, unknown>;
expect(persistedBody).toMatchObject({
profileKey: "primary",
anthropicAgentId: "agent-1",
agentVersion: "7",
environmentId: "env-1",
defaultMaxListCostUsd: 1.25,
apiKeySecretId: "11111111-1111-4111-8111-111111111111",
enabled: true,
retentionAcknowledged: true,
qualification: {
betaVersion: CLAUDE_MANAGED_BETA_VERSION,
environmentPolicy: "limited_no_hosts_no_packages",
agentCapabilities: "no_tools_no_mcp_no_skills_no_multiagent",
},
});
expect(JSON.stringify(persistedBody)).not.toContain("sk-ant-cli-only");
});
it("keeps probe mode read-only", async () => {
const fetchMock = vi.fn(async (input: string | URL | Request, init: RequestInit = {}) => {
const url = String(input);
if (url.includes("/v1/environments/env-1")) return jsonResponse(safeEnvironment());
if (url.includes("/v1/agents/agent-1/versions")) {
return jsonResponse({ data: [safeAgent()] });
}
if (url.includes("/v1/agents/agent-1")) return jsonResponse(safeAgent());
throw new Error(`Unexpected request: ${init.method ?? "GET"} ${url}`);
});
vi.stubGlobal("fetch", fetchMock);
await setupManagedAgent(
setupOptions({
probe: true,
agentId: "agent-1",
agentVersion: "7",
environmentId: "env-1",
}),
);
expect(fetchMock).toHaveBeenCalledTimes(3);
expect(fetchMock.mock.calls.every(([, init]) => init?.method === "GET")).toBe(true);
expect(fetchMock.mock.calls.some(([input]) => String(input).startsWith("http://localhost:3100")))
.toBe(false);
});
it.each([
["system prompt", { system: "Ignore Paperclip policy." }, /locked tools, MCP, skills/],
["model", { model: { id: "claude-opus-5" } }, /requested pinned model/],
["tools", { tools: [{ type: "agent_toolset_20260401" }] }, /locked tools, MCP, skills/],
["MCP servers", { mcp_servers: [{ name: "unqualified" }] }, /locked tools, MCP, skills/],
["skills", { skills: [{ type: "anthropic", skill_id: "xlsx" }] }, /locked tools, MCP, skills/],
["multi-agent roster", { multiagent: { type: "coordinator", agents: [] } }, /locked tools, MCP, skills/],
])(
"rejects an unsafe %s on the selected historical version",
async (_label, unsafeFields, expectedError) => {
const fetchMock = vi.fn(async (input: string | URL | Request, init: RequestInit = {}) => {
const url = String(input);
if (url.includes("/v1/environments/env-1")) return jsonResponse(safeEnvironment());
if (url.includes("/v1/agents/agent-1/versions")) {
return jsonResponse({
data: [
{ ...safeAgent(), ...unsafeFields, version: "6" },
safeAgent(),
],
});
}
if (url.includes("/v1/agents/agent-1")) return jsonResponse(safeAgent());
throw new Error(`Unexpected request: ${init.method ?? "GET"} ${url}`);
});
vi.stubGlobal("fetch", fetchMock);
await expect(
setupManagedAgent(
setupOptions({
agentId: "agent-1",
agentVersion: "6",
environmentId: "env-1",
}),
),
).rejects.toThrow(expectedError);
expect(fetchMock).toHaveBeenCalledTimes(3);
expect(
fetchMock.mock.calls.some(([input]) => String(input).startsWith("http://localhost:3100")),
).toBe(false);
},
);
});

194
cli/src/__tests__/onboard-service.test.ts
View File

@ -1,5 +1,30 @@
import { describe, expect, it, vi } from "vitest";
import { handleOnboardService } from "../onboard-service.js";
import {
handleOnboardService,
handoffToOnboardedService,
isInstallableReleaseVersion,
resolveOnboardServiceDashboardUrl,
shouldOfferForegroundStart,
} from "../onboard-service.js";
function dashboardConfig(overrides: {
host?: string;
port?: number;
baseUrlMode?: "auto" | "explicit";
publicBaseUrl?: string;
} = {}) {
return {
server: {
host: overrides.host ?? "127.0.0.1",
port: overrides.port ?? 3100,
},
auth: {
baseUrlMode: overrides.baseUrlMode ?? "auto",
disableSignUp: false,
...(overrides.publicBaseUrl ? { publicBaseUrl: overrides.publicBaseUrl } : {}),
},
};
}
function supportedDetection() {
return {
@ -24,6 +49,7 @@ function supportedDetection() {
pid: 123,
})),
logs: vi.fn(async () => undefined),
installedExecutablePath: vi.fn(async () => null),
},
};
}
@ -48,7 +74,11 @@ describe("onboard service policy", () => {
const installed = await handleOnboardService(
{ yes: true, installService: true },
{ detect: vi.fn(async () => detection), isInteractive: () => false },
{
detect: vi.fn(async () => detection),
isInteractive: () => false,
ensureServiceShim: vi.fn(async () => ({ ok: true, installedNow: false })),
},
);
expect(installed).toBe(true);
@ -61,7 +91,12 @@ describe("onboard service policy", () => {
const installed = await handleOnboardService(
{},
{ detect: vi.fn(async () => detection), isInteractive: () => true, confirm },
{
detect: vi.fn(async () => detection),
isInteractive: () => true,
confirm,
ensureServiceShim: vi.fn(async () => ({ ok: true, installedNow: false })),
},
);
expect(confirm).toHaveBeenCalledOnce();
@ -81,4 +116,157 @@ describe("onboard service policy", () => {
expect(detect).not.toHaveBeenCalled();
expect(info).not.toHaveBeenCalled();
});
it("materializes the managed shim before installing the service", async () => {
const detection = supportedDetection();
const success = vi.fn();
const ensureServiceShim = vi.fn(async () => ({ ok: true, installedNow: true }));
const installed = await handleOnboardService(
{ yes: true, installService: true },
{ detect: vi.fn(async () => detection), isInteractive: () => false, ensureServiceShim, success },
);
expect(installed).toBe(true);
expect(ensureServiceShim).toHaveBeenCalledOnce();
expect(success).toHaveBeenCalledWith(expect.stringContaining("managed paperclipai payload"));
expect(detection.manager.install).toHaveBeenCalledWith({ startNow: true, startOnLogin: true });
});
it("declines instead of installing a service without a binary", async () => {
const detection = supportedDetection();
const warn = vi.fn();
const installed = await handleOnboardService(
{ yes: true, installService: true },
{
detect: vi.fn(async () => detection),
isInteractive: () => false,
ensureServiceShim: vi.fn(async () => ({ ok: false, installedNow: false, reason: "npm exploded" })),
warn,
},
);
expect(installed).toBe(false);
expect(detection.manager.install).not.toHaveBeenCalled();
expect(warn).toHaveBeenCalledWith(expect.stringContaining("npm exploded"));
expect(warn).toHaveBeenCalledWith(expect.stringContaining("paperclipai install"));
});
});
describe("isInstallableReleaseVersion", () => {
it("accepts calendar releases and rejects placeholders", () => {
expect(isInstallableReleaseVersion("2026.824.1")).toBe(true);
expect(isInstallableReleaseVersion("2026.818.0-beta.1")).toBe(true);
expect(isInstallableReleaseVersion("0.3.1")).toBe(false);
expect(isInstallableReleaseVersion("not-a-version")).toBe(false);
});
});
describe("onboarded service dashboard handoff", () => {
it("resolves a reachable local dashboard URL", () => {
expect(resolveOnboardServiceDashboardUrl(dashboardConfig({ host: "0.0.0.0", port: 4321 })))
.toBe("http://127.0.0.1:4321");
expect(resolveOnboardServiceDashboardUrl(dashboardConfig({ host: "::1" })))
.toBe("http://[::1]:3100");
});
it("uses the configured public URL when auth requires one", () => {
expect(resolveOnboardServiceDashboardUrl(dashboardConfig({
baseUrlMode: "explicit",
publicBaseUrl: "https://paperclip.example.com/",
}))).toBe("https://paperclip.example.com");
});
it("prints the dashboard URL without opening a browser in non-interactive runs", async () => {
const info = vi.fn();
const waitUntilReady = vi.fn(async () => ({
schemaVersion: 1 as const,
instanceId: "default",
pid: 123,
host: "127.0.0.1",
port: 3100,
dashboardUrl: "http://127.0.0.1:3100",
startedAt: "2026-08-25T00:00:00.000Z",
}));
const openDashboard = vi.fn(async () => true);
await handoffToOnboardedService(dashboardConfig(), {
isInteractive: () => false,
waitUntilReady,
openDashboard,
info,
});
expect(info).toHaveBeenCalledWith(expect.stringContaining("http://127.0.0.1:3100"));
expect(waitUntilReady).toHaveBeenCalledOnce();
expect(openDashboard).not.toHaveBeenCalled();
});
it("uses the ready service runtime port before opening the dashboard", async () => {
const waitUntilReady = vi.fn(async () => ({
schemaVersion: 1 as const,
instanceId: "default",
pid: 123,
host: "127.0.0.1",
port: 3101,
dashboardUrl: "http://127.0.0.1:3101",
startedAt: "2026-08-25T00:00:00.000Z",
}));
const openDashboard = vi.fn(async () => true);
const success = vi.fn();
await handoffToOnboardedService(dashboardConfig(), {
isInteractive: () => true,
waitUntilReady,
openDashboard,
info: vi.fn(),
success,
});
expect(waitUntilReady).toHaveBeenCalledOnce();
expect(openDashboard).toHaveBeenCalledWith("http://127.0.0.1:3101");
expect(success).toHaveBeenCalledWith(expect.stringContaining("Sent"));
});
it("keeps the manual link and warns when service health does not become ready", async () => {
const openDashboard = vi.fn(async () => true);
const warn = vi.fn();
await handoffToOnboardedService(dashboardConfig(), {
isInteractive: () => true,
waitUntilReady: vi.fn(async () => null),
openDashboard,
info: vi.fn(),
warn,
});
expect(openDashboard).not.toHaveBeenCalled();
expect(warn).toHaveBeenCalledWith(expect.stringContaining("paperclipai service logs"));
});
});
describe("shouldOfferForegroundStart", () => {
const base = { serviceInstalled: false, startAlreadyDecided: false, invokedByRun: false, interactive: true };
it("offers a foreground start on a plain interactive onboard", () => {
expect(shouldOfferForegroundStart(base)).toBe(true);
});
it("never prompts after the service was installed and started", () => {
expect(shouldOfferForegroundStart({ ...base, serviceInstalled: true })).toBe(false);
});
it("never prompts when the start decision was already made by flags", () => {
expect(shouldOfferForegroundStart({ ...base, startAlreadyDecided: true })).toBe(false);
});
it("never prompts when run itself invoked onboarding", () => {
expect(shouldOfferForegroundStart({ ...base, invokedByRun: true })).toBe(false);
});
it("never prompts without an interactive terminal", () => {
expect(shouldOfferForegroundStart({ ...base, interactive: false })).toBe(false);
});
});

76
cli/src/__tests__/onboard.test.ts
View File

@ -1,10 +1,16 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { onboard } from "../commands/onboard.js";
import type { PaperclipConfig } from "../config/schema.js";
const runCommandMock = vi.hoisted(() => vi.fn());
vi.mock("../commands/run.js", () => ({
runCommand: runCommandMock,
}));
const ORIGINAL_ENV = { ...process.env };
const ORIGINAL_CWD = process.cwd();
const ORIGINAL_PATH = process.env.PATH;
@ -86,6 +92,7 @@ describe("onboard", () => {
beforeEach(() => {
process.env = { ...ORIGINAL_ENV };
delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
delete process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET;
delete process.env.PAPERCLIP_SECRETS_MASTER_KEY;
delete process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE;
delete process.env.PAPERCLIP_DB_BACKUP_DIR;
@ -102,7 +109,10 @@ describe("onboard", () => {
delete process.env.PAPERCLIP_BIND;
delete process.env.PAPERCLIP_BIND_HOST;
delete process.env.PAPERCLIP_TAILNET_BIND_HOST;
delete process.env.PAPERCLIP_OPEN_ON_LISTEN;
delete process.env.PAPERCLIP_NO_BROWSER;
delete process.env.HOST;
runCommandMock.mockReset();
});
afterEach(() => {
@ -131,6 +141,68 @@ describe("onboard", () => {
expect(fs.existsSync(path.join(path.dirname(fixture.configPath), ".env"))).toBe(true);
});
it("does not opt into opening a browser for a non-interactive existing setup", async () => {
const fixture = createExistingConfigFixture();
await onboard({ config: fixture.configPath, yes: true });
expect(runCommandMock).toHaveBeenCalledWith({ config: fixture.configPath, repair: true, yes: true });
expect(process.env.PAPERCLIP_OPEN_ON_LISTEN).toBeUndefined();
});
it.each([
["existing", () => createExistingConfigFixture().configPath],
["fresh", () => createFreshConfigPath()],
])("opens the browser once while an interactive %s setup starts", async (_label, configPathForTest) => {
const configPath = configPathForTest();
const stdinIsTTY = process.stdin.isTTY;
const stdoutIsTTY = process.stdout.isTTY;
let openOnListenDuringRun: string | undefined;
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: true });
Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: true });
runCommandMock.mockImplementation(async () => {
openOnListenDuringRun = process.env.PAPERCLIP_OPEN_ON_LISTEN;
});
try {
await onboard({ config: configPath, yes: true });
} finally {
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: stdinIsTTY });
Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: stdoutIsTTY });
}
expect(runCommandMock).toHaveBeenCalledWith({ config: configPath, repair: true, yes: true });
expect(openOnListenDuringRun).toBe("true");
expect(process.env.PAPERCLIP_OPEN_ON_LISTEN).toBeUndefined();
});
it.each([
["PAPERCLIP_NO_BROWSER", "1"],
["PAPERCLIP_OPEN_ON_LISTEN", "false"],
])("respects the interactive browser opt-out %s", async (key, value) => {
const configPath = createFreshConfigPath();
const stdinIsTTY = process.stdin.isTTY;
const stdoutIsTTY = process.stdout.isTTY;
let openOnListenDuringRun: string | undefined;
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: true });
Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: true });
process.env[key] = value;
runCommandMock.mockImplementation(async () => {
openOnListenDuringRun = process.env.PAPERCLIP_OPEN_ON_LISTEN;
});
try {
await onboard({ config: configPath, yes: true });
} finally {
Object.defineProperty(process.stdin, "isTTY", { configurable: true, value: stdinIsTTY });
Object.defineProperty(process.stdout, "isTTY", { configurable: true, value: stdoutIsTTY });
}
expect(runCommandMock).toHaveBeenCalledWith({ config: configPath, repair: true, yes: true });
expect(openOnListenDuringRun).not.toBe("true");
expect(process.env[key]).toBe(value);
});
it("backs up invalid config bytes and refuses --yes replacement", async () => {
const configPath = createFreshConfigPath();
const invalidBytes = Buffer.from('{"database": invalid}\n', "utf8");
@ -177,6 +249,8 @@ describe("onboard", () => {
expect(raw.storage.localDisk.baseDir).toBe(path.join(instanceRoot, "data", "storage"));
expect(raw.secrets.localEncrypted.keyFilePath).toBe(path.join(instanceRoot, "secrets", "master.key"));
expect(fs.existsSync(path.join(instanceRoot, ".env"))).toBe(true);
expect(fs.readFileSync(path.join(instanceRoot, ".env"), "utf8"))
.toContain("PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=");
expect(fs.existsSync(path.join(instanceRoot, "secrets", "master.key"))).toBe(true);
});

56
cli/src/__tests__/runtime-info.test.ts Normal file
View File

@ -0,0 +1,56 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
readRuntimeInfo,
removeRuntimeInfoForPid,
writeRuntimeInfo,
type PaperclipRuntimeInfo,
} from "../runtime-info.js";
const roots: string[] = [];
afterEach(() => {
for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true });
});
function fixture(): { filePath: string; info: PaperclipRuntimeInfo } {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-runtime-info-"));
roots.push(root);
return {
filePath: path.join(root, "runtime-info.json"),
info: {
schemaVersion: 1,
instanceId: "default",
pid: 123,
host: "127.0.0.1",
port: 3101,
dashboardUrl: "http://127.0.0.1:3101",
startedAt: "2026-08-25T00:00:00.000Z",
},
};
}
describe("runtime info", () => {
it("writes and reads the selected runtime endpoint", () => {
const { filePath, info } = fixture();
writeRuntimeInfo(info, filePath);
expect(readRuntimeInfo("default", filePath)).toEqual(info);
});
it("does not remove runtime info owned by a replacement process", () => {
const { filePath, info } = fixture();
writeRuntimeInfo(info, filePath);
removeRuntimeInfoForPid(999, "default", filePath);
expect(readRuntimeInfo("default", filePath)).toEqual(info);
removeRuntimeInfoForPid(info.pid, "default", filePath);
expect(readRuntimeInfo("default", filePath)).toBeNull();
});
it("rejects malformed runtime info", () => {
const { filePath } = fixture();
fs.writeFileSync(filePath, JSON.stringify({ schemaVersion: 1, port: 70_000 }));
expect(readRuntimeInfo("default", filePath)).toBeNull();
});
});

113
cli/src/__tests__/service-health-check.test.ts
View File

@ -3,6 +3,13 @@ import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { serviceHealthChecks } from "../checks/service-health-check.js";
import {
extractExecutableFromLaunchdPlist,
extractExecutableFromSystemdUnit,
isExecutableFile,
renderLaunchdPlist,
renderSystemdUnit,
} from "../services/service-manager.js";
import { resolveRestartExpectedVersion, withHotRestartLock } from "../commands/service.js";
import type { PaperclipConfig } from "../config/schema.js";
import { buildLocalHealthUrl } from "../utils/health-url.js";
@ -52,6 +59,7 @@ function managerFixture(active = true) {
linger: true,
})),
logs: vi.fn(async () => undefined),
installedExecutablePath: vi.fn(async () => null),
};
}
@ -129,6 +137,7 @@ describe("service health doctor checks", () => {
const results = await serviceHealthChecks(config, {
detect: vi.fn(async () => ({ supported: true as const, manager })),
probe: vi.fn(async () => ({ ok: true, version: "1.2.3" })),
shimPresent: vi.fn(async () => true),
});
expect(results).toContainEqual(
@ -140,3 +149,107 @@ describe("service health doctor checks", () => {
);
});
});
describe("isExecutableFile", () => {
it("accepts only executable regular files", async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "shim-check-"));
const executable = path.join(dir, "exec");
const plain = path.join(dir, "plain");
fs.writeFileSync(executable, "#!/bin/sh\n", { mode: 0o755 });
fs.writeFileSync(plain, "data", { mode: 0o644 });
await expect(isExecutableFile(executable)).resolves.toBe(true);
await expect(isExecutableFile(plain)).resolves.toBe(false);
await expect(isExecutableFile(dir)).resolves.toBe(false);
await expect(isExecutableFile(path.join(dir, "missing"))).resolves.toBe(false);
});
});
describe("service runtime shim awareness", () => {
function inactiveManager() {
return {
platform: "launchd" as const,
instanceId: "default",
serviceName: "ing.paperclip.paperclipai",
definitionPath: "/tmp/nonexistent-definition.plist",
renderDefinition: () => "plist",
install: vi.fn(async () => ({ changed: false })),
uninstall: vi.fn(async () => undefined),
start: vi.fn(async () => undefined),
stop: vi.fn(async () => undefined),
restart: vi.fn(async () => undefined),
status: vi.fn(async () => ({
platform: "launchd" as const,
serviceName: "ing.paperclip.paperclipai",
installed: true,
active: false,
enabled: true,
pid: null,
detail: "loaded",
})),
logs: vi.fn(async () => undefined),
installedExecutablePath: vi.fn(async (): Promise<string | null> => null),
};
}
it("blames the missing binary, not a port conflict, when the shim is gone", async () => {
const results = await serviceHealthChecks({} as never, {
detect: vi.fn(async () => ({ supported: true as const, manager: inactiveManager() as never })),
probe: vi.fn(async () => ({ ok: false, version: null, error: "fetch failed" })),
shimPresent: vi.fn(async () => false),
});
const runtime = results.find((r) => r.name === "Service runtime");
expect(runtime?.status).toBe("fail");
expect(runtime?.message).toContain("no executable exists at");
expect(runtime?.repairHint).toContain("paperclipai install");
});
it("diagnoses against the executable recorded in the definition, not the current env", async () => {
const manager = inactiveManager();
manager.installedExecutablePath = vi.fn(async () => "/custom/bin/paperclipai");
const shimPresent = vi.fn(async () => false);
const results = await serviceHealthChecks({} as never, {
detect: vi.fn(async () => ({ supported: true as const, manager: manager as never })),
probe: vi.fn(async () => ({ ok: false, version: null, error: "fetch failed" })),
shimPresent,
});
const runtime = results.find((r) => r.name === "Service runtime");
expect(shimPresent).toHaveBeenCalledWith("/custom/bin/paperclipai");
expect(runtime?.message).toContain("/custom/bin/paperclipai");
expect(runtime?.repairHint).toContain("/custom/bin/paperclipai");
expect(runtime?.repairHint).toContain("unset PAPERCLIP_SHIM_PATH");
expect(runtime?.repairHint).toContain("`paperclipai install` followed by `paperclipai service install`");
});
it("attributes a healthy foreign responder instead of reporting Healthy", async () => {
const results = await serviceHealthChecks({} as never, {
detect: vi.fn(async () => ({ supported: true as const, manager: inactiveManager() as never })),
probe: vi.fn(async () => ({ ok: true, version: "9.9.9" })),
shimPresent: vi.fn(async () => true),
});
const healthResult = results.find((r) => r.name === "Service health");
expect(healthResult?.status).toBe("warn");
expect(healthResult?.message).toContain("but not from ing.paperclip.paperclipai");
const runtime = results.find((r) => r.name === "Service runtime");
expect(runtime?.message).toContain("serving another Paperclip process");
});
});
describe("definition executable extraction", () => {
it("round-trips through both renderers", () => {
const unit = renderSystemdUnit({ instanceId: "default", shimPath: "/custom/bin/paperclipai", homeDir: "/home/x/.paperclip" });
expect(extractExecutableFromSystemdUnit(unit)).toBe("/custom/bin/paperclipai");
const plist = renderLaunchdPlist({ instanceId: "default", shimPath: "/custom/bin/paperclipai", homeDir: "/home/x/.paperclip", stdoutPath: "/tmp/o.log", stderrPath: "/tmp/e.log" });
expect(extractExecutableFromLaunchdPlist(plist)).toBe("/custom/bin/paperclipai");
expect(extractExecutableFromSystemdUnit("garbage")).toBe(null);
expect(extractExecutableFromLaunchdPlist("garbage")).toBe(null);
});
it("round-trips paths the renderers escape", () => {
const hostile = '/tmp/we"ird $pa%th & <x>/paperclipai';
const unit = renderSystemdUnit({ instanceId: "default", shimPath: hostile, homeDir: "/home/x/.paperclip" });
expect(extractExecutableFromSystemdUnit(unit)).toBe(hostile);
const plist = renderLaunchdPlist({ instanceId: "default", shimPath: hostile, homeDir: "/home/x/.paperclip", stdoutPath: "/tmp/o.log", stderrPath: "/tmp/e.log" });
expect(extractExecutableFromLaunchdPlist(plist)).toBe(hostile);
});
});

602
cli/src/__tests__/test-drive.test.ts Normal file
View File

@ -0,0 +1,602 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { Agent, Company, InstanceExperimentalSettings } from "@paperclipai/shared";
import {
assertTestDriveDatabaseIsolation,
bootstrapTestDrive,
prepareTestDriveEnvironment,
reconcileTestDriveWorktreeExecution,
redactTestDriveArgv,
redactTestDriveText,
resolveTestDriveBootstrap,
resolveTestDriveDataDir,
testDriveCommand,
type TestDriveApi,
type TestDriveHarness,
} from "../commands/test-drive.js";
import type { RunOptions, StartedServer } from "../commands/run.js";
import type { PaperclipConfig } from "../config/schema.js";
const ORIGINAL_ENV = { ...process.env };
const cleanupDirectories: string[] = [];
function company(id = "company-1", name = "Test Company"): Company {
return { id, name } as Company;
}
function agent(overrides: Partial<Agent> = {}): Agent {
return {
id: "agent-1",
companyId: "company-1",
name: "CEO",
role: "ceo",
adapterType: "claude_local",
adapterConfig: {},
...overrides,
} as Agent;
}
function settings(overrides: Partial<InstanceExperimentalSettings> = {}): InstanceExperimentalSettings {
return {
enableWorktreeRunExecution: false,
worktreeRunExecutionActivatedAt: null,
worktreeRunExecutionActivationInstanceId: null,
...overrides,
} as InstanceExperimentalSettings;
}
function freshBootstrapApi(input?: { failAgent?: boolean }) {
const calls: Array<{ method: string; path: string; body?: unknown }> = [];
const api = {
get: vi.fn(async <T>(requestPath: string) => {
calls.push({ method: "GET", path: requestPath });
return [] as T;
}),
post: vi.fn(async <T>(requestPath: string, body?: unknown) => {
calls.push({ method: "POST", path: requestPath, body });
if (requestPath === "/api/companies") return company() as T;
if (requestPath.endsWith("/agents")) {
if (input?.failAgent) throw new Error("agent setup failed");
const payload = body as { name: string; adapterType: Agent["adapterType"]; adapterConfig: Record<string, unknown> };
return agent({
name: payload.name,
adapterType: payload.adapterType,
adapterConfig: payload.adapterConfig,
}) as T;
}
return { ok: true } as T;
}),
patch: vi.fn(async <T>() => ({ ok: true }) as T),
delete: vi.fn(async <T>(requestPath: string) => {
calls.push({ method: "DELETE", path: requestPath });
return { ok: true } as T;
}),
} as TestDriveApi;
return { api, calls };
}
afterEach(() => {
process.env = { ...ORIGINAL_ENV };
while (cleanupDirectories.length > 0) {
fs.rmSync(cleanupDirectories.pop()!, { recursive: true, force: true });
}
vi.restoreAllMocks();
});
describe("test-drive data isolation", () => {
it("creates unique retained OS temporary directories and reports absolute paths", () => {
const first = resolveTestDriveDataDir();
const second = resolveTestDriveDataDir();
cleanupDirectories.push(first, second);
expect(path.isAbsolute(first)).toBe(true);
expect(path.dirname(first)).toBe(os.tmpdir());
expect(first).not.toBe(second);
expect(fs.existsSync(first)).toBe(true);
expect(fs.existsSync(second)).toBe(true);
});
it("resolves an explicit reusable directory without resetting it", () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-test-drive-explicit-"));
cleanupDirectories.push(root);
const marker = path.join(root, "keep.txt");
fs.writeFileSync(marker, "keep");
expect(resolveTestDriveDataDir(root)).toBe(path.resolve(root));
expect(fs.readFileSync(marker, "utf8")).toBe("keep");
});
it("discards inherited Paperclip routing while preserving a custom key source", async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-test-drive-env-"));
cleanupDirectories.push(root);
process.env.PAPERCLIP_HOME = "/normal/home";
process.env.PAPERCLIP_CONFIG = "/normal/config.json";
process.env.PAPERCLIP_IN_WORKTREE = "true";
process.env.PAPERCLIP_TEST_PROVIDER_KEY = "secret-value";
process.env.DATABASE_URL = "postgres://normal-instance";
const prepared = await prepareTestDriveEnvironment(
{ dataDir: root, apiKeyEnv: "PAPERCLIP_TEST_PROVIDER_KEY" },
os.tmpdir(),
);
expect(prepared.dataDir).toBe(path.resolve(root));
expect(prepared.linkedWorktree).toBe(false);
expect(process.env.PAPERCLIP_HOME).toBe(path.resolve(root));
expect(process.env.PAPERCLIP_CONFIG).toBe(
path.join(path.resolve(root), "instances", "default", "config.json"),
);
expect(process.env.PAPERCLIP_IN_WORKTREE).toBe("false");
expect(process.env.PAPERCLIP_DISABLE_CWD_ENV_FILE).toBe("true");
expect(process.env.PAPERCLIP_DEPLOYMENT_MODE).toBe("local_trusted");
expect(process.env.PAPERCLIP_DEPLOYMENT_EXPOSURE).toBe("private");
expect(process.env.PAPERCLIP_BIND).toBe("loopback");
expect(process.env.HOST).toBe("127.0.0.1");
expect(process.env.PAPERCLIP_TEST_PROVIDER_KEY).toBe("secret-value");
expect(process.env.DATABASE_URL).toBeUndefined();
expect(Number(process.env.PORT)).toBeGreaterThanOrEqual(3100);
});
it.each(["DATABASE_URL", "DATABASE_MIGRATION_URL"])(
"rejects %s loaded from the isolated directory",
(variable) => {
const readConfigFile = vi.fn(() => null);
expect(() => assertTestDriveDatabaseIsolation(
undefined,
{ [variable]: "postgres://external-database" },
readConfigFile,
)).toThrow(/requires its isolated embedded database/);
expect(readConfigFile).not.toHaveBeenCalled();
},
);
it("rejects an explicitly reused PostgreSQL configuration", () => {
const externalConfig = {
database: { mode: "postgres" },
} as PaperclipConfig;
expect(() => assertTestDriveDatabaseIsolation(
"/tmp/reused/config.json",
{},
() => externalConfig,
)).toThrow(/cannot reuse.*external PostgreSQL database/);
});
it("accepts an embedded configuration", () => {
const embeddedConfig = {
database: { mode: "embedded-postgres" },
} as PaperclipConfig;
expect(() => assertTestDriveDatabaseIsolation(
"/tmp/reused/config.json",
{},
() => embeddedConfig,
)).not.toThrow();
});
});
describe("test-drive bootstrap validation", () => {
it("uses Claude defaults and the canonical environment credential", () => {
const resolved = resolveTestDriveBootstrap({}, { ANTHROPIC_API_KEY: "anthropic-secret" });
expect(resolved).toMatchObject({
companyName: "Test Company",
agentName: "CEO",
adapterType: "claude_local",
credentialTarget: "ANTHROPIC_API_KEY",
credential: "anthropic-secret",
});
expect(resolved.model).toBeUndefined();
});
it("maps all harnesses and leaves Claude/Codex models optional", () => {
const cases: Array<[TestDriveHarness, string, string]> = [
["claude", "claude_local", "ANTHROPIC_API_KEY"],
["codex", "codex_local", "OPENAI_API_KEY"],
["opencode", "opencode_local", "OPENROUTER_API_KEY"],
];
for (const [harness, adapterType, credentialTarget] of cases) {
const resolved = resolveTestDriveBootstrap(
{
harness,
...(harness === "opencode" ? { model: "openrouter/anthropic/claude-sonnet-4.5" } : {}),
},
{ [credentialTarget]: "provider-secret" },
);
expect(resolved.adapterType).toBe(adapterType);
expect(resolved.credentialTarget).toBe(credentialTarget);
}
});
it("requires an OpenRouter OpenCode model and preserves every model path segment", () => {
expect(() => resolveTestDriveBootstrap(
{ harness: "opencode" },
{ OPENROUTER_API_KEY: "secret" },
)).toThrow(/require --model openrouter/);
for (const model of ["anthropic/claude", "openrouter/", "openrouter//claude", "openrouter/a/"]) {
expect(() => resolveTestDriveBootstrap(
{ harness: "opencode", model },
{ OPENROUTER_API_KEY: "secret" },
)).toThrow(/require --model openrouter/);
}
const model = "openrouter/publisher/family/model";
expect(resolveTestDriveBootstrap(
{ harness: "opencode", model },
{ OPENROUTER_API_KEY: "secret" },
).model).toBe(model);
});
it("supports custom source variables while retaining the canonical target", () => {
const resolved = resolveTestDriveBootstrap(
{
harness: "opencode",
model: "openrouter/anthropic/claude-sonnet-4.5",
apiKeyEnv: "MY_OPENROUTER_KEY",
},
{ MY_OPENROUTER_KEY: "custom-secret" },
);
expect(resolved.credential).toBe("custom-secret");
expect(resolved.credentialSource).toBe("MY_OPENROUTER_KEY");
expect(resolved.credentialTarget).toBe("OPENROUTER_API_KEY");
});
it("accepts a literal key and gives it precedence over canonical environment lookup", () => {
const resolved = resolveTestDriveBootstrap(
{ apiKey: "literal-secret" },
{ ANTHROPIC_API_KEY: "environment-secret" },
);
expect(resolved.credential).toBe("literal-secret");
expect(resolved.credentialSource).toBe("--api-key");
});
it("rejects mutually exclusive key inputs", () => {
expect(() => resolveTestDriveBootstrap({
apiKey: "literal-secret",
apiKeyEnv: "ANTHROPIC_API_KEY",
}, { ANTHROPIC_API_KEY: "environment-secret" })).toThrow(/mutually exclusive/);
});
it("rejects invalid key variable names and redacts credentials", () => {
expect(() => resolveTestDriveBootstrap({
apiKeyEnv: "NOT-A-VALID-NAME",
}, { ANTHROPIC_API_KEY: "env-secret" })).toThrow(/valid environment variable/);
expect(redactTestDriveText(
"literal-secret, custom-secret, and env-secret must never appear",
["literal-secret", "custom-secret", "env-secret"],
)).toBe("[REDACTED], [REDACTED], and [REDACTED] must never appear");
});
it("removes literal keys from the JavaScript argv view", () => {
const splitArgv = ["node", "paperclipai", "test-drive", "--api-key", "literal-secret"];
const joinedArgv = ["node", "paperclipai", "test-drive", "--api-key=literal-secret"];
redactTestDriveArgv("literal-secret", splitArgv);
redactTestDriveArgv("literal-secret", joinedArgv);
expect(splitArgv).toEqual(["node", "paperclipai", "test-drive", "--api-key", "[REDACTED]"]);
expect(joinedArgv).toEqual(["node", "paperclipai", "test-drive", "--api-key=[REDACTED]"]);
});
});
describe("test-drive API bootstrap", () => {
it.each([
["claude", "claude_local", "ANTHROPIC_API_KEY", undefined],
["codex", "codex_local", "OPENAI_API_KEY", undefined],
["opencode", "opencode_local", "OPENROUTER_API_KEY", "openrouter/anthropic/claude-sonnet-4.5"],
] as const)("creates exactly one company and one CEO for %s", async (
harness,
adapterType,
credentialTarget,
model,
) => {
const { api, calls } = freshBootstrapApi();
const result = await bootstrapTestDrive({
api,
options: { harness, ...(model ? { model } : {}) },
linkedWorktree: false,
instanceId: "default",
env: { [credentialTarget]: "secret" },
});
expect(result.reused).toBe(false);
expect(result.agent?.role).toBe("ceo");
expect(calls.map((call) => `${call.method} ${call.path}`)).toEqual([
"GET /api/companies",
"POST /api/companies",
"POST /api/companies/company-1/user-secret-definitions",
"POST /api/companies/company-1/me/user-secrets",
"POST /api/companies/company-1/agents",
]);
const secretValueCall = calls.find((call) => call.path.endsWith("/me/user-secrets"));
expect(secretValueCall?.body).toEqual({ definitionKey: credentialTarget, value: "secret" });
const agentCall = calls.find((call) => call.path.endsWith("/agents"));
expect(agentCall?.body).toEqual({
name: "CEO",
role: "ceo",
adapterType,
adapterConfig: {
...(model ? { model } : {}),
env: {
[credentialTarget]: {
type: "user_secret_ref",
key: credentialTarget,
version: "latest",
required: true,
},
},
},
});
expect(calls.some((call) => /issues|projects|goals|tasks|heartbeat/.test(call.path))).toBe(false);
});
it("rejects invalid OpenCode configuration before creating a company", async () => {
const { api, calls } = freshBootstrapApi();
await expect(bootstrapTestDrive({
api,
options: { harness: "opencode" },
linkedWorktree: false,
instanceId: "default",
env: { OPENROUTER_API_KEY: "secret" },
})).rejects.toThrow(/require --model openrouter/);
expect(calls).toEqual([{ method: "GET", path: "/api/companies" }]);
});
it("preserves seeded data and ignores every bootstrap flag", async () => {
const get = vi.fn(async <T>(requestPath: string) => {
if (requestPath === "/api/companies") return [company("existing", "Existing Company")] as T;
throw new Error(`Unexpected GET ${requestPath}`);
});
const api = {
get,
post: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
const result = await bootstrapTestDrive({
api,
options: { harness: "opencode", companyName: "Ignored" },
linkedWorktree: false,
instanceId: "default",
env: {},
});
expect(result).toMatchObject({ reused: true, company: { id: "existing" }, agent: null });
expect(api.post).not.toHaveBeenCalled();
expect(api.patch).not.toHaveBeenCalled();
expect(api.delete).not.toHaveBeenCalled();
});
it("deletes only the newly-created company when fresh bootstrap fails", async () => {
const { api, calls } = freshBootstrapApi({ failAgent: true });
await expect(bootstrapTestDrive({
api,
options: {},
linkedWorktree: false,
instanceId: "default",
env: { ANTHROPIC_API_KEY: "secret" },
})).rejects.toThrow("agent setup failed");
expect(calls.at(-1)).toEqual({ method: "DELETE", path: "/api/companies/company-1" });
});
});
describe("test-drive worktree setting reconciliation", () => {
function worktreeApi(initial: InstanceExperimentalSettings, instanceId = "test-instance") {
let current = initial;
const patchBodies: unknown[] = [];
const api = {
get: vi.fn(async <T>() => current as T),
patch: vi.fn(async <T>(_path: string, body?: unknown) => {
patchBodies.push(body);
const enabled = (body as { enableWorktreeRunExecution: boolean }).enableWorktreeRunExecution;
current = settings({
...current,
enableWorktreeRunExecution: enabled,
worktreeRunExecutionActivatedAt: enabled ? "2026-09-05T12:00:00.000Z" : null,
worktreeRunExecutionActivationInstanceId: enabled ? instanceId : null,
});
return current as T;
}),
post: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
return { api, patchBodies };
}
it("enables a disabled setting", async () => {
const { api, patchBodies } = worktreeApi(settings());
await reconcileTestDriveWorktreeExecution(api, "test-instance");
expect(patchBodies).toEqual([{ enableWorktreeRunExecution: true }]);
});
it("leaves a correctly armed setting unchanged", async () => {
const { api, patchBodies } = worktreeApi(settings({
enableWorktreeRunExecution: true,
worktreeRunExecutionActivatedAt: "2026-09-05T11:00:00.000Z",
worktreeRunExecutionActivationInstanceId: "test-instance",
}));
await reconcileTestDriveWorktreeExecution(api, "test-instance");
expect(patchBodies).toEqual([]);
});
it.each([
settings({ enableWorktreeRunExecution: true }),
settings({
enableWorktreeRunExecution: true,
worktreeRunExecutionActivatedAt: "2026-09-05T11:00:00.000Z",
worktreeRunExecutionActivationInstanceId: "another-instance",
}),
])("rearms missing or mismatched activation metadata", async (initial) => {
const { api, patchBodies } = worktreeApi(initial);
await reconcileTestDriveWorktreeExecution(api, "test-instance");
expect(patchBodies).toEqual([
{ enableWorktreeRunExecution: false },
{ enableWorktreeRunExecution: true },
]);
});
it("fails when the setting cannot be armed for this instance", async () => {
const api = {
get: vi.fn(async <T>() => settings() as T),
patch: vi.fn(async <T>() => settings() as T),
post: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
await expect(reconcileTestDriveWorktreeExecution(api, "test-instance"))
.rejects.toThrow(/Could not arm/);
});
});
describe("test-drive foreground lifecycle", () => {
const server: StartedServer = {
apiUrl: "http://127.0.0.1:3100/api",
databaseUrl: "postgres://embedded",
host: "127.0.0.1",
listenPort: 3100,
};
it("skips service-manager integration for an auto-created directory and opens after initialization", async () => {
process.env.PAPERCLIP_HOME = "/tmp/test-drive-lifecycle";
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
process.env.ANTHROPIC_API_KEY = "secret";
const events: string[] = [];
let runOptions: RunOptions | undefined;
const api = {
get: vi.fn(async <T>() => {
events.push("initialized");
return [company("existing", "Existing")] as T;
}),
post: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
await testDriveCommand({}, {
run: async (options) => {
runOptions = options;
events.push("listening");
await options.afterStart?.(server);
},
createApi: () => api,
openBrowser: async () => {
events.push("browser");
return true;
},
});
expect(runOptions).toMatchObject({
yes: true,
bind: "loopback",
installService: false,
skipServiceManagerCheck: true,
introLabel: "paperclipai test-drive",
});
expect(events).toEqual(["listening", "initialized", "browser"]);
});
it("retains the managed-instance collision guard for an explicitly reused directory", async () => {
process.env.PAPERCLIP_HOME = "/tmp/test-drive-reused";
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
let runOptions: RunOptions | undefined;
const api = {
get: vi.fn(async <T>() => [company("existing", "Existing")] as T),
post: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
await testDriveCommand({ dataDir: "/tmp/test-drive-reused", browser: false }, {
run: async (options) => {
runOptions = options;
await options.afterStart?.(server);
},
createApi: () => api,
openBrowser: vi.fn(async () => true),
});
expect(runOptions?.skipServiceManagerCheck).toBe(false);
});
it("uses the credential snapshot captured before downstream server initialization", async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-test-drive-credential-"));
cleanupDirectories.push(root);
process.env.PAPERCLIP_HOME = root;
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
process.env.ANTHROPIC_API_KEY = "upstream-secret";
const { api, calls } = freshBootstrapApi();
await testDriveCommand({ browser: false }, {
run: async (options) => {
delete process.env.ANTHROPIC_API_KEY;
await options.afterStart?.(server);
},
createApi: () => api,
openBrowser: vi.fn(async () => true),
});
const secretValueCall = calls.find((call) => call.path.endsWith("/me/user-secrets"));
expect(secretValueCall?.body).toEqual({
definitionKey: "ANTHROPIC_API_KEY",
value: "upstream-secret",
});
});
it("redacts a literal key from downstream errors", async () => {
process.env.PAPERCLIP_HOME = "/tmp/test-drive-redaction";
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
await expect(testDriveCommand({
apiKey: "literal-secret",
browser: false,
}, {
run: async () => {
throw new Error("downstream rejected literal-secret");
},
createApi: () => freshBootstrapApi().api,
openBrowser: vi.fn(async () => true),
})).rejects.toThrow("downstream rejected [REDACTED]");
});
it("redacts a custom environment key when its option name has whitespace", async () => {
process.env.PAPERCLIP_HOME = "/tmp/test-drive-custom-env-redaction";
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
process.env.CUSTOM_TEST_DRIVE_KEY = "custom-secret";
await expect(testDriveCommand({
apiKeyEnv: " CUSTOM_TEST_DRIVE_KEY ",
browser: false,
}, {
run: async () => {
throw new Error("downstream rejected custom-secret");
},
createApi: () => freshBootstrapApi().api,
openBrowser: vi.fn(async () => true),
})).rejects.toThrow("downstream rejected [REDACTED]");
});
it("honors --no-browser after successful initialization", async () => {
process.env.PAPERCLIP_HOME = "/tmp/test-drive-no-browser";
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_IN_WORKTREE = "false";
const api = {
get: vi.fn(async <T>() => [company("existing", "Existing")] as T),
post: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
} as unknown as TestDriveApi;
const openBrowser = vi.fn(async () => true);
await testDriveCommand({ browser: false }, {
run: async (options) => options.afterStart?.(server),
createApi: () => api,
openBrowser,
});
expect(openBrowser).not.toHaveBeenCalled();
});
});

60
cli/src/__tests__/update-command.test.ts
View File

@ -2,7 +2,7 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { flipCurrentAtomic, initializeInstallStore, payloadPathFor, readInstallManifest, resolveInstallStorePaths, writeInstallManifestAtomic, type InstallManifest, type InstallRecord } from "../install-store.js";
import { writeManagedShim, flipCurrentAtomic, initializeInstallStore, payloadPathFor, readInstallManifest, resolveInstallStorePaths, writeInstallManifestAtomic, type InstallManifest, type InstallRecord } from "../install-store.js";
import type { CommandRunner } from "../commands/install.js";
import { compareVersions, detectInstallMode, resolveUpdateRequest, rollbackManagedInstall, updateCommand } from "../commands/update.js";
@ -36,6 +36,56 @@ afterEach(() => {
});
describe("update command", () => {
it.each(["npm", "git", "global-npm"] as const)("rejects %s updates on unsupported Node before any update work", async (source) => {
const paths = resolveInstallStorePaths(); initializeInstallStore(paths);
const payload = payloadPathFor(paths, "npm", "1.0.0");
const entrypoint = createPayload(payload, "1.0.0");
flipCurrentAtomic(payload, paths);
const manifest: InstallManifest = { schemaVersion: 1, ...record(payload, "1.0.0"), source: source === "git" ? "git" : "npm", previous: [] };
writeInstallManifestAtomic(manifest, paths);
const runCommand = vi.fn<CommandRunner>();
const backup = vi.fn();
const restartActiveService = vi.fn();
const nodeVersion = Object.getOwnPropertyDescriptor(process.versions, "node")!;
Object.defineProperty(process.versions, "node", { ...nodeVersion, value: "22.22.2" });
try {
await expect(updateCommand({ yes: true }, {
paths,
executablePath: source === "global-npm" ? path.join(root, "lib", "node_modules", "paperclipai", "dist", "index.js") : entrypoint,
runCommand, backup, restartActiveService,
})).rejects.toThrow("npx paperclipai@latest install --yes");
expect(runCommand).not.toHaveBeenCalled();
expect(backup).not.toHaveBeenCalled();
expect(restartActiveService).not.toHaveBeenCalled();
expect(readInstallManifest(paths)).toEqual(manifest);
expect(fs.realpathSync(paths.currentPath)).toBe(fs.realpathSync(payload));
} finally {
Object.defineProperty(process.versions, "node", nodeVersion);
}
});
it("keeps update checks, dry runs, and rollback available on unsupported Node", async () => {
const paths = resolveInstallStorePaths(); initializeInstallStore(paths);
const oldPayload = payloadPathFor(paths, "npm", "1.0.0"); createPayload(oldPayload, "1.0.0");
const payload = payloadPathFor(paths, "npm", "2.0.0"); const executablePath = createPayload(payload, "2.0.0");
flipCurrentAtomic(payload, paths);
writeInstallManifestAtomic({ schemaVersion: 1, ...record(payload, "2.0.0"), previous: [record(oldPayload, "1.0.0")] }, paths);
const runCommand = vi.fn(async () => ({ stdout: '\"3.0.0\"\n', stderr: "" }));
const restartActiveService = vi.fn(async () => false);
const nodeVersion = Object.getOwnPropertyDescriptor(process.versions, "node")!;
Object.defineProperty(process.versions, "node", { ...nodeVersion, value: "22.22.2" });
try {
await updateCommand({ check: true }, { paths, executablePath, runCommand });
await updateCommand({ dryRun: true }, { paths, executablePath, runCommand });
expect(readInstallManifest(paths)?.version).toBe("2.0.0");
await updateCommand({ rollback: true }, { paths, executablePath, restartActiveService });
expect(readInstallManifest(paths)?.version).toBe("1.0.0");
expect(restartActiveService).toHaveBeenCalledWith("1.0.0");
} finally {
Object.defineProperty(process.versions, "node", nodeVersion);
}
});
it("orders SemVer prerelease identifiers numerically", () => {
expect(compareVersions("1.0.0-canary.10", "1.0.0-canary.2")).toBeGreaterThan(0);
expect(compareVersions("1.0.0-1", "1.0.0-alpha")).toBeLessThan(0);
@ -72,12 +122,16 @@ describe("update command", () => {
fs.writeFileSync(path.join(newPayload, "node_modules", "paperclipai", "package.json"), JSON.stringify({ version: "0.3.1" }));
flipCurrentAtomic(oldPayload, paths);
writeInstallManifestAtomic({ schemaVersion: 1, source: "git", version: "0.3.1", channel: "pinned", repo: "paperclipai/paperclip", ref: "master", sha: oldSha, payloadPath: oldPayload, installedAt: "2026-07-22T00:00:00.000Z", previous: [] }, paths);
writeManagedShim(paths);
// Simulate a launcher generated before child-runtime PATH pinning existed.
fs.writeFileSync(paths.shimPath, fs.readFileSync(paths.shimPath, "utf8").replace(/^export PATH=.*\n/m, ""));
const backup = vi.fn(async () => undefined);
const confirm = vi.fn(async () => true);
const restartActiveService = vi.fn(async () => true);
const runCommand = vi.fn(async (file: string) => file === "curl" ? { stdout: JSON.stringify({ sha: newSha }), stderr: "" } : { stdout: "0.3.1\n", stderr: "" });
await updateCommand({}, { paths, executablePath: executable, runCommand, backup, confirm, restartActiveService, hasInstanceData: () => true, now: () => new Date("2026-07-22T12:00:00Z") });
expect(confirm).toHaveBeenCalledWith(expect.stringContaining(`commit ${newSha.slice(0, 12)}`));
expect(fs.readFileSync(paths.shimPath, "utf8")).toContain(`export PATH='${path.dirname(process.execPath)}'`);
expect(backup).toHaveBeenCalledOnce();
expect(restartActiveService).toHaveBeenCalledWith("0.3.1");
expect(readInstallManifest(paths)?.sha).toBe(newSha);
@ -134,6 +188,9 @@ describe("update command", () => {
const paths = resolveInstallStorePaths(); initializeInstallStore(paths);
const oldPayload = payloadPathFor(paths, "npm", "1.0.0"); const executable = createPayload(oldPayload, "1.0.0"); flipCurrentAtomic(oldPayload, paths);
writeInstallManifestAtomic({ schemaVersion: 1, ...record(oldPayload, "1.0.0"), previous: [] }, paths);
writeManagedShim(paths);
// Simulate a launcher generated before child-runtime PATH pinning existed.
fs.writeFileSync(paths.shimPath, fs.readFileSync(paths.shimPath, "utf8").replace(/^export PATH=.*\n/m, ""));
const backup = vi.fn(async () => undefined);
const restartActiveService = vi.fn(async () => true);
const runCommand = vi.fn(async (file: string, args: string[]) => {
@ -142,6 +199,7 @@ describe("update command", () => {
return { stdout: "2.0.0\n", stderr: "" };
});
await updateCommand({}, { paths, executablePath: executable, runCommand, backup, restartActiveService, hasInstanceData: () => true, now: () => new Date("2026-07-22T12:00:00Z") });
expect(fs.readFileSync(paths.shimPath, "utf8")).toContain(`export PATH='${path.dirname(process.execPath)}'`);
expect(backup).toHaveBeenCalledOnce();
expect(restartActiveService).toHaveBeenCalledWith("2.0.0");
expect(readInstallManifest(paths)?.version).toBe("2.0.0");

456
cli/src/__tests__/worktree.test.ts
View File

@ -5,7 +5,9 @@ import { execFileSync } from "node:child_process";
import { randomUUID } from "node:crypto";
import { createServer } from "node:net";
import { eq } from "drizzle-orm";
import { afterEach, describe, expect, it, vi } from "vitest";
import { drizzle } from "drizzle-orm/postgres-js";
import { migrate } from "drizzle-orm/postgres-js/migrator";
import { afterEach, describe, expect, it, onTestFinished, vi } from "vitest";
import {
agents,
authAccounts,
@ -13,6 +15,8 @@ import {
companies,
companyMemberships,
createDb,
closeRegisteredClients,
ensurePostgresDatabase,
executionWorkspaces,
inspectMigrations,
issueComments,
@ -64,6 +68,7 @@ import {
} from "../commands/worktree-lib.js";
import type { PaperclipConfig } from "../config/schema.js";
import {
EMBEDDED_POSTGRES_TEST_TIMEOUT_MS,
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
@ -71,7 +76,16 @@ import {
const ORIGINAL_CWD = process.cwd();
const ORIGINAL_ENV = { ...process.env };
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const itEmbeddedPostgres = embeddedPostgresSupport.supported ? it : it.skip;
// Every test in the embedded-Postgres cost class shares one time budget
// (see EMBEDDED_POSTGRES_TEST_TIMEOUT_MS) instead of a hand-written number
// per call site.
function itEmbeddedPostgres(name: string, fn: () => Promise<void>): void {
if (!embeddedPostgresSupport.supported) {
it.skip(name, fn);
return;
}
it(name, fn, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
}
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
function mockVerifiedSeedResult() {
@ -125,6 +139,8 @@ async function seedValidWorktreeSource(
if (options.includeCredentialAccount !== false) {
await db.insert(authAccounts).values({
id: "credential-existing",
// The issuer Better Auth stamps on an email/password account.
issuer: "local:credential",
accountId: "existing@paperclip.ing",
providerId: "credential",
userId,
@ -149,15 +165,12 @@ async function seedValidWorktreeSource(
principalId: userId,
status: "active",
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Representative seed issue",
status: "backlog",
priority: "medium",
issueNumber: 1,
identifier: "SEED-1",
});
// This helper also seeds an intentionally older schema. Current Drizzle
// insert builders include defaults for newly added columns absent there.
await db.$client`
insert into issues (id, company_id, title, status, priority, issue_number, identifier)
values (${issueId}, ${companyId}, 'Representative seed issue', 'backlog', 'medium', 1, 'SEED-1')
`;
await db.$client.end({ timeout: 5 });
return { companyId, issueId };
}
@ -591,6 +604,7 @@ describe("worktree helpers", () => {
itEmbeddedPostgres("recognizes positive legacy database schema evidence", async () => {
const tempDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-legacy-evidence-");
onTestFinished(() => tempDb.cleanup());
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-legacy-config-"));
try {
const configPath = path.join(tempRoot, "config.json");
@ -621,9 +635,8 @@ describe("worktree helpers", () => {
});
} finally {
fs.rmSync(tempRoot, { recursive: true, force: true });
await tempDb.cleanup();
}
}, 30000);
});
it("ensure-seeded seeds once and fast-exits on the verified manifest", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-ensure-seeded-"));
@ -1158,6 +1171,7 @@ describe("worktree helpers", () => {
itEmbeddedPostgres("quarantines copied live execution state in seeded worktree databases", async () => {
const tempDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-quarantine-");
onTestFinished(() => tempDb.cleanup());
const db = createDb(tempDb.connectionString);
const companyId = randomUUID();
const agentId = randomUUID();
@ -1390,9 +1404,8 @@ describe("worktree helpers", () => {
expect(runtimeService?.stoppedAt).toBeInstanceOf(Date);
} finally {
await db.$client?.end?.({ timeout: 5 }).catch(() => undefined);
await tempDb.cleanup();
}
}, 20_000);
});
it("copies the source local_encrypted secrets key into the seeded worktree instance", () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-secrets-"));
@ -1459,10 +1472,12 @@ describe("worktree helpers", () => {
const repoRoot = path.join(tempRoot, "repo");
const originalCwd = process.cwd();
const originalJwtSecret = process.env.PAPERCLIP_AGENT_JWT_SECRET;
const originalToolActionSigningSecret = process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET;
try {
fs.mkdirSync(repoRoot, { recursive: true });
process.env.PAPERCLIP_AGENT_JWT_SECRET = "worktree-shared-secret";
process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET = "worktree-tool-action-secret";
process.chdir(repoRoot);
await worktreeInitCommand({
@ -1474,6 +1489,7 @@ describe("worktree helpers", () => {
const envPath = path.join(repoRoot, ".paperclip", ".env");
const envContents = fs.readFileSync(envPath, "utf8");
expect(envContents).toContain("PAPERCLIP_AGENT_JWT_SECRET=worktree-shared-secret");
expect(envContents).toContain("PAPERCLIP_TOOL_ACTION_SIGNING_SECRET=worktree-tool-action-secret");
expect(envContents).toContain("PAPERCLIP_WORKTREE_NAME=repo");
expect(envContents).toMatch(/PAPERCLIP_WORKTREE_COLOR=\"#[0-9a-f]{6}\"/);
} finally {
@ -1483,6 +1499,11 @@ describe("worktree helpers", () => {
} else {
process.env.PAPERCLIP_AGENT_JWT_SECRET = originalJwtSecret;
}
if (originalToolActionSigningSecret === undefined) {
delete process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET;
} else {
process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET = originalToolActionSigningSecret;
}
fs.rmSync(tempRoot, { recursive: true, force: true });
}
});
@ -1530,93 +1551,93 @@ describe("worktree helpers", () => {
"seeds a local-trusted implicit board user without a credential account",
async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-local-board-seed-"));
const originalCwd = process.cwd();
onTestFinished(() => {
process.chdir(originalCwd);
fs.rmSync(tempRoot, { recursive: true, force: true });
});
const worktreeRoot = path.join(tempRoot, "PAP-17696-local-board-seed");
const sourceConfigDir = path.join(tempRoot, "source");
const sourceConfigPath = path.join(sourceConfigDir, "config.json");
const sourceKeyPath = path.join(sourceConfigDir, "secrets", "master.key");
const worktreeHome = path.join(tempRoot, ".paperclip-worktrees");
const originalCwd = process.cwd();
const sourceDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-local-board-source-");
onTestFinished(() => sourceDb.cleanup());
try {
await seedValidWorktreeSource(sourceDb.connectionString, {
includeCredentialAccount: false,
userId: "local-board",
});
fs.mkdirSync(path.dirname(sourceKeyPath), { recursive: true });
fs.mkdirSync(worktreeRoot, { recursive: true });
await seedValidWorktreeSource(sourceDb.connectionString, {
includeCredentialAccount: false,
userId: "local-board",
});
fs.mkdirSync(path.dirname(sourceKeyPath), { recursive: true });
fs.mkdirSync(worktreeRoot, { recursive: true });
const sourceConfig = buildSourceConfig();
sourceConfig.database = {
...sourceConfig.database,
mode: "postgres",
connectionString: sourceDb.connectionString,
};
sourceConfig.server.deploymentMode = "local_trusted";
sourceConfig.server.exposure = "private";
sourceConfig.auth.baseUrlMode = "auto";
delete sourceConfig.auth.publicBaseUrl;
sourceConfig.secrets.localEncrypted.keyFilePath = sourceKeyPath;
const sourceConfig = buildSourceConfig();
sourceConfig.database = {
...sourceConfig.database,
mode: "postgres",
connectionString: sourceDb.connectionString,
};
sourceConfig.server.deploymentMode = "local_trusted";
sourceConfig.server.exposure = "private";
sourceConfig.auth.baseUrlMode = "auto";
delete sourceConfig.auth.publicBaseUrl;
sourceConfig.secrets.localEncrypted.keyFilePath = sourceKeyPath;
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig, null, 2)}\n`, "utf8");
fs.writeFileSync(sourceKeyPath, "source-master-key", "utf8");
fs.writeFileSync(sourceConfigPath, `${JSON.stringify(sourceConfig, null, 2)}\n`, "utf8");
fs.writeFileSync(sourceKeyPath, "source-master-key", "utf8");
process.chdir(worktreeRoot);
await worktreeInitCommand({
name: "PAP-17696-local-board-seed",
home: worktreeHome,
fromConfig: sourceConfigPath,
force: true,
});
process.chdir(worktreeRoot);
await worktreeInitCommand({
name: "PAP-17696-local-board-seed",
home: worktreeHome,
fromConfig: sourceConfigPath,
force: true,
});
const targetConfigPath = path.join(worktreeRoot, ".paperclip", "config.json");
const targetConfig = JSON.parse(fs.readFileSync(targetConfigPath, "utf8")) as PaperclipConfig;
expect(readWorktreeSeedManifest(targetConfigPath)).toMatchObject({
state: "verified",
phase: "complete",
});
const targetConfigPath = path.join(worktreeRoot, ".paperclip", "config.json");
const targetConfig = JSON.parse(fs.readFileSync(targetConfigPath, "utf8")) as PaperclipConfig;
expect(readWorktreeSeedManifest(targetConfigPath)).toMatchObject({
state: "verified",
phase: "complete",
});
const { default: EmbeddedPostgres } = await import("embedded-postgres");
const targetPg = new EmbeddedPostgres({
databaseDir: targetConfig.database.embeddedPostgresDataDir,
user: "paperclip",
password: "paperclip",
port: targetConfig.database.embeddedPostgresPort,
persistent: true,
initdbFlags: ["--encoding=UTF8", "--locale=C", "--lc-messages=C"],
onLog: () => {},
onError: () => {},
});
const { default: EmbeddedPostgres } = await import("embedded-postgres");
const targetPg = new EmbeddedPostgres({
databaseDir: targetConfig.database.embeddedPostgresDataDir,
user: "paperclip",
password: "paperclip",
port: targetConfig.database.embeddedPostgresPort,
persistent: true,
initdbFlags: ["--encoding=UTF8", "--locale=C", "--lc-messages=C"],
onLog: () => {},
onError: () => {},
});
await targetPg.start();
try {
const targetDb = createDb(
`postgres://paperclip:paperclip@127.0.0.1:${targetConfig.database.embeddedPostgresPort}/paperclip`,
);
const [seededLocalBoard] = await targetDb
.select({ id: authUsers.id })
.from(authUsers)
.where(eq(authUsers.id, "local-board"));
const seededAccounts = await targetDb.select().from(authAccounts);
expect(seededLocalBoard?.id).toBe("local-board");
expect(seededAccounts).toHaveLength(0);
await targetDb.$client.end({ timeout: 5 });
} finally {
await targetPg.stop();
}
} finally {
process.chdir(originalCwd);
await sourceDb.cleanup();
fs.rmSync(tempRoot, { recursive: true, force: true });
}
await targetPg.start();
onTestFinished(() => targetPg.stop());
const targetDb = createDb(
`postgres://paperclip:paperclip@127.0.0.1:${targetConfig.database.embeddedPostgresPort}/paperclip`,
);
const [seededLocalBoard] = await targetDb
.select({ id: authUsers.id })
.from(authUsers)
.where(eq(authUsers.id, "local-board"));
const seededAccounts = await targetDb.select().from(authAccounts);
expect(seededLocalBoard?.id).toBe("local-board");
expect(seededAccounts).toHaveLength(0);
await targetDb.$client.end({ timeout: 5 });
},
30_000,
);
itEmbeddedPostgres(
"seeds a lagging source whose migration application order differs from filename order",
async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-auth-seed-"));
const originalCwd = process.cwd();
onTestFinished(() => {
process.chdir(originalCwd);
fs.rmSync(tempRoot, { recursive: true, force: true });
});
const worktreeRoot = path.join(tempRoot, "PAP-999-auth-seed");
const sourceHome = path.join(tempRoot, "source-home");
const sourceConfigDir = path.join(sourceHome, "instances", "source");
@ -1624,139 +1645,144 @@ describe("worktree helpers", () => {
const sourceEnvPath = path.join(sourceConfigDir, ".env");
const sourceKeyPath = path.join(sourceConfigDir, "secrets", "master.key");
const worktreeHome = path.join(tempRoot, ".paperclip-worktrees");
const originalCwd = process.cwd();
const sourceDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-auth-source-");
try {
await seedValidWorktreeSource(sourceDb.connectionString);
const sourceDbClient = createDb(sourceDb.connectionString);
await sourceDbClient.$client.unsafe(`
DELETE FROM "drizzle"."__drizzle_migrations"
WHERE "id" = (
SELECT max("id") FROM "drizzle"."__drizzle_migrations"
);
WITH pair AS (
SELECT
array_agg("id" ORDER BY "id" DESC) AS ids,
array_agg("hash" ORDER BY "id" DESC) AS hashes
FROM (
SELECT "id", "hash"
FROM "drizzle"."__drizzle_migrations"
ORDER BY "id" DESC
LIMIT 2
) latest
)
UPDATE "drizzle"."__drizzle_migrations" migrations
SET "hash" = CASE
WHEN migrations."id" = pair.ids[1] THEN pair.hashes[2]
WHEN migrations."id" = pair.ids[2] THEN pair.hashes[1]
ELSE migrations."hash"
END
FROM pair
WHERE migrations."id" IN (pair.ids[1], pair.ids[2]);
INSERT INTO "drizzle"."__drizzle_migrations" ("hash", "created_at")
VALUES ('stale-unresolvable-migration-hash', 0)
`);
await sourceDbClient.$client.end({ timeout: 5 });
const laggingMigrationState = await inspectMigrations(sourceDb.connectionString);
expect(laggingMigrationState.status).toBe("needsMigrations");
if (laggingMigrationState.status !== "needsMigrations") {
throw new Error("Expected the source migration journal to lag the code journal");
}
expect(laggingMigrationState.pendingMigrations).toHaveLength(1);
const expectedAppliedPrefix = laggingMigrationState.availableMigrations.slice(
0,
laggingMigrationState.appliedMigrations.length,
);
expect(laggingMigrationState.appliedMigrations).not.toEqual(expectedAppliedPrefix);
expect([...laggingMigrationState.appliedMigrations].sort()).toEqual(
[...expectedAppliedPrefix].sort(),
);
expect(laggingMigrationState.journalEntryCount).toBeGreaterThan(
laggingMigrationState.appliedMigrations.length,
);
const sourceMigrationRevision = expectedAppliedPrefix.at(-1);
expect(sourceMigrationRevision).toBeTruthy();
fs.mkdirSync(path.dirname(sourceKeyPath), { recursive: true });
fs.mkdirSync(worktreeRoot, { recursive: true });
const sourceConfig = buildSourceConfig();
sourceConfig.database = {
mode: "postgres",
embeddedPostgresDataDir: path.join(sourceConfigDir, "db"),
embeddedPostgresPort: 54329,
backup: {
enabled: true,
intervalMinutes: 60,
retentionDays: 30,
dir: path.join(sourceConfigDir, "backups"),
},
connectionString: sourceDb.connectionString,
};
sourceConfig.logging.logDir = path.join(sourceConfigDir, "logs");
sourceConfig.storage.localDisk.baseDir = path.join(sourceConfigDir, "storage");
sourceConfig.secrets.localEncrypted.keyFilePath = sourceKeyPath;
fs.writeFileSync(sourceConfigPath, JSON.stringify(sourceConfig, null, 2) + "\n", "utf8");
fs.writeFileSync(sourceEnvPath, "", "utf8");
fs.writeFileSync(sourceKeyPath, "source-master-key", "utf8");
process.chdir(worktreeRoot);
await worktreeInitCommand({
name: "PAP-999-auth-seed",
home: worktreeHome,
fromConfig: sourceConfigPath,
force: true,
});
const targetConfig = JSON.parse(
fs.readFileSync(path.join(worktreeRoot, ".paperclip", "config.json"), "utf8"),
) as PaperclipConfig;
const manifestText = fs.readFileSync(
path.join(worktreeRoot, ".paperclip", "seed-manifest.json"),
"utf8",
);
expect(JSON.parse(manifestText)).toMatchObject({
version: 2,
seedMode: "minimal",
state: "verified",
phase: "complete",
});
expect(manifestText).toContain(`Validated migration ${sourceMigrationRevision}`);
expect(manifestText).not.toContain("fixture-password-hash");
expect(manifestText).not.toContain("source-master-key");
const { default: EmbeddedPostgres } = await import("embedded-postgres");
const targetPg = new EmbeddedPostgres({
databaseDir: targetConfig.database.embeddedPostgresDataDir,
user: "paperclip",
password: "paperclip",
port: targetConfig.database.embeddedPostgresPort,
persistent: true,
initdbFlags: ["--encoding=UTF8", "--locale=C", "--lc-messages=C"],
onLog: () => {},
onError: () => {},
});
await targetPg.start();
try {
const targetDb = createDb(
`postgres://paperclip:paperclip@127.0.0.1:${targetConfig.database.embeddedPostgresPort}/paperclip`,
);
const seededUsers = await targetDb.select().from(authUsers);
expect(seededUsers.some((row) => row.email === "existing@paperclip.ing")).toBe(true);
} finally {
await targetPg.stop();
}
} finally {
process.chdir(originalCwd);
await sourceDb.cleanup();
fs.rmSync(tempRoot, { recursive: true, force: true });
const sourceCluster = await startEmbeddedPostgresTestDatabase("paperclip-worktree-auth-source-");
const sourceUrl = new URL(sourceCluster.connectionString);
sourceUrl.pathname = "/lagging_source";
const sourceDb = { connectionString: sourceUrl.toString() };
onTestFinished(async () => {
await closeRegisteredClients(sourceDb.connectionString);
await sourceCluster.cleanup();
});
await ensurePostgresDatabase(sourceCluster.connectionString, "lagging_source");
// A lagging source must also have the prior schema. Deleting only the
// newest receipt from a fully migrated schema relied on that particular
// migration being idempotent and breaks when the new migration creates a
// table. Build the actual all-but-last schema before shuffling its history.
const migrationsRoot = new URL("../../../packages/db/src/migrations/", import.meta.url);
const journal = JSON.parse(fs.readFileSync(new URL("meta/_journal.json", migrationsRoot), "utf8"));
const priorEntries = journal.entries.slice(0, -1);
const priorMigrations = path.join(tempRoot, "prior-migrations");
fs.mkdirSync(path.join(priorMigrations, "meta"), { recursive: true });
fs.writeFileSync(path.join(priorMigrations, "meta", "_journal.json"), JSON.stringify({ ...journal, entries: priorEntries }));
for (const entry of priorEntries) {
fs.copyFileSync(new URL(`${entry.tag}.sql`, migrationsRoot), path.join(priorMigrations, `${entry.tag}.sql`));
}
const sourceDbClient = createDb(sourceDb.connectionString);
await migrate(drizzle(sourceDbClient.$client), { migrationsFolder: priorMigrations });
await seedValidWorktreeSource(sourceDb.connectionString);
await sourceDbClient.$client.unsafe(`
WITH pair AS (
SELECT
array_agg("id" ORDER BY "id" DESC) AS ids,
array_agg("hash" ORDER BY "id" DESC) AS hashes
FROM (
SELECT "id", "hash"
FROM "drizzle"."__drizzle_migrations"
ORDER BY "id" DESC
LIMIT 2
) latest
)
UPDATE "drizzle"."__drizzle_migrations" migrations
SET "hash" = CASE
WHEN migrations."id" = pair.ids[1] THEN pair.hashes[2]
WHEN migrations."id" = pair.ids[2] THEN pair.hashes[1]
ELSE migrations."hash"
END
FROM pair
WHERE migrations."id" IN (pair.ids[1], pair.ids[2]);
INSERT INTO "drizzle"."__drizzle_migrations" ("hash", "created_at")
VALUES ('stale-unresolvable-migration-hash', 0)
`);
await sourceDbClient.$client.end({ timeout: 5 });
const laggingMigrationState = await inspectMigrations(sourceDb.connectionString);
expect(laggingMigrationState.status).toBe("needsMigrations");
if (laggingMigrationState.status !== "needsMigrations") {
throw new Error("Expected the source migration journal to lag the code journal");
}
expect(laggingMigrationState.pendingMigrations).toHaveLength(1);
const expectedAppliedPrefix = laggingMigrationState.availableMigrations.slice(
0,
laggingMigrationState.appliedMigrations.length,
);
expect(laggingMigrationState.appliedMigrations).not.toEqual(expectedAppliedPrefix);
expect([...laggingMigrationState.appliedMigrations].sort()).toEqual(
[...expectedAppliedPrefix].sort(),
);
expect(laggingMigrationState.journalEntryCount).toBeGreaterThan(
laggingMigrationState.appliedMigrations.length,
);
const sourceMigrationRevision = expectedAppliedPrefix.at(-1);
expect(sourceMigrationRevision).toBeTruthy();
fs.mkdirSync(path.dirname(sourceKeyPath), { recursive: true });
fs.mkdirSync(worktreeRoot, { recursive: true });
const sourceConfig = buildSourceConfig();
sourceConfig.database = {
mode: "postgres",
embeddedPostgresDataDir: path.join(sourceConfigDir, "db"),
embeddedPostgresPort: 54329,
backup: {
enabled: true,
intervalMinutes: 60,
retentionDays: 30,
dir: path.join(sourceConfigDir, "backups"),
},
connectionString: sourceDb.connectionString,
};
sourceConfig.logging.logDir = path.join(sourceConfigDir, "logs");
sourceConfig.storage.localDisk.baseDir = path.join(sourceConfigDir, "storage");
sourceConfig.secrets.localEncrypted.keyFilePath = sourceKeyPath;
fs.writeFileSync(sourceConfigPath, JSON.stringify(sourceConfig, null, 2) + "\n", "utf8");
fs.writeFileSync(sourceEnvPath, "", "utf8");
fs.writeFileSync(sourceKeyPath, "source-master-key", "utf8");
process.chdir(worktreeRoot);
await worktreeInitCommand({
name: "PAP-999-auth-seed",
home: worktreeHome,
fromConfig: sourceConfigPath,
force: true,
});
const targetConfig = JSON.parse(
fs.readFileSync(path.join(worktreeRoot, ".paperclip", "config.json"), "utf8"),
) as PaperclipConfig;
const manifestText = fs.readFileSync(
path.join(worktreeRoot, ".paperclip", "seed-manifest.json"),
"utf8",
);
expect(JSON.parse(manifestText)).toMatchObject({
version: 2,
seedMode: "minimal",
state: "verified",
phase: "complete",
});
expect(manifestText).toContain(`Validated migration ${sourceMigrationRevision}`);
expect(manifestText).not.toContain("fixture-password-hash");
expect(manifestText).not.toContain("source-master-key");
const { default: EmbeddedPostgres } = await import("embedded-postgres");
const targetPg = new EmbeddedPostgres({
databaseDir: targetConfig.database.embeddedPostgresDataDir,
user: "paperclip",
password: "paperclip",
port: targetConfig.database.embeddedPostgresPort,
persistent: true,
initdbFlags: ["--encoding=UTF8", "--locale=C", "--lc-messages=C"],
onLog: () => {},
onError: () => {},
});
await targetPg.start();
onTestFinished(() => targetPg.stop());
const targetDb = createDb(
`postgres://paperclip:paperclip@127.0.0.1:${targetConfig.database.embeddedPostgresPort}/paperclip`,
);
const seededUsers = await targetDb.select().from(authUsers);
expect(seededUsers.some((row) => row.email === "existing@paperclip.ing")).toBe(true);
},
30000,
);
it("avoids ports already claimed by sibling worktree instance configs", async () => {
@ -2041,6 +2067,7 @@ describe("worktree helpers", () => {
const currentDatabaseReservation = await reserveTestPort();
const currentDatabasePort = currentDatabaseReservation.port;
const sourceDb = await startEmbeddedPostgresTestDatabase("paperclip-worktree-reseed-source-");
onTestFinished(() => sourceDb.cleanup());
try {
fs.mkdirSync(path.dirname(currentPaths.configPath), { recursive: true });
@ -2114,7 +2141,6 @@ describe("worktree helpers", () => {
).toBe(true);
} finally {
await currentDatabaseReservation.release();
await sourceDb.cleanup();
process.chdir(originalCwd);
if (originalPaperclipConfig === undefined) {
delete process.env.PAPERCLIP_CONFIG;
@ -2123,7 +2149,7 @@ describe("worktree helpers", () => {
}
fs.rmSync(tempRoot, { recursive: true, force: true });
}
}, 30_000);
});
it("restores the current worktree config and instance data if reseed fails", async () => {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-worktree-reseed-rollback-"));

58
cli/src/checks/service-health-check.ts
View File

@ -1,9 +1,12 @@
import fs from "node:fs/promises";
import path from "node:path";
import type { PaperclipConfig } from "../config/schema.js";
import { resolvePaperclipInstanceId } from "../config/home.js";
import { readInstallManifest } from "../install-store.js";
import { readInstallManifest, resolveInstallStorePaths } from "../install-store.js";
import {
detectServiceManager,
isExecutableFile,
resolveServiceShimPath,
type ServiceManagerDetection,
} from "../services/service-manager.js";
import { buildLocalHealthUrl } from "../utils/health-url.js";
@ -13,6 +16,7 @@ type HealthResult = { ok: boolean; version: string | null; error?: string };
type ServiceCheckDependencies = {
detect: (instanceId: string) => Promise<ServiceManagerDetection>;
probe: (config: PaperclipConfig) => Promise<HealthResult>;
shimPresent: (executablePath: string) => Promise<boolean>;
};
async function probeHealth(config: PaperclipConfig): Promise<HealthResult> {
@ -45,6 +49,7 @@ export async function serviceHealthChecks(
const deps: ServiceCheckDependencies = {
detect: (instanceId) => detectServiceManager({ instanceId }),
probe: probeHealth,
shimPresent: (executablePath) => isExecutableFile(executablePath),
...dependencies,
};
const instanceId = resolvePaperclipInstanceId();
@ -84,17 +89,36 @@ export async function serviceHealthChecks(
);
const health = await deps.probe(config);
// The installed definition is the truth about what the service executes;
// fall back to the environment-derived path only when it is unreadable.
const serviceExecutable = (await manager.installedExecutablePath()) ?? resolveServiceShimPath();
const shimPresent = status.active ? true : await deps.shimPresent(serviceExecutable);
results.push(
status.active
? { name: "Service runtime", status: "pass", message: `${status.serviceName} is active` }
: {
name: "Service runtime",
status: "fail",
message: health.ok
? `${status.serviceName} is inactive but the configured port is serving another Paperclip process`
: `${status.serviceName} is ${status.detail ?? "inactive"}`,
repairHint: "Run `paperclipai service start`, or stop the conflicting foreground process first",
},
: !shimPresent
? {
name: "Service runtime",
status: "fail",
message: `${status.serviceName} cannot start: no executable exists at ${serviceExecutable}`,
repairHint:
path.resolve(serviceExecutable) === path.resolve(resolveInstallStorePaths().shimPath)
? "Run `paperclipai install` to restore the managed payload and shim, then `paperclipai service start`"
: `Restore the executable at ${serviceExecutable}, or unset PAPERCLIP_SHIM_PATH and run \`paperclipai install\` followed by \`paperclipai service install\` to re-point the service at the managed shim`,
}
: health.ok
? {
name: "Service runtime",
status: "fail",
message: `${status.serviceName} is inactive but the configured port is serving another Paperclip process`,
repairHint: "Run `paperclipai service start`, or stop the conflicting foreground process first",
}
: {
name: "Service runtime",
status: "fail",
message: `${status.serviceName} is ${status.detail ?? "inactive"}`,
repairHint: "Run `paperclipai service start`; inspect `paperclipai service logs` if it does not stay up",
},
);
let expectedVersion: string | null = null;
@ -116,11 +140,17 @@ export async function serviceHealthChecks(
message: `Running ${health.version ?? "unknown"}; managed install is ${expectedVersion}`,
repairHint: "Run `paperclipai service restart --expected-version " + expectedVersion + "`",
}
: {
name: "Service health",
status: "pass",
message: `Healthy${health.version ? ` at version ${health.version}` : ""}`,
},
: status.active
? {
name: "Service health",
status: "pass",
message: `Healthy${health.version ? ` at version ${health.version}` : ""}`,
}
: {
name: "Service health",
status: "warn",
message: `The configured port answers healthy${health.version ? ` (version ${health.version})` : ""}, but not from ${status.serviceName} — the service is inactive`,
},
);
if (status.enabled && status.linger === false) {

1
cli/src/commands/client/adapter.ts
View File

@ -122,7 +122,6 @@ export function registerAdapterCommands(program: Command): void {
{ includeCompany: false },
);
addCompanyAdapterGet(adapter, "model-profiles", "List adapter model profiles", "model-profiles");
addCompanyAdapterGet(adapter, "detect-model", "Detect adapter model", "detect-model");
addCompanyAdapterPost(adapter, "test-environment", "Test adapter environment configuration", "test-environment");
}

7
cli/src/commands/client/company.ts
View File

@ -14,6 +14,7 @@ import type {
CompanyPortabilityImportResult,
} from "@paperclipai/shared";
import {
buildAlreadyImportedMessage,
companyImportTransferApplyPath,
companyImportTransferPartPath,
companyImportTransferPreviewPath,
@ -1160,9 +1161,9 @@ export async function uploadCompanyImportTransfer(
if (created.alreadyCompleted) {
// The server keys transfers by content, and this exact zip already
// finished an apply — its spooled parts are gone, so it cannot re-run.
throw new Error(
"This exact package was already imported by a completed transfer. Re-export the package to import it again.",
);
// Name the company that apply created so the rejection points at the
// existing import instead of reading as data loss.
throw new Error(buildAlreadyImportedMessage(created.company));
}
const missing = new Set(created.missingParts);
let uploadedParts = manifest.parts.length - missing.size;

74
cli/src/commands/client/connections.ts Normal file
View File

@ -0,0 +1,74 @@
import { Command } from "commander";
import {
CONNECTION_INTENT_AGENT_GUIDANCE,
connectionRequestInputSchema,
connectionsSearchInputSchema,
} from "@paperclipai/shared";
interface RuntimeConnectionOptions {
json?: boolean;
}
async function callRuntimeConnectionTool(
endpointEnv: "PAPERCLIP_RUNTIME_TOOLS_CONNECTIONS_SEARCH_URL" | "PAPERCLIP_RUNTIME_TOOLS_CONNECTION_REQUEST_URL",
body: unknown,
) {
const endpoint = process.env[endpointEnv]?.trim();
const token = process.env.PAPERCLIP_RUNTIME_TOOLS_TOKEN?.trim();
if (!endpoint || !token) {
throw new Error("This command requires the runtime connection environment from an active heartbeat run");
}
const response = await fetch(endpoint, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
Accept: "application/json",
},
body: JSON.stringify(body),
});
const text = await response.text();
const parsed = text ? JSON.parse(text) as unknown : null;
if (!response.ok) {
const message = parsed && typeof parsed === "object" && "error" in parsed
? String((parsed as { error: unknown }).error)
: `Runtime connection request failed with ${response.status}`;
throw new Error(message);
}
return parsed;
}
function writeResult(value: unknown, options: RuntimeConnectionOptions) {
process.stdout.write(`${JSON.stringify(value, null, options.json ? 2 : 0)}\n`);
}
export function registerConnectionIntentCommands(program: Command) {
const connections = program
.command("connections")
.description("Search or request connections from an active heartbeat run")
.addHelpText("after", `\n${CONNECTION_INTENT_AGENT_GUIDANCE}\n`);
connections
.command("search")
.argument("[query]", "Service name or capability")
.option("--json", "Print formatted JSON")
.action(async (query: string | undefined, options: RuntimeConnectionOptions) => {
const input = connectionsSearchInputSchema.parse({ query: query ?? "" });
writeResult(await callRuntimeConnectionTool(
"PAPERCLIP_RUNTIME_TOOLS_CONNECTIONS_SEARCH_URL",
input,
), options);
});
connections
.command("request")
.argument("<service>", "Connectable service slug")
.option("--json", "Print formatted JSON")
.action(async (service: string, options: RuntimeConnectionOptions) => {
const input = connectionRequestInputSchema.parse({ service });
writeResult(await callRuntimeConnectionTool(
"PAPERCLIP_RUNTIME_TOOLS_CONNECTION_REQUEST_URL",
input,
), options);
});
}

76
cli/src/commands/client/email.ts Normal file
View File

@ -0,0 +1,76 @@
import { readFile } from "node:fs/promises";
import { Command } from "commander";
import { emailSendSchema } from "@paperclipai/shared";
import {
addCommonClientOptions,
resolveCommandContext,
printOutput,
type BaseClientOptions,
} from "./common.js";
export function registerEmailCommands(program: Command) {
const email = program
.command("email")
.description(
"Explicitly send and inspect task-bound AgentMail conversations",
);
addCommonClientOptions(email.command("inboxes"), {
includeCompany: true,
}).action(async (opts: BaseClientOptions) => {
const ctx = resolveCommandContext(opts, { requireCompany: true });
printOutput(
await ctx.api.get(`/api/companies/${ctx.companyId}/email/inboxes`),
{ json: true },
);
});
for (const verb of ["send", "reply"] as const) {
addCommonClientOptions(
email
.command(verb)
.requiredOption(
"--file <path>",
"JSON request file, including a stable idempotencyKey",
),
{ includeCompany: true },
).action(async (opts: BaseClientOptions & { file: string }) => {
const ctx = resolveCommandContext(opts, { requireCompany: true });
const input = emailSendSchema.parse(
JSON.parse(await readFile(opts.file, "utf8")),
);
if ((verb === "reply") !== Boolean(input.conversationId))
throw new Error(
`${verb} requires ${verb === "reply" ? "an existing conversation" : "a parent task and a new conversation"}`,
);
printOutput(
await ctx.api.post(`/api/companies/${ctx.companyId}/email/send`, input),
{ json: true },
);
});
}
addCommonClientOptions(
email.command("thread").argument("<issueId>", "Email task ID"),
{ includeCompany: true },
).action(async (issueId: string, opts: BaseClientOptions) => {
const ctx = resolveCommandContext(opts, { requireCompany: true });
printOutput(
await ctx.api.get(
`/api/companies/${ctx.companyId}/email/tasks/${encodeURIComponent(issueId)}`,
),
{ json: true },
);
});
addCommonClientOptions(
email
.command("delivery")
.argument("<publicationId>", "Publication ID returned by send"),
{ includeCompany: true },
).action(async (publicationId: string, opts: BaseClientOptions) => {
const ctx = resolveCommandContext(opts, { requireCompany: true });
printOutput(
await ctx.api.get(
`/api/companies/${ctx.companyId}/email/deliveries/${encodeURIComponent(publicationId)}`,
),
{ json: true },
);
});
}

6
cli/src/commands/client/issue.ts
View File

@ -80,6 +80,7 @@ interface IssueUpdateOptions extends BaseClientOptions {
interface IssueCommentOptions extends BaseClientOptions {
body: string;
attachmentId?: string[];
reopen?: boolean;
resume?: boolean;
}
@ -361,6 +362,10 @@ export function registerIssueCommands(program: Command): void {
.description("Add comment to issue")
.argument("<issueId>", "Issue ID")
.requiredOption("--body <text>", "Comment body")
.option(
"--attachment-id <id...>",
"Bind uploaded issue attachments to this comment",
)
.option("--reopen", "Reopen if issue is done/cancelled")
.option("--resume", "Request explicit follow-up and wake the assignee when resumable")
.action(async (issueId: string, opts: IssueCommentOptions) => {
@ -368,6 +373,7 @@ export function registerIssueCommands(program: Command): void {
const ctx = resolveCommandContext(opts);
const payload = addIssueCommentSchema.parse({
body: opts.body,
attachmentIds: opts.attachmentId,
reopen: opts.reopen,
resume: opts.resume,
});

51
cli/src/commands/git-workspace.ts Normal file
View File

@ -0,0 +1,51 @@
import path from "node:path";
import { execFileSync } from "node:child_process";
export type GitWorkspaceInfo = {
root: string;
commonDir: string;
gitDir: string;
hooksPath: string;
};
/**
* Resolve the repository metadata Git exposes for both primary checkouts and
* linked worktrees. Returns null outside a Git working tree.
*/
export function detectGitWorkspaceInfo(cwd: string): GitWorkspaceInfo | null {
try {
const root = execFileSync("git", ["rev-parse", "--show-toplevel"], {
cwd,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const commonDirRaw = execFileSync("git", ["rev-parse", "--git-common-dir"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const gitDirRaw = execFileSync("git", ["rev-parse", "--git-dir"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const hooksPathRaw = execFileSync("git", ["rev-parse", "--git-path", "hooks"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
return {
root: path.resolve(root),
commonDir: path.resolve(root, commonDirRaw),
gitDir: path.resolve(root, gitDirRaw),
hooksPath: path.resolve(root, hooksPathRaw),
};
} catch {
return null;
}
}
export function isLinkedGitWorktree(cwd: string): boolean {
const workspace = detectGitWorkspaceInfo(cwd);
return Boolean(workspace && workspace.gitDir !== workspace.commonDir);
}

4
cli/src/commands/install.ts
View File

@ -84,9 +84,9 @@ export function resolveGitInstallWorkspacePackages(checkoutPath: string): Releas
return ordered;
}
function assertSupportedNodeVersion(): void {
export function assertSupportedNodeVersion(): void {
if (!isSupportedNodeVersion(process.versions.node)) {
throw new Error(`Managed installs require Node.js ${MINIMUM_NODE_VERSION} or newer (found ${process.version}).`);
throw new Error(`Installing or updating Paperclip requires Node.js ${MINIMUM_NODE_VERSION} or newer (found ${process.version} at ${process.execPath}). Put a supported Node bin directory first on PATH and run 'npx paperclipai@latest install --yes' to re-pin an existing managed install.`);
}
}

433
cli/src/commands/managed-agent.ts Normal file
View File

@ -0,0 +1,433 @@
import { Command } from "commander";
import {
addCommonClientOptions,
apiPath,
handleCommandError,
printOutput,
resolveCommandContext,
type BaseClientOptions,
} from "./client/common.js";
const ANTHROPIC_ORIGIN = "https://api.anthropic.com";
const ANTHROPIC_VERSION = "2023-06-01";
export const CLAUDE_MANAGED_BETA_VERSION = "managed-agents-2026-04-01" as const;
export const CLAUDE_MANAGED_QUALIFIED_MODEL = "claude-sonnet-5" as const;
export const CLAUDE_MANAGED_SYSTEM_PROMPT =
"You are a Paperclip remote agent. Follow the current user turn and use only the custom tools supplied for that session. Paperclip tool authority, completion, blocking, review, and yielding are enforced by the runner. Never request or infer a Paperclip endpoint or credential.";
export interface ManagedAgentSetupOptions extends BaseClientOptions {
companyId?: string;
profileKey: string;
displayName: string;
apiKeySecretId: string;
model: string;
maxSessionListCostUsd: string;
agentId?: string;
agentVersion?: string;
environmentId?: string;
probe?: boolean;
acknowledgeRetention?: boolean;
}
interface RemoteResource {
id?: string;
[key: string]: unknown;
}
interface ValidatedSetup {
anthropicApiKey: string;
profileKey: string;
displayName: string;
apiKeySecretId: string;
model: string;
agentId?: string;
agentVersion?: string;
environmentId?: string;
defaultMaxListCostUsd: number;
}
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
function record(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
function required(value: string | undefined, label: string): string {
const normalized = value?.trim() ?? "";
if (!normalized) throw new Error(`${label} is required`);
return normalized;
}
export function validateManagedAgentSetup(
options: ManagedAgentSetupOptions,
env: NodeJS.ProcessEnv = process.env,
): ValidatedSetup {
const anthropicApiKey = env.ANTHROPIC_API_KEY?.trim();
if (!anthropicApiKey) {
throw new Error("ANTHROPIC_API_KEY is required in the CLI process environment");
}
if (!options.acknowledgeRetention) {
throw new Error(
"Pass --acknowledge-retention to enable the stateful beta Managed Agents service",
);
}
const profileKey = required(options.profileKey, "--profile-key");
const displayName = required(options.displayName, "--display-name");
const apiKeySecretId = required(options.apiKeySecretId, "--api-key-secret-id");
const model = required(options.model, "--model");
if (model !== CLAUDE_MANAGED_QUALIFIED_MODEL) {
throw new Error(
`--model must be the qualified Managed Agents model ${CLAUDE_MANAGED_QUALIFIED_MODEL}`,
);
}
if (!UUID_RE.test(apiKeySecretId)) {
throw new Error("--api-key-secret-id must be a UUID");
}
const defaultMaxListCostUsd = Number(options.maxSessionListCostUsd);
const cents = Math.round(defaultMaxListCostUsd * 100);
if (
!Number.isFinite(defaultMaxListCostUsd)
|| defaultMaxListCostUsd <= 0
|| !Number.isSafeInteger(cents)
|| cents <= 0
) {
throw new Error("--max-session-list-cost-usd must resolve to at least one cent");
}
return {
anthropicApiKey,
profileKey,
displayName,
apiKeySecretId,
model,
agentId: options.agentId?.trim() || undefined,
agentVersion: options.agentVersion?.trim() || undefined,
environmentId: options.environmentId?.trim() || undefined,
defaultMaxListCostUsd,
};
}
async function anthropicRequest(
key: string,
method: "GET" | "POST",
path: string,
body?: Record<string, unknown>,
): Promise<Record<string, unknown>> {
const response = await fetch(`${ANTHROPIC_ORIGIN}${path}`, {
method,
headers: {
"x-api-key": key,
"anthropic-version": ANTHROPIC_VERSION,
"anthropic-beta": CLAUDE_MANAGED_BETA_VERSION,
...(body ? { "content-type": "application/json" } : {}),
},
...(body ? { body: JSON.stringify(body) } : {}),
signal: AbortSignal.timeout(15_000),
});
if (!response.ok) {
throw new Error(`Anthropic Managed Agents request failed with HTTP ${response.status}`);
}
if (response.status === 204) return {};
return record(await response.json());
}
async function listAll(key: string, path: string): Promise<RemoteResource[]> {
const rows: RemoteResource[] = [];
let page: string | null = null;
do {
const suffix = page ? `${path.includes("?") ? "&" : "?"}page=${encodeURIComponent(page)}` : "";
const response = await anthropicRequest(key, "GET", `${path}${suffix}`);
for (const value of Array.isArray(response.data) ? response.data : []) {
rows.push(record(value) as RemoteResource);
}
page = typeof response.next_page === "string" && response.next_page
? response.next_page
: null;
} while (page);
return rows;
}
function resourceByProfile(
resources: RemoteResource[],
profileKey: string,
resourceLabel: string,
): RemoteResource | null {
const matches = resources.filter(
(resource) =>
typeof resource.id === "string"
&& record(resource.metadata).paperclip_profile === profileKey,
);
if (matches.length > 1) {
throw new Error(
`Multiple Anthropic ${resourceLabel} resources use Paperclip profile ${profileKey}; pass an explicit resource ID`,
);
}
return matches[0] ?? null;
}
export function assertSafeManagedEnvironment(environment: Record<string, unknown>): void {
const config = record(environment.config);
const networking = record(config.networking);
const packages = record(config.packages);
const installed = Object.entries(packages)
.filter(([key]) => key !== "type")
.flatMap(([, value]) => (Array.isArray(value) ? value : [value]))
.filter((value) => value !== undefined && value !== null);
if (
environment.archived_at !== null
|| config.type !== "cloud"
|| networking.type !== "limited"
|| networking.allow_mcp_servers !== false
|| networking.allow_package_managers !== false
|| !Array.isArray(networking.allowed_hosts)
|| networking.allowed_hosts.length > 0
|| installed.length > 0
) {
throw new Error(
"Existing Anthropic Environment does not match Paperclip's no-network, no-package profile",
);
}
}
export function assertSafeManagedAgent(agent: Record<string, unknown>): void {
const model = typeof agent.model === "string" ? agent.model : record(agent.model).id;
if (
agent.archived_at !== null
|| agent.system !== CLAUDE_MANAGED_SYSTEM_PROMPT
|| typeof model !== "string"
|| !model
|| !Array.isArray(agent.tools)
|| agent.tools.length > 0
|| !Array.isArray(agent.mcp_servers)
|| agent.mcp_servers.length > 0
|| !Array.isArray(agent.skills)
|| agent.skills.length > 0
|| agent.multiagent != null
) {
throw new Error(
"Existing Anthropic Agent enables or omits the locked tools, MCP, skills, or multi-agent profile",
);
}
}
async function resolveEnvironment(
key: string,
options: ManagedAgentSetupOptions,
): Promise<Record<string, unknown>> {
if (options.environmentId) {
const environment = await anthropicRequest(
key,
"GET",
`/v1/environments/${encodeURIComponent(options.environmentId)}`,
);
assertSafeManagedEnvironment(environment);
return environment;
}
const existing = resourceByProfile(
await listAll(key, "/v1/environments"),
options.profileKey,
"Environment",
);
if (existing) {
assertSafeManagedEnvironment(existing);
return existing;
}
if (options.probe) throw new Error("Probe found no matching Anthropic Environment");
const environment = await anthropicRequest(key, "POST", "/v1/environments", {
name: `Paperclip · ${options.displayName}`,
description: "Paperclip remote-agent environment: no network or added packages.",
config: {
type: "cloud",
networking: {
type: "limited",
allow_mcp_servers: false,
allow_package_managers: false,
allowed_hosts: [],
},
packages: {
apt: [],
cargo: [],
gem: [],
go: [],
npm: [],
pip: [],
},
},
metadata: { paperclip_profile: options.profileKey },
});
assertSafeManagedEnvironment(environment);
return environment;
}
async function resolveAgent(
key: string,
options: ManagedAgentSetupOptions,
): Promise<Record<string, unknown>> {
if (options.agentId) {
const agent = await anthropicRequest(
key,
"GET",
`/v1/agents/${encodeURIComponent(options.agentId)}`,
);
assertSafeManagedAgent(agent);
assertManagedAgentModel(agent, options.model);
return agent;
}
const existing = resourceByProfile(
await listAll(key, "/v1/agents"),
options.profileKey,
"Agent",
);
if (existing) {
assertSafeManagedAgent(existing);
assertManagedAgentModel(existing, options.model);
return existing;
}
if (options.probe) throw new Error("Probe found no matching Anthropic Agent");
const agent = await anthropicRequest(key, "POST", "/v1/agents", {
name: `Paperclip · ${options.displayName}`,
description: "Versioned Paperclip remote agent; runnerd supplies session tools.",
model: options.model,
system: CLAUDE_MANAGED_SYSTEM_PROMPT,
tools: [],
mcp_servers: [],
skills: [],
metadata: { paperclip_profile: options.profileKey },
});
assertSafeManagedAgent(agent);
assertManagedAgentModel(agent, options.model);
return agent;
}
function assertManagedAgentModel(agent: Record<string, unknown>, expectedModel: string): void {
const model = typeof agent.model === "string" ? agent.model : record(agent.model).id;
if (model !== expectedModel) {
throw new Error(
`Existing Anthropic Agent model does not match the requested pinned model ${expectedModel}`,
);
}
}
export async function setupManagedAgent(options: ManagedAgentSetupOptions): Promise<void> {
const validated = validateManagedAgentSetup(options);
const normalizedOptions: ManagedAgentSetupOptions = {
...options,
profileKey: validated.profileKey,
displayName: validated.displayName,
apiKeySecretId: validated.apiKeySecretId,
model: validated.model,
agentId: validated.agentId,
agentVersion: validated.agentVersion,
environmentId: validated.environmentId,
};
const [environment, agent] = await Promise.all([
resolveEnvironment(validated.anthropicApiKey, normalizedOptions),
resolveAgent(validated.anthropicApiKey, normalizedOptions),
]);
const agentId = String(agent.id ?? "");
const environmentId = String(environment.id ?? "");
if (!agentId || !environmentId) {
throw new Error("Anthropic did not return usable Agent and Environment identities");
}
const versions = await listAll(
validated.anthropicApiKey,
`/v1/agents/${encodeURIComponent(agentId)}/versions`,
);
const version = normalizedOptions.agentVersion
?? String(agent.version ?? versions.at(-1)?.version ?? "");
const pinnedAgent = version
? versions.find((entry) => String(entry.version) === version)
: undefined;
if (!version || !pinnedAgent) {
throw new Error("Anthropic did not return a usable pinned Agent version");
}
if (String(pinnedAgent.id ?? "") !== agentId) {
throw new Error("Anthropic pinned Agent version identity does not match the selected Agent");
}
assertSafeManagedAgent(pinnedAgent);
assertManagedAgentModel(pinnedAgent, normalizedOptions.model);
const qualification = {
probedAt: new Date().toISOString(),
betaVersion: CLAUDE_MANAGED_BETA_VERSION,
environmentPolicy: "limited_no_hosts_no_packages",
agentCapabilities: "no_tools_no_mcp_no_skills_no_multiagent",
};
const profile = {
profileKey: normalizedOptions.profileKey,
displayName: normalizedOptions.displayName,
anthropicAgentId: agentId,
agentVersion: version,
environmentId,
defaultModel: normalizedOptions.model,
defaultMaxListCostUsd: validated.defaultMaxListCostUsd,
apiKeySecretId: normalizedOptions.apiKeySecretId,
enabled: !options.probe,
retentionAcknowledged: true,
qualification,
};
if (options.probe) {
printOutput({ mode: "probe", qualified: true, profile }, { json: options.json });
return;
}
const context = resolveCommandContext(options, { requireCompany: true });
const stored = await context.api.post(
apiPath`/api/companies/${context.companyId}/managed-agent-profiles`,
profile,
);
printOutput(stored, { json: context.json });
}
export function registerManagedAgentCommands(program: Command): void {
const command = program
.command("managed-agent")
.description("Provision and qualify remote managed-agent providers");
addCommonClientOptions(
command
.command("setup")
.description(
"Create or adopt a locked-down Anthropic Agent and Environment, then store a company profile",
)
.requiredOption("--profile-key <key>", "Stable company profile key")
.requiredOption("--display-name <name>", "Profile display name")
.requiredOption(
"--api-key-secret-id <id>",
"Existing company secret containing ANTHROPIC_API_KEY",
)
.option("--model <id>", "Pinned Claude model", CLAUDE_MANAGED_QUALIFIED_MODEL)
.option(
"--max-session-list-cost-usd <usd>",
"Default hard session ceiling",
"1.00",
)
.option("--agent-id <id>", "Adopt an existing Anthropic Agent")
.option("--agent-version <version>", "Pin an existing Agent version")
.option("--environment-id <id>", "Adopt an existing Anthropic Environment")
.option("--probe", "Read-only qualification; create or persist nothing", false)
.option(
"--acknowledge-retention",
"Acknowledge beta retention and non-ZDR/non-HIPAA status",
false,
)
.action(async (options: ManagedAgentSetupOptions) => {
try {
await setupManagedAgent(options);
} catch (error) {
handleCommandError(error);
}
}),
{ includeCompany: true },
);
}

61
cli/src/commands/onboard.ts
View File

@ -28,7 +28,7 @@ import {
findPaperclipConfigKeyWarnings,
type PaperclipConfig,
} from "../config/schema.js";
import { ensureAgentJwtSecret, resolveAgentJwtEnvFile } from "../config/env.js";
import { ensureAgentJwtSecret, ensureToolActionSigningSecret, resolveAgentJwtEnvFile } from "../config/env.js";
import { ensureLocalSecretsKeyFile } from "../config/secrets-key.js";
import { promptDatabase } from "../prompts/database.js";
import { promptLlm } from "../prompts/llm.js";
@ -52,7 +52,11 @@ import {
trackInstallStarted,
trackInstallCompleted,
} from "../telemetry.js";
import { handleOnboardService } from "../onboard-service.js";
import {
handleOnboardService,
handoffToOnboardedService,
shouldOfferForegroundStart,
} from "../onboard-service.js";
import { readInstallManifest, isManagedExecutable } from "../install-store.js";
type SetupMode = "quickstart" | "advanced";
@ -111,6 +115,33 @@ function parseBooleanFromEnv(rawValue: string | undefined): boolean | null {
return null;
}
async function runOnboardedForeground(configPath: string): Promise<void> {
const previousOpenOnListen = process.env.PAPERCLIP_OPEN_ON_LISTEN;
const browserDisabled = parseBooleanFromEnv(process.env.PAPERCLIP_NO_BROWSER) === true;
const interactive = Boolean(process.stdin.isTTY && process.stdout.isTTY);
// The server consumes this flag in its listen callback. Keep it scoped to
// this foreground start so a later in-process restart does not open another
// tab. Explicit configuration wins over the interactive default, while the
// broad no-browser switch wins over an earlier explicit opt-in.
if (browserDisabled) {
process.env.PAPERCLIP_OPEN_ON_LISTEN = "false";
} else if (interactive && previousOpenOnListen === undefined) {
process.env.PAPERCLIP_OPEN_ON_LISTEN = "true";
}
try {
const { runCommand } = await import("./run.js");
await runCommand({ config: configPath, repair: true, yes: true });
} finally {
if (previousOpenOnListen === undefined) {
delete process.env.PAPERCLIP_OPEN_ON_LISTEN;
} else {
process.env.PAPERCLIP_OPEN_ON_LISTEN = previousOpenOnListen;
}
}
}
function parseNumberFromEnv(rawValue: string | undefined): number | null {
if (!rawValue) return null;
const parsed = Number(rawValue);
@ -422,6 +453,10 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
} else {
p.log.info(`Using existing ${pc.cyan("PAPERCLIP_AGENT_JWT_SECRET")} in ${pc.dim(envFilePath)}`);
}
const toolActionSigningSecret = ensureToolActionSigningSecret(configPath);
if (toolActionSigningSecret.created) {
p.log.success(`Created ${pc.cyan("PAPERCLIP_TOOL_ACTION_SIGNING_SECRET")} in ${pc.dim(envFilePath)}`);
}
const keyResult = ensureLocalSecretsKeyFile(existingConfig, configPath);
if (keyResult.status === "created") {
@ -457,9 +492,12 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
printManagedInstallHint();
const serviceInstalled = await handleOnboardService(opts);
if (serviceInstalled) {
await handoffToOnboardedService(existingConfig);
}
let shouldRunNow = !serviceInstalled && (opts.run === true || opts.yes === true);
if (!shouldRunNow && !opts.invokedByRun && process.stdin.isTTY && process.stdout.isTTY) {
if (shouldOfferForegroundStart({ serviceInstalled, startAlreadyDecided: shouldRunNow, invokedByRun: opts.invokedByRun === true, interactive: Boolean(process.stdin.isTTY && process.stdout.isTTY) })) {
const answer = await p.confirm({
message: "Start Paperclip now?",
initialValue: true,
@ -470,9 +508,7 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
}
if (shouldRunNow && !opts.invokedByRun) {
process.env.PAPERCLIP_OPEN_ON_LISTEN = "true";
const { runCommand } = await import("./run.js");
await runCommand({ config: configPath, repair: true, yes: true });
await runOnboardedForeground(configPath);
return;
}
@ -657,6 +693,10 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
} else {
p.log.info(`Using existing ${pc.cyan("PAPERCLIP_AGENT_JWT_SECRET")} in ${pc.dim(envFilePath)}`);
}
const toolActionSigningSecret = ensureToolActionSigningSecret(configPath);
if (toolActionSigningSecret.created) {
p.log.success(`Created ${pc.cyan("PAPERCLIP_TOOL_ACTION_SIGNING_SECRET")} in ${pc.dim(envFilePath)}`);
}
const config: PaperclipConfig = {
$meta: {
@ -723,9 +763,12 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
}
const serviceInstalled = await handleOnboardService(opts);
if (serviceInstalled) {
await handoffToOnboardedService(config);
}
let shouldRunNow = !serviceInstalled && (opts.run === true || opts.yes === true);
if (!shouldRunNow && !opts.invokedByRun && process.stdin.isTTY && process.stdout.isTTY) {
if (shouldOfferForegroundStart({ serviceInstalled, startAlreadyDecided: shouldRunNow, invokedByRun: opts.invokedByRun === true, interactive: Boolean(process.stdin.isTTY && process.stdout.isTTY) })) {
const answer = await p.confirm({
message: "Start Paperclip now?",
initialValue: true,
@ -736,9 +779,7 @@ export async function onboard(opts: OnboardOptions): Promise<void> {
}
if (shouldRunNow && !opts.invokedByRun) {
process.env.PAPERCLIP_OPEN_ON_LISTEN = "true";
const { runCommand } = await import("./run.js");
await runCommand({ config: configPath, repair: true, yes: true });
await runOnboardedForeground(configPath);
return;
}

49
cli/src/commands/run.ts
View File

@ -17,29 +17,41 @@ import {
resolvePaperclipInstanceId,
} from "../config/home.js";
import { assertForegroundRunAllowed } from "../services/service-manager.js";
import { removeRuntimeInfoForPid, writeRuntimeInfo } from "../runtime-info.js";
import { printUpdateNotice } from "../update-notice.js";
import { ensureWorktreeSeeded } from "./worktree.js";
interface RunOptions {
export interface RunOptions {
config?: string;
instance?: string;
repair?: boolean;
yes?: boolean;
bind?: "loopback" | "lan" | "tailnet";
force?: boolean;
/** Internal lifecycle option used by foreground-only commands. */
installService?: boolean;
/** Internal lifecycle option for isolated instances that cannot collide with a managed service. */
skipServiceManagerCheck?: boolean;
/** Internal label override for commands that reuse the foreground run path. */
introLabel?: string;
/** Runs after the server is listening and all normal post-start initialization has completed. */
afterStart?: (server: StartedServer) => Promise<void>;
}
interface StartedServer {
export interface StartedServer {
apiUrl: string;
databaseUrl: string;
host: string;
listenPort: number;
shutdown?: (signal?: "SIGINT" | "SIGTERM") => Promise<void>;
}
export async function runCommand(opts: RunOptions): Promise<void> {
const instanceId = resolvePaperclipInstanceId(opts.instance);
process.env.PAPERCLIP_INSTANCE_ID = instanceId;
await assertForegroundRunAllowed(instanceId, opts.force);
if (!opts.skipServiceManagerCheck) {
await assertForegroundRunAllowed(instanceId, opts.force);
}
const homeDir = resolvePaperclipHomeDir();
fs.mkdirSync(homeDir, { recursive: true });
@ -52,20 +64,26 @@ export async function runCommand(opts: RunOptions): Promise<void> {
loadPaperclipEnvFile(configPath);
await printUpdateNotice(configPath);
p.intro(pc.bgCyan(pc.black(" paperclipai run ")));
p.intro(pc.bgCyan(pc.black(` ${opts.introLabel ?? "paperclipai run"} `)));
p.log.message(pc.dim(`Home: ${paths.homeDir}`));
p.log.message(pc.dim(`Instance: ${paths.instanceId}`));
p.log.message(pc.dim(`Config: ${configPath}`));
if (!configExists(configPath)) {
if (!process.stdin.isTTY || !process.stdout.isTTY) {
if ((!process.stdin.isTTY || !process.stdout.isTTY) && !opts.yes) {
p.log.error("No config found and terminal is non-interactive.");
p.log.message(`Run ${pc.cyan("paperclipai onboard")} once, then retry ${pc.cyan("paperclipai run")}.`);
process.exit(1);
}
p.log.step("No config found. Starting onboarding...");
await onboard({ config: configPath, invokedByRun: true, bind: opts.bind });
await onboard({
config: configPath,
invokedByRun: true,
bind: opts.bind,
yes: opts.yes,
installService: opts.installService,
});
}
const seedResult = await ensureWorktreeSeeded({ config: configPath });
@ -93,6 +111,16 @@ export async function runCommand(opts: RunOptions): Promise<void> {
p.log.step("Starting Paperclip server...");
const startedServer = await importServerEntry();
writeRuntimeInfo({
schemaVersion: 1,
instanceId,
pid: process.pid,
host: startedServer.host,
port: startedServer.listenPort,
dashboardUrl: startedServer.apiUrl.replace(/\/api\/?$/, ""),
startedAt: new Date().toISOString(),
});
process.once("exit", () => removeRuntimeInfoForPid(process.pid, instanceId));
if (shouldGenerateBootstrapInviteAfterStart(config)) {
p.log.step("Generating bootstrap CEO invite");
@ -102,6 +130,15 @@ export async function runCommand(opts: RunOptions): Promise<void> {
baseUrl: resolveBootstrapInviteBaseUrl(config, startedServer),
});
}
if (opts.afterStart) {
try {
await opts.afterStart(startedServer);
} catch (error) {
await startedServer.shutdown?.("SIGTERM");
throw error;
}
}
}
function resolveBootstrapInviteBaseUrl(

507
cli/src/commands/test-drive.ts Normal file
View File

@ -0,0 +1,507 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { createServer } from "node:net";
import * as p from "@clack/prompts";
import pc from "picocolors";
import { Option, type Command } from "commander";
import type { Agent, Company, InstanceExperimentalSettings } from "@paperclipai/shared";
import { PaperclipApiClient } from "../client/http.js";
import { openUrl } from "../client/board-auth.js";
import {
expandHomePrefix,
resolveDefaultConfigPath,
resolveDefaultContextPath,
} from "../config/home.js";
import { readConfig } from "../config/store.js";
import type { PaperclipConfig } from "../config/schema.js";
import { runCommand, type StartedServer } from "./run.js";
import { isLinkedGitWorktree } from "./git-workspace.js";
export const TEST_DRIVE_HARNESSES = ["claude", "codex", "opencode"] as const;
export type TestDriveHarness = (typeof TEST_DRIVE_HARNESSES)[number];
export interface TestDriveOptions {
dataDir?: string;
companyName?: string;
agentName?: string;
harness?: TestDriveHarness;
model?: string;
apiKeyEnv?: string;
apiKey?: string;
browser?: boolean;
}
export type TestDriveApi = Pick<PaperclipApiClient, "get" | "post" | "patch" | "delete">;
type HarnessDefinition = {
adapterType: "claude_local" | "codex_local" | "opencode_local";
credentialTarget: "ANTHROPIC_API_KEY" | "OPENAI_API_KEY" | "OPENROUTER_API_KEY";
credentialName: string;
};
export type ResolvedTestDriveBootstrap = HarnessDefinition & {
companyName: string;
agentName: string;
model?: string;
credential: string;
credentialSource: string;
};
export type TestDriveBootstrapResult = {
reused: boolean;
company: Company;
agent: Agent | null;
};
export interface TestDriveDependencies {
run: typeof runCommand;
createApi: (apiBase: string) => TestDriveApi;
openBrowser: (url: string) => Promise<boolean>;
}
const HARNESS_DEFINITIONS: Record<TestDriveHarness, HarnessDefinition> = {
claude: {
adapterType: "claude_local",
credentialTarget: "ANTHROPIC_API_KEY",
credentialName: "Anthropic API Key",
},
codex: {
adapterType: "codex_local",
credentialTarget: "OPENAI_API_KEY",
credentialName: "OpenAI API Key",
},
opencode: {
adapterType: "opencode_local",
credentialTarget: "OPENROUTER_API_KEY",
credentialName: "OpenRouter API Key",
},
};
const NON_PAPERCLIP_ISOLATED_ENV_KEYS = [
"DATABASE_URL",
"DATABASE_MIGRATION_URL",
"HOST",
"PORT",
"SERVE_UI",
"BETTER_AUTH_URL",
"BETTER_AUTH_BASE_URL",
] as const;
function requiredApiResult<T>(value: T | null, action: string): T {
if (value === null) {
throw new Error(`Paperclip returned no result while ${action}.`);
}
return value;
}
function errorMessage(error: unknown): string {
if (error instanceof Error) return error.message || error.name;
if (typeof error === "string") return error;
try {
return JSON.stringify(error);
} catch {
return String(error);
}
}
export function redactTestDriveText(text: string, credentials: Array<string | undefined>): string {
let redacted = text;
for (const credential of credentials) {
if (!credential) continue;
redacted = redacted.replaceAll(credential, "[REDACTED]");
}
return redacted;
}
export function redactTestDriveArgv(
apiKey: string | undefined,
argv: string[] = process.argv,
): void {
if (!apiKey) return;
for (let index = 0; index < argv.length; index += 1) {
if (argv[index] === "--api-key" && argv[index + 1] === apiKey) {
argv[index + 1] = "[REDACTED]";
index += 1;
continue;
}
if (argv[index] === `--api-key=${apiKey}`) {
argv[index] = "--api-key=[REDACTED]";
}
}
}
export function resolveTestDriveDataDir(dataDir?: string): string {
const explicit = dataDir?.trim();
if (explicit) {
return path.resolve(expandHomePrefix(explicit));
}
return fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-test-drive-"));
}
async function loopbackPortAvailable(port: number): Promise<boolean> {
return await new Promise<boolean>((resolve) => {
const server = createServer();
server.unref();
server.once("error", () => resolve(false));
server.listen(port, "127.0.0.1", () => {
server.close(() => resolve(true));
});
});
}
export async function resolveTestDriveServerPort(preferredPort = 3100): Promise<number> {
for (let port = preferredPort; port <= 65_535; port += 1) {
if (await loopbackPortAvailable(port)) return port;
}
throw new Error(`No available loopback port found at or above ${preferredPort}.`);
}
/**
* Establish isolation before the CLI's normal config and .env loading hook.
* The selected credential source is preserved in case its name happens to use
* a PAPERCLIP_ prefix; all other Paperclip routing/configuration is discarded.
*/
export async function prepareTestDriveEnvironment(
options: Pick<TestDriveOptions, "dataDir" | "apiKeyEnv">,
cwd = process.cwd(),
): Promise<{ dataDir: string; linkedWorktree: boolean }> {
const sourceEnvName = options.apiKeyEnv?.trim();
const preservedCredential = sourceEnvName ? process.env[sourceEnvName] : undefined;
for (const key of Object.keys(process.env)) {
if (key.startsWith("PAPERCLIP_")) {
delete process.env[key];
}
}
for (const key of NON_PAPERCLIP_ISOLATED_ENV_KEYS) {
delete process.env[key];
}
if (sourceEnvName && preservedCredential !== undefined) {
process.env[sourceEnvName] = preservedCredential;
}
const dataDir = resolveTestDriveDataDir(options.dataDir);
const linkedWorktree = isLinkedGitWorktree(cwd);
process.env.PAPERCLIP_HOME = dataDir;
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.PAPERCLIP_CONFIG = resolveDefaultConfigPath("default");
process.env.PAPERCLIP_CONTEXT = resolveDefaultContextPath();
process.env.PAPERCLIP_IN_WORKTREE = linkedWorktree ? "true" : "false";
process.env.PAPERCLIP_OPEN_ON_LISTEN = "false";
process.env.PAPERCLIP_DISABLE_CWD_ENV_FILE = "true";
process.env.PAPERCLIP_DEPLOYMENT_MODE = "local_trusted";
process.env.PAPERCLIP_DEPLOYMENT_EXPOSURE = "private";
process.env.PAPERCLIP_BIND = "loopback";
process.env.HOST = "127.0.0.1";
process.env.PORT = String(await resolveTestDriveServerPort());
return { dataDir, linkedWorktree };
}
export function assertTestDriveDatabaseIsolation(
configPath?: string,
env: NodeJS.ProcessEnv = process.env,
readConfigFile: (path?: string) => PaperclipConfig | null = readConfig,
): void {
if (env.DATABASE_URL?.trim() || env.DATABASE_MIGRATION_URL?.trim()) {
throw new Error(
"test-drive requires its isolated embedded database. Remove DATABASE_URL and " +
"DATABASE_MIGRATION_URL from the selected data directory's .env, or choose a fresh --data-dir.",
);
}
const config = readConfigFile(configPath);
if (config?.database.mode === "postgres") {
throw new Error(
"test-drive cannot reuse a data directory configured for an external PostgreSQL database. " +
"Choose a fresh data directory or change database.mode to embedded-postgres.",
);
}
}
export function resolveTestDriveBootstrap(
options: TestDriveOptions,
env: NodeJS.ProcessEnv = process.env,
): ResolvedTestDriveBootstrap {
if (options.apiKey !== undefined && options.apiKeyEnv !== undefined) {
throw new Error("--api-key and --api-key-env are mutually exclusive.");
}
const harness = options.harness ?? "claude";
const definition = HARNESS_DEFINITIONS[harness];
if (!definition) {
throw new Error(`Unsupported test-drive harness: ${String(harness)}.`);
}
const companyName = (options.companyName ?? "Test Company").trim();
const agentName = (options.agentName ?? "CEO").trim();
if (!companyName) throw new Error("--company-name cannot be empty.");
if (!agentName) throw new Error("--agent-name cannot be empty.");
const model = options.model;
if (model !== undefined && (!model || model.trim() !== model)) {
throw new Error("--model cannot be empty or have surrounding whitespace.");
}
if (
harness === "opencode" &&
(!model || !/^openrouter\/[^/\s]+(?:\/[^/\s]+)*$/.test(model))
) {
throw new Error(
"OpenCode test drives require --model openrouter/<model>, with no empty path segments.",
);
}
const sourceEnvName = options.apiKeyEnv?.trim() || definition.credentialTarget;
if (options.apiKeyEnv !== undefined && !/^[A-Za-z_][A-Za-z0-9_]*$/.test(sourceEnvName)) {
throw new Error("--api-key-env must name a valid environment variable.");
}
const credential = options.apiKey ?? env[sourceEnvName];
if (!credential || credential.trim().length === 0) {
throw new Error(
`No credential found. Set ${sourceEnvName}, pass --api-key-env <variable>, or pass --api-key <value>.`,
);
}
return {
...definition,
companyName,
agentName,
...(model ? { model } : {}),
credential,
credentialSource: options.apiKey !== undefined ? "--api-key" : sourceEnvName,
};
}
function worktreeExecutionArmed(
settings: InstanceExperimentalSettings,
instanceId: string,
): boolean {
return settings.enableWorktreeRunExecution === true
&& Boolean(settings.worktreeRunExecutionActivatedAt)
&& settings.worktreeRunExecutionActivationInstanceId === instanceId;
}
export async function reconcileTestDriveWorktreeExecution(
api: TestDriveApi,
instanceId: string,
): Promise<void> {
const current = requiredApiResult(
await api.get<InstanceExperimentalSettings>("/api/instance/settings/experimental"),
"reading experimental settings",
);
if (!current.enableWorktreeRunExecution) {
await api.patch<InstanceExperimentalSettings>("/api/instance/settings/experimental", {
enableWorktreeRunExecution: true,
});
} else if (!worktreeExecutionArmed(current, instanceId)) {
await api.patch<InstanceExperimentalSettings>("/api/instance/settings/experimental", {
enableWorktreeRunExecution: false,
});
await api.patch<InstanceExperimentalSettings>("/api/instance/settings/experimental", {
enableWorktreeRunExecution: true,
});
}
const verified = requiredApiResult(
await api.get<InstanceExperimentalSettings>("/api/instance/settings/experimental"),
"verifying experimental settings",
);
if (!worktreeExecutionArmed(verified, instanceId)) {
throw new Error(
`Could not arm “Run tasks in this worktree” for Paperclip instance ${instanceId}. ` +
"Check that PAPERCLIP_IN_WORKTREE=true and retry the command.",
);
}
}
export async function bootstrapTestDrive(input: {
api: TestDriveApi;
options: TestDriveOptions;
linkedWorktree: boolean;
instanceId: string;
env?: NodeJS.ProcessEnv;
}): Promise<TestDriveBootstrapResult> {
const companies = requiredApiResult(
await input.api.get<Company[]>("/api/companies"),
"reading companies",
);
const existingCompany = companies[0];
if (existingCompany) {
if (input.linkedWorktree) {
await reconcileTestDriveWorktreeExecution(input.api, input.instanceId);
}
return { reused: true, company: existingCompany, agent: null };
}
// Resolve every bootstrap input before the first mutation. In particular,
// OpenCode model validation and credential lookup happen before company
// creation so an invalid invocation leaves the database untouched.
const resolved = resolveTestDriveBootstrap(input.options, input.env);
let company: Company | null = null;
try {
company = requiredApiResult(
await input.api.post<Company>("/api/companies", { name: resolved.companyName }),
"creating the test company",
);
await input.api.post(`/api/companies/${company.id}/user-secret-definitions`, {
key: resolved.credentialTarget,
name: resolved.credentialName,
});
await input.api.post(`/api/companies/${company.id}/me/user-secrets`, {
definitionKey: resolved.credentialTarget,
value: resolved.credential,
});
const adapterConfig: Record<string, unknown> = {
env: {
[resolved.credentialTarget]: {
type: "user_secret_ref",
key: resolved.credentialTarget,
version: "latest",
required: true,
},
},
};
if (resolved.model) adapterConfig.model = resolved.model;
const agent = requiredApiResult(
await input.api.post<Agent>(`/api/companies/${company.id}/agents`, {
name: resolved.agentName,
role: "ceo",
adapterType: resolved.adapterType,
adapterConfig,
}),
"creating the CEO agent",
);
if (input.linkedWorktree) {
await reconcileTestDriveWorktreeExecution(input.api, input.instanceId);
}
return { reused: false, company, agent };
} catch (error) {
if (company) {
try {
await input.api.delete(`/api/companies/${company.id}`);
} catch (cleanupError) {
throw new Error(
`${errorMessage(error)} Cleanup also failed for newly-created company ${company.id}: ${errorMessage(cleanupError)}`,
{ cause: error },
);
}
}
throw error;
}
}
function dashboardUrl(server: StartedServer): string {
return server.apiUrl.replace(/\/api\/?$/, "");
}
export async function testDriveCommand(
options: TestDriveOptions,
dependencies: TestDriveDependencies = {
run: runCommand,
createApi: (apiBase) => new PaperclipApiClient({ apiBase }),
openBrowser: openUrl,
},
): Promise<void> {
// Commander has already copied the value into options. Remove it from the
// JavaScript argv view before logging, telemetry, diagnostics, or startup.
redactTestDriveArgv(options.apiKey);
const dataDir = path.resolve(process.env.PAPERCLIP_HOME ?? resolveTestDriveDataDir(options.dataDir));
const linkedWorktree = process.env.PAPERCLIP_IN_WORKTREE === "true";
const instanceId = process.env.PAPERCLIP_INSTANCE_ID ?? "default";
// Resolve environment-backed credentials against the CLI environment as it
// exists before server startup. In-process server initialization must not
// change which credential the post-listen bootstrap observes.
const bootstrapEnv = { ...process.env };
const possibleCredentials = [
options.apiKey,
options.apiKeyEnv ? bootstrapEnv[options.apiKeyEnv.trim()] : undefined,
bootstrapEnv[HARNESS_DEFINITIONS[options.harness ?? "claude"].credentialTarget],
];
p.log.message(pc.dim(`Data directory: ${dataDir}`));
p.log.message(pc.dim("The data directory is retained when Paperclip exits."));
if (options.apiKey !== undefined) {
p.log.warn("A key passed with --api-key may be visible in process arguments and shell history.");
}
try {
await dependencies.run({
repair: true,
yes: true,
bind: "loopback",
installService: false,
// Auto-created directories are private to this process. Explicitly reused
// directories retain the normal guard against an already-managed instance.
skipServiceManagerCheck: !options.dataDir?.trim(),
introLabel: "paperclipai test-drive",
afterStart: async (server) => {
const api = dependencies.createApi(server.apiUrl);
const result = await bootstrapTestDrive({
api,
options,
linkedWorktree,
instanceId,
env: bootstrapEnv,
});
if (result.reused) {
p.log.message(
`Using existing data for ${pc.cyan(result.company.name)}; bootstrap flags were ignored.`,
);
} else {
p.log.success(
`Created ${pc.cyan(result.company.name)} with agent ${pc.cyan(result.agent?.name ?? "CEO")}.`,
);
}
if (linkedWorktree) {
p.log.success("Run tasks in this worktree is enabled for this instance.");
}
const url = dashboardUrl(server);
if (options.browser === false) {
p.log.success(`Paperclip is ready at ${pc.cyan(url)}.`);
return;
}
const opened = await dependencies.openBrowser(url);
if (opened) {
p.log.success(`Paperclip is ready and opened at ${pc.cyan(url)}.`);
} else {
p.log.warn(`Paperclip is ready, but the browser could not be opened. Visit ${url}.`);
}
},
});
} catch (error) {
throw new Error(redactTestDriveText(errorMessage(error), possibleCredentials), { cause: error });
}
}
export function registerTestDriveCommand(program: Command): void {
program
.command("test-drive")
.description("Start an isolated, initialized Paperclip instance for manual testing")
.option("-d, --data-dir <path>", "Paperclip data directory to create or reuse")
.option("--company-name <name>", "Initial company name", "Test Company")
.option("--agent-name <name>", "Initial CEO agent name", "CEO")
.addOption(
new Option("--harness <harness>", "Initial agent harness")
.choices(TEST_DRIVE_HARNESSES)
.default("claude"),
)
.option("--model <model-id>", "Initial agent model")
.addOption(
new Option("--api-key-env <variable>", "Read the provider key from an environment variable")
.conflicts("apiKey"),
)
.addOption(
new Option("--api-key <value>", "Provider API key")
.conflicts("apiKeyEnv"),
)
.option("--no-browser", "Do not open the initialized instance in a browser")
.action(async (options: TestDriveOptions) => {
await testDriveCommand(options);
});
}

9
cli/src/commands/update.ts
View File

@ -5,9 +5,9 @@ import { execFile } from "node:child_process";
import { promisify } from "node:util";
import * as p from "@clack/prompts";
import pc from "picocolors";
import { buildNextManifest, flipCurrentAtomic, isManagedExecutable, pruneInstallPayloads, readInstallManifest, resolveInstallStorePaths, withInstallStoreLock, writeInstallManifestAtomic, type InstallChannel, type InstallManifest, type InstallRecord, type InstallStorePaths } from "../install-store.js";
import { assertManagedShimWritable, writeManagedShim, buildNextManifest, flipCurrentAtomic, isManagedExecutable, pruneInstallPayloads, readInstallManifest, resolveInstallStorePaths, withInstallStoreLock, writeInstallManifestAtomic, type InstallChannel, type InstallManifest, type InstallRecord, type InstallStorePaths } from "../install-store.js";
import { dbBackupCommand } from "./db-backup.js";
import { installGitPayload, installNpmPayload, PUBLIC_NPM_REGISTRY, resolveGitHubRef, resolvePublishedVersion, type CommandRunner } from "./install.js";
import { assertSupportedNodeVersion, installGitPayload, installNpmPayload, PUBLIC_NPM_REGISTRY, resolveGitHubRef, resolvePublishedVersion, type CommandRunner } from "./install.js";
import { resolvePaperclipInstanceId, resolvePaperclipInstanceRoot } from "../config/home.js";
import { resolveConfigPath } from "../config/store.js";
import { detectServiceManager } from "../services/service-manager.js";
@ -180,6 +180,7 @@ export async function updateCommand(options: UpdateOptions, overrides: Partial<D
}
if (mode === "npx") { emit(options, { mode, action: "install" }, "This is an ephemeral npx install. Run `paperclipai install`, then use `paperclipai update` from the managed shim."); return; }
if (mode === "source" || mode === "unknown") { emit(options, { mode, action: "manual" }, "This appears to be a source checkout. Update it with `git pull` followed by `pnpm install`; Paperclip will not mutate the repository."); return; }
if (!options.check && !options.dryRun) assertSupportedNodeVersion();
const request = resolveUpdateRequest(mode === "managed" ? manifest : null, options);
if (mode === "managed" && manifest?.source === "git") {
if (!manifest.repo || !manifest.ref || !manifest.sha) throw new Error("Managed git install metadata is incomplete.");
@ -193,7 +194,9 @@ export async function updateCommand(options: UpdateOptions, overrides: Partial<D
}
if (options.backup !== false) await runPreUpdateBackup(options, overrides.backup ?? (() => dbBackupCommand({})), overrides.hasInstanceData);
const installed = await withInstallStoreLock(async () => {
assertManagedShimWritable(paths);
const payload = await installGitPayload(manifest.repo!, targetSha, runCommand, paths);
writeManagedShim(paths);
const record: InstallRecord = { source: "git", version: payload.version, channel: "pinned", repo: manifest.repo, ref: manifest.ref, sha: targetSha, payloadPath: payload.payloadPath, installedAt: (overrides.now?.() ?? new Date()).toISOString() };
const next = buildNextManifest(record, manifest); const oldTarget = fs.readlinkSync(paths.currentPath); flipCurrentAtomic(payload.payloadPath, paths);
try { writeInstallManifestAtomic(next, paths); } catch (error) { flipCurrentAtomic(path.resolve(paths.cliRoot, oldTarget), paths); throw error; }
@ -247,7 +250,9 @@ export async function updateCommand(options: UpdateOptions, overrides: Partial<D
if (options.dryRun) { emit(options, { mode, currentVersion, targetVersion, action: comparison < 0 ? "downgrade" : "update", backup: options.backup !== false, dryRun: true }, `Would ${comparison < 0 ? "downgrade" : "update"} paperclipai ${currentVersion} → ${targetVersion}${options.backup === false ? " without a backup" : " after a database backup"}.`); return; }
if (options.backup !== false) await runPreUpdateBackup(options, overrides.backup ?? (() => dbBackupCommand({})), overrides.hasInstanceData);
const installed = await withInstallStoreLock(async () => {
assertManagedShimWritable(paths);
const payload = await installNpmPayload(targetVersion, runCommand, paths);
writeManagedShim(paths);
const record: InstallRecord = { source: "npm", version: targetVersion, channel: request.channel, payloadPath: payload.payloadPath, installedAt: (overrides.now?.() ?? new Date()).toISOString() };
const next = buildNextManifest(record, manifest); const oldTarget = fs.readlinkSync(paths.currentPath); flipCurrentAtomic(payload.payloadPath, paths);
try { writeInstallManifestAtomic(next, paths); } catch (error) { flipCurrentAtomic(path.resolve(paths.cliRoot, oldTarget), paths); throw error; }

48
cli/src/commands/worktree.ts
View File

@ -67,7 +67,7 @@ import {
prepareEmbeddedPostgresNativeRuntime,
} from "@paperclipai/db";
import type { Command } from "commander";
import { ensureAgentJwtSecret, loadPaperclipEnvFile, mergePaperclipEnvEntries, readPaperclipEnvEntries, resolvePaperclipEnvFile } from "../config/env.js";
import { ensureAgentJwtSecret, ensureToolActionSigningSecret, loadPaperclipEnvFile, mergePaperclipEnvEntries, readPaperclipEnvEntries, resolvePaperclipEnvFile } from "../config/env.js";
import { expandHomePrefix } from "../config/home.js";
import type { PaperclipConfig } from "../config/schema.js";
import { readConfig, resolveConfigPath, writeConfig } from "../config/store.js";
@ -104,6 +104,7 @@ import {
type PlannedIssueDocumentMerge,
type PlannedIssueInsert,
} from "./worktree-merge-history-lib.js";
import { detectGitWorkspaceInfo } from "./git-workspace.js";
type WorktreeInitOptions = {
name?: string;
@ -204,13 +205,6 @@ type EmbeddedPostgresHandle = {
stop: () => Promise<void>;
};
type GitWorkspaceInfo = {
root: string;
commonDir: string;
gitDir: string;
hooksPath: string;
};
type CopiedGitHooksResult = {
sourceHooksPath: string;
targetHooksPath: string;
@ -717,39 +711,6 @@ function resolveRepairWorktreeDirName(branchName: string): string {
return normalized || "worktree";
}
function detectGitWorkspaceInfo(cwd: string): GitWorkspaceInfo | null {
try {
const root = execFileSync("git", ["rev-parse", "--show-toplevel"], {
cwd,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const commonDirRaw = execFileSync("git", ["rev-parse", "--git-common-dir"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const gitDirRaw = execFileSync("git", ["rev-parse", "--git-dir"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
const hooksPathRaw = execFileSync("git", ["rev-parse", "--git-path", "hooks"], {
cwd: root,
encoding: "utf8",
stdio: ["ignore", "pipe", "ignore"],
}).trim();
return {
root: path.resolve(root),
commonDir: path.resolve(root, commonDirRaw),
gitDir: path.resolve(root, gitDirRaw),
hooksPath: path.resolve(root, hooksPathRaw),
};
} catch {
return null;
}
}
function copyDirectoryContents(sourceDir: string, targetDir: string): boolean {
if (!existsSync(sourceDir)) return false;
@ -2530,14 +2491,19 @@ async function runWorktreeInit(opts: WorktreeInitOptions): Promise<void> {
const existingAgentJwtSecret =
nonEmpty(sourceEnvEntries.PAPERCLIP_AGENT_JWT_SECRET) ??
nonEmpty(process.env.PAPERCLIP_AGENT_JWT_SECRET);
const existingToolActionSigningSecret =
nonEmpty(sourceEnvEntries.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET) ??
nonEmpty(process.env.PAPERCLIP_TOOL_ACTION_SIGNING_SECRET);
mergePaperclipEnvEntries(
{
...buildWorktreeEnvEntries(paths, branding),
...(existingAgentJwtSecret ? { PAPERCLIP_AGENT_JWT_SECRET: existingAgentJwtSecret } : {}),
...(existingToolActionSigningSecret ? { PAPERCLIP_TOOL_ACTION_SIGNING_SECRET: existingToolActionSigningSecret } : {}),
},
paths.envPath,
);
ensureAgentJwtSecret(paths.configPath);
ensureToolActionSigningSecret(paths.configPath);
loadPaperclipEnvFile(paths.configPath);
const copiedGitHooks = copyGitHooksToWorktreeGitDir(cwd);

20
cli/src/config/env.ts
View File

@ -6,6 +6,7 @@ import { updateEnvFileContents, writeEnvFileAtomicallyIfChanged } from "@papercl
import { resolveConfigPath } from "./store.js";
const JWT_SECRET_ENV_KEY = "PAPERCLIP_AGENT_JWT_SECRET";
const TOOL_ACTION_SIGNING_SECRET_ENV_KEY = "PAPERCLIP_TOOL_ACTION_SIGNING_SECRET";
const PAPERCLIP_OWNED_ENV_KEY_PATTERN = /^PAPERCLIP_[A-Z0-9_]+$/;
function resolveEnvFilePath(configPath?: string) {
return path.resolve(path.dirname(resolveConfigPath(configPath)), ".env");
@ -93,6 +94,25 @@ export function ensureAgentJwtSecret(configPath?: string): { secret: string; cre
return { secret, created };
}
export function ensureToolActionSigningSecret(configPath?: string): { secret: string; created: boolean } {
loadAgentJwtEnvFile(resolveEnvFilePath(configPath));
const existingEnv = process.env[TOOL_ACTION_SIGNING_SECRET_ENV_KEY];
if (isNonEmpty(existingEnv)) {
return { secret: existingEnv.trim(), created: false };
}
const envFilePath = resolveEnvFilePath(configPath);
const existingFile = readPaperclipEnvEntries(envFilePath)[TOOL_ACTION_SIGNING_SECRET_ENV_KEY];
const secret = isNonEmpty(existingFile) ? existingFile.trim() : randomBytes(32).toString("hex");
const created = !isNonEmpty(existingFile);
if (created) {
mergePaperclipEnvEntries({ [TOOL_ACTION_SIGNING_SECRET_ENV_KEY]: secret }, envFilePath);
}
return { secret, created };
}
export function writeAgentJwtEnv(secret: string, filePath = resolveEnvFilePath()): void {
mergePaperclipEnvEntries({ [JWT_SECRET_ENV_KEY]: secret }, filePath);
}

33
cli/src/index.ts
View File

@ -1,3 +1,4 @@
import { registerEmailCommands } from "./commands/client/email.js";
import { Command } from "commander";
import { warnIfUnsupportedNodeVersion } from "@paperclipai/shared/node-version";
import { onboard } from "./commands/onboard.js";
@ -41,6 +42,7 @@ import { registerWorkspaceCommands } from "./commands/client/workspace.js";
import { registerAccessCommands } from "./commands/client/access.js";
import { registerRoutineApiCommands } from "./commands/client/routine-api.js";
import { registerAdapterCommands } from "./commands/client/adapter.js";
import { registerManagedAgentCommands } from "./commands/managed-agent.js";
import { registerAssetCommands } from "./commands/client/asset.js";
import { registerSkillCommands } from "./commands/client/skill.js";
import { cliVersion } from "./version.js";
@ -48,6 +50,14 @@ import { installCommand } from "./commands/install.js";
import { uninstallCommand } from "./commands/uninstall.js";
import { updateCommand } from "./commands/update.js";
import { registerServiceCommands } from "./commands/service.js";
import { registerConnectionIntentCommands } from "./commands/client/connections.js";
import {
assertTestDriveDatabaseIsolation,
prepareTestDriveEnvironment,
redactTestDriveArgv,
registerTestDriveCommand,
type TestDriveOptions,
} from "./commands/test-drive.js";
const program = new Command();
const DATA_DIR_OPTION_HELP =
@ -90,17 +100,31 @@ program
.option("--no-backup", "Skip the pre-update database backup")
.action(updateCommand);
program.hook("preAction", (_thisCommand, actionCommand) => {
const options = actionCommand.optsWithGlobals() as DataDirOptionLike;
program.hook("preAction", async (_thisCommand, actionCommand) => {
const options = actionCommand.optsWithGlobals() as DataDirOptionLike & TestDriveOptions;
let dataDirOptions: DataDirOptionLike = options;
if (actionCommand.name() === "test-drive") {
redactTestDriveArgv(options.apiKey);
const prepared = await prepareTestDriveEnvironment({
dataDir: options.dataDir,
apiKeyEnv: options.apiKeyEnv,
});
dataDirOptions = { ...options, dataDir: prepared.dataDir };
}
const optionNames = new Set(actionCommand.options.map((option) => option.attributeName()));
applyDataDirOverride(options, {
applyDataDirOverride(dataDirOptions, {
hasConfigOption: optionNames.has("config"),
hasContextOption: optionNames.has("context"),
});
loadPaperclipEnvFile(options.config);
if (actionCommand.name() === "test-drive") {
assertTestDriveDatabaseIsolation(options.config);
}
initTelemetryFromConfigFile(options.config);
});
registerTestDriveCommand(program);
program
.command("onboard")
.description("Interactive first-run setup wizard")
@ -209,6 +233,8 @@ heartbeat
registerContextCommands(program);
registerConnectCommand(program);
registerConnectionIntentCommands(program);
registerEmailCommands(program);
registerCompanyCommands(program);
registerIssueCommands(program);
registerAgentCommands(program);
@ -224,6 +250,7 @@ registerWorkspaceCommands(program);
registerAccessCommands(program);
registerRoutineApiCommands(program);
registerAdapterCommands(program);
registerManagedAgentCommands(program);
registerAssetCommands(program);
registerSkillCommands(program);
registerRoutineCommands(program);

14
cli/src/install-store.ts
View File

@ -380,13 +380,17 @@ function shellQuote(value: string): string {
function isManagedShimContents(contents: string): boolean {
const lines = contents.split("\n");
// Accept the original pinned-runtime shim so upgrades can replace it.
const withRuntimePath = lines.length === 6;
const execIndex = withRuntimePath ? 4 : 3;
return (
lines.length === 5 &&
(lines.length === 5 || withRuntimePath) &&
lines[0] === "#!/bin/sh" &&
lines[1] === `# ${MANAGED_SHIM_MARKER}` &&
lines[2] === "set -eu" &&
/^exec '(?:[^']|'"'"')+' '(?:[^']|'"'"')+' "\$@"$/.test(lines[3]) &&
lines[4] === ""
(!withRuntimePath || /^export PATH='(?:[^']|'"'"')+':"\$\{PATH:-\/usr\/local\/bin:\/usr\/bin:\/bin\}"$/.test(lines[3])) &&
/^exec '(?:[^']|'"'"')+' '(?:[^']|'"'"')+' "\$@"$/.test(lines[execIndex]) &&
lines[execIndex + 1] === ""
);
}
@ -399,7 +403,9 @@ export function writeManagedShim(paths = resolveInstallStorePaths()): void {
fs.mkdirSync(path.dirname(paths.shimPath), { recursive: true, mode: 0o755 });
assertManagedShimWritable(paths);
const entrypoint = path.join(paths.currentPath, "node_modules", "paperclipai", "dist", "index.js");
const contents = `#!/bin/sh\n# ${MANAGED_SHIM_MARKER}\nset -eu\nexec ${shellQuote(process.execPath)} ${shellQuote(entrypoint)} "\$@"\n`;
// ACP servers and package-manager shims use /usr/bin/env node. Pin their
// runtime too, even when systemd/launchd supplies a different PATH.
const contents = `#!/bin/sh\n# ${MANAGED_SHIM_MARKER}\nset -eu\nexport PATH=${shellQuote(path.dirname(process.execPath))}:"\${PATH:-/usr/local/bin:/usr/bin:/bin}"\nexec ${shellQuote(process.execPath)} ${shellQuote(entrypoint)} "\$@"\n`;
writeFileAtomic(paths.shimPath, contents, 0o755);
}

2
cli/src/node-version.test.ts
View File

@ -29,6 +29,8 @@ describe("isSupportedNodeVersion", () => {
expect(warning).toContain(NODE_VERSION_INSTALL_GUIDE_URL);
expect(warning).toContain("piped install.sh form cannot upgrade");
expect(warning).toContain("Restart Paperclip after upgrading");
expect(warning).toContain(process.execPath);
expect(warning).toContain("startup executable and PATH");
});
it("emits at most one warning when CLI and server boot in the same process", () => {

199
cli/src/onboard-service.ts
View File

@ -1,18 +1,130 @@
import path from "node:path";
import * as p from "@clack/prompts";
import pc from "picocolors";
import type { PaperclipConfig } from "./config/schema.js";
import { openUrl } from "./client/board-auth.js";
import { installCommand } from "./commands/install.js";
import { resolvePaperclipInstanceId } from "./config/home.js";
import { readRuntimeInfo, type PaperclipRuntimeInfo } from "./runtime-info.js";
import {
readInstallManifest,
resolveInstallStorePaths,
type InstallManifest,
} from "./install-store.js";
import {
detectServiceManager,
isExecutableFile,
resolveServiceShimPath,
type ServiceManagerDetection,
} from "./services/service-manager.js";
import { buildLocalAppUrl, buildLocalHealthUrl } from "./utils/health-url.js";
import { packageVersion } from "./version.js";
export type OnboardServiceOptions = {
yes?: boolean;
installService?: boolean;
};
type EnsureShimResult = { ok: boolean; installedNow: boolean; reason?: string };
type OnboardServiceDashboardConfig = {
auth: Pick<PaperclipConfig["auth"], "baseUrlMode" | "publicBaseUrl">;
server: Pick<PaperclipConfig["server"], "host" | "port">;
};
type OnboardServiceDashboardDependencies = {
isInteractive: () => boolean;
waitUntilReady: () => Promise<PaperclipRuntimeInfo | null>;
openDashboard: (url: string) => Promise<boolean>;
info: (message: string) => void;
success: (message: string) => void;
warn: (message: string) => void;
};
function envDisablesBrowser(value = process.env.PAPERCLIP_NO_BROWSER): boolean {
const normalized = value?.trim().toLowerCase();
return normalized === "1" || normalized === "true" || normalized === "yes";
}
async function waitUntilDashboardReady(timeoutMs = 60_000): Promise<PaperclipRuntimeInfo | null> {
const instanceId = resolvePaperclipInstanceId();
const detection = await detectServiceManager({ instanceId });
if (!detection.supported) return null;
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
const info = readRuntimeInfo(instanceId);
if (info) {
const status = await detection.manager.status().catch(() => null);
if (status?.active && status.pid === info.pid) {
try {
const response = await fetch(buildLocalHealthUrl(info.host, info.port), {
signal: AbortSignal.timeout(2_000),
});
const body = await response.json() as { status?: unknown };
if (response.ok && body.status === "ok") return info;
} catch {}
}
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
return null;
}
const defaultDashboardDependencies: OnboardServiceDashboardDependencies = {
isInteractive: () => process.stdin.isTTY === true && process.stdout.isTTY === true,
waitUntilReady: waitUntilDashboardReady,
openDashboard: openUrl,
info: (message) => p.log.info(message),
success: (message) => p.log.success(message),
warn: (message) => p.log.warn(message),
};
export function resolveOnboardServiceDashboardUrl(
config: OnboardServiceDashboardConfig,
runtime?: Pick<PaperclipRuntimeInfo, "dashboardUrl"> | null,
): string {
if (runtime?.dashboardUrl.trim()) return runtime.dashboardUrl.trim().replace(/\/+$/, "");
if (config.auth.baseUrlMode === "explicit" && config.auth.publicBaseUrl?.trim()) {
return config.auth.publicBaseUrl.trim().replace(/\/+$/, "");
}
return buildLocalAppUrl(config.server.host, config.server.port);
}
export async function handoffToOnboardedService(
config: OnboardServiceDashboardConfig,
dependencies: Partial<OnboardServiceDashboardDependencies> = {},
): Promise<void> {
const deps = { ...defaultDashboardDependencies, ...dependencies };
const runtime = await deps.waitUntilReady();
const dashboardUrl = resolveOnboardServiceDashboardUrl(config, runtime);
deps.info(`Paperclip dashboard: ${pc.cyan(dashboardUrl)}`);
if (!runtime) {
deps.warn(
`The background service started, but the dashboard is not ready yet. ` +
`Open ${dashboardUrl} after checking \`paperclipai service logs\`.`,
);
return;
}
if (!deps.isInteractive() || envDisablesBrowser()) return;
if (await deps.openDashboard(dashboardUrl)) {
deps.success("Sent the Paperclip dashboard to your browser.");
} else {
deps.warn(`Could not open a browser automatically. Open ${dashboardUrl} manually.`);
}
}
// Source checkouts carry the repository placeholder version; installing
// that as an npm spec would fetch an ancient release (or nothing) instead
// of the running code. Only real calendar versions are installable.
export function isInstallableReleaseVersion(version: string): boolean {
return /^\d{4}\.\d{1,4}\.\d+(?:-[0-9A-Za-z.-]+)?$/.test(version);
}
type OnboardServiceDependencies = {
detect: (instanceId: string) => Promise<ServiceManagerDetection>;
ensureServiceShim: () => Promise<EnsureShimResult>;
confirm: () => Promise<boolean>;
confirmLinger: () => Promise<boolean>;
isInteractive: () => boolean;
@ -23,6 +135,60 @@ type OnboardServiceDependencies = {
const defaultDependencies: OnboardServiceDependencies = {
detect: (instanceId) => detectServiceManager({ instanceId }),
// The service definition targets the managed shim. An ephemeral run (npx)
// never lays it down, so installing the service without this step creates
// a definition that crash-loops on a missing binary.
ensureServiceShim: async () => {
const shimPath = resolveServiceShimPath();
if (await isExecutableFile(shimPath)) {
return { ok: true, installedNow: false };
}
const storeShimPath = resolveInstallStorePaths().shimPath;
if (path.resolve(shimPath) !== path.resolve(storeShimPath)) {
return {
ok: false,
installedNow: false,
reason: `no executable exists at ${shimPath} (PAPERCLIP_SHIM_PATH), and it is outside the managed install store`,
};
}
let manifest: InstallManifest | null = null;
try {
manifest = readInstallManifest();
} catch {}
try {
if (manifest?.source === "git" && manifest.repo) {
// A managed git payload must be preserved as-is: reinstall the
// exact revision the manifest records, not an npm release.
await installCommand({ repo: manifest.repo, ref: manifest.sha ?? manifest.ref, yes: true });
} else if (isInstallableReleaseVersion(packageVersion)) {
// packageVersion, not cliVersion: a managed executable's cliVersion
// carries provenance text that is not an installable npm spec.
await installCommand({ version: packageVersion, yes: true });
} else {
return {
ok: false,
installedNow: false,
reason:
`this build reports version ${packageVersion}, which is not an installable release; ` +
"run `paperclipai install` (or `paperclipai install --repo <repo> --ref <ref>` for source builds) first",
};
}
} catch (error) {
return {
ok: false,
installedNow: false,
reason: error instanceof Error ? error.message : String(error),
};
}
if (await isExecutableFile(shimPath)) {
return { ok: true, installedNow: true };
}
return {
ok: false,
installedNow: false,
reason: `the managed install completed but no executable shim appeared at ${shimPath}`,
};
},
confirm: async () => {
const answer = await p.confirm({
message: "Install Paperclip as a background service?",
@ -68,6 +234,22 @@ export async function handleOnboardService(
if (!explicitlyRequested && !(await deps.confirm())) return false;
// A definition pointing at a missing binary crash-loops in the platform
// supervisor's penalty box while doctor blames a port conflict.
// Materialize the managed install first, or decline with the repair path
// instead of installing a corpse.
const shim = await deps.ensureServiceShim();
if (!shim.ok) {
deps.warn(
`Background service not installed: ${shim.reason ?? "the managed install could not be completed"}. ` +
"Run `paperclipai install`, then `paperclipai service install`.",
);
return false;
}
if (shim.installedNow) {
deps.success("Installed the managed paperclipai payload and command shim for the service.");
}
await detection.manager.install({ startNow: true, startOnLogin: true });
if (!explicitlyRequested && detection.manager.enableLinger && await deps.confirmLinger()) {
await detection.manager.enableLinger();
@ -75,3 +257,20 @@ export async function handleOnboardService(
deps.success(`Installed and started ${detection.manager.serviceName}.`);
return true;
}
// Onboarding falls back to offering a foreground start when nothing else
// will serve. A just-installed service is already serving, so offering the
// start would only run the user into the already-running instance guard.
export function shouldOfferForegroundStart(options: {
serviceInstalled: boolean;
startAlreadyDecided: boolean;
invokedByRun: boolean;
interactive: boolean;
}): boolean {
return (
!options.startAlreadyDecided &&
!options.serviceInstalled &&
!options.invokedByRun &&
options.interactive
);
}

82
cli/src/runtime-info.ts Normal file
View File

@ -0,0 +1,82 @@
import fs from "node:fs";
import path from "node:path";
import { resolvePaperclipInstanceRoot } from "./config/home.js";
export const PAPERCLIP_RUNTIME_INFO_FILENAME = "runtime-info.json";
export type PaperclipRuntimeInfo = {
schemaVersion: 1;
instanceId: string;
pid: number;
host: string;
port: number;
dashboardUrl: string;
startedAt: string;
};
export function resolveRuntimeInfoPath(instanceId?: string): string {
return path.join(resolvePaperclipInstanceRoot(instanceId), PAPERCLIP_RUNTIME_INFO_FILENAME);
}
function parseRuntimeInfo(value: unknown): PaperclipRuntimeInfo | null {
if (!value || typeof value !== "object") return null;
const record = value as Record<string, unknown>;
if (
record.schemaVersion !== 1 ||
typeof record.instanceId !== "string" ||
!Number.isInteger(record.pid) ||
(record.pid as number) <= 0 ||
typeof record.host !== "string" ||
!Number.isInteger(record.port) ||
(record.port as number) <= 0 ||
(record.port as number) > 65_535 ||
typeof record.dashboardUrl !== "string" ||
typeof record.startedAt !== "string"
) {
return null;
}
return record as PaperclipRuntimeInfo;
}
export function readRuntimeInfo(instanceId?: string, filePath = resolveRuntimeInfoPath(instanceId)): PaperclipRuntimeInfo | null {
try {
const info = parseRuntimeInfo(JSON.parse(fs.readFileSync(filePath, "utf8")));
if (!info) return null;
if (instanceId && info.instanceId !== instanceId) return null;
return info;
} catch {
return null;
}
}
export function writeRuntimeInfo(
info: PaperclipRuntimeInfo,
filePath = resolveRuntimeInfoPath(info.instanceId),
): void {
const directoryPath = path.dirname(filePath);
fs.mkdirSync(directoryPath, { recursive: true, mode: 0o700 });
const temporaryPath = path.join(
directoryPath,
`.${path.basename(filePath)}.tmp-${process.pid}-${Date.now()}`,
);
try {
fs.writeFileSync(temporaryPath, `${JSON.stringify(info, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
flag: "wx",
});
fs.renameSync(temporaryPath, filePath);
} finally {
fs.rmSync(temporaryPath, { force: true });
}
}
export function removeRuntimeInfoForPid(
pid: number,
instanceId?: string,
filePath = resolveRuntimeInfoPath(instanceId),
): void {
const current = readRuntimeInfo(instanceId, filePath);
if (current?.pid !== pid) return;
fs.rmSync(filePath, { force: true });
}

57
cli/src/services/service-manager.ts
View File

@ -1,4 +1,5 @@
import fs from "node:fs/promises";
import { constants as fsConstants } from "node:fs";
import os from "node:os";
import path from "node:path";
import { execFile } from "node:child_process";
@ -33,6 +34,7 @@ export interface ServiceManager {
restart(): Promise<void>;
status(): Promise<ServiceStatus>;
logs(follow: boolean, lines: number): Promise<void>;
installedExecutablePath(): Promise<string | null>;
enableLinger?(): Promise<void>;
}
@ -75,6 +77,45 @@ export function resolveServiceShimPath(homeDir = os.homedir()): string {
return process.env.PAPERCLIP_SHIM_PATH?.trim() || path.join(homeDir, ".local", "bin", "paperclipai");
}
// The installed definition, not the current environment, is the truth
// about what the service executes: PAPERCLIP_SHIM_PATH may have changed
// or been unset since the definition was written.
function unescapeSystemd(value: string): string {
return value.replace(/\\\\|\\"|\$\$|%%/g, (m) =>
m === "\\\\" ? "\\" : m === '\\"' ? '"' : m === "$$" ? "$" : "%",
);
}
function unescapeXml(value: string): string {
return value.replace(/&(amp|lt|gt|quot|apos);/g, (_, name: string) =>
name === "amp" ? "&" : name === "lt" ? "<" : name === "gt" ? ">" : name === "quot" ? '"' : "'",
);
}
export function extractExecutableFromSystemdUnit(content: string): string | null {
const match = content.match(/^ExecStart="((?:\\.|[^"\\])*)"/m);
return match ? unescapeSystemd(match[1]) : null;
}
export function extractExecutableFromLaunchdPlist(content: string): string | null {
const match = content.match(/<key>ProgramArguments<\/key>\s*<array>\s*<string>([^<]+)<\/string>/);
return match ? unescapeXml(match[1]) : null;
}
// The service definition executes this path directly: existence is not
// enough — a directory or a non-executable file would satisfy fs.access's
// default mode and still crash the supervisor at spawn.
export async function isExecutableFile(filePath: string): Promise<boolean> {
try {
const stats = await fs.stat(filePath);
if (!stats.isFile()) return false;
await fs.access(filePath, fsConstants.X_OK);
return true;
} catch {
return false;
}
}
export function systemdServiceName(instanceId: string): string {
return instanceId === "default" ? "paperclipai.service" : `paperclipai-${instanceId}.service`;
}
@ -174,6 +215,14 @@ export class SystemdServiceManager implements ServiceManager {
return renderSystemdUnit({ instanceId: this.instanceId, shimPath: this.shimPath, homeDir: this.homeDir });
}
async installedExecutablePath(): Promise<string | null> {
try {
return extractExecutableFromSystemdUnit(await fs.readFile(this.definitionPath, "utf8"));
} catch {
return null;
}
}
private async ensureCurrent(): Promise<boolean> {
const changed = await writeIfChanged(this.definitionPath, this.renderDefinition());
if (changed) await this.runner("systemctl", ["--user", "daemon-reload"]);
@ -242,6 +291,14 @@ export class LaunchdServiceManager implements ServiceManager {
renderDefinition(): string { return renderLaunchdPlist({ instanceId: this.instanceId, shimPath: this.shimPath, homeDir: this.homeDir, stdoutPath: this.stdoutPath, stderrPath: this.stderrPath }); }
async installedExecutablePath(): Promise<string | null> {
try {
return extractExecutableFromLaunchdPlist(await fs.readFile(this.definitionPath, "utf8"));
} catch {
return null;
}
}
async install(options: ServiceInstallOptions): Promise<{ changed: boolean }> {
await fs.mkdir(path.dirname(this.stdoutPath), { recursive: true });
const changed = await writeIfChanged(this.definitionPath, this.renderDefinition());

8
cli/src/utils/health-url.ts
View File

@ -1,4 +1,4 @@
export function buildLocalHealthUrl(host: string | undefined, port: number): string {
export function buildLocalAppUrl(host: string | undefined, port: number): string {
const configuredHost = host?.trim();
const reachableHost = !configuredHost || configuredHost === "0.0.0.0" || configuredHost === "::"
? "127.0.0.1"
@ -6,5 +6,9 @@ export function buildLocalHealthUrl(host: string | undefined, port: number): str
const urlHost = reachableHost.includes(":") && !reachableHost.startsWith("[")
? `[${reachableHost}]`
: reachableHost;
return `http://${urlHost}:${port}/api/health`;
return `http://${urlHost}:${port}`;
}
export function buildLocalHealthUrl(host: string | undefined, port: number): string {
return `${buildLocalAppUrl(host, port)}/api/health`;
}

2
cli/tsconfig.json
View File

@ -4,5 +4,5 @@
"outDir": "dist",
"rootDir": ".."
},
"include": ["src", "../packages/shared/src", "../packages/plugins/create-paperclip-plugin/src"]
"include": ["src", "../packages/shared/src", "../server/src/types/express.d.ts", "../packages/plugins/create-paperclip-plugin/src"]
}

5
doc/AGENT-ARTIFACTS.md
View File

@ -79,6 +79,11 @@ When a task produces a user-inspectable deliverable file:
4. Link the printed attachment URL in the final issue comment.
5. Then set the final issue status.
For a response that is explicitly intended for an external chat conversation,
also pass each intended file with `paperclipai issue comment --attachment-id
<id>`. Paperclip binds only those exact uploaded files to that comment; other
task attachments remain internal.
Final comments should name and link the uploaded artifact or work product, not
just the local filesystem path. For workspace-only files, include the work
product title and recorded relative path. Local paths can be included as

99
doc/CLI.md
View File

@ -154,6 +154,90 @@ Choose local instance:
npx paperclipai run --instance dev
```
## Isolated Manual Test Drives
`paperclipai test-drive` creates or reuses an isolated local data directory,
ensures one usable CEO agent exists in a fresh database, starts Paperclip in the
foreground, and opens the browser after initialization succeeds. It never
installs a background service and never creates a goal, project, issue, task,
or heartbeat.
```sh
npx paperclipai test-drive \
[-d, --data-dir <path>] \
[--company-name <name>] \
[--agent-name <name>] \
[--harness <claude|codex|opencode>] \
[--model <model-id>] \
[--api-key-env <variable> | --api-key <value>] \
[--no-browser]
```
Defaults are `Test Company`, a `CEO` agent with the `ceo` role, and the Claude
harness. Without `--data-dir`, every invocation creates a unique OS temporary
directory and prints its absolute path. The directory is retained after exit
for inspection. An explicit data directory is reused and is never reset. The
reused directory must use Paperclip's embedded database; `DATABASE_URL`,
`DATABASE_MIGRATION_URL`, and configs with `database.mode: postgres` are
rejected so test-drive cannot mutate an external database. The server also
ignores the invocation directory's `.env` for test-drive launches, while still
loading the selected instance's own environment file. Reused directories also
retain the normal guard against colliding with a managed Paperclip service. The
server uses the first available loopback port at or above `3100`, so an
unrelated local Paperclip process can remain running.
Harness configuration:
| Harness | Agent adapter | Agent credential variable | Model |
| --- | --- | --- | --- |
| `claude` | `claude_local` | `ANTHROPIC_API_KEY` | Optional; omitted uses the adapter default |
| `codex` | `codex_local` | `OPENAI_API_KEY` | Optional; omitted uses the adapter default |
| `opencode` | `opencode_local` | `OPENROUTER_API_KEY` | Required and must begin with `openrouter/` |
OpenCode model references retain their complete path, including additional
slashes:
```sh
OPENROUTER_API_KEY=... npx paperclipai test-drive \
--harness opencode \
--model openrouter/anthropic/claude-sonnet-4.5
```
Credentials come from `--api-key`, the variable named by `--api-key-env`, or
the harness's canonical environment variable shown in the table. `--api-key`
and `--api-key-env` are mutually exclusive. A custom source variable is still
stored and projected under the canonical target variable:
```sh
MY_ROUTER_KEY=... npx paperclipai test-drive \
--harness opencode \
--model openrouter/openai/gpt-5.4 \
--api-key-env MY_ROUTER_KEY
```
Credentials are stored through Paperclip's user-secret reference path and are
redacted from Paperclip command output. Paperclip does not print `--api-key`,
and it removes the value from its JavaScript argument view immediately after
Commander parses it. Paperclip does not put the raw argument list in telemetry,
API metadata, or diagnostics. Command wrappers, operating-system process
listings, and shell history can still expose values passed in arguments. This
is an explicit tradeoff for the local test-drive workflow. Prefer an exported
canonical variable or `--api-key-env` when that matters. Provider connectivity,
local harness installation, credential validity, and model availability are
intentionally checked only when the agent first runs.
When invoked inside a linked Git worktree, the command ignores inherited
`PAPERCLIP_IN_WORKTREE` state, launches in worktree mode, and verifies **Run
tasks in this worktree** is armed for the current instance before opening the
browser. In a primary checkout or non-Git directory it launches without
worktree mode and does not alter the setting. On reuse, if any company already
exists, all bootstrap flags are ignored and companies, agents, and secrets are
left untouched; worktree-setting reconciliation is the only permitted
mutation.
Use `--no-browser` for a foreground instance that prints its ready URL without
opening it.
## Install, Update, And Uninstall
Managed installs keep CLI payloads under `~/.paperclip/cli`, expose a stable
@ -297,7 +381,7 @@ npx paperclipai context set --api-key-env-var-name PAPERCLIP_API_KEY
export PAPERCLIP_API_KEY=...
```
## Company Commands
## Organization Commands
```sh
npx paperclipai company list
@ -343,7 +427,7 @@ npx paperclipai issue get <issue-id-or-identifier>
npx paperclipai issue create --company-id <company-id> --title "..." [--description "..."] [--status todo] [--priority high]
npx paperclipai issue update <issue-id> [--status in_progress] [--comment "..."]
npx paperclipai issue delete <issue-id> --yes
npx paperclipai issue comment <issue-id> --body "..." [--reopen]
npx paperclipai issue comment <issue-id> --body "..." [--attachment-id <id...>] [--reopen]
npx paperclipai issue comments <issue-id> [--limit 50]
npx paperclipai issue comment:get <issue-id> <comment-id>
npx paperclipai issue comment:delete <issue-id> <comment-id>
@ -633,7 +717,7 @@ npx paperclipai skills install paperclipai:optional:browser:agent-browser --comp
External GitHub, skills.sh, local-path, and URL sources still go through
`skills import`; catalog commands are for the app-shipped catalog only.
### Company library
### Organization library
```sh
npx paperclipai skills list --company-id <company-id>
@ -716,8 +800,10 @@ Preview/install options:
`paperclipai company current --json`, or `PAPERCLIP_COMPANY_ID` to select the
target company. `company list` falls back to the scoped current company when
board-wide listing is forbidden. `teams install` creates agents and therefore
requires board authentication, an `agents:create` grant, or an agent with
explicit `canCreateAgents` permission.
requires board authentication, an `agents:create` grant, or an agent with the
`canCreateAgents` permission (enabled by default for newly created
standard-trust agents; low-trust agents and pre-existing agents without an
explicit value stay disabled).
- `--request-approval-on-forbidden` turns a 403 install denial into a linked
board approval request instead of a raw failed command; use
`--approval-issue-id <id>` to attach it to a specific issue. During Paperclip
@ -762,7 +848,7 @@ bootstrap credentials in Paperclip secrets.
Per-company provider vaults (multiple vault instances per provider, default
vault selection, coming-soon GCP/Vault) can be configured from the board UI under
`Company Settings → Secrets → Provider vaults` or through the provider-config CLI
`Organization Settings → Secrets → Provider vaults` or through the provider-config CLI
commands above. See the
[secrets deploy guide](../docs/deploy/secrets.md#provider-vaults) and
[API reference](../docs/api/secrets.md#provider-vaults) for the contract.
@ -899,7 +985,6 @@ npx paperclipai adapter delete <adapter-type>
npx paperclipai adapter config-schema <adapter-type>
npx paperclipai adapter ui-parser <adapter-type>
npx paperclipai adapter models <adapter-type> --company-id <company-id> [--refresh] [--environment-id <id>]
npx paperclipai adapter model-profiles <adapter-type> --company-id <company-id>
npx paperclipai adapter detect-model <adapter-type> --company-id <company-id>
npx paperclipai adapter test-environment <adapter-type> --company-id <company-id> --payload-json '{...}'
```

129
doc/DATABASE.md
View File

@ -122,8 +122,10 @@ All of these are optional; when unset, the driver defaults apply and behavior is
```sh
DATABASE_PREPARED_STATEMENTS=false # required for transaction-mode poolers; default: enabled
DATABASE_POOL_MAX=25 # connection pool size; default: 10
DATABASE_IDLE_TIMEOUT_SECONDS=60 # close idle pooled connections; default: keep open
DATABASE_IDLE_TIMEOUT_SECONDS=60 # close idle pooled connections; default: 60 (0 = keep open)
DATABASE_CONNECT_TIMEOUT_SECONDS=10 # default: 30
DATABASE_MAX_LIFETIME_SECONDS=1800 # recycle a pooled connection after this long; default: 30-60 min (random)
DATABASE_APPLICATION_NAME=paperclip # application_name in pg_stat_activity; default: paperclip
```
### Push the schema
@ -167,6 +169,26 @@ When authoring migrations or one-time backfills:
- Split schema changes, index creation, and data backfill into separate phases so each step has clear locking and rollback behavior.
- Treat the `check:migrations` CI gate as the enforcement backstop for these rules. If it flags a migration, rewrite the migration or add a suppression comment with the indexed predicate, batch bound, and reason the remaining scan is safe.
## Migration snapshots
`drizzle-kit generate` diffs `packages/db/src/schema/` against the newest snapshot in `packages/db/src/migrations/meta/`. That snapshot must describe the schema that every migration produces when they run in order. A snapshot that drifts from the schema makes the *next* migration wrong, because `generate` folds the drift into it. The drift can add a column that an earlier migration already created, which makes that migration fail on a fresh database. It can also drop a column that the schema still uses.
- Create every migration with `pnpm --filter @paperclipai/db generate`. Do not hand-write a snapshot.
- Do not hand-edit a snapshot to resolve a merge conflict. Renumber your migration and run `generate` again, as `packages/db/.gitattributes` describes.
- `packages/db/src/migration-snapshot-drift.test.ts` is the enforcement backstop. It repeats the diff that `generate` performs and fails when the newest snapshot no longer matches `packages/db/src/schema/`.
## Cloud runtime identity singleton
The private `instance_settings` row whose singleton key is
`cloud-runtime-identity/v1` records the immutable Cloud stack id, warm-pool
claim id, previous pool origin, canonical origin, and stack slug accepted from
Cloud's signed pre-activation assertion. It is separate from the normal
`default` settings row and never appears in the settings API. This is
intentionally instance-scoped rather than company-scoped: an instance has one
public identity, and the existing unique singleton-key index makes concurrent
or later attempts to replace it fail closed. The server loads the row before
constructing URL-dependent runtime services on every boot.
## Resource membership tables
Paperclip stores current-user sidebar membership state in:
@ -198,6 +220,96 @@ Triage writes serialize on the company and attention-source identity so concurre
`decision_retention` tracks the last observed source `activityAt`, Keep, reversible archive provenance, and monotonic source/archive versions. `decision_archive_notification_outbox` has a unique key over company, source identity, archive version, and immutable origin agent so repeated sweeps cannot enqueue duplicate notifications; delivery claims are retryable and coalesced per agent.
## Native runner persistence
Native runner state is additive to the existing heartbeat tables. Every existing
`heartbeat_runs` row defaults to `runtime_mode = 'legacy'`; adding these columns
does not select the native runtime or start a runner process. Native execution can
record its resolved runtime profile, provider session, driver, completion
contract, durable event cursor, and finalization phase on the run when a later
rollout explicitly selects it.
`completion_contracts`, `native_run_results`, `native_run_finalizations`,
`work_assessments`, `status_decisions`, and `status_decision_effects` form the
append-oriented evidence and status-decision chain. Unique fingerprints,
versions, ordinals, and idempotency keys make retries deterministic. Composite
foreign keys bind every contract, result, assessment, decision, effect, and
finalization to one company, issue, and run. The database rejects mixed-owner
evidence even when every referenced ID exists. Native source identities on
`heartbeat_run_events` are nullable so legacy events remain readable without
rewriting historical rows. Per-run native source identifiers are unique, while
the existing legacy sequence behavior remains unchanged. The hidden native
coordinator serializes on its bound `heartbeat_runs` row, allocates
`next_event_seq`, and commits a validated PRP event before the transport sends
its cumulative ACK. Byte-equivalent source retries return the existing cursor;
gaps and conflicting replays fail closed. Accepted structured results enter the
finalization ledger, whose retry time and owner lease are checked under a row
lock. None of these writes selects a runtime or changes a legacy run's execution
path.
Durable agent session goals are an additive projection on
`agent_task_sessions`, distinct from the business-goal hierarchy. The row stores
the negotiated goal capability, normalized snapshot and status, desired state,
provider source cursor, monotonic projection revision, and observation time.
`agent_session_goal_actions` is the control outbox: `(session_id, request_id)`
is unique, so retries return the original accepted action. Provider source
ordering fences duplicate and stale updates, and a cleared projection retains
its revision/cursor tombstone so an older provider event cannot resurrect it.
Issue `status_version` advances only when `status` changes. The JavaScript backup
path includes user-defined functions and triggers so a restored database keeps
that invariant. Removing or disabling a future native rollout flag must not
delete these records; persisted experimental runs remain available for recovery
and inspection.
`native_run_finalizations` also stores restart ownership and recovery state.
The controller owner is a server boot id, PID, operating-system process-start
timestamp, and monotonically increasing controller generation. Recovery writes
its correlated request id, current state, and a bounded JSON history. A
successor can take the lease immediately only when coordinated handoff or PID
and process-start evidence proves the prior controller is gone, or when the
lease expires. Recovery generation changes do not increment the independent
provider-attempt counter.
## Telegram private draft identities
`chat_telegram_draft_ids` is a content-free, instance-wide PostgreSQL sequence,
not a company-owned record. Telegram's native Stop callback carries a draft ID
but no actor or Paperclip generation. IDs therefore must not be recycled when
a transaction rolls back or an endpoint/company is deleted and its bot is
connected again. The sequence allocates positive 31-bit IDs without cycling;
exhaustion refuses new draft allocation rather than wrapping or falling back to
random IDs. Never reset it as part of chat cleanup.
The matching `chat_actions` entry remains company/endpoint-scoped and binds the
draft to its exact conversation, publication attempt, runtime, credential and
approved text. Stop can suppress that private draft's final publication; it
cannot cancel a task or model run. Logical backups preserve the sequence, but
restoring an older database may roll back its high-water mark: disaster recovery
must not assume stale provider Stop events are safe to reuse. That restore
boundary is not qualified by the rollback/concurrency regression.
## Attachment upload provenance
`issue_attachments.originating_run_id` records server-derived run attribution at
upload time. It is not writable through attachment or work-product update APIs.
Legacy attachments and uploads without a registered run keep a null value; the
migration deliberately does not infer attribution from mutable work products.
Deleting the originating run clears the reference and fails closed for automatic
chat handoff. An agent's external file selection must match the attachment's
company, task, agent, and originating run. Editing or recreating a work-product
record cannot reassign that authority to a later run.
## Question-response delivery receipts
`issue_question_response_deliveries` is the retry-safe, content-free outbox for
answered `ask_user_questions` interactions. Its unique interaction and correlation
indexes enforce one causal delivery per response. It records source and target
run/turn ids, payload digest, attempt/acknowledgement state, and one of `steered`,
`coalesced`, or `wake_fallback`; answer content remains only in
`issue_thread_interactions.result`. Deleting the interaction cascades its receipt,
while deleting a referenced run clears that run pointer without deleting history.
## Plugin database namespaces
The plugin runtime tracks plugin-owned database namespaces and migrations in `plugin_database_namespaces` and `plugin_migrations`. Hosted deployments that separate runtime and migration connections should set `DATABASE_MIGRATION_URL`; plugin namespace migration work uses the migration connection when present.
@ -274,3 +386,18 @@ pnpm secrets:migrate-inline-env --apply
```
Hosted AWS provider notes live in [SECRETS-AWS-PROVIDER.md](./SECRETS-AWS-PROVIDER.md).
### Persistent agent conversations
Migration `0274_agent_chat.sql` adds conversation identity/state and session generation/boundary columns to `issues`, plus idempotent client request IDs and processed session-boundary generations to `issue_comments`. The company/agent/user unique index resolves concurrent first writes to one issue. A check constraint preserves the assigned-agent identity and prevents terminal conversation status. Comment request IDs are unique per issue and user. There is no separate chat/message store. Provider sessions continue to use `agent_task_sessions`; `/new` removes only the matching conversation session, and session writers fence stale generations against the issue row.
## Legacy controller ownership
Legacy run claims atomically record `controller_boot_id`, a database-clock
`controller_lease_expires_at`, and `execution_stage` before workspace provisioning.
The lease renews independently of output. A different container must not infer
controller death from its own process map or numeric PIDs. Expiration grants
cleanup authority; it does not prove that remote inference has stopped. Recovery
revokes the previous boot identity with a conditional update. Its own claim also
expires so another sweep can finish cleanup after a restart. Historical rows keep
null ownership fields and follow the previous recovery path.

26
doc/DEPLOYMENT-MODES.md
View File

@ -64,6 +64,24 @@ Paperclip now treats **bind** as a separate concern from auth:
- recommended bind is `loopback` behind a reverse proxy; direct `lan/custom` is advanced
- local stdio MCP runtime slots fail closed by default; set `PAPERCLIP_TRUSTED_MCP_RUNTIME_HOST` only when a trusted worker/runtime host is configured to supervise those processes. Remote HTTP MCP remains the preferred public-hosted path.
### Paperclip Cloud warm-pool identity
A Cloud-managed warm-pool process initially boots under a `pool-*` origin. It
receives only Cloud's public verification set in
`PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS`. Before Cloud activates a claimed stack,
the existing server-to-server health request carries a short-lived Ed25519 JWS
that binds the immutable `PAPERCLIP_CLOUD_STACK_ID`, pool claim, previous
origin, canonical HTTPS origin, and slug. Paperclip verifies and persists that
one-time assertion, updates its live public/API URL provider, and acknowledges
the exact origin in `/api/health` before the first user request is admitted.
The Harness signing private key is never present in Paperclip, browsers, or
other tenant stacks. A different claim or destination cannot replace the
persisted identity. On restart, the durable identity is loaded before auth,
routes, and child-runtime configuration, even when provider variables are
temporarily stale. Self-hosted deployments continue to use their configured
`PAPERCLIP_PUBLIC_URL` and do not participate in this protocol.
## 4. Onboarding UX Contract
Default onboarding remains interactive and flagless:
@ -145,6 +163,12 @@ only to real browser session actors in `authenticated/private`; unauthenticated
requests, agent keys, board API keys, and local implicit board actors are
rejected.
This is intentionally a first-claim bootstrap contract: before an instance
admin exists, the first authenticated browser session that completes the claim
wins. Operators must keep a `bootstrap_pending` private deployment on a trusted
network and complete setup before admitting untrusted users. This behavior is
not an account-recovery or public-deployment mechanism.
The CLI fallback remains supported in all authenticated setup states:
```sh
@ -173,7 +197,7 @@ future public-hosted setup design explicitly changes this policy.
## 11. Relationship to Other Docs
- implementation plan: `doc/plans/deployment-auth-mode-consolidation.md`
- implementation plan: `doc/plans/2026-02-23-deployment-auth-mode-consolidation.md`
- V1 contract: `doc/SPEC-implementation.md`
- operator workflows: `doc/DEVELOPING.md` and `doc/CLI.md`
- invite/join state map: `doc/spec/invite-flow.md`

Some files were not shown because too many files have changed in this diff Show More