## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Every master push publishes a canary through release.yml, gated by
release-verify.yml — the fleet's staging deploys and the
nightly/beta/stable chain all start from those canaries
> - release-verify splits the server test suite across three shards with
a 20-minute job cap, while pr-trusted splits the same suite across five
> - The server suite grew on 2026-09-10 and the three shards moved to
17-19 minutes; that evening every push-triggered canary run was
cancelled by the 20-minute cap mid-verify, and no canary published after
18:50 UTC
> - This pull request mirrors pr-trusted's five-way server split in
release-verify, putting shards back at the 10-15 minute range with real
headroom
> - The benefit is a canary lane that reports test verdicts instead of
dying on an infrastructure cap
## Linked Issues or Issue Description
**What happened?**
Push-triggered Release runs stopped publishing canaries on 2026-09-10.
Runs at 19:34, 22:30, and 22:37 UTC were all cancelled by "The job has
exceeded the maximum execution time of 20m0s" on a `verify_canary /
General tests (server (N/3))` shard. No canary published after 18:50
UTC, which also starves the staging fleet's continuous deploys.
**Expected behavior**
release-verify's server shards finish well inside the 20-minute cap and
runs conclude with a test verdict, as pr-trusted's five-way split of the
same suite does (10-15 minutes per shard).
**Steps to reproduce**
1. Compare server shard durations in the `verify_canary` job across
2026-09-10: 11-14 minutes in the morning, 17-19 minutes from 15:06 UTC,
over 20 minutes by evening.
2. Observe runs 34521169020, 34537798488, and 34538332689 cancelled at
the cap.
**Paperclip version or commit**
`master` at `d1ba17eec` (current tip; its canary run was one of the
cancelled ones).
## What Changed
- `release-verify.yml`: the `general-server` matrix goes from three
shards to five, byte-for-byte the shape `pr-trusted.yml` already runs,
with a comment recording why.
## Verification
- The identical five-way split runs green on every pr-trusted run (10-15
minutes per shard today, including on PRs merged this evening).
- The suite's own growth (slower chat-connector tests) is being
addressed separately; this PR only removes the artificial cliff.
## Risks
- Low risk: two more runners per verify run; no test content changes. If
shard durations regress further, the cap fires again — which is the
correct signal once shards have honest headroom.
## Model Used
- Claude (Anthropic), model ID `claude-fable-5` (Claude Fable 5),
extended thinking, tool use via Claude Code CLI.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - People also ask agents for work in their existing chat tools.
> - Each external conversation needs one task and a current authorized
source.
> - Retries, Stop, and provider failures must not duplicate work or
expose private data.
> - The first chat PR establishes the opt-in provider and data
contracts.
> - This PR adds experimental channel integration and its durable
control plane.
> - Users can request work from connected channels and inspect delivery
in Paperclip.
## Linked Issues or Issue Description
Refs #13100 and #13092. This is the second of exactly two chat PRs.
Foundation #13100 is merged and changed 143 files. Runner prerequisite
#13092 is also merged. This PR changes 400 files against master, below
the 500-file review limit. It contains no wireframe images or HTML
galleries.
## What Changed
- Add native Slack, GitHub, Microsoft Teams, Telegram, and Discord chat
connections. Keep chat disabled unless the operator enables experimental
chat connectors. Preserve the production GitHub tool connection and its
normal setup path.
- Bind each provider bot identity to one immutable Paperclip agent. Bind
each admitted external conversation to one task. Paperclip owns tasks,
runs, permissions, and audit records.
- Add durable admission, per-conversation queues, questions, task
controls, progress, final replies, images, files, and delivery receipts.
Board comments remain internal unless explicitly sent to the channel.
- Check current identity, provider reach, resource access, credentials,
runtime generation, and exact source before provider effects. Keep
private responses private. Never send raw reasoning, private logs,
credentials, or tool arguments.
- Hold uncertain sends for explicit audited resolution. Make Board
Send-to-channel atomic and idempotent. Keep reconnect and setup
credentials in Paperclip secret storage.
- Preserve current native-runner authority across retries, lost
acknowledgements, and recovery. Keep immutable input and completion
contracts separate from newer user input. Receipt reconciliation cannot
launch a provider.
- Reconcile chat close/new ordering and provider-effect lock order.
Audit resource access changes in the same transaction. Submit only the
selected resource from each UI toggle so stale pages cannot undo
unrelated access changes.
- Drain Codex stdout before certifying process exit. Bound the drain
with the existing shutdown grace. Preserve observed terminal authority
without treating an undrained process as successful or reusable.
- Incorporate master `018ca5da` with its ACP Stop, mobile task layout,
runner packaging, and official lock changes. Preserve dedicated
chat-answer continuations in both directions when ordinary queued
comments are adopted after Stop.
- Fence late adapter readiness behind an earlier Stop for the same run.
Preserve verified cleanup for registered adapters. Handle single Stop,
agent pause, duplicate Stops, and failure release without creating a
false cancellation receipt.
- Incorporate master's `6dd48cad4` wake-queue extraction. Preserve exact
failed-chat retry authorization and lineage, retired question-source
suppression, and the block on generic recovery that would discard the
admitted source. Fresh deferred input retains its separate promotion
path.
- Incorporate master `2a05b5ed3` and its queue-admission extraction,
simplified transaction ports, and separate runner CI job. Preserve exact
durable receipts, actor separation, and dedicated-answer isolation
through the new module. A failed receipt insert rolls back the
accompanying deferred-wake merge.
## Verification
Current head: `afe19299d06253cb628eb398e91d1200ea9f412a`, incorporating
master `2a05b5ed3457ea33efd6895520447d1d97fe98d8`. The conflicts are
resolved. This successor fixes two test-harness boundaries exposed by
CI: per-case route-module preparation and actual durable-save completion
before intentional runner termination. Production code and all existing
test/turn deadlines are unchanged. [Exact-head Greptile
review](https://github.com/paperclipai/paperclip/pull/13038#issuecomment-5587250594)
is **5/5**, completed September 10 at 13:20:55 UTC, with no actionable
findings or open review threads. [Fresh exact-head
CI](https://github.com/paperclipai/paperclip/actions/runs/34481724341)
passes **all 24 jobs**, including Build and both required aggregates.
Normal exact-head guarded merge was attempted and rejected by the
remaining branch approval policy: CODEOWNER review is required and no
human approval is present. Normal **squash auto-merge is enabled** as of
September 10 at 13:36:26 UTC. Requested CODEOWNERS have been notified;
no approval bypass or self-approval was used. Earlier-head results below
remain historical evidence, not qualification of this successor.
- Final exact-head Linux evidence: 995/995 chat integration cases; 36/36
agent-skills routes; 35/35 runner live-session cases, including real
process kill/resume; 1948 runner Vitest cases with three existing
benchmark/platform guards; 870/870 API-authority cases; and 104 browser
cases with four existing optional skips. Rust, conformance/replay, full
repository build, typecheck, canary, all server/workspace shards, and
both required aggregates pass with normal CI concurrency. Earlier failed
attempts remain recorded below.
- Latest test-only qualification: 141/141
route/permissions/authentication cases pass in separate cold forks, with
plain server types and independent review clear. The real-runner suite
passes 35/35, with plain runner types and independent review clear. A
controlled premature-save acknowledgement fails as expected; matching
ownership/effect/process evidence, rejected saves, real turn outcome,
test abort, and pre-kill liveness are covered. No local reproduction of
the original CI scheduling failure is claimed. The preceding [CI
run](https://github.com/paperclipai/paperclip/actions/runs/34479680858)
passes 21/24 jobs, including all 995 Linux chat cases and browser
aggregate (104 passed, four existing optional skips); only Build, the
skills serialized shard, and the required verification aggregate fail.
Its exact-head Greptile review was 5/5. Both failed job logs are
retained.
- Final fixture qualification: all eight focused Discord cases and all
995 chat integration cases pass. The exact modal statement/PID is
observed before taking the real connection lock; the test then proves
its actual blocking relationship before mutation. Original SQL
execution, provider behavior, negative assertions, and 1s/15s timeouts
remain unchanged. Independent review is clear and test/production hashes
remain frozen. The preceding [CI
attempt](https://github.com/paperclipai/paperclip/actions/runs/34477184777)
passed 22 jobs, including Build/runner, typecheck, canary, all other
test shards, and browser aggregate (104 passed, four existing optional
skips); the two fixture failures and failed verification aggregate
remain recorded, not relabeled as a pass.
- Current queue-module composition: 308/308 recovery/batching/queue/Stop
tests; 995/995 full chat integration; 89/89 module tests, including real
PostgreSQL receipt-insert rollback; 24/24 workflow/module-boundary
tests; plain server and UI types. All four actual local process/ACP
browser paths pass in 1.4 minutes. Fresh databases, no skips or retries,
stable reviewed source hashes. The initial boundary failure is retained;
its no-op service wrapper was removed without changing recovery context
or weakening the check. An exploratory standalone test-directory
typecheck fails because its new upstream transformation config is not a
standalone typechecking project; standard CI/build does not invoke it,
and no configuration was weakened to suppress those diagnostics.
- The preceding head `e02a63d462ce5d47433b0aeb632bb6fd20aab1ba` passed
[all 24 CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34436462958)
and exact-head Greptile review at 5/5. Required CODEOWNER review
prevented its normal merge before master advanced again.
- Final extracted-module composition: 307/307 recovery, batching, queue
and Stop-control tests; 995/995 full chat integration; 49/49 module
tests including eight PostgreSQL adapter cases; and 19/19 issue-update
tests. Plain server types pass. All four actual local process/ACP
browser paths pass in 1.3 minutes. Fresh databases, no skips or retries
in these cohorts, frozen source hashes, and independent review clear.
- The preceding head `3e4e1c1c` passes [all PR CI
jobs](https://github.com/paperclipai/paperclip/actions/runs/34415826820),
including Build and required `ci / verify` and `ci / e2e`. Both the
original Rust failure and the previously load-sensitive lineage fixture
pass with unchanged Linux concurrency. Master advanced afterward and
required this reconciliation.
- Final master composition: 448/448 focused UI tests, 186/186 adapter
tests, 24/24 queue/control tests, and 11/11 packaging tests. Plain UI,
server, shared, and adapter types pass. Token gates and diff checks
pass. Independent server and UI reviews are clear.
- Stop-registration regression: both real-service cases fail against
exact `a95` source and pass with the fix. The full corrected
recovery/control suite passes 265/265. Duplicate-owner and failed-Stop
controls also pass. Plain server types pass. The readiness barrier
prevents provider startup without adding an acknowledgment to an already
terminal run.
- Final qualification strengthens terminal-field equality and repeats
both affected cases successfully on a fresh database. All four actual
local process/ACP browser paths pass again in 1.3 minutes, without skips
or retries. The final screenshot shows Cancelled, a paused subtree,
retained input, and no error toast.
- Two new actual-service regressions fail before the merge fix. They
prove that queued-comment adoption could consume a dedicated chat answer
or add unrelated input to that answer. The fixed four-case cohort
passes, including ordinary upstream continuation and adapter Stop
controls. Full recovery passes 257/257. All four actual local
process/ACP Stop browser flows pass in 1.4 minutes, without skips or
retries, on a fresh database.
- The unchanged runner artifact was qualified with 171/171 transport
tests, 870/870 API-authority tests, conformance 1/1, and replay 11/11.
Six controlled reader tests prove the exit/drain repair. Its local
serial Rust workspace passed 546 top-level cases plus two invoked
helpers; the later passing Linux CI supplies default-concurrency
evidence.
- Prior exact-source full chat integration passes 995/995. Settings
regressions cover concurrent stale pages, 501 destinations, pending
state, rejected updates, and explicit retry. These deterministic tests
do not prove live provider behavior.
- Retained failed attempts and their causes are in the [qualification
log](afe19299d0/doc/plans/chat-adapters/2026-09-08-chat-queue-and-webhook-repair.md).
The first merge adapter run timed out while macOS slept for 290 seconds.
Its unchanged repeat passed with a temporary sleep guard. No assertion,
deadline, or CI gate was weakened.
Review commands include `pnpm --filter @paperclipai/server exec vitest
run src/__tests__/heartbeat-process-recovery.test.ts
src/__tests__/issue-queued-comments-routes.test.ts` and `pnpm exec
playwright test --config tests/e2e/playwright.config.ts
tests/e2e/acp-stop-continuation.spec.ts`. Database suites require fresh
disposable databases. See the [browser
runbook](afe19299d0/doc/plans/chat-adapters/2026-09-04-chat-adapters-browser-e2e-runbook.md)
for provider setup and separate live acceptance steps.
## Risks
- This remains experimental. Deterministic tests and bounded live
evidence do not establish every provider feature, tenant, permission
layout, or media shape. Teams work-tenant qualification is still open.
- Failed and uncertain provider effects remain visible and can require
operator action. A transport receipt does not prove recipient
visibility.
- Native controller and runner artifacts must remain compatible.
Preserve lease ownership, terminal authority, source binding, and
quarantine during future changes.
- Access and audit rows commit together, but activity notifications
remain best-effort. This is not a new durable event outbox.
- The PR operation does not deploy a live server, replace its runner, or
change provider permissions. Remaining live qualification is documented
in the [temporary
handoff](afe19299d0/doc/plans/chat-adapters/2026-09-08-open-qualification-followups.md).
## Model Used
OpenAI Codex assisted with implementation, tool execution, testing, and
review. The work records `gpt-6-astra` assistance. The environment does
not report a context-window size. No private reasoning traces are
included.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip uses GitHub Actions to verify changes before release.
> - The Paperclip Runner has a separate verification boundary.
> - The build job currently runs this verification before the workspace
build.
> - This pull request moves runner verification into its own parallel
job.
> - The benefit is clearer CI results and less wait time for independent
work.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The trusted PR and release verification workflows run Paperclip Runner
verification inside the Build job.
**Subsystem affected**
Cross-cutting (GitHub Actions CI workflows).
**Current behavior**
The Build job runs `pnpm --filter @paperclipai/paperclip-runner
check:all` before it builds the workspace. A runner verification failure
appears as a Build failure. The workspace build cannot run in parallel
with runner verification.
**Proposed behavior**
Each workflow has a `Verify Paperclip Runner` job with the same
checkout, dependency install, and command. The Build job only builds its
required outputs. Both jobs run after the same gate and policy jobs.
**Reason and benefit**
The runner command is an independent verification boundary. A dedicated
job gives it a clear status and allows it to run in parallel with Build.
**Breaking changes**
None. The same runner verification command still runs in both workflows.
## What Changed
- Added a dedicated `Verify Paperclip Runner` job to the trusted PR
workflow.
- Added a dedicated `Verify Paperclip Runner` job to the release
verification workflow.
- Kept the Build jobs independent and retained their existing build
commands.
- Updated the trusted-workflow policy test for the additional
dependency-install job.
## Verification
- Ran `git diff --check`.
- Ran `node --test ./scripts/__tests__/e2e-shard.test.mjs`.
- Ran `pnpm exec prettier --check .github/workflows/pr-trusted.yml
.github/workflows/release-verify.yml`.
- Confirmed both jobs retain their prior runner, dependency, and policy
prerequisites.
## Risks
Low risk. The runner verification job repeats the existing setup. It
adds one parallel GitHub Actions runner to each affected workflow.
## Model Used
OpenAI Codex, GPT-5.6, 128k context window, reasoning and tool-use
capabilities.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip runs agent work in isolated worktrees.
> - Each worktree needs dependencies that match its source and patches.
> - A failed install currently loses its exit status after an `if`
statement.
> - The provisioner can then record a successful dependency fingerprint.
> - This pull request preserves failures and bounds lockfile recovery.
> - Agents receive a usable workspace or an accurate provisioning
failure.
## Linked Issues or Issue Description
**What happened?** A nonzero pnpm install could return success and save
a fingerprint. Patch changes alone also did not invalidate the
fingerprint.
**Expected behavior:** Fail provisioning on an unsuccessful install.
Retry known frozen-lockfile mismatches once and record success only
after installation succeeds.
**Steps to reproduce:** Run the provisioner in a worktree with a pnpm
install that exits nonzero. The regression suite uses real shell
execution and a controlled pnpm fixture.
**Paperclip version or commit:** Reproduced on master before this
change. **Deployment mode:** Self-hosted. **Installation method:** Git
checkout. **Agent adapters involved:** Core workspace provisioning.
**Database mode:** Not relevant. **Access context:** Execution host.
**Node.js version:** 26.4.0 locally; supported minimum remains
unchanged. **Operating system:** macOS locally and Linux execution
hosts.
**Relevant logs or output:** `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`,
`ERR_PNPM_OUTDATED_LOCKFILE`, and ordinary installation failures.
**Additional context:** Related lockfile maintenance: #13061. No
lockfile or workflow changes are included.
## What Changed
- Capture the failed install status inside the `else` branch.
- Use the existing single retry for both frozen-lockfile mismatch
errors.
- Include patch contents in the dependency fingerprint.
- Add executable regression coverage and document the behavior.
## Verification
- All CI checks passed, including build, typecheck, tests, browser
suites, canary dry run, and security scans. Greptile: 5/5 with no
remaining findings.
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`:
19 passed; one existing test requires Linux flock and was skipped on
macOS.
- `bash -n scripts/provision-worktree.sh` and `git diff --check` passed.
- Full workspace typecheck and build passed in the companion runner-fix
worktree at the same base revision. This change only touches shell
provisioning, its tests, and documentation.
- CI and review are pending.
## Risks
The existing non-frozen recovery can update a worktree-local lockfile.
Committed lockfile updates remain bot-owned. Ordinary failures now
correctly stop provisioning and may expose previously hidden
installation problems.
## Model Used
OpenAI GPT-6 through Codex, with code editing, shell execution, tests,
and browser inspection. The exact model variant and context-window size
are not exposed in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Runner executes agents through native and managed provider
drivers.
> - The direct live eval layer had drifted from the current Runner
contracts.
> - The old local workflow did not provide a complete parallel campaign
or durable report history.
> - The Runner also needed current native OpenCode and OpenRouter
qualification.
> - This pull request restores the direct campaign, corrects the runtime
gaps that the campaign found, and adds safe hosted Evalbook history.
> - The benefit is repeatable model comparison against an immutable
Runner and eval source revision.
## Linked Issues or Issue Description
Refs #11297
Refs #11634
**What existing behavior does this improve?**
This improves the direct live `paperclip-runner` eval workflow, provider
execution contract, and static Evalbook reporting path.
**Current behavior**
The direct evals do not have one maintained full campaign on current
`master`. OpenCode has no qualified multi-model OpenRouter roster.
Parallel provider bursts can compact committed events before the
transport observes them. Local reports do not have a separate safe S3
history index.
**Proposed behavior**
Run one immutable roster-plus-case matrix. Use the shared paid AWS
runner fleet. Keep raw artifacts access-controlled. Publish a sanitized
canonical Evalbook report under the separate `runner-protocol-evals` S3
prefix. Keep immutable campaign directories plus root history, latest,
and latest-green pointers.
**Reason and benefit**
Maintainers can compare native Codex, native OpenCode, ACPX, Claude
Managed, and AWS AgentCore behavior over time. They can inspect failures
without mixing this direct protocol layer with browser full-stack E2E.
**Breaking changes**
None. The new workflow and S3 prefix are additive. The existing Runner
full-stack E2E workflow and report remain separate.
## What Changed
- Added a trusted two-shard direct live workflow for up to 393
roster-plus-case cells.
- Reused the numeric actor allowlist, protected paid environment, and
RunsOn fleet controls from Runner full-stack E2E.
- Added immutable Runner and eval revision resolution, exact credential
boundaries, bounded retries, and cost ceilings.
- Added a public report projection that removes sessions, transcripts,
tool payloads, state, traces, raw failures, remote profile identities,
and credential-shaped values.
- Added additive S3 history under `runner-protocol-evals`, with
immutable campaigns and mutable root index pointers.
- Added native OpenCode model injection and current OpenRouter pricing
contracts.
- Fixed direct eval completion, workflow execution, semantic discovery,
warm-attach state reset, executable binding, and event-burst handling.
- Kept Runner browser full-stack E2E behavior and publication separate.
- Documented local and hosted direct eval operation.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `pnpm --filter @paperclipai/paperclip-runner build:typescript` —
passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml
.github/workflows/runner-full-stack-e2e.yml` — passed.
- Local current matrix at the revision in
[paperclip-evals#17](https://github.com/paperclipai/paperclip-evals/pull/17)
— 323 cells across 10 enabled configurations completed.
- Final local current matrix — 269 passed, 11 behavior failures, and 43
expected macOS-only ACPX platform failures.
- Targeted Runner checks — 13/13 eval-session tests, 15/15
publisher/security tests, and package typecheck passed; complete PR CI
is green, including all browser E2E shards.
## Risks
- Paid live campaigns can consume provider budget. Actor authorization,
exact per-cell ceilings, protected environments, and explicit schedule
enablement bound this risk.
- Public reports can leak provider data. The workflow publishes only a
separately projected report and validates every file before upload.
- The new workflow cannot publish until it is present on the default
branch. This pull request does not change the existing
`runner-full-stack-e2e` publication path.
- The campaign is large. It uses two GitHub matrices and caps combined
concurrency at the shared fleet limit.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, code editing, browser
inspection, repository tools, and live provider execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub Actions builds the Docker images that ship Paperclip, and
downstream deployments consume the `-cloud` image variant on every
master merge.
> - PR #12769 slimmed the Docker build context with a broad
`.dockerignore` block for `packages/paperclip-runner`, and the block
also removed three files the image build itself reads.
> - The image build re-runs the runner's generated-file drift checks, so
it found no committed capability contract in the context and failed on
every master commit after the merge.
> - PR CI never runs those checks against the Docker context, so the
pull request stayed green and the breakage only appeared post-merge, on
every image build.
> - This pull request restores the three files with narrow
`.dockerignore` exceptions and adds a PR CI job that runs the drift
checks against the exact Docker build context.
> - The benefit is that image publishing works again now, and the next
context-slimming regression fails the pull request instead of every
post-merge image build.
## Linked Issues or Issue Description
Refs #12769 (the context-slimming change that exposed this) and #12608
(which committed the generated contract outputs the image build checks).
**What happened?**
Every `Docker` workflow run on master failed from 2026-09-04 12:58Z
onward, in both the `build-and-push` and `build-and-push-cloud` jobs.
The failing step reported `Generated contract drift:
generated/capability/capability-contract.md` from
`check:capability-contract` inside `pnpm --filter @paperclipai/server
build`. The committed contract file is current — regeneration on a full
checkout is a no-op. The file was simply absent from the build context:
the new `packages/paperclip-runner/**/*.md` ignore rule strips the
committed drift-check outputs
(`generated/capability/capability-contract.md`,
`generated/capability/downstream-handoff.md`), and the
`packages/paperclip-runner/docs` rule also strips
`docs/capability-contract.md`, which `check:capability-inventory` reads
next in the chain. No cloud image published for eight hours, which
stalled every downstream deployment that consumes the canary images.
**Expected behavior**
The Docker build context must contain every file the image build reads,
and a change that removes one must fail the pull request that introduces
it, not every image build after the merge.
**Steps to reproduce**
1. Check out master at any commit from `af3023f1` onward.
2. Run `docker buildx build -f .github/docker-context-checks.Dockerfile
.` (the probe added by this PR), or start the real `Docker` workflow
build.
3. Observe `Generated contract drift:
generated/capability/capability-contract.md` — while `node
packages/paperclip-runner/scripts/generate-capability-contract.mjs
--check` passes on the same checkout outside Docker.
**Paperclip version or commit**
`d593463ab` (master tip at diagnosis time; first failing commit
`af3023f1`).
**Deployment mode**
GitHub Actions image builds (`docker.yml`), consumed by managed cloud
deployments.
## What Changed
- `.dockerignore`: narrow exceptions (last match wins) re-include the
committed drift-check outputs
(`!packages/paperclip-runner/generated/**`) and the inventory check's
documentation input
(`!packages/paperclip-runner/docs/capability-contract.md`). Every other
exclusion from #12769 stays: no crate declares an explicit `[[test]]`
target, so cargo builds without the `tests` directories, and the image
build chain never runs the excluded smoke scripts.
- `.github/docker-context-checks.Dockerfile` (new): a small probe that
COPYs the real build context — identical `.dockerignore` semantics — and
runs the dependency-independent drift checks inside it
(`generate-capability-contract.mjs --check`,
`check-capability-inventory.mjs`). ajv installs in an isolated directory
for schema validation only; codegen checks such as
`generate-protocol-schema-module` stay out because their emitted bytes
vary with the ajv release and would raise false drift alarms outside the
locked dependency tree.
- `.github/workflows/pr-trusted.yml`: new `docker_context_integrity` job
builds the probe on every full-CI pull request, and the existing
`verify` aggregate now requires its result, so the guard gates merges
through the same required check as the other lanes.
- Activation note: `pr.yml` pins `pr-trusted.yml` by commit SHA, so the
new job starts gating pull requests after the usual follow-up `ci:
activate ...` pin bump once this merges. The `.dockerignore` fix needs
no activation — `docker.yml` reads it directly, so image builds recover
on the first master commit after this merges.
## Verification
- `docker buildx build -f .github/docker-context-checks.Dockerfile .` on
master (before the `.dockerignore` fix): fails with the exact production
error, `Generated contract drift:
generated/capability/capability-contract.md`.
- Same command with the `.dockerignore` exceptions applied: passes,
which also proves BuildKit honors the `!` exceptions, including the file
inside the excluded `docs` directory.
- `node scripts/generate-capability-contract.mjs --check` on a full
checkout: passes both before and after, which confirms the committed
contract was never stale — only missing from the context.
- Static sweep of every script in the image build chain (`build`,
`build:typescript` and their `check:*` steps) against the ignore rules:
the three restored files are the only build inputs the #12769 block
strips.
- YAML for `pr-trusted.yml` lints clean.
## Risks
- Low. The `.dockerignore` exceptions only re-add three committed files
to the build context; image contents do not change otherwise.
- The probe job adds one context transfer and two Node scripts per
full-CI pull request run (about one to two minutes, no dependency
install beyond one isolated ajv package).
- The `verify` aggregate now also requires the new job, mirroring the
existing pattern for the other lanes; on non-full-CI runs the job skips
and `verify` asserts the skip, unchanged from how the other lanes
behave.
- The new job only takes effect for pull requests after a follow-up pin
bump in `pr.yml` (same two-step flow as every `pr-trusted.yml` change).
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
Claude Fable 5 (Anthropic, model id `claude-fable-5`), extended
thinking, agentic tool use in Claude Code: GitHub Actions log forensics
to isolate the failing check, static analysis of the build-chain scripts
against the ignore rules, and local docker buildx runs to reproduce the
failure and verify the fix.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (the docker probe, both
failing-before and passing-after; the drift checks themselves on a full
checkout)
- [x] I have added or updated tests where applicable (the probe IS the
regression test for this class)
- [x] I have updated relevant documentation to reflect my changes
(inline comments in `.dockerignore` and the probe explain the invariant)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
Paid cells now reuse the AWS image's system Chrome, but Playwright video
recording still resolves its revision-pinned FFmpeg helper from the
Playwright cache. Run 33875618534 proved Chrome qualification succeeds
and then failed before provider startup because that helper was absent.
The same run also exposed that generic lock repair can churn unrelated
package platform metadata, so the automated repair paths need
resolution-only regeneration rather than lockfile-only metadata refresh.
## What Changed
- install Playwright FFmpeg only on the AWS/system-Chrome path
- retry the small helper installation up to three times before provider
secrets are exposed
- keep the GitHub-hosted Chromium fallback unchanged
- bind static coverage to the exact FFmpeg step block and its pre-secret
ordering
- add pnpm `--resolution-only` to all four automated lock-repair paths
while retaining full transitive resolution
- require resolution-only repair in the shared workflow regression
The actual generated lockfile correction remains bot-owned by PR #12828
and is intentionally not committed here.
## Verification
- `node --test
.github/scripts/tests/lockfile-refresh-workflows.test.mjs`
- `actionlint -ignore SC2012` on all modified workflows
- focused Prettier checks
- `git diff --check`
- prior run 33875618534: system Chrome 151 qualified; missing Playwright
FFmpeg was the sole cell startup failure
## Risks
Low. The new network operation is limited to Playwright's pinned FFmpeg
payload, happens before paid credentials are exposed, and leaves the
hosted-runner path unchanged. Resolution-only is still a full
dependency-resolution pass, unlike lockfile-only, while avoiding
unrelated current-platform metadata churn.
## Model Used
GPT-5
Use full dependency resolution in automated lockfile repair paths, add regression coverage, and refresh the stale Rollup snapshot.
Co-Authored-By: Dotta <cryppadotta@users.noreply.github.com>
Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: lockfile-bot <lockfile-bot@users.noreply.github.com>
## Thinking Path
> - Paperclip is the control plane for AI-agent companies.
> - Agent outputs must remain visible after a run and easy to inspect
from a task.
> - The thread and artifact inventory need one consistent rich-card
vocabulary.
> - Run uploads also need durable artifact registration and
producing-run context.
> - Reviewers need deterministic examples for each rich-card kind and
state.
> - This pull request adds the shared presentation, registration,
inventory, and Storybook review coverage.
> - The benefit is a complete output path that reviewers can inspect
without seeded data.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
This change improves work-product presentation in task threads and the
task Artifacts tab.
**Subsystem affected**
The change affects shared work-product contracts, the runner diff path,
server attachment and work-product services, GitHub metadata refresh,
the React board UI, and Storybook.
**Current behavior**
The thread used generic cards. Some files uploaded by a run existed only
as message attachments. The Artifacts tab showed a flat list without run
context or filters. Storybook showed only one resting card per kind.
**Proposed behavior**
The thread uses rich cards for supported work-product types. Each
run-produced file registers one attachment-backed artifact work product.
The Artifacts tab groups outputs by run and supports filters. Storybook
shows every kind and requested state, PR lifecycle states, stats
variants, truncation, mobile layout, and message-tail media.
**Reason and benefit**
Users can identify outputs quickly. Reviewers can inspect all card
permutations without creating task data.
**Breaking changes**
None. The metadata fields and automatic artifact registration are
additive. Existing attachments and work products keep their current
behavior.
## What Changed
- Added a shared rich work-product card with kind-specific content and a
compact inventory variant.
- Added pull-request and commit diff metadata plus bounded GitHub state
refresh.
- Added media strips and typed file chips to message-tail attachments.
- Registered each run-produced attachment as an artifact work product in
the same server transaction.
- Grouped task artifacts by run with agent and timestamp headings.
- Added type and run filters, image thumbnails, compact cards, and a
company Artifacts link.
- Added a Storybook kind-by-state matrix with stats variants for all
eight visual kinds.
- Added PR open, draft, merged, and closed examples, long-title
truncation, an exact 375-pixel viewport, and message-tail overflow
coverage.
- Closed reconciled runtime work products when the linked runtime stops
or disappears, so the card shows `Stopped` instead of `Unhealthy`.
### Screenshots
Before: one resting card per kind.
After: the kind and state matrix.
After: message-tail media at 375 pixels.
[Open the Storybook evidence
viewer](https://pages.paperclip.ing/rich-work-product-storybook-20260902/).
The earlier artifact inventory comparison remains available in the
[artifact inventory
viewer](https://pages.paperclip.ing/rich-artifacts-inventory-proof-20260902/).
## Verification
- `pnpm --filter @paperclipai/ui typecheck` passed.
- `pnpm check:token-gates` passed.
- `pnpm build-storybook` passed.
- `pnpm exec vitest run
server/src/__tests__/work-product-runtime-reconciliation.test.ts` passed
with 5 tests.
- Chromium visual checks passed at desktop and 375-pixel widths.
- All 30 latest-head GitHub checks passed. One unrelated annotation test
was flaky and passed on its single retry.
- Greptile passed at 5/5 with zero unresolved threads.
## Risks
- Low risk. The Storybook change adds review fixtures only. The runtime
fix changes read-time reconciliation without database writes.
- The matrix is intentionally large so every permutation stays visible
in one review surface.
> I checked `ROADMAP.md`. This work does not duplicate planned core
work.
## Model Used
- OpenAI Codex with GPT-5 and GPT-5.6-sol across this pull request.
Reasoning, tool use, and code execution were enabled. The context-window
size is not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My public branch name describes the change and contains no
internal task id
- [x] I have run tests locally and the changed-path tests pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.
**Problem or motivation**
The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.
**Proposed solution**
Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.
**Alternatives considered**
We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.
**Roadmap alignment**
This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.
Related stack:
- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.
## What Changed
- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.
## Verification
Passed locally:
- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.
I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.
## Risks
The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.
Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.
Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.
The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.
This pull request does not enable the experimental native runner for
existing instances.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The onboarding command starts the local server and opens the
first-run wizard.
> - Interactive onboarding stopped opening the browser by default.
> - Organization creation could also succeed in the API while the wizard
stayed on the name step.
> - The npm canary workflow did not prove that the published package
could complete this path.
> - This pull request repairs the startup and organization transitions
and adds an exact-version canary smoke gate.
> - The benefit is a quickstart that works for users and is tested after
each canary publish.
## Linked Issues or Issue Description
Related: #12557 covers a separate final-route onboarding handoff.
**What happened?**
Interactive `paperclipai onboard` runs did not open the onboarding page.
The organization API request could succeed while a same-company context
update caused the wizard to stay on the organization step. The canary
release lane did not test the exact published npm package through this
path.
**Expected behavior**
Interactive onboarding must open the browser once. A successful
organization request must advance to the first-agent step when the
surrounding context adopts the same organization. Each published canary
must install in a clean environment and reach the model connection step.
**Steps to reproduce**
1. Run `npx paperclipai@canary onboard --data-dir "$(mktemp -d
/tmp/paperclip-canary.XXXXXX)"` in an interactive terminal.
2. Enter an organization name while the company context refreshes from
the create response.
3. Observe that the browser does not open or that the wizard can remain
on the organization step after the API creates it.
4. Inspect the canary release lane and observe that no post-publish
onboarding test runs against the exact npm version.
**Paperclip version or commit**
The issue reproduced with `2026.901.0-canary.8` and the source state
before this pull request.
**Deployment mode**
Local trusted quickstart with embedded PostgreSQL. The install source
can be npm or a source checkout.
## What Changed
- Open the browser once for interactive foreground onboarding.
- Preserve explicit browser opt-outs and restore the prior environment
value after startup.
- Accept a same-company context update after organization creation and
reject a different-company takeover with an explicit error.
- Export the exact canary version from the publish job.
- Install and test that exact npm version in a clean Playwright smoke
job through the "Connect a model" step.
- Upload server logs, traces, screenshots, and the Playwright report
when the canary smoke fails.
- Document the interactive default and headless opt-outs.
## Verification
- `pnpm exec vitest run cli/src/__tests__/onboard.test.ts
ui/src/components/OnboardingWizard.step.test.tsx --reporter=dot` passes
with 37 tests.
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
passes with 9 tests.
- `pnpm exec playwright test --config tests/e2e/playwright.config.ts
tests/e2e/onboarding.spec.ts` passes with 2 tests.
- `PAPERCLIPAI_VERSION=2026.901.0-canary.8 pnpm run
test:canary-onboarding-smoke` passes against the published npm package.
- `pnpm check:token-gates` passes.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- A fresh interactive source run opens the browser and reaches "Connect
a model" after organization and agent naming.
## Risks
- Low risk. Automatic browser opening only applies to interactive
foreground onboarding.
- `PAPERCLIP_NO_BROWSER=1` and `PAPERCLIP_OPEN_ON_LISTEN=false` keep
headless runs silent.
- A different organization context still blocks the pending create
transition.
- The canary package is immutable before the smoke runs. A smoke failure
leaves the package published but makes the release workflow red.
- This change does not modify REST APIs, database schemas, or shared
data types.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex based on GPT-5. The runtime does not expose the exact
deployment snapshot or context-window size. The model used reasoning,
browser automation, repository tools, shell commands, code editing, and
test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip uses GitHub Actions to protect changes before they enter
`master`.
> - GitHub evaluates every pull request in a native stack against the
stack base.
> - The current workflow therefore starts the complete CI matrix for
every layer in a stack.
> - A large stack can queue many copies of the same integrated
verification and delay every pull request.
> - GitHub provides stack position and base metadata so workflows can
select merge-relevant layers.
> - This pull request keeps policy and required check names on every
layer, but runs full CI only for ordinary pull requests, the top layer,
and the lowest unmerged layer.
> - The benefit is much lower CI load without weakening the
required-check contract.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The trusted pull request workflow currently runs every test, build,
canary, and E2E lane for every pull request in a native stack.
**Subsystem affected**
GitHub Actions pull request verification.
**Current behavior**
A stack with 61 pull requests can start 61 complete CI matrices after a
cascading rebase.
**Proposed behavior**
Run the always-on policy job and stable required-check aggregators for
every layer. Run the complete verification matrix only for ordinary pull
requests, the top stack layer, and the lowest unmerged stack layer.
**Reason and benefit**
The top layer verifies the integrated stack. The lowest unmerged layer
verifies the current merge candidate. Middle layers keep
branch-protection checks without consuming the complete runner matrix.
**Breaking changes**
Middle stack layers no longer run the complete CI matrix. Their `ci /
verify` and `ci / e2e` checks still require the policy job to pass and
require every expensive lane to be intentionally skipped.
## What Changed
- Add a fail-safe stack scope decision to the trusted PR runner gate.
- Run typecheck, general tests, build, serialized tests, canary, and E2E
shards only for ordinary, top, and lowest-unmerged pull requests.
- Preserve the required `ci / verify` and `ci / e2e` names on every
layer.
- Make the required aggregators distinguish valid middle-layer skips
from failures or missing scope decisions.
- Add regression coverage for ordinary, top, bottom, middle, and
malformed stack metadata.
## Verification
- `node --test scripts/__tests__/e2e-shard.test.mjs` — 11 tests passed.
- `actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml`
— passed.
- `git diff --check origin/master...HEAD` — passed.
- The caller remains pinned to the current trusted workflow. A separate
activation change must advance the immutable SHA after this pull request
lands.
## Risks
- Incorrect stack classification could skip important jobs. Missing or
malformed stack metadata defaults to full CI.
- Middle-layer required checks depend on the policy job and verify that
all expensive jobs have the `skipped` result.
- The reusable workflow change does not become active until the
immutable caller SHA advances in a separate change.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with GPT-5. The exact deployment suffix and context
window are not exposed. The model used reasoning, repository tools, code
execution, Git, and GitHub API access.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip uses trusted GitHub Actions workflows to verify every pull
request
> - Native stacked pull requests use another pull request branch as
their base
> - A parent layer can change a package manifest without committing
`pnpm-lock.yaml`
> - A child layer can inherit that manifest change without changing a
manifest itself
> - The current policy skips lockfile regeneration for that child and
downstream frozen installs fail
> - This pull request validates the complete merge tree and shares a
regenerated lockfile only when needed
> - The benefit is reliable stacked pull request verification without
weakening the trusted workflow boundary
## Linked Issues or Issue Description
**What happened?**
A stacked child pull request inherited a package manifest change from
its parent. The child did not change a manifest itself. The policy job
skipped lockfile regeneration. Downstream jobs tried to restore an
artifact that did not exist and then failed during frozen dependency
installation.
**Expected behavior**
The policy job must validate the complete pull request merge tree. It
must upload a regenerated lockfile when the checked-in lockfile is
stale, including on a stacked child layer.
**Steps to reproduce**
1. Create a parent pull request that changes `package.json` without
committing `pnpm-lock.yaml`.
2. Create a child pull request on that branch without another manifest
change.
3. Run the trusted pull request workflow for the child.
4. Observe that frozen dependency installation fails because no
`pr-lockfile` artifact exists.
**Paperclip version or commit**
`f173ee09fa5c2ced7806bba47b54c3df853ab4df`
**Deployment mode**
GitHub Actions trusted pull request workflow.
**Agent adapter(s) involved**
Not adapter-specific. This is a core CI workflow bug.
## What Changed
- Regenerate the lockfile from every checked-out merge tree.
- Compare the generated lockfile with the checked-in copy before upload.
- Download the artifact only when the policy job reports that it
uploaded one.
- Fail closed when a reported artifact is missing.
- Add a workflow contract test for stacked lockfile handling.
- Keep the caller pinned to the last merged trusted SHA; after this
implementation merges, a separate activation PR will advance the
immutable pin to its merge commit.
## Verification
- `actionlint .github/workflows/pr-trusted.yml`
- `node --test scripts/__tests__/e2e-shard.test.mjs`
## Risks
- The policy job runs one lockfile-only install for every pull request.
This can add a small amount of CI time.
- A missing artifact now fails immediately when the policy job reports
an upload. This is intentional because it exposes workflow corruption.
- No runtime or product behavior changes.
- The implementation/activation split is intentional: unmerged
PR-authored workflow code must never execute on trusted runners.
## Model Used
OpenAI Codex with model `gpt-5`, reasoning, tool use, and code
execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip uses pull request CI to validate each proposed change
> - The existing workflow defines every heavy job in a PR-controlled
file
> - A trusted reusable workflow now contains the synchronized CI
definition
> - The caller must use an immutable default-branch SHA
> - This pull request replaces the duplicate job list with that pinned
caller
> - The benefit is automatic secure runner selection without workflow
drift
## Linked Issues or Issue Description
Refs #12436
Refs #12438
**What existing behavior does this improve?**
This improves how the pull request workflow selects trusted CI capacity.
**Subsystem affected**
Cross-cutting CI automation.
**Current behavior**
The active workflow contains a duplicate list of all heavy jobs. It
cannot use the administrator-controlled runner gate.
**Proposed behavior**
The active workflow calls the synchronized trusted workflow at an
immutable SHA. The trusted workflow selects GitHub-hosted or isolated
AWS capacity from the validated contributor identity.
**Reason and benefit**
The thin caller prevents pull request changes from replacing the
external-runner security gate. It also keeps runner selection automatic.
**Breaking changes**
The check names gain the reusable workflow job prefix. AWS routing
remains disabled until the canary starts.
## What Changed
- Replaced the duplicated heavy CI job list with one reusable-workflow
call.
- Pinned the call to the reviewed default-branch commit.
- Limited the caller token to actions, contents, and pull request read
access.
## Verification
- actionlint on both workflow files
- Trusted-routing tests
- Confirmed the pinned SHA contains the workflow and is an ancestor of
master
- Full AWS and GitHub runner-boundary verification with routing disabled
## Risks
The check context names change when GitHub expands the reusable
workflow. The rollout verifies the new aggregate contexts before branch
rules change. The repository kill switch remains off during this pull
request.
## Model Used
OpenAI Codex with GPT-5, tool use, and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked a related public PR or described the issue
with the matching template fields
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Worktree provisioning prepares the dependencies and tools that these
agents need.
> - The pinned pnpm version calls the deprecated `url.parse()` function
during each install.
> - Node.js 24 reports this call as `DeprecationWarning [DEP0169]`.
> - The provisioning scripts run more than one install, so the warning
repeats in each run.
> - This pull request disables only `DEP0169` at each affected pnpm
install call site.
> - The benefit is a clear provisioning log while other deprecation
warnings remain visible.
## Linked Issues or Issue Description
**What happened?**
The worktree provisioning scripts printed `DeprecationWarning [DEP0169]`
during each pnpm install. The warning came from pnpm 9.15.4 and its
`toNerfDart` call to `url.parse()`.
**Expected behavior**
The provisioning scripts should hide this known warning from the pinned
pnpm version. They should keep other deprecation warnings visible.
**Steps to reproduce**
1. Use Node.js 24 with pnpm 9.15.4.
2. Run worktree provisioning with a base-workspace repair or dependency
install.
3. Observe the repeated `DeprecationWarning [DEP0169]` output.
**Paperclip version or commit**
Commit `5cd41b1a9996713efdfdc62373da8045664c7f30`.
**Deployment mode**
Built from source.
**Installation method**
Built from source with pnpm.
**Agent adapter(s) involved**
Not adapter-specific (core bug).
**Database mode**
Not database-related.
## What Changed
- Add `--disable-warning=DEP0169` to each affected pnpm install call
site.
- Append the flag to `NODE_OPTIONS` so the scripts keep existing values.
- Add comments that name the source of the warning and the removal
condition.
- Add a regression test for all affected scripts and warning codes.
## Verification
- `bash -n scripts/provision-worktree.sh` passes.
- `bash -n scripts/provision-worktree-runtime.sh` passes.
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`
passes with 15 tests.
- GitHub Actions must pass all required checks before merge.
## Risks
This change has low risk. It changes warning output only for `DEP0169`.
It does not overwrite existing `NODE_OPTIONS` values. Revert commit
`5cd41b1a9996713efdfdc62373da8045664c7f30` to restore the prior output.
## Model Used
OpenAI Codex, GPT-5. The model used tool calls and code execution. The
runtime did not expose a context-window value.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner package now has protocol, transport, provider, catalog,
and authorization foundations.
> - Its first upstream package boundary should expose only the
implemented runtime and test-helper surfaces.
> - Rust correctness belongs in the repository existing build
verification, without introducing a parallel release process.
> - Direct package creation must build the files declared by the package
manifest.
> - This pull request defines the minimal package API and verifies the
optimized runner binaries in the existing PR and release Build jobs.
> - The benefit is a production-ready runner package boundary with
minimal build-process change.
## Linked Issues or Issue Description
Refs #11962
This pull request replaces one bounded part of the archived large runner
change. It follows the package-local authorization change in #12126.
## What Changed
- Export only `@paperclipai/paperclip-runner` and
`@paperclipai/paperclip-runner/testing`.
- Keep Node-only fixture loading and semantic conformance helpers out of
the runtime root.
- Add a provider-neutral semantic conformance kit with stable JSON
comparison and fail-closed input checks.
- Keep deferred SDK, eval, browser, React, lab, and command surfaces
private.
- Pin the runner Rust toolchain to 1.97.1 with the minimal profile and
`rustfmt`.
- Run the Rust workspace tests in release mode.
- Launch the optimized `paperclip-runnerd` and fake-harness binaries in
process-level integration coverage.
- Add one `pnpm --filter @paperclipai/paperclip-runner check:all` step
to each existing PR and release Build job.
- Make the existing server `prepack` lifecycle run its existing build
after it prepares UI assets.
- Document that no production adapter starts runnerd yet.
This revision adds no standalone GitHub Actions job. It adds no server
runner dependency or runner vendoring. It adds no Docker bootstrap or
clean-consumer harness. It does not change `pnpm-lock.yaml`.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner check:all`
- 66 TypeScript tests
- 8 protocol contract tests
- 56 Rust unit and integration tests
- Release-mode integration coverage launches the optimized runnerd and
fake-harness binaries.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/server-package-build-script.test.ts` (2 tests)
- Clean `pnpm pack` from `server/` rebuilt the server and produced both
`package/dist/index.js` and `package/dist/index.d.ts`.
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` (8
tests)
- `pnpm -r typecheck`
- `pnpm build`
- `pnpm check:token-gates`
- `git diff --check`
- No `pnpm-lock.yaml` diff.
- The diff changes 12 files.
## Risks
The runner adds Rust work to the existing Build jobs. These jobs can
take longer on a cold cache. The pinned toolchain makes contributor and
CI behavior reproducible. Cargo tests use `--release` to verify
optimized executables. The server prepack lifecycle now performs the
build that its published entry points require. This can make direct
server packing slower. This pull request does not wire runnerd into the
server. It does not select runnerd for any adapter. Existing application
execution and finalization paths remain unchanged.
## Model Used
OpenAI Codex with GPT-5. Agentic coding mode used repository tools, code
execution, and automated tests.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents can run in isolated worktrees with a separate Paperclip
runtime.
> - Runtime provisioning uses a Bash script on macOS hosts.
> - macOS ships Bash 3.2, where an empty array expansion fails under
`set -u`.
> - The source-config argument array is empty when the base workspace
already has a config.
> - This pull request guards that expansion and tests the normal
base-config path on Bash 3.2.
> - The benefit is that managed worktree provisioning no longer fails
before database seeding.
## Linked Issues or Issue Description
No public GitHub issue exists for this problem. PR #11752 added the
conditional source-config argument that exposed the failure.
**What happened?**
`scripts/provision-worktree-runtime.sh` expands an empty
`source_config_args` array while `set -u` is active. Bash 3.2 reports
`source_config_args[@]: unbound variable` and stops provisioning when
the registered base workspace already has `.paperclip/config.json`.
**Expected behavior**
Runtime provisioning must call `worktree ensure-seeded` without a source
override when the base workspace config exists. It must work with the
Bash 3.2 version that macOS supplies.
**Steps to reproduce**
1. Use macOS system Bash 3.2.
2. Create a base workspace with `.paperclip/config.json`.
3. Run `scripts/provision-worktree-runtime.sh` with `set -u` active in
the script.
4. Observe the unbound-variable error before `worktree ensure-seeded`
runs.
**Paperclip version or commit**
Reproduced on `origin/master` before this change.
**Deployment mode**
Local managed worktree runtime on macOS.
## What Changed
- Guard all three optional source-config array expansions with Bash
3.2-compatible parameter expansion.
- Add a regression test that uses the base-config path and verifies that
no `--from-config` argument is sent.
- Document the Bash 3.2 compatibility requirement in the runtime script.
## Verification
- `/bin/bash -n scripts/provision-worktree-runtime.sh`
- `node --test --test-name-pattern='runtime provisioning invokes
ensure-seeded once|runtime provisioning omits the source
override|runtime provisioning guards every optional source-config
expansion' scripts/__tests__/provision-worktree-self-heal.test.mjs`
- `git diff --check`
## Risks
Low risk. The change only affects expansion of an optional two-element
CLI argument array. The regression tests cover both the empty and
non-empty paths.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with model ID `gpt-5`. The runtime did not expose the
context-window size. Reasoning, tool use, and code execution were
enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents do that work in isolated git worktrees, and a managed
worktree runs its own Paperclip instance with a cloned database
> - That clone needs a seed source, and the source must come from
server-owned registration, never from state the workspace itself can
rewrite
> - The seed-source resolver requires the registered base project
workspace to hold its own `.paperclip/config.json`
> - A managed project workspace is a plain `git clone`, and no code
writes that file into it
> - Every isolated worktree provision, deferred seed, and workspace
repair therefore fails on a managed checkout
> - This pull request lets a named source supply the config when the
base checkout has none
> - The benefit is that managed worktrees provision again, and the seed
source stays server-owned
## Linked Issues or Issue Description
No public GitHub issue exists for this problem. It is described below.
**What happened?**
Agent runs that need an isolated worktree fail during provisioning. The
provision command exits with this error (paths redacted):
```
Execution workspace provision command "bash ./scripts/provision-worktree.sh" failed:
Registered base project workspace has no canonical Paperclip config:
<instance-home>/instances/default/projects/<company-id>/<project-id>/<repo>/.paperclip/config.json
```
`resolveRegisteredWorktreeSeedSource` sets `registeredConfigPath` to
`<baseCwd>/.paperclip/config.json` whenever the caller names a
registered base workspace. It then requires that file to exist.
`scripts/provision-worktree.sh` applies the same rule.
A managed project workspace never has that file.
`materializeManagedProjectWorkspace` creates it with `git clone` and a
rename, so the checkout holds repository content only. The control plane
keeps its config at `<home>/instances/<id>/config.json` instead.
The failure reaches three paths: worktree provisioning, deferred seeding
through `worktree ensure-seeded`, and workspace repair.
The behavior changed in #11671. That pull request replaced a fallback
chain with a single hard requirement. Fixture code in
`scripts/__tests__/provision-worktree-self-heal.test.mjs` writes a
config into the fake base workspace, so tests kept passing.
**Expected behavior**
A managed worktree provisions and seeds from the registered source. The
seed manifest still never selects that source.
**Steps to reproduce**
1. Register the Paperclip repository as a project with a `repoUrl`, so
the server materializes a managed checkout.
2. Assign an issue to an agent whose workspace strategy is
`git_worktree`.
3. Watch the workspace operation log for the provision command.
4. The command exits non-zero with the error above.
**Paperclip version or commit**
Reproduced on `master` at 01ddc26a3.
**Deployment mode**
`local_trusted`, single instance.
**Database mode**
Embedded PostgreSQL.
**Operating system**
Linux, Node.js 22.
**Related pull requests**
- Refs #11671 — introduced the requirement this pull request relaxes.
- Refs #11733 — open work on seed-source preflight. It reads the same
base-workspace config path and skips when the file is absent. It does
not change source selection.
- Refs #11735 — open work on provisioning reliability. It edits the same
four files and will need a rebase after either lands.
## What Changed
- `resolveRegisteredWorktreeSeedSource` sets the registered config path
only when `<baseCwd>/.paperclip/config.json` exists. This makes the
existing `registeredConfigPath ?? explicitSource` branch reachable for a
plain checkout.
- A base workspace that does hold its own config stays authoritative. A
mismatched explicit source is still rejected.
- The resolver throws a named error when the base workspace has no
config and no source is named.
- `readInstanceId` accepts an instance-root config at
`<home>/instances/<id>/config.json`. That layout names its instance by
directory and has no adjacent `.env`. Validation reuses
`resolvePaperclipInstanceId`.
- `scripts/provision-worktree.sh` and
`scripts/provision-worktree-runtime.sh` name the control plane's
instance config as the source when the base workspace has none. The
canonical-path and symlink checks stay.
- The workspace repair route supplies the same fallback, and only when
the base workspace has no config of its own.
- `doc/DEVELOPING.md` records the two source layouts.
## Verification
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`
— 10 tests pass. The fixture no longer writes a config into the base
workspace, so it models a real managed checkout. One test now creates
that config mid-test, which covers both layouts.
- `npx vitest run src/worktree-seed-source.test.ts` in `packages/shared`
— 4 tests pass. Two are new: one resolves an instance-root source, and
one still fails closed when no source exists.
- `npx vitest run src/__tests__/workspace-runtime.test.ts
src/__tests__/execution-workspaces-routes.test.ts
src/__tests__/execution-workspace-runtime-control-conflict.test.ts
src/__tests__/workspace-operations-reconciliation.test.ts
src/__tests__/worktree-seed-server-spawn.test.ts` in `server` — all
pass. Run them one file at a time. They share one test database, and
concurrent runs fail teardown.
- `npx vitest run src/__tests__/worktree.test.ts` in `cli` — 63 tests
pass.
- `pnpm --filter @paperclipai/shared typecheck` — clean.
- Manual check on a live instance: the resolver now returns the instance
config as the source for a managed checkout, with the source instance
`default` and a distinct target instance.
## Risks
Low to moderate.
- The relaxed rule applies only when the base workspace holds no config.
A base workspace that holds one keeps full authority, so the trust model
from #11671 is unchanged. The seed manifest still never selects the
source.
- The instance-id fallback reads a directory name. It applies only to
the `<home>/instances/<id>/config.json` layout, and
`resolvePaperclipInstanceId` rejects an unsafe segment.
- #11735 edits the same four files. Whichever pull request lands second
needs a rebase.
- `pnpm --filter @paperclipai/server typecheck` currently fails on this
checkout with duplicate `drizzle-orm` type instantiations. The failure
is present with and without this change, and the error count is
identical. It comes from an unrelated lockfile state, not from this pull
request.
## Model Used
Claude Opus 5 (`claude-opus-5`), by Anthropic, running in Claude Code.
Extended thinking was on. The model used file, search, and shell tools
to diagnose the failure on a live instance and to run the test suites.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Execution workspaces need isolated databases, ports, and runtime
services
> - Concurrent workspaces could reuse ports or lose service ownership
after a restart
> - A markerless worktree also needed seed recovery, but normal
markerless instances still needed to boot
> - This pull request makes seed, port, and service ownership state
explicit and recoverable
> - It also checks live process and listener identity before it reclaims
shared resources
> - The benefit is reliable workspace startup, restart, adoption, and
concurrent provisioning
## Linked Issues or Issue Description
**What happened?**
Managed workspaces could lose runtime service ownership after a
control-plane restart. Concurrent worktrees could also reuse a port when
their parent paths differed. A seed recovery change made every
markerless instance resolve a worktree seed source, so normal instances
without a source could not start.
**Expected behavior**
Paperclip must preserve healthy managed services across restarts. It
must reserve unique ports across worktree parents. It must provision a
registered markerless worktree, but it must skip seed work for a normal
markerless instance.
**Steps to reproduce**
1. Start two managed worktrees under different parent paths at the same
time.
2. Restart the control plane while a managed service stays alive.
3. Start Paperclip with a config that has no seed markers and no
registered worktree source.
4. Observe duplicate port selection, lost service adoption, or a
seed-source startup error.
**Paperclip version or commit**
Current `master` plus the workspace runtime reliability changes in this
pull request.
**Deployment mode**
Local development with managed execution workspaces and embedded
Postgres.
## What Changed
- Added a shared port registry with lease heartbeats, process identity
checks, and live listener probes.
- Reserved worktree ports across custom parent paths and repaired
duplicate legacy assignments.
- Preserved and adopted healthy managed services across control-plane
restarts.
- Reconciled guest bind modes and verified listener ownership before
termination or reuse.
- Provisioned registered markerless worktree databases and kept normal
markerless instance startup as a no-op.
- Added CLI, shared, server, and shell regression tests for seed, port,
listener, restart, and adoption behavior.
- Updated the worktree development documentation.
## Verification
- `pnpm exec vitest run cli/src/__tests__/worktree.test.ts
--reporter=verbose` — 63 tests passed.
- `pnpm exec vitest run
packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` —
5 tests passed.
- Focused runtime Vitest set — 199 tests passed across 37 suites.
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`
— 10 tests passed.
- `git diff --check` passed.
## Risks
- Port reservation now depends on lease and process identity data. The
fallback listener probe prevents early reclamation when process metadata
is incomplete.
- Runtime adoption is stricter about bind and owner identity. The tests
cover healthy adoption, stale records, PID reuse, and unrelated
listeners.
- Markerless seed detection now separates registered worktrees from
normal instances. The tests cover both paths.
- There are no database schema migrations.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with the `gpt-5` model family. The serving snapshot and
context-window size are not exposed. The agent used reasoning,
repository tools, code execution, and test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Dev Agent <dev@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Managed worktree services run isolated Paperclip instances with
cloned databases.
> - A reachable service was reported as ready even when its database,
runtime identity, or login path was not usable.
> - The first candidate added verified database seeding and managed
repair in #11665.
> - This pull request consolidates that candidate with signed login
handoff and a complete readiness contract.
> - Post-QA fixes close five defects in repair identity, repair
responses, UI retry, seed journal handling, and seed-source trust.
> - The benefit is a workspace that either opens safely or reports one
accurate recovery action.
## Linked Issues or Issue Description
No public GitHub issue exists for this work, so the problem is described
here.
**What happened**
Managed workspace URLs could return HTTP 200 and report ready while
login failed. QA also found cases where repair used the wrong instance
identity, returned a generic error, left the UI stuck, rejected a safe
journal lag, or trusted a mutable workspace manifest.
**Expected behavior**
Opening a ready workspace signs the board user in to the correct
isolated instance. Provisioning and repair use a registered source and
report a structured recovery state.
**Actual behavior**
Entry depended on a password copied into the clone. Several failure
paths could publish stale readiness, hide the repair precondition, or
trust state that the workspace could modify.
**Additional context**
This pull request includes the commits first published in #11665. That
pull request keeps the original base head for review history. This
consolidated pull request is the merge candidate. Related open readiness
work includes #11575 and #11621.
## What Changed
- Adds a short-lived, signed, single-use login ticket. It binds the
user, workspace, instance, and runtime origin.
- Exchanges the ticket through Better Auth. It creates the session and
cookie through the supported adapter path.
- Adds protected workspace readiness fields for the database, clone
data, login handoff, seed phase, and runtime identity.
- Fails readiness closed when the guest has no company or
execution-workspace binding.
- Binds ticket issuance to the exact cloned user and active company
membership selected for the handoff.
- Verifies every current active board identity through the exact-user
handoff before publication or reuse.
- Gates managed runtime publication on the readiness contract and the
recorded worktree instance identity.
- Refreshes runtime work products from the live runtime row after a port
change.
- Adds one workspace access card with ready, degraded, repairing, and
failed states.
- Uses the runtime response identity for repair. It returns structured
repair precondition errors.
- Lets a valid source journal lag converge during provisioning.
- Binds seed and repair manifests to a source registered outside the
agent-writable worktree.
- Clears recovered UI errors so a successful retry can open the
workspace.
- Makes runtime tests register canonical sources and avoid ports owned
by live host listeners.
- Keeps Vitest on source suites when compiled `dist` trees exist.
- Isolates CLI and adapter tests from ambient AWS and runtime API
environment variables.
- Preserves a 404 response for cross-company workspace ID lookups before
runtime authorization.
- Makes concurrent single-flight coverage independent of
path-canonicalization scheduling order.
## Verification
The following checks passed on the integrated head:
```sh
pnpm -r typecheck
pnpm build
pnpm check:token-gates
pnpm --filter @paperclipai/db check:migrations
```
- The server source lane passed 420 files and 4,953 tests. Five tests
were skipped.
- The CLI lane passed 57 files and 385 tests.
- The database lane passed 26 files and 97 tests.
- The shared package passed 58 files and 506 tests.
- The adapter utility lane passed 640 tests. Four tests were skipped.
- The Claude adapter passed 220 tests. One test was skipped.
- The Codex adapter passed 323 tests.
- The OpenClaw adapter passed 13 tests.
- The OpenCode adapter passed 42 tests.
- The plugin SDK passed 45 tests.
- The workspace runtime suite passed 124 tests.
- The caller-scoped readiness and handoff suite passed 52 tests.
- The workspace provisioning shell suite passed 7 tests.
- The runtime exposure suite passed 17 tests while live host mappings
occupied fixed test ports.
- `git diff --check` passed and the worktree is clean.
The serialized route lane will run in GitHub CI with its normal shards.
No deployment or active-workspace migration was performed.
## Risks
- This is a medium-risk authentication and runtime-readiness change.
- The login ticket uses exact origin, workspace, instance, and user
binding. It has a short expiry and a one-time nonce.
- Runtime publication is stricter. A real readiness, identity, per-user
handoff, or control-plane database disagreement now blocks publication.
- This pull request supersedes #11665 as the merge candidate. Close
#11665 after this pull request merges.
- No new database migration is included. The lockfile and workflow files
are unchanged.
- Deployment and active-workspace migration are intentionally outside
this pull request.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool
use, and code execution produced the main candidate. OpenAI GPT-5
(`gpt-5`) through Codex, with agentic reasoning, tool use, and code
execution, integrated the post-QA fixes and hardened the test gates. The
Codex context-window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100). -->
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The PR verify workflow gates every pull request; its wall-clock time
sets the feedback loop for all contributors
> - In a recent successful PR run (actions run 32012408876), the slowest
check was "Verify serialized server suites (1/5)" at 337s, while its
four sibling shards finished in 212-238s
> - The serialized lane assigns suites to shards round-robin over an
alphabetical list, so the heavy heartbeat and issues suites cluster on
one runner
> - The general-server lane already solves this with a duration-aware
LPT partition backed by a recorded manifest
> - This pull request reuses that partitioner for the serialized lane
with a fresh per-suite duration manifest
> - The benefit is a balanced serialized matrix: the measured 968s suite
total levels to about 194s per shard, which removes about 80-100s from
the run's slowest check
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The `Verify serialized server suites` shard matrix in
`.github/workflows/pr.yml` distributes route/authz test suites across
five runners.
**Subsystem affected**
CI / test infrastructure (`scripts/run-vitest-stable.mjs`).
**Current behavior**
`selectSerializedSuites` assigns suites round-robin (`index %
shardCount`) over the alphabetically sorted file list. The heavy suites
cluster on shard 1/5. In actions run 32012408876, shard 1/5 spent 291s
in its test step while the other shards spent 170-201s, which made that
job (337s total) the slowest check of the whole PR run.
**Proposed behavior**
Partition the serialized suites with the same duration-aware LPT
algorithm the general-server lane already uses
(`scripts/general-server-shard.mjs`), backed by a new per-suite duration
manifest. All five shards then carry about 194s of measured test time.
**Reason and benefit**
The slowest check bounds PR feedback time. Balancing the serialized
matrix removes about 80-100s from that bound without adding runners.
**Breaking changes**
None. The partition remains deterministic, complete, and
non-overlapping; suites missing from the manifest get the median weight.
## What Changed
- Added `scripts/serialized-shard-durations.json`: per-suite wall-clock
durations (ms) for all 134 serialized suites, sampled from actions run
32012408876 by diffing consecutive per-suite label timestamps in the
shard logs (captures vitest spawn overhead, not just reported test time)
- `scripts/run-vitest-stable.mjs`: `selectSerializedSuites` now uses the
existing LPT partitioner (`selectGeneralServerShard`) with the new
manifest instead of round-robin
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: added a
manifest-freshness test and a shard-balance test for the serialized
lane, mirroring the general-server ones
- `.github/workflows/pr.yml`: updated the serialized matrix comment with
the new measurement and mechanism
## Verification
- `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs`
passes (13 tests), including the existing test that the serialized
shards form a complete, non-overlapping partition
- Dry-run of all five shards shows estimated totals of
194/194/194/194/193s (round-robin was 276/175/160/172/187s):
`node scripts/run-vitest-stable.mjs --mode serialized --shard-index N
--shard-count 5 --dry-run`
- The `Verify serialized server suites` jobs on this PR run the real
partition end to end
## Risks
- Low risk. Selection logic only; the vitest invocation per suite is
unchanged
- A stale manifest degrades gracefully: unknown suites get the median
weight, and a dedicated test fails if fewer than half the current suites
have recorded durations
## Model Used
- Claude (Anthropic), model ID `claude-fable-5`, agentic coding session
with tool use (Claude Code / Claude Agent SDK); no extended-thinking
mode
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Related prior work: #10923 (split serialized tests into five shards),
#10925 (general-server duration manifest), #11156 (workspaces-a native
shards).
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release channel system re-smokes every published beta as
post-publish verification (`smoke_beta`)
> - The candidate-branch beta lane (#11209) added
`verify_beta_candidate` to `publish_beta`'s needs; that job is skipped
on every normal promote-mode beta
> - `smoke_beta`'s condition has no status-check function, so GitHub
attaches an implicit `success()` that evaluates the needs chain
transitively — a skipped ancestor makes it false
> - This pull request makes the condition explicit so promote-mode betas
smoke again, and pins the shape in the workflow wiring test
> - The benefit is that the post-publish beta gate actually runs instead
of silently skipping
## Linked Issues or Issue Description
**What happened?**
Beta `2026.818.0-beta.0` (run 32082007439) published successfully, but
its post-publish `smoke_beta` job was skipped. No configuration or input
asked for that: the run was a plain `channel: beta` dispatch with
`dry_run` at its default `false`, and the same expression
`!inputs.dry_run` evaluated true inside `publish_beta`'s own steps (the
Docker dispatch step ran).
**Expected behavior**
Every non-dry-run beta publish is followed by the release smoke suite
against the exact published version, as documented in `doc/RELEASING.md`
and `doc/RELEASE-CHECKLIST.md`.
**Steps to reproduce**
Dispatch `release.yml` with `channel: beta` promoting a nightly (promote
mode). `verify_beta_candidate` is skipped by design; `publish_beta` runs
through its explicit `!cancelled()` condition; `smoke_beta` then skips
because its implicit `success()` sees the skipped ancestor in the
transitive needs chain (actions/runner#2205 semantics). The beta
published on 2026-08-11 predated #11209, so this never surfaced before.
**Paperclip version or commit**
master at `43ab441f0` (workflow file, current head).
Related (not duplicates): #11209 introduced the candidate lane whose
skipped job triggers this; #11208 covers the adjacent tag-push failure
playbooks.
## What Changed
- `smoke_beta`'s condition becomes `!cancelled() &&
needs.publish_beta.result == 'success' && !inputs.dry_run` — an explicit
status-check function suppresses the implicit `success()`, and the
result check keeps the dependency on a successful publish.
- A comment above the job records why the explicit form is load-bearing.
- `scripts/__tests__/release-verify-workflow.test.mjs` pins the new
shape so the implicit form cannot silently return.
## Verification
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 8
pass, including the new assertion.
- `release.yml` re-parsed as YAML.
- The exact skip is visible on run 32082007439 (`smoke_beta: skipped`
after `publish_beta: success`); the coverage gap for that beta was
closed manually by dispatching `release-smoke.yml` with
`paperclip_version: beta` (run 32084880767).
- Not exercised end-to-end: the corrected condition needs the next real
promote-mode beta to demonstrate; the expression change is minimal and
the semantics are the documented actions/runner behavior.
## Risks
- Low risk: condition-only change on one job plus a test. Dry runs still
skip the smoke (`!inputs.dry_run` retained). Candidate-mode betas, where
`verify_beta_candidate` actually runs, behave as before.
## Model Used
Claude Fable 5 (Claude Code)
## Pre-submission checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
> Follow-up to #11208 (merged): rebased onto master and ready for
review.
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release channels promote artifacts along canary → nightly → beta
→ stable, with the happy path being promotion of an existing build
> - When one or two targeted fixes are needed before a beta or stable,
the only options today are waiting for the next nightly or absorbing a
whole day of unrelated master changes
> - The channel model was designed with an escape hatch for exactly
this: short-lived candidate branches carrying only cherry-picked fixes
> - This pull request implements candidate-branch beta builds with full
verification, documents the stable fix path through the
soak-justification gate, and adds the release captain's checklist
> - The benefit is that a surgical fix can ship forward without either
delay or blast radius, with its provenance recorded
## Linked Issues or Issue Description
Refs #11008 — completes the fix-path half of the channel model
introduced there.
**Subsystem affected**
Release automation: `scripts/release.sh`,
`.github/workflows/release.yml`, `doc/RELEASING.md`, new
`doc/RELEASE-CHECKLIST.md`, tests.
**Problem or motivation**
Beta promotion only accepts commits that already shipped as a nightly,
and stable promotion expects a soaked beta. There is no supported way to
ship one or two cherry-picked fixes between lanes: an urgent fix must
wait for the nightly cycle or pull in every unrelated master change from
the day. The original channel design called for candidate branches to
cover this, and they were deferred from the initial implementation.
**Proposed solution**
Candidate-branch beta builds: cut `candidate/beta-<target>` from a
nightly's source commit, cherry-pick the fixes, and dispatch `channel:
beta` with the new `candidate_branch` input. Selection enforces the
naming convention, rejects heads that already shipped as a beta or
predate the candidate tooling, and records the cherry-picked commits in
the job summary. Because candidate heads never went through a canary or
nightly, publication is gated on a full `release-verify` run (promoted
nightlies keep skipping re-verification). The stable fix path
(`candidate/release-<target>` as `source_ref`) works through the
existing soak gate: the justification requirement is the deliberate,
recorded trade-off for shipping unsoaked bits, and is now documented as
such.
## What Changed
- `scripts/release.sh`: `--from-candidate` flag (beta only) waives the
shipped-a-nightly requirement while keeping the duplicate-beta guard
- `.github/workflows/release.yml`: `candidate_branch` dispatch input;
candidate mode in `select_beta` (naming validation, duplicate and
tooling-era rejection, cherry-pick recording); new
`verify_beta_candidate` job gating candidate publishes on full
verification
- `doc/RELEASING.md`: beta fix-path and stable fix-path sections
- `doc/RELEASE-CHECKLIST.md` (new): the release captain's checklist for
all four lanes as built
- Tests: dry-run fixture coverage for `--from-candidate` (waives the
nightly guard, keeps the duplicate guard, rejected outside beta) and
wiring tests for candidate validation plus the verification gate
## Verification
- `node --test` on the four affected suites: 42 pass in total (17 + 25
across the two runs), including the 5 new tests
- `bash -n` on `release.sh`; YAML parse of the workflow
- After merge: exercise the path end to end the first time a real
cherry-picked beta is needed — dispatch with a `candidate/beta-*` branch
and confirm the summary records the picks and verification runs
## Risks
- Candidate builds bypass the smoke-tested-nightly provenance by design;
the compensating controls are full verification before publish, the
post-publish beta smoke, the human `npm-beta` gate, and recorded
cherry-picks
- The stable fix path rides the existing justification mechanism rather
than adding a second bypass — one recorded escape hatch, not two
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's promotion lanes publish to npm, then push a
lane tag and dispatch the Docker image build at that tag
> - The first nightly of the beta-tooling merge published to npm and
then died at the tag push: GITHUB_TOKEN may not create refs pointing at
workflow-modifying commits from dispatch or scheduled runs
> - The failure was a bare `remote rejected` with no guidance, leaving
the release half-finished (npm live, no tag, no images) until an
operator reverse-engineered the recovery
> - This pull request makes every lane's tag push degrade into exact
recovery instructions in the job summary
> - The benefit is that a rare platform-permission rejection becomes a
two-minute runbook operation instead of a forensic exercise
## Linked Issues or Issue Description
Refs #11008 — the incident occurred promoting that change's own merge
commit, the first workflow-modifying commit to flow through the lanes it
introduced.
**Subsystem affected**
Release automation: `.github/workflows/release.yml`, `doc/RELEASING.md`,
workflow wiring tests.
**Problem or motivation**
Run 31445344811 published `2026.811.0-nightly.0` to npm, then failed
pushing `nightly/v2026.811.0-nightly.0`: `refusing to allow a GitHub App
to create or update workflow .github/workflows/release.yml without
workflows permission`. The tagged commit modifies workflow files, and
GITHUB_TOKEN may not create refs pointing at such commits from dispatch
or scheduled runs (push-event runs are exempt, which is why the canary
tag on the same commit succeeded). The job failed with no explanation
and the Docker dispatch never ran.
**Proposed solution**
Wrap the nightly, beta, and stable tag pushes: on rejection, write the
exact recovery commands into the job summary — create and push the tag
with maintainer credentials, dispatch `docker.yml` at the tag, and for
stable also run `create-github-release.sh` — then fail the job. Document
the cause and recovery in the failure playbooks and pin the three
recovery blocks with a wiring test.
## What Changed
- `.github/workflows/release.yml`: recovery-summary wrappers on the
nightly, beta, and stable tag-push steps
- `doc/RELEASING.md`: failure-playbook entry for the
workflows-permission rejection
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
asserting all three lanes carry the recovery summary
## Verification
- Wiring tests: 6 pass; YAML parse of the workflow
- The recovery commands are exactly the ones used to resolve the real
incident (tag push + `docker.yml` dispatch for
`nightly/v2026.811.0-nightly.0`)
## Risks
- Low. The happy path is unchanged (a successful push skips the
wrapper); the failure path trades a bare error for actionable output and
still fails the job, since the release state is genuinely incomplete
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
Split the slow workspaces-a CI lane into two Vitest native shards and keep release verification in parity.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem promotes builds along canary → nightly → beta
→ stable, and each publish job checks out the promotion's source commit
and runs that tree's release tooling
> - The first beta dispatch failed with `unexpected argument: beta`: the
selected nightly's source predated the beta channel, so its `release.sh`
did not know the argument
> - The failure was clean (argument parsing, nothing published) but
cryptic, and the same trap waits for any promotion of a source older
than its target channel's tooling
> - This pull request makes the selection jobs reject such sources with
an actionable error and documents the property
> - The benefit is that a bootstrapping or old-source promotion fails in
seconds with instructions, instead of mid-publish with a parser error
## Linked Issues or Issue Description
Refs #11008 — the guard hardens the beta promotion flow introduced
there, after its first dispatch surfaced the gap described below.
**Subsystem affected**
Release automation: `.github/workflows/release.yml`, `doc/RELEASING.md`,
workflow wiring tests.
**Problem or motivation**
Run 31444045044 (first beta dispatch) failed in `publish_beta` with
`unexpected argument: beta`. Promotions deliberately build from the
pinned source commit, which means they also run that commit's
`scripts/release.sh` — and a source that predates the target channel's
introduction cannot publish it. Nothing guards this today; the error
surfaces deep in the publish job with no explanation.
**Proposed solution**
Guard at selection time: `select_nightly` requires the source canary's
`release.sh` to know the nightly channel, and `select_beta` requires the
source nightly's `release.sh` to know the beta channel. Each guard
literally matches the channel case arm and fails closed with a clear
message naming the remedy (promote a newer source). Document the
tooling-era property in `RELEASING.md` and pin the guards with a wiring
test.
## What Changed
- `.github/workflows/release.yml`: tooling-era guards in
`select_nightly` and `select_beta`
- `doc/RELEASING.md`: documents that promotions run the source commit's
release tooling
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
pinning both guards
## Verification
- Wiring tests: 5 pass
- Guard expressions exercised against real commits: accepts the
beta-capable merge commit of the beta-channel change, rejects a pre-beta
commit
- YAML parse of the workflow
- After merge: the next beta dispatch selects a beta-capable nightly and
passes the guard
## Risks
- Low. Selection-time check only; the guards match the channel case arm
literally and fail closed (with the same actionable message) if that
line is ever reformatted
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
> Follow-up to #11006 (merged): rebased onto master and ready for
review.
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem now publishes canary (every master push),
nightly (scheduled, smoke-gated, added in #11006), and stable (manual)
> - There is still no human-approved release-candidate lane between
nightly and stable, and nothing enforces that a stable actually soaked
anywhere before shipping
> - Betas need a real approval gate, and stables need a soak policy that
is data, not prose
> - This pull request adds the beta channel: a manual promotion of a
chosen nightly behind the `npm-beta` environment gate, re-smoked after
publish, plus a stable preflight that enforces a 3-day beta soak with a
written-justification bypass
> - The benefit is a complete canary → nightly → beta → stable train
where every stable shipped as a beta first, and emergencies leave a
written trace
## Linked Issues or Issue Description
**Subsystem affected**
Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.
**Problem or motivation**
After #11006 the project has canary and nightly prerelease lanes, but no
release-candidate lane. Stable promotion has no enforced soak: any ref
can ship as stable directly. There is no approval boundary for a
broader-audience prerelease, and no structured way to record why an
emergency release skipped validation.
**Proposed solution**
Add a `beta` channel: a manual dispatch that promotes a chosen nightly's
source commit, publishes behind the `npm-beta` GitHub environment
(required reviewers are the gate), re-smokes the published beta, and
tags `beta/vX`. Enforce in the stable path that the source commit
shipped as a beta at least 3 days earlier (measured from the beta's npm
publish time), with a `skip_soak_justification` input as the recorded
emergency bypass.
**Alternatives considered**
Codifying the soak policy in docs only. Rejected: an unenforced policy
decays; the preflight makes the policy executable while the
justification input keeps the emergency path usable and auditable.
## What Changed
- `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel —
requires HEAD to carry a `nightly/v*` tag, publishes the package set as
`YYYY.MDD.P-beta.N` under dist-tag `beta`, tags
`beta/vYYYY.MDD.P-beta.N`
- `.github/workflows/release.yml`:
- `channel: beta` dispatch path: `select_beta` resolves the newest (or
an explicit `source_version`) nightly and fails loudly on selection
problems; `publish_beta` runs behind the `npm-beta` environment, pushes
the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release
smoke suite against the exact published beta version
- stable path: new `preflight_stable` job enforces the 3-day beta soak
from the beta's npm publish time; `skip_soak_justification` bypasses
with the reason echoed into the job summary; dry runs report without
blocking
- `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both
images, with exact version stamping
- `.github/workflows/release-smoke.yml`: `beta` added to the dispatch
choice list
- Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate,
and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta`
environment setup, including the warning to create the environment
before the first beta dispatch (GitHub auto-creates unprotected
environments on first reference)
- Tests: beta version-counting coverage in
`scripts/release-registry-versions.test.mjs`; beta identity and
nightly-tag guard coverage in
`scripts/__tests__/release-dry-run-notes.test.mjs`
## Verification
- `node --test` on the two touched suites: 17 pass, including the 3 new
beta tests
- `bash -n` on both shell scripts and YAML parse of all three workflows
- After merge, in order: create the `npm-beta` environment, dispatch
`channel: beta` with `dry_run: true` to preview, then a real promotion
of a published nightly through the approval gate, then a stable dry-run
against a young beta to see the soak gate report
## Risks
- If the `npm-beta` environment does not exist when the first beta
dispatch runs, GitHub creates it with no protection rules and the beta
publishes without approval. Mitigated by documentation and by creating
the environment before merge (operator step)
- Until the first beta exists, every stable dispatch requires
`skip_soak_justification`. This is deliberate — the first beta ships
immediately after this merges — but it is a behavior change to the
stable dispatch
- The soak clock reads the beta's npm publish time from the registry; a
registry outage makes the preflight fall back to requiring justification
(fail-closed)
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's nightly lane gates every nightly on the
release smoke suite, which boots the published artifact in a Docker
container and drives real onboarding
> - The gate kept failing even after the readiness budget fix (#11187),
and the new container-log dump revealed the server was healthy but
listening on 127.0.0.1 inside the container, unreachable through
Docker's port mapping
> - `onboard --yes` without an explicit `--bind` prefers trusted-local
quickstart defaults: it writes a loopback bind into the instance config
and ignores the deployment env vars the harness passes, and that config
outranks `HOST` at runtime
> - This pull request pins the smoke container to the `lan` bind preset
and adds a wiring test for it
> - The benefit is a working nightly gate, verified end to end against a
real published canary
## Linked Issues or Issue Description
**Subsystem affected**
Release smoke testing: `docker/Dockerfile.onboard-smoke`,
`scripts/__tests__/release-verify-workflow.test.mjs`.
**Problem or motivation**
Nightly run 31428558684 failed in smoke with the server unreachable at
the mapped port for the full 420 second budget. The container logs
(captured thanks to #11187) show a fully booted server with `Bind
loopback (127.0.0.1)`. The harness sets `HOST=0.0.0.0` and the
deployment env vars, but `onboard --yes` without `--bind` deliberately
prefers trusted-local defaults, writes `bind: loopback` into the
instance config, and the config outranks `HOST` at runtime. A loopback
listener inside a container is invisible to the port mapping, so the
health check can never pass. This behavior predates the current stable,
so the harness was silently broken against every recent version — it
only surfaced now because the nightly lane is the suite's first CI
consumer.
**Proposed solution**
Pass `--bind lan` in the smoke container command (the flag is supported
by `latest` and canary alike; it selects the all-interfaces preset and
keeps the env-driven authenticated deployment), and pin the flag with a
wiring test so it cannot regress silently.
## What Changed
- `docker/Dockerfile.onboard-smoke`: the onboard command is now `onboard
--yes --bind lan --data-dir ...`, with a comment explaining why the flag
is load-bearing
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
asserting the smoke Dockerfile pins a non-loopback bind preset
## Verification
- Full local harness run against the real nightly candidate
`2026.810.0-canary.1`: container healthy, bind banner shows `lan
(0.0.0.0)`, authenticated bootstrap completed (admin created, bootstrap
invite accepted, board session verified), `/api/health` returns
`bootstrapStatus: ready`
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`: 4
pass
- After merge: dispatch `release.yml` with `channel: nightly` to run the
gate end to end in CI
## Risks
- Low. The change only affects the smoke container. `--bind lan` inside
a container exposes the port to the container network only; reachability
from outside still goes through Docker's explicit port mapping
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (CI log forensics, upstream source
tracing, local Docker reproduction and verification). All changes
model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's nightly lane (#11006) gates every nightly
publish on the release smoke suite, which boots the published artifact
in a Docker container
> - The suite's first CI execution failed at the health readiness check:
the harness hard-codes a 90 second budget, but a CI container
cold-installs paperclipai from npm and initializes embedded postgres
with no warm caches
> - When the timeout expired with the container still running, the
harness printed no container logs, so the failure gave no diagnostics
> - This pull request makes the readiness budget configurable, raises it
for CI, and dumps container logs on timeout
> - The benefit is that the nightly gate measures the artifact, not the
runner's cold caches, and a red smoke run is diagnosable from its logs
## Linked Issues or Issue Description
**Subsystem affected**
Release smoke testing: `scripts/docker-onboard-smoke.sh`,
`.github/workflows/release-smoke.yml`.
**Problem or motivation**
Run 31426044332 (first forced nightly after #11006) failed in
`smoke_nightly` with `server did not become ready at
http://localhost:3232/api/health` after exactly 90 seconds. The
harness's readiness window is hard-coded to 90 attempts at 1 second.
Locally that works because the npm cache is warm; in CI the container
downloads the full package set and embedded postgres first. The timeout
path also printed no container logs when the container was still
running, so there was no way to see how far boot had progressed.
**Proposed solution**
Make the readiness budget an environment variable
(`SMOKE_READY_TIMEOUT_SECONDS`, default unchanged at 90 for local use),
set it to 420 in the CI workflow, and dump the last 150 container log
lines when the readiness check times out on a still-running container.
## What Changed
- `scripts/docker-onboard-smoke.sh`: `SMOKE_READY_TIMEOUT_SECONDS` env
var (default 90) replaces the hard-coded readiness budget; timeout with
a still-running container now prints the tail of `docker logs`
- `.github/workflows/release-smoke.yml`: sets
`SMOKE_READY_TIMEOUT_SECONDS=420` for CI runs
## Verification
- `bash -n` on the harness and YAML parse of the workflow
- The real proof is the next `channel: nightly` dispatch of
`release.yml`, which re-runs this suite in CI with the new budget
## Risks
- Low. The local default is unchanged; CI runs simply wait longer before
declaring failure, and a genuinely broken artifact still fails (with
logs now)
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. Diagnosis from CI run logs; patch
model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem publishes the `paperclipai` npm package set
and the Docker images on two lanes: canary on every master push, and
stable on manual promotion
> - There is no middle ground between those lanes. Users must track
every merge or wait weeks for a stable. Docker `:latest` also tracks
master, so Docker users have no stable image at all
> - A calm prerelease lane needs to exist, and it must never ship a
build that failed its checks
> - This pull request adds the nightly channel: a scheduled job that
selects the newest master commit with a green canary publish, runs the
full release smoke suite against that exact published canary, and only
then republishes it as the nightly. It also separates Docker tags by
lane, so `:latest` finally means stable
> - The benefit is that users can follow prereleases at a nightly
cadence with a smoke-tested guarantee, and Docker users get real
`:canary`, `:nightly`, and stable image tags
## Linked Issues or Issue Description
**Subsystem affected**
Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.
**Problem or motivation**
The project publishes only `canary` (every master push) and `latest`
(manual stable). Users who want prereleases without per-merge churn have
no option. Docker has a second problem: master builds overwrite
`:latest`, and CI-published stables never produced Docker images,
because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger
in `docker.yml`. No stable-versioned image exists in ghcr today.
**Proposed solution**
Add a `nightly` channel. A scheduled job selects the newest
canary-tagged master commit, smoke-tests that exact published canary,
and republishes the same commit as `YYYY.MDD.P-nightly.N` under the
`nightly` dist-tag. Separate Docker tags by lane (`:canary` for master,
`:nightly` for nightly tags, `:latest` plus version tags for stable tags
only), and have the release jobs dispatch `docker.yml` at the new tag so
lane images actually build.
**Alternatives considered**
Moving the `nightly` dist-tag to the existing canary version without a
republish. Rejected: the version string would say `canary` while the
user is on nightly, which breaks at-a-glance lane identification in bug
reports and `--version` output.
## What Changed
- `scripts/release-lib.sh`: channel-parameterized
`next_prerelease_version` and `prerelease_tag_name` helpers (canary
helpers delegate to them), a `require_channel_tag_at_head` guard, and
the no-provenance retry for Sigstore transparency-log duplicates now
covers the `nightly` dist-tag as well as `canary`
- `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry
a `canary/v*` tag, publishes the full public package set as
`YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source
commit `nightly/vYYYY.MDD.P-nightly.N`
- `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) —
select candidate, smoke it via `release-smoke.yml`, publish on green
under the existing `npm-canary` environment, push the tag, dispatch
`docker.yml`. New `channel` dispatch input (default `stable`, so
existing stable dispatches are unchanged) with `nightly_source_version`
and `dry_run` support for forced runs. The stable path now also
dispatches `docker.yml` at the new `v*` tag
- `.github/workflows/docker.yml`: lane tag mapping for both image jobs —
master pushes publish `:canary` and no longer move `:latest`;
`nightly/v*` tags publish `:nightly`; only stable `v*` tags publish
`:latest` and the versioned tags. New `workflow_dispatch` trigger for
the release-job dispatches. Build-version stamping uses the exact
nightly version on nightly tag builds
- `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch
choice list
- `doc/CHANNELS.md` (new): user-facing guide to the channels
- `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping
table, and a nightly failure playbook
- `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses
`npm-canary` and needs no npm trusted-publisher changes
- Tests: channel-parameterized version helper coverage in
`scripts/release-registry-versions.test.mjs`, and nightly flow coverage
(publish identity, notes not required, canary-tag guard) in
`scripts/__tests__/release-dry-run-notes.test.mjs`
## Verification
- `node --test` on the release script suites: 68 pass, including 6 new
tests. The only failure, `acpx-patch-packaging.test.mjs`, needs
installed `node_modules` and fails identically on a pristine checkout of
master in the same environment
- `bash -n` on both shell scripts and YAML parse of all three workflows
- Live fail-path check: `./scripts/release.sh nightly --print-version`
from a master tip with no canary tag fails with `HEAD has no canary/v*
tag`
- Live success-path check: the same command from the
`canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0`
- Live selection check: the candidate-selection shell logic run against
the real repository selects the commit of `canary/v2026.806.0-canary.7`,
which matches the current npm `canary` dist-tag exactly
- After merge: dispatch `release.yml` with `channel: nightly` and
`dry_run: true` to preview, then a real forced run to validate end to
end before the first scheduled run
## Risks
- Docker `:latest` changes meaning from "latest master build" to "latest
stable release". This is deliberate and will be announced. Users who
want the old behavior pull `:canary`. Until the first stable release
after this change, `:latest` stays at its current (master-built) image
- The nightly is a rebuild of the same source commit, not the
byte-identical canary artifact that was smoked. The lockfile pins
dependencies, and the publish path's registry-visibility and
clean-prefix install gates still run on the nightly artifacts
- All npm publishing must stay inside `release.yml` because npm trusted
publishing pins that workflow file per package. The nightly jobs were
added to `release.yml` for exactly that reason; this constraint is now
documented in `RELEASING.md`
- The stable-lane Docker dispatch fails gracefully (a warning with
manual instructions) when the source ref predates `docker.yml`'s
`workflow_dispatch` trigger
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The PR workflow runs the server vitest suite across sharded runners
because the suite is pinned to one worker.
> - In successful PR run 30930345729 (2026-08-04), shard `server (3/4)`
took 311 seconds of wall time and was the slowest check in the run.
> - The suite has grown to about 946 seconds of serial vitest time, but
the duration manifest was last sampled on 2026-08-01 at about 882
seconds.
> - This pull request refreshes the per-suite duration manifest from
that run's logs and splits the lane into five shards.
> - The benefit is a shorter PR critical path: each shard carries about
196 seconds of suite time, level with the other lanes.
## Linked Issues or Issue Description
Refs #10663 (previous split of this lane into four shards).
Related: #10923 splits the separate serialized-suites lane into five
shards. Both PRs touch `.github/workflows/pr.yml` in different matrix
blocks; whichever merges second needs a trivial rebase.
**What existing behavior does this improve?**
The `general-server` vitest lane runs in four shards with a duration
manifest sampled on 2026-08-01.
**Current behavior**
In PR run 30930345729, shard 3/4 ran for 311 seconds (273 seconds in the
test step) and was the longest check in the run. The suite now totals
about 946 seconds of serial vitest time.
**Proposed behavior**
Run the same suite set in five shards, balanced with a per-suite
duration manifest refreshed from that run's shard logs (279 suites
measured by diffing consecutive completion timestamps).
**Reason and benefit**
The refreshed LPT partition balances at about 196 seconds of suite time
per shard (about 240 seconds per job), level with the other PR lanes. No
test coverage is lost.
**Breaking changes**
None. The change only alters the CI partition size and the duration
manifest.
## What Changed
- Bump the `general-server` shard matrix in `.github/workflows/pr.yml`
from four to five shards.
- Refresh `scripts/general-server-shard-durations.json` from the
2026-08-04 run's shard logs.
- Update `SHARD_COUNT` in
`scripts/__tests__/run-vitest-stable-shard.test.mjs` to five.
## Verification
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9
pass, including the complete non-overlapping partition proof and the
duration-balance check.
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2
pass.
- `node --test scripts/__tests__/e2e-shard.test.mjs` — 7/7 pass.
- A 5-way dry-run partition covers all suites exactly once with equal
projected weights.
## Risks
- Low risk. The change only alters CI partition size and duration
weights; the suite set is unchanged.
- One more runner is used per PR run for this lane.
- Stale duration weights degrade gracefully: suites missing from the
manifest get the median weight.
## Model Used
- Claude (Anthropic), Claude Code CLI, model ID `claude-fable-5`,
extended thinking with tool use enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(workflow comments explain the new shard math)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Claude <claude@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip uses CI to keep control-plane changes safe and mergeable.
> - The PR workflow splits serialized server tests across isolated
runners.
> - A recent successful run spent 305 seconds in serialized shard 2/4.
> - That job was the slowest check in the run.
> - The four shards reported about 739 seconds of Vitest suite time.
> - This pull request adds a fifth serialized shard and keeps release
verification aligned.
> - The benefit is a shorter PR critical path with no loss of test
coverage.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The PR and release verification workflows run serialized server tests in
four shards.
**Current behavior**
Successful PR run 30876682788 spent 305 seconds in `Verify serialized
server suites (2/4)`. The test step used 256 seconds and made this job
the slowest check.
**Proposed behavior**
Run the same serialized suite set in five complete and non-overlapping
shards.
**Reason and benefit**
The measured suites reported about 739 seconds of total Vitest time.
Five runners reduce the expected average suite time from about 185
seconds to about 148 seconds before setup overhead.
**Breaking changes**
None. The change only alters CI partition size.
## What Changed
- Split serialized server tests into five shards in the PR workflow.
- Apply the same five-shard layout to release verification.
- Add a partition test that proves complete and non-overlapping
serialized coverage.
- Update release workflow coverage tests for five shards.
## Verification
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs`
- `git diff --check`
## Risks
- Low risk. CI uses one additional runner for the serialized lane.
- Round-robin partition weights can still vary as suite timings change.
> This change does not overlap with planned core work in `ROADMAP.md`.
Related PR #10663 optimized the separate general-server lane.
## Model Used
- OpenAI Codex, GPT-5, agentic coding with reasoning, tool use, and code
execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Isolated workspaces give each task a safe and reproducible checkout.
> - The existing setup cloned the development database before an agent
needed to run the app.
> - This made worktree creation slower and heavier for tasks that never
start a service.
> - Runtime services already use one server start path for heartbeat,
operator, and startup recovery flows.
> - This pull request moves heavy setup to that start path and keeps
worktree creation lean.
> - The benefit is faster isolated workspace creation with the same
reliable runtime setup when a service starts.
## Linked Issues or Issue Description
Related pull request: #10652 covers the initial deferred
database-seeding slice. This pull request supersedes it with end-to-end
runtime provisioning and safe cleanup.
**What existing behavior does this improve?**
This improves isolated worktree creation, runtime service startup, and
isolated instance cleanup.
**Subsystem affected**
Cross-cutting: CLI worktree setup, server runtime orchestration, shared
workspace contracts, and development scripts.
**Current behavior**
Paperclip seeds an isolated development database during worktree
creation. It can also leave an isolated instance directory after
workspace teardown. This work happens even when no runtime service
starts.
**Proposed behavior**
Paperclip creates the worktree with a lean eager setup. It runs an
idempotent runtime provision command before the first managed service
spawn. Concurrent starts share one provision attempt. Teardown removes
the isolated instance safely.
**Reason and benefit**
Many agent tasks only edit and test code. They do not need a running
Paperclip instance. Deferring the database seed reduces workspace
startup cost while preserving automatic setup for tasks that start the
app.
**Breaking changes**
None. The new runtime provision command is optional. Existing workspace
behavior is unchanged when it is absent.
## What Changed
- Split Paperclip worktree setup into a lean eager script and an
idempotent runtime provision script.
- Added `runtimeProvisionCommand` to project, issue, realized workspace,
and persisted workspace contracts.
- Added a per-workspace provision mutex before local service spawn for
heartbeat, operator, and startup recovery flows.
- Added a persisted `provisioning` service state and the
`workspace_runtime_provision` operation phase.
- Kept provision time outside the service readiness timeout and made
failed attempts visible and retryable.
- Reclaimed isolated instance data during safe workspace teardown.
- Serialized deferred database seeding across processes and bound
teardown to the instance root captured in persisted workspace metadata.
- Added tests for config flow, concurrency, retry, no-op behavior,
readiness timing, scripts, CLI commands, and cleanup.
- Documented the eager and runtime provisioning contracts.
## Verification
- `pnpm -r typecheck`
- `pnpm build`
- `pnpm test:run` (server: 3,201 passed; UI: 3,345 passed; the CLI phase
exposed one environment-sensitive AWS doctor assertion because the agent
runtime injects static AWS credentials)
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest
run cli/src/__tests__/secrets.test.ts -t 'passes AWS doctor checks when
non-secret provider config is present'`
- Focused runtime tests cover serialized provisioning, retry after
stderr failure, absent-command no-op behavior, operation logging,
persisted state order, and readiness timeout exclusion.
- Focused CLI and cleanup tests cover concurrent seed serialization,
stale-lock fail-closed behavior, persisted instance ownership, and
rewritten sibling pointers.
## Risks
- A faulty runtime provision script blocks service startup. Paperclip
records stderr, marks the service failed, and retries on the next start.
- Concurrent service requests share an in-process provision attempt,
while the seed command uses an atomic filesystem lock across processes.
A stale lock fails closed and requires an operator to verify no seed is
running before removing it.
- Isolated instance cleanup is destructive. The cleanup service
validates ownership and path containment before removal.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, `gpt-5.6-sol`, with agentic reasoning, tool use, and
code execution. The service does not expose the context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request CI must give contributors fast and stable feedback.
> - The `general-server` Vitest lane runs many single-worker server
suites.
> - A recent completed PR run showed this lane as the slowest completed
check.
> - Three shards still left one runner with the largest share of work.
> - This pull request splits that lane into four duration-balanced
shards.
> - The benefit is a shorter critical path for the same server test
coverage.
## Linked Issues or Issue Description
No public GitHub issue exists for this CI maintenance change.
**Pre-submission checklist**
- I confirmed this improves existing behavior. It does not add a new
command, endpoint, or concept.
- I searched open public issues and pull requests for related CI
sharding work.
**What existing behavior does this improve?**
The pull request workflow's `general-server` Vitest lane.
**Subsystem affected**
Cross-cutting. This affects GitHub Actions CI and the Vitest shard
duration manifest.
**Current behavior**
The `general-server` lane uses three shards. The server suites now total
about 880 seconds of serial Vitest wall time. The slowest shard was
about 313 seconds in the measured run.
**Proposed behavior**
The `general-server` lane uses four shards. Each shard receives about
220 seconds of predicted suite weight from the refreshed duration
manifest.
**Reason and benefit**
The slowest PR check controls how soon a reviewer can trust the PR. Four
balanced shards reduce the slowest `general-server` shard while keeping
the same suite selection rules.
**Breaking changes**
None. This only changes CI partitioning and duration data for existing
test suites.
**Additional context**
Related public searches found no exact open issue or pull request for
this `general-server` sharding change.
## What Changed
- Split the `general-server` CI matrix from three shards to four shards.
- Refreshed `scripts/general-server-shard-durations.json` with wall-time
weights from a recent completed PR run.
## Verification
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check origin/master...HEAD`
- Dry-ran the four `general-server` shards locally during
implementation. The partition covers 300 unique suites with about 220.56
seconds of predicted weight per shard.
- Ran a local sensitive-data scan before push. It found only test
filenames that contain words such as `secret` or `token`, not credential
values.
## Risks
Low risk. The main risk is that the duration manifest becomes stale as
suite costs move. Missing suites fall back to the median weight, so the
lane still runs if the manifest is incomplete.
## Model Used
OpenAI Codex, GPT-5, with tool use and local command execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip uses pull request CI to test changes before merge.
> - The e2e PR lane runs Playwright specs in a shard matrix.
> - Each shard builds a list of spec files for its matrix entry.
> - The workflow passed that list after a literal `--` separator.
> - Playwright did not receive the list as file filters.
> - This pull request removes the separator and adds a guard test.
> - The benefit is that each e2e shard runs only its assigned specs.
## Linked Issues or Issue Description
Refs #10629.
**What happened?**
The e2e shard step used `pnpm run test:e2e -- $specs`. The shard spec
list was not applied as Playwright file filters.
**Expected behavior**
Each e2e shard should pass only its selected specs to Playwright.
**Steps to reproduce**
1. Inspect `.github/workflows/pr.yml` at the merge commit for #10629.
2. Find the `e2e_shards` command that invokes `pnpm run test:e2e`.
3. See the literal `--` before `$specs`.
**Paperclip version or commit**
`86767951`
**Deployment mode**
GitHub Actions PR CI.
## What Changed
- Removed the literal `--` from the e2e shard `pnpm run test:e2e $specs`
invocation.
- Added a regression test that checks the workflow passes `$specs`
without that separator.
## Verification
- `node --test scripts/__tests__/e2e-shard.test.mjs`
## Risks
Low risk. This changes one CI command and one workflow guard test. The
main risk is shell argument handling in the workflow, and the guard now
covers the expected command shape.
## Model Used
OpenAI GPT-5 through Codex. The run used shell and GitHub CLI tool
access. The runtime did not expose a context window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The pull request workflow protects changes with a Playwright e2e
lane.
> - That lane already uses a weighted file partition so slow specs do
not cluster by test count.
> - Recent green PR runs showed the two e2e shard jobs were slower than
the next slow required lane.
> - The largest spec is indivisible, so a third shard lets that spec run
alone and lets the rest split by duration.
> - This pull request changes only the PR e2e shard matrix and the guard
test.
> - The benefit is a shorter expected PR critical path while the
required `e2e` aggregate check name stays stable.
## Linked Issues or Issue Description
Refs #9923
**What existing behavior does this improve?**
The `pull_request` workflow Playwright e2e lane.
**Subsystem affected**
Cross-cutting: GitHub Actions CI and test scripts.
**Current behavior**
The PR workflow runs the weighted Playwright e2e partition across two
jobs. Recent green runs showed those jobs as the slowest required
checks.
**Proposed behavior**
The PR workflow runs the same e2e spec set across three weighted jobs.
The aggregate required check stays named `e2e`.
**Reason and benefit**
The third shard lets the slow smoke-lab spec run alone while the rest of
the catalog stays balanced. This should shorten the PR critical path.
The win is bounded by fixed per-job setup time.
**Breaking changes**
None. The required aggregate check contract is preserved.
## What Changed
- Change the PR e2e shard matrix from two entries to three entries.
- Update the shard guard test to expect three shards.
- Floor the balance bound at the largest single spec weight.
- Assert that the workflow does not define more shard indexes than
`SHARD_COUNT`.
## Verification
- `node --test ./scripts/__tests__/e2e-shard.test.mjs` passes with 6
tests.
- The recorded-weight partition is complete and non-overlapping: 168.0s,
116.5s, and 114.4s.
- I checked `ROADMAP.md` and found no overlapping roadmap-level core
feature.
- I searched public GitHub PRs and issues for related e2e shard work. I
found related PR #9923 and no open duplicate for this branch or change.
## Risks
- This adds one extra GitHub Actions runner to the PR e2e lane.
- The wall-clock win is bounded by fixed per-job setup.
- Behavior risk is low because the aggregate required check remains
named `e2e`.
## Model Used
OpenAI Codex, GPT-5, tool-enabled coding agent in this repository. The
runtime did not expose the context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Cody <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Managed updates can change both the application payload and its
database schema
> - Unit tests cannot prove that an older live install upgrades through
a real migration and remains recoverable
> - The managed-install work in #10045 needs a repeatable cross-version
system test
> - This pull request adds an isolated end-to-end harness for update,
migration, backup, service restart, and rollback behavior
> - The benefit is a direct proof that managed upgrades preserve the
existing database and service lifecycle across versions
## Linked Issues or Issue Description
Refs #10045
This test is a focused follow-up to the managed install integration.
Merge #10045 first so the tested install, update, service, backup, and
rollback commands are available.
## What Changed
- Added a cross-version managed-update E2E script.
- Installed an older Git ref, initialized its embedded PostgreSQL
database, and updated to a ref with one additional migration.
- Verified the pre-update backup, payload switch, service recovery,
migration result, database-cluster reuse, and rollback behavior.
- Isolated Paperclip state under a dedicated test home and cleaned up
the service and managed install on success or failure.
- Added regression tests for shell syntax, required-ref validation,
side-effect-free preflight failure, and complete failure cleanup.
## Verification
- `node --test scripts/__tests__/e2e-update-migrations.test.mjs`
- `bash -n scripts/e2e-update-migrations.sh`
- GitHub latest-head CI: build, typecheck, release registry, canary
dry-run, general tests, serialized suites, and both browser E2E shards
passed.
- Full harness execution needs an isolated macOS or Linux host with a
real launchd or systemd user service. It is intentionally not run on a
live Paperclip server host.
## Risks
- The script manages a real user service and downloads two Git refs. Run
it only on an isolated test host.
- The test needs #10045 because `origin/master` does not yet contain the
managed install lifecycle.
- The script uses a dedicated `PAPERCLIP_HOME`, refuses a pre-existing
shim or test home, and removes its service and install during cleanup.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex based on GPT-5. The runtime did not expose a more
specific deployment ID or context-window size. Reasoning, repository
access, shell execution, and GitHub tooling were enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents run tasks in isolated execution workspaces that are
provisioned as git worktrees by `scripts/provision-worktree.sh`
> - The script runs the base workspace's CLI (`cli/src/index.ts` via the
base `tsx` install) to seed each new worktree, and it only checked that
those files exist
> - pnpm links each package's `node_modules` into a hash-versioned
virtual store; a lockfile change followed by a partial or filtered
install prunes old hashed dirs without relinking every package, leaving
dangling symlinks
> - A CLI with dangling symlinks fails ESM resolution
(`ERR_MODULE_NOT_FOUND`) at boot, so provisioning aborts with
`setup_failed` — deterministically, on every retry, with no self-heal
path
> - This pull request makes provisioning health-check the CLI by
actually booting it, repair the base install when the check fails, and
degrade to the no-CLI fallback config instead of failing the run
> - The benefit is that a class of permanent `setup_failed` loops
becomes self-healing, and workspace provisioning survives a broken base
CLI
## Linked Issues or Issue Description
No public GitHub issue exists; the underlying bug is described here per
`bug_report.yml`. Related PR: #10578 self-heals the sibling
workspace-validation failure loop uncovered by the same incident
diagnosis.
**What happened?**
Isolated-workspace runs failed at provision time with `setup_failed`.
Every retry failed identically. One observed incident burned 4 runs
across two adapters before the task was stranded.
**Expected behavior**
Provisioning either succeeds or degrades gracefully; a broken base CLI
install repairs itself instead of permanently blocking all new
worktrees.
**Steps to reproduce**
In the base workspace, cause a lockfile-affecting dependency bump plus a
partial/filtered `pnpm install` so a package symlink (e.g.
`cli/node_modules/drizzle-orm`) dangles into a pruned virtual-store dir.
Start any isolated-workspace run. Provision fails with
`ERR_MODULE_NOT_FOUND` and the run ends `setup_failed`; retries never
recover.
**Paperclip version or commit**
master as of the branch point of this PR.
**Deployment mode**
Local trusted deployment with git-worktree isolated workspaces.
## What Changed
- `base_cli_healthy` now boots the base CLI (`--help`) instead of only
testing file existence, which exercises the top-level import graph.
- New `repair_base_workspace_install`: when the health check fails, run
a non-interactive `pnpm install --prod=false --force --frozen-lockfile`
in the base workspace. `--force` guarantees relinking when pnpm's
up-to-date heuristics would skip dangling symlinks; `--frozen-lockfile`
keeps the repair from mutating the shared lockfile.
- The repair install is serialized with `flock` on a lock file inside
the resolved git dir (`git rev-parse --absolute-git-dir`), so locking
also covers base workspaces that are linked worktrees, where `.git` is a
file.
- If every CLI candidate is unusable (including a base CLI the repair
could not fix), provisioning falls back to the existing no-CLI fallback
config writer (loudly, on stderr) instead of failing the run. A CLI that
runs and fails `worktree init` still fails provisioning with its real
exit code — that deliberate fail-closed policy is unchanged and covered
by an existing server regression test.
- Fixed a latent bug: `run_isolated_worktree_init` returned 0
unconditionally after the init subshell, so callers treated a failed
init as success. Exit codes now propagate.
## Verification
- Reproduced the incident state (dangling `cli/node_modules/drizzle-orm`
symlink); the base CLI failed with the exact `ERR_MODULE_NOT_FOUND` seen
in the incident run logs.
- Ran the patched script against a fresh scratch worktree: health check
failed → locked repair install ran (~26 s warm) → symlink relinked →
`worktree init` completed → exit 0 with `.paperclip/config.json` and
`.env` written.
- Happy path (healthy base CLI): provisioning behavior unchanged, exit
0.
- Verified `git rev-parse --absolute-git-dir` resolves a real directory
for both a normal checkout and a linked worktree.
- New hermetic tests: `node --test
./scripts/__tests__/provision-worktree-self-heal.test.mjs` (4 tests:
healthy CLI used, broken CLI degrades, locked repair end-to-end with a
fake pnpm, init failure propagates). Not yet wired into a CI workflow.
- `server`: the existing `realizeExecutionWorkspace` fail-closed
regression test ("fails instead of writing an unseeded fallback config
when worktree init errors after CLI detection succeeds") passes against
the new script.
- `bash -n scripts/provision-worktree.sh` is clean.
## Risks
- Low risk overall: the script only adds recovery paths; the happy path
is unchanged.
- The repair install runs in the shared base workspace. It is bounded by
`--frozen-lockfile` (no lockfile mutation) and serialized by `flock`,
but it can add ~30 s to the first provision after a base install breaks.
- If the repair cannot fix the CLI and no other CLI candidate exists,
runs now continue with an unseeded fallback config instead of failing;
that is intentional, and the fallback path already existed. Genuine
`worktree init` failures from a working CLI still fail the run.
## Model Used
Claude Fable 5 (Anthropic, model ID `claude-fable-5`), extended
thinking, agentic tool use (Claude Code harness).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The pull request workflow is the main merge gate for changes to that
app.
> - The Playwright e2e lane is expensive because every spec shares one
isolated server and runs serially.
> - Splitting that lane across runners shortens the critical path, but
the public required-check contract still needs a check named exactly
`e2e`.
> - This pull request shards the real e2e work while preserving a fast
aggregate `e2e` job for branch protection.
> - The benefit is a faster PR workflow without making otherwise-good
PRs unmergeable because a legacy required check disappeared.
## Linked Issues or Issue Description
No public GitHub issue exists for this CI follow-up.
Related prior CI work:
- Refs #8360
- Refs #9168
- Refs #9516
Bug report:
### What happened?
Sharding the PR e2e lane directly at the workflow job level changes the
emitted check names to shard-specific names, while existing branch
protection expects a check named exactly `e2e`.
### Expected behavior
The PR workflow should be able to run e2e specs across multiple runners
while still emitting a stable aggregate check named `e2e`.
### Steps to reproduce
1. Open a PR against `master`.
2. Run the PR workflow with the e2e lane split only as a matrix job.
3. Observe that the shard checks complete, but a required check named
exactly `e2e` never appears.
### Paperclip version or commit
Current `master`.
### Deployment mode
GitHub Actions pull request workflow.
## What Changed
- Added `scripts/e2e-shard.mjs`, which partitions default Playwright e2e
specs by recorded per-spec duration.
- Added `scripts/e2e-shard-durations.json` with measured e2e spec
durations so the slow smoke-lab spec does not dominate one runner.
- Split the PR workflow e2e lane into two `e2e_shards` matrix jobs and
added a fast aggregate job named exactly `e2e`.
- Added `scripts/__tests__/e2e-shard.test.mjs` to lock the shard
partition, ignored-spec sync, manifest coverage, and aggregate
required-check contract.
## Verification
- `node --test scripts/__tests__/e2e-shard.test.mjs`
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check upstream/master..HEAD`
- Searched GitHub for duplicate or related e2e-shard / required-check
PRs and issues before opening this PR; no direct duplicate was found.
## Risks
Low risk. The main risk is that the duration manifest can drift as specs
are added or runtimes change; missing specs fall back to the median
known duration, and the focused shard test catches empty, overlapping,
or badly imbalanced partitions.
## Model Used
OpenAI GPT-5 via Codex CLI coding agent, with shell/tool execution and
repository inspection.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release subsystem uses GitHub Actions dry-run previews so
maintainers can validate stable and canary release behavior before
publishing.
> - Stable release publishing needs a release notes gate so real
`latest` publishes never happen without authored notes.
> - The same gate was also blocking stable dry-run previews, which made
release QA fail before publish-sensitive work could be previewed.
> - This pull request narrows the notes requirement to real non-dry-run
stable publishes.
> - The benefit is that stable dry-run dispatches can preview the
release without same-day notes, while real stable publishes remain
protected.
## Linked Issues or Issue Description
No public GitHub issue exists for this release-QA blocker, so the bug is
described inline below.
### What happened?
Stable dry-run release previews fail when the same-day release notes
file is absent. The release script runs the stable notes-file gate
before release preview work even when `--dry-run` is set.
### Expected behavior
`./scripts/release.sh stable --dry-run` should preview the stable
release without requiring `releases/vYYYY.MDD.P.md`. Real non-dry-run
stable publishes must still fail before build/publish work starts when
the notes file is missing.
### Steps to reproduce
1. Check out current `master` before this fix.
2. Ensure the computed same-day stable release notes file does not exist
under `releases/`.
3. Run `./scripts/release.sh stable --skip-verify --dry-run`.
4. Observe that the script exits with `stable release notes file is
required` instead of reaching the release preview plan.
### Paperclip version or commit
Reproduced on `master` at `9a1d4b7983dfd50e8eb40ee9770e44999d405f60`.
### Deployment mode
Built from source / GitHub Actions release workflow.
## What Changed
- Narrowed the stable release notes gate to `channel=stable` and
`dry_run=false`.
- Updated the release script usage note to say the notes file is
required for non-dry-run stable releases.
- Added a targeted Node test covering dry-run allowed behavior and
non-dry-run blocked behavior.
- Stubbed release fixture registry-version checks so the test isolates
the notes gate without hitting npm.
## Verification
- `node --test scripts/__tests__/release-dry-run-notes.test.mjs` passed
with 2/2 subtests.
- `bash -n scripts/release.sh` exited 0.
## Risks
Low risk. The behavior change only relaxes the notes-file gate for
stable dry-runs. The new test verifies real non-dry-run stable publish
still fails before build/publish work starts when notes are missing.
## Model Used
OpenAI Codex, GPT-5-based coding agent, with shell/tool execution in a
local repository workspace.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Its PR CI runs the general-server vitest lane pinned to
`maxWorkers=1` and sharded across 3 runners (introduced in #8360)
> - Suites were assigned to shards round-robin by sorted file index, so
shard test time was unbalanced: a recent PR run split 73s / 153s / 115s,
and the heaviest shard made "General tests (server 2/3)" the slowest
check in the whole workflow at 314s wall
> - The slowest shard sets the lane's wall time, so unbalanced
partitions waste the other two runners and stretch the PR critical path
> - This pull request replaces the round-robin assignment with a
deterministic longest-processing-time partition weighted by a checked-in
per-suite duration manifest
> - The benefit is near-even shard weights (projected 113s / 113s / 113s
with the current manifest), taking roughly 40s off the PR critical path
with no reduction in coverage
## Linked Issues or Issue Description
- Refs #8360 (introduced the 3-way general-server sharding this PR
rebalances)
- No public issue exists. Problem: the general-server test lane's
round-robin shard assignment ignores per-suite duration, so one shard
can carry multiple 30s+ suites while another finishes in half the time;
the slowest shard alone determines the check's wall time.
## What Changed
- `scripts/general-server-shard.mjs` (new): manifest loader and
deterministic LPT (longest-processing-time) partitioner; suites missing
from the manifest get the median recorded weight, and a missing or
malformed manifest degrades to uniform weights so the lane never fails
on stale data
- `scripts/general-server-shard-durations.json` (new): per-suite
duration manifest sampled from a real PR run (240 suites); the
`$comment` field documents how to regenerate it
- `scripts/run-vitest-stable.mjs`: both shard-selection sites (run and
`--dry-run`) now use the balanced partition instead of index round-robin
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: 6 new tests
covering skew-balance vs round-robin, determinism, median fallback for
unlisted suites, malformed-manifest degradation, manifest coverage of
the current suite set, and real-partition balance
- `server/src/__tests__/heartbeat-issue-rewake-throttle.test.ts`:
hardened the `afterEach` sweep — post-run bookkeeping (run-event
records, follow-up wake scheduling) can still insert rows briefly after
a run reaches a terminal status, and a late insert landing between the
`agent_wakeup_requests` and `agents` deletes failed teardown with a
foreign-key violation on the first CI attempt of this PR; the sweep now
retries so a late background write cannot take down the shard
- `release-verify.yml` shares the same runner script and inherits the
balancing with no workflow change
## Verification
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9
pass (run against current master)
- `npx vitest run src/__tests__/heartbeat-issue-rewake-throttle.test.ts`
— 6/6 pass against embedded Postgres with the hardened teardown
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2
pass
- `node scripts/run-vitest-stable.mjs --dry-run` with each shard flag
shows every suite assigned exactly once across the 3 shards, with
projected weights ~113s each
## Risks
- Low risk: partition changes which runner executes which suite, not
what runs; a completeness test asserts every suite is assigned to
exactly one shard
- The duration manifest will drift as suites are added/changed; unlisted
suites get the median weight and a coverage test flags when the manifest
covers less than half the suite set, so drift degrades balance
gracefully rather than breaking the lane
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Fable 5 (`claude-fable-5`, Anthropic), extended thinking
enabled, agentic tool use (file edits, shell, test execution) via Claude
Code
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude (Paperclip SWE) <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server package ships runtime asset trees used by built-in agents
and onboarding templates.
> - A prior server build omitted those source asset trees from `dist`,
allowing a built artifact to differ from runtime expectations.
> - The copy step is now present, but the existing build-gap gate only
checked TypeScript coverage for packages whose build skips `tsc`.
> - This pull request extends that standing gate so source asset files
under the server runtime asset trees must exist at the matching `dist`
paths after build.
> - The benefit is that future server asset additions fail loudly in CI
instead of silently shipping an incomplete `dist`.
## Linked Issues or Issue Description
### What happened?
After a server build, runtime asset files under
`server/src/built-ins/**` and `server/src/onboarding-assets/**` could be
missing from `dist/` with no build failure. The existing build-gap gate
only checked TypeScript coverage for packages that skip `tsc`; it did
not verify that non-TypeScript source assets were copied to `dist`. A
server build that forgot the `cp -R` step, or that added a new asset
tree without updating the copy command, would produce an incomplete
`dist` without any CI signal.
### Expected behavior
After `pnpm --filter @paperclipai/server build`, every
non-TypeScript/non-JavaScript source file under `server/src/built-ins/`
and `server/src/onboarding-assets/` must exist at the matching path
under `server/dist/`. If any file is missing, the build-gap gate must
exit non-zero with a diagnostic listing the missing files and the
command to fix them.
### Steps to reproduce
1. Remove a copied runtime asset: `rm
server/dist/built-ins/agents/reflection-coach/AGENTS.md`
2. Run the guard: `node scripts/run-typecheck-build-gaps.mjs
--runtime-assets-only`
3. Before this fix: the command exits 0 and the missing file goes
undetected.
### Paperclip version or commit
Reproduced on `master` at `c36f1a4af` (`@paperclipai/server` 0.3.1).
### Deployment mode
Not deployment-specific — the build-gap check runs in CI on any
checkout.
## What Changed
- Extended `scripts/run-typecheck-build-gaps.mjs` with a source-derived
server runtime asset parity check for non-`.ts`/non-`.js` files under
`server/src/built-ins/**` and `server/src/onboarding-assets/**`.
- Added a guard-only mode, `--runtime-assets-only`, for focused
pass/fail verification after a server build.
- Wired `pnpm run typecheck:build-gaps` to prepare plugin SDK build
deps, build the server package, then run the existing build-gap gate
plus the new asset check.
## Verification
Pass path:
```text
$ pnpm --filter @paperclipai/plugin-sdk ensure-build-deps
> @paperclipai/plugin-sdk@1.0.0 ensure-build-deps .../packages/plugins/sdk
> node ../../../scripts/ensure-plugin-build-deps.mjs
$ pnpm --filter @paperclipai/server build
> @paperclipai/server@0.3.1 build .../server
> tsc && mkdir -p dist/onboarding-assets dist/built-ins && cp -R src/onboarding-assets/. dist/onboarding-assets/ && cp -R src/built-ins/. dist/built-ins/
$ node scripts/run-typecheck-build-gaps.mjs --runtime-assets-only
[typecheck:build-gaps] server runtime assets present in dist: 7 file(s)
```
Regression simulation (guard catches the missing file):
```text
$ rm server/dist/built-ins/agents/reflection-coach/AGENTS.md
$ node scripts/run-typecheck-build-gaps.mjs --runtime-assets-only
[typecheck:build-gaps] Missing server runtime asset(s) in dist:
- source: server/src/built-ins/agents/reflection-coach/AGENTS.md
expected dist: server/dist/built-ins/agents/reflection-coach/AGENTS.md
Run pnpm --filter @paperclipai/server build and ensure source runtime asset trees are copied into dist.
```
Standing gate (full end-to-end):
```text
$ pnpm run typecheck:build-gaps
[typecheck:build-gaps] typechecking 4 workspace(s): paperclipai, @paperclipai/plugin-authoring-smoke-example, @paperclipai/plugin-llm-wiki, @paperclipai/ui
[typecheck:build-gaps] server runtime assets present in dist: 7 file(s)
```
## Risks
Low risk. The check only reads source and dist files during the
build-gap gate. The main tradeoff is that the gate now builds
`@paperclipai/server` so a clean checkout has generated `dist` content
to validate.
## Model Used
OpenAI Codex, GPT-5 based coding agent with repository tool use and
shell execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Releases publish the same app and package set that operators
install, so release verification should keep full release-strength
coverage.
> - The release workflow currently verifies stable and canary releases
with one serial job that typechecks, runs all tests, and builds.
> - The PR workflow already proves the test surface can be split into
grouped general suites and serialized shards without changing coverage.
> - This pull request extracts the release verify work into a reusable
workflow and fans out the independent lanes.
> - The benefit is faster stable and canary release verification while
preserving the existing publish and preview gates.
## Linked Issues or Issue Description
No public GitHub issue exists for this CI improvement.
**Subsystem affected**
Cross-cutting (multiple of the above)
**Problem or motivation**
Release verification spends most of its wall time in a single serial
test step even though the same stable test surface is already
partitioned for PR CI. Stable dispatches and master-push canaries
therefore wait on one long runner after setup, typecheck, tests, and
build run sequentially.
**Proposed solution**
Add a reusable release verification workflow with parallel typecheck,
grouped general tests, serialized test shards, and build lanes. Have
both stable and canary release verification call it with the ref they
need to verify.
**Alternatives considered**
Keeping the serial `pnpm test:run` job preserves the old shape but keeps
stable and canary releases waiting on one long runner. Skipping
verification when a source SHA already has green CI would be faster, but
adds stale-check and lookup risk beyond this change.
**Roadmap alignment**
No overlapping item found in `ROADMAP.md`; this is release CI
maintenance.
**Additional context**
The new workflow keeps the release-strength full `pnpm -r typecheck`,
uses the existing stable test grouping/sharding entry points, and leaves
publish/preview jobs unchanged.
## What Changed
- Added `.github/workflows/release-verify.yml` as a `workflow_call`
workflow accepting a `ref` input.
- Split release verification into parallel `typecheck`, `general_tests`,
`serialized_tests`, and `build` jobs with 20-minute lane timeouts.
- Mirrored the PR workflow's stable test partition: `general-server`
shards 1-3, `general-workspaces-a`, `general-workspaces-b`, and four
serialized shards.
- Replaced `release.yml` `verify_canary` and `verify_stable` job bodies
with calls to the reusable workflow while leaving publish and preview
jobs unchanged.
- Added a Node test that guards the release workflow delegation and
split verify surface.
## Verification
- `actionlint 1.7.12 .github/workflows/release.yml
.github/workflows/release-verify.yml`
- `node ./scripts/release-package-map.mjs check`
- `node --test ./scripts/__tests__/release-verify-workflow.test.mjs
./scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check`
## Risks
- Release verification now starts more jobs per release event,
increasing total runner setup/install minutes. This matches the existing
PR CI tradeoff and should reduce release wall time substantially.
- The called workflow checks out the requested ref shallowly. That is
intentional for verify lanes; publish and preview jobs still retain
their existing full-history checkouts.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex, GPT-5-class coding agent in local tool-use mode with shell
execution, repository editing, GitHub connector access, and medium
reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Storybook visual baselines protect UI surfaces from unintended
visual drift.
> - Pixel-perfect screenshot baselines are sensitive to OS, font
rasterization, and browser environment.
> - The suite already stores external baseline artifacts and has an
opt-in CI path.
> - Local runs on non-matching platforms can report false-positive diffs
unless the platform lock is explicit.
> - This pull request documents the Linux/Ubuntu baseline constraint and
makes the local static server port explicit.
> - The benefit is clearer visual-review guidance and more predictable
Playwright web server startup.
## Linked Issues or Issue Description
No public GitHub issue exists.
### What happened?
The Storybook visual baseline suite requires a matching Linux capture
environment for pixel-exact comparisons, but the docs did not clearly
warn local users that non-Linux environments can produce false-positive
diffs. The Playwright web server command also relied on the static
server's default port instead of passing the configured port explicitly.
### Expected behavior
Developers should see clear Linux/Ubuntu baseline guidance before
running the visual suite locally, and Playwright should start the
Storybook static server on the same explicit port that the test config
expects.
### Steps to reproduce
1. Review the Storybook visual docs before this PR.
2. Run or inspect the Storybook visual Playwright config.
3. Notice the missing platform guidance and implicit static server port
coupling.
### Paperclip version or commit
Reproducible on `master` before this branch.
### Deployment mode
Local dev (pnpm dev) / built from source.
## What Changed
- Documents the Linux/Ubuntu-only baseline limitation in the developer
docs and visual-suite README.
- Adds `--port` parsing and validation to the Storybook static server
helper.
- Adds regression coverage for `--port` followed by another flag.
- Passes the Playwright web server port explicitly from the Storybook
visual config.
## Verification
- Passed: `node --check scripts/serve-storybook-static.mjs`
- Passed: `node --test
scripts/__tests__/serve-storybook-static.test.mjs`
- Passed: `node --test
scripts/__tests__/storybook-visual-baseline.test.mjs`
- Greptile: 5/5 with no unresolved review threads after commit
`94a649755a2ae7c4a34a3e8a1f16ec4d26d738fd`.
- Not run: full `pnpm test:storybook-visual`, because it builds
Storybook and runs the browser visual suite; this PR only changes docs
plus server port plumbing.
## Risks
Low risk. The server still defaults to port 6106 when no explicit port
is provided, and invalid port values now fail fast with a clear error
before the Playwright server waits for an unreachable URL.
## Model Used
OpenAI GPT-5 Codex coding agent with local command execution and
repository editing tools.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Its UI is the operator's daily surface: task lists, boards, budgets,
agent status — all built on shadcn components and Tailwind
> - Visual values (colors, spacing, type sizes, radii) were hardcoded at
~1,600 call sites: the same "small gray label" was 9/10/11px depending
on the file, charts disagreed with chips about status colors, two
toggle-switch implementations coexisted in two greens, and there was no
visual regression coverage
> - This made the UI drift-prone and made any restyle a
hundreds-of-files project, which discourages design iteration
> - This pull request extracts visual values into a single token layer
in `ui/src/index.css`, adds a Storybook visual regression suite backed
by external immutable baseline archives, and then applies a deliberate
retune reviewed change-by-change on screenshot diffs
> - The benefit is that Paperclip's look becomes a config surface:
retheming is a token edit reviewed as a snapshot diff, drift is blocked
by a token gate, and future UI PRs can prove exactly what changed
visually without committing hundreds of PNGs
## Linked Issues or Issue Description
No existing public issue covers this work (searched "design tokens",
"visual regression", "design system" across issues and PRs). Related in
spirit: Refs #8982 (theming a hardcoded panel — a one-off instance of
the same problem class this PR addresses systematically).
**Problem (feature-request form):** UI visual values are hardcoded per
call site with no source of truth and no regression coverage;
consistency depends on reviewer memory, and restyling requires mass file
edits.
**Proposed solution (this PR):** a single token layer + enforcement gate
+ externally stored visual snapshot suite, then an intentional restyle
on top of that foundation.
## What Changed
- **Token extraction (zero visual change, machine-verified during
development):** committed codemods (`scripts/codemod-*.mjs`) moved
~1,600 hardcoded color/type/spacing/radius/shadow/misc values into named
tokens in a non-inline `:root` block of `ui/src/index.css`.
- **Visual regression suite:** `pnpm test:storybook-visual` covers 255
stories × light/dark = 510 Playwright screenshots at `maxDiffPixels: 0`,
plus new primitive-coverage stories and deterministic-render fixes.
- **External visual baselines:** committed PNG snapshots were removed.
`tests/storybook-visual/baseline-manifest.json` pins an immutable
archive URL/hash/size/count, and `scripts/storybook-visual-baseline.mjs`
handles `download`, `verify`, `pack`, and trusted maintainer `upload`
flows.
- **Opt-in visual CI artifacts:** added a `Storybook Visual` workflow
that runs on manual dispatch or PRs labeled `storybook-visual`,
downloads/verifies the baseline, runs Playwright, and uploads Playwright
report/test-result artifacts for review. Normal PR runs do not mutate
baseline objects.
- **Token gate:** `pnpm check:token-gates` — zero hex literals, zero
arbitrary bracket values, zero raw font-sizes in `ui/src/components/**`
and `ui/src/pages/**`, with a documented inline allowlist for legitimate
opt-outs.
- **Theme retune (intentional, snapshot-reviewed):** new base theme
values; radius ladder derived from a single `--radius` knob; micro-type
cluster collapsed to a named ladder (`--text-nano/micro/compact` +
Tailwind `text-xs`/`text-sm`); letter-spacing collapsed to named steps.
- **One status-color vocabulary:** charts, quota/budget bar fills,
RUNNING/live chips, and liveness indicators all use the canonical
`--status-*` hues. Light-mode legibility fixes for red alert surfaces
that used dark-tuned text classes.
- **One switch:** `ToggleSwitch` restyled to the registry capsule form,
second hand-rolled implementation removed, and all call sites unified.
- **Docs:** `DESIGN.md` is the design contract; `doc/design/` holds
audit reports, decision logs, and updated guidance for external baseline
review/update workflows.
- Dead code removed (`agentStatusBadge` duplicate map), byte-identical
contrast constants consolidated, semantic renames
(`--project-seed`/`--project-none`, `--liveness-blue`).
## Verification
- `pnpm check:token-gates` — 3/3 gates CLEAN during the design-system
run
- `pnpm typecheck` && `pnpm --filter @paperclipai/ui build` — green
during the design-system run
- `node --test scripts/__tests__/storybook-visual-baseline.test.mjs` —
pass after external-baseline rework
- `pnpm exec tsc --noEmit --pretty false --module NodeNext
--moduleResolution NodeNext --target ES2022 --types
node,@playwright/test tests/storybook-visual/playwright.config.ts
tests/storybook-visual/storybook-visual.spec.ts` — pass after
external-baseline rework
- `git diff --check origin/pr/9134..HEAD` — pass after external-baseline
rework
- `find tests/storybook-visual -type f -name '*.png' -print | wc -l` —
`0`
- `node scripts/storybook-visual-baseline.mjs verify` — intentionally
fails closed until the first trusted maintainer publishes the baseline
archive and updates `baseline-manifest.json`
## Risks
- **Large but shallow:** the PR still touches many UI files due to
mechanical token extraction and retune work, but committed PNG snapshot
churn has been removed from the branch.
- **Baseline publication required before the visual suite can pass in
clean clones:** the manifest currently has placeholder archive metadata.
A trusted maintainer must publish the first immutable archive, then
update `baseline-manifest.json`.
- **Rendering platform variance:** the external baseline should be
captured in the documented Linux/Chromium environment. Future CI runs
verify against the pinned archive and fail closed on checksum/count
mismatch.
- **Visual CI is opt-in while stabilizing:** add the `storybook-visual`
label or dispatch the workflow manually to produce downloadable
Playwright report/test-result artifacts.
- **Scheduled follow-ups, deliberately out of scope:** Tailwind palette
classes map to semantic tokens in a dedicated pass; card/pill component
consolidation; ESLint ratchet. Tracked in
`doc/design/DECISION-SHEET.md`.
## Model Used
Claude Fable 5 (Anthropic, `claude-fable-5`, Mythos-class tier) with
extended thinking, running in Claude Code with tool use; mechanical
phases delegated to Claude Sonnet subagents. Follow-up external-baseline
rework assisted by OpenAI Codex (`gpt-5` coding agent with repository,
terminal, and GitHub tool use). All bulk rewrites executed via
deterministic, idempotent scripts committed in `scripts/`; intentional
visual changes were human-reviewed on screenshot contact sheets.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run targeted local verification and documented the
intentional baseline-publication failure above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green *(pending new CI run after this
rework)*
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
*(pending review)*
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code) and
OpenAI Codex
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Dotta <bippadotta@protonmail.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - CI runs a Canary Dry Run job that exercises `release.sh`, which
builds the standalone sandbox-provider packages for publish
> - That step (`scripts/build-standalone-public-packages.mjs`) built the
7 provider plugins serially — each doing `rm -rf dist && tsc` — making
it the dominant cost (~49s) inside the slowest PR check (~4.9m wall)
after the general-server lane was already sharded
> - The packages are independent (their own `node_modules` via
`--ignore-workspace`, their own `dist`), so the serial build is pure
latency with no correctness benefit
> - This pull request builds them with a bounded-concurrency pool sized
to the runner CPU count (overridable via
`STANDALONE_BUILD_CONCURRENCY`), buffering each package's output and
flushing it as one block so parallel logs stay readable, and aggregating
failures by original index
> - The benefit is a faster Canary Dry Run / PR feedback loop without
changing what gets built or published
## Linked Issues or Issue Description
No public GitHub issue exists. Inline feature/perf description:
### Problem or motivation
`build-standalone-public-packages.mjs` builds standalone provider
packages serially, making it the largest single cost inside the slowest
PR check.
### Proposed solution
Run independent per-package builds through a bounded-concurrency worker
pool sized to runner CPU count, with an env override and readable
buffered logs.
### Alternatives considered
Keep the serial build for simpler logs, but that preserves the avoidable
CI latency.
### Roadmap alignment
This is CI maintenance and does not overlap planned core roadmap work.
## What Changed
- `scripts/build-standalone-public-packages.mjs`: replaced the serial
per-package build loop with a bounded-concurrency pool (default = runner
CPU count, override via `STANDALONE_BUILD_CONCURRENCY`); per-package
stdout/stderr is buffered and flushed as a single block; failures are
aggregated by original package index so one failure neither aborts the
others mid-flight nor obscures which package broke.
- `scripts/__tests__/build-standalone-concurrency.test.mjs`: new
`node:test` unit suite covering the pool (limit respected, all items
run, ordered failure aggregation, env-override resolution).
- `.github/workflows/pr.yml`: wired the new unit test into the policy
job.
## Verification
- `node --test
./scripts/__tests__/build-standalone-concurrency.test.mjs` → 6/6 pass
- `node ./scripts/release-package-map.mjs check` → OK (29 enabled for CI
publish)
- `git diff --check origin/master..HEAD` → clean
## Risks
- Low risk. Build inputs/outputs are unchanged; only scheduling differs.
The concurrency is bounded by CPU count and overridable; output is
buffered per package so logs remain attributable. If a package fails,
all failures are still reported with their package index.
## Model Used
- Claude (Anthropic), `claude-opus-4-8`, extended thinking with tool
use.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Every PR runs the `PR` GitHub Actions workflow, whose `verify` gate
fans out into parallel test lanes (general tests, serialized server
route suites, build, typecheck)
> - The `General tests (server)` lane had grown into the run's critical
path: it executed all ~213 non-route server suites serially in a single
job (~7.2m of test time), more than 2x any other job
> - It runs serially because `server/vitest.config.ts` pins `maxWorkers:
1`, so server suites cannot parallelize within a single runner — the
only lever is spreading them across runners
> - This pull request shards that lane into 3 even partitions that run
on separate runners, mirroring the 4-way sharding already used for the
serialized route suites
> - The benefit is the lane drops from ~7.7m to ~2.4m/shard, cutting
overall PR wall time roughly in half (~8.5m → ~4.2m)
## Linked Issues or Issue Description
No public GitHub issue exists for this work, so the underlying issue is
described inline following the feature-request template.
### Problem or motivation
PR CI wall time had crept back up to ~8.5m. On a recent fully-green run,
the `General tests (server)` job took 7.72m — more than double any other
job and the clear critical path. Of that, 7.23m was pure test execution
(dependency install was a cached 0.27m). The job ran all server suites
that are not route/authz tests (213 files) one after another, because
the server vitest project pins `maxWorkers: 1`, making these suites
inherently serial within a single runner.
### Proposed solution
Shard the general-server lane across 3 parallel runners — the same
technique the route/authz suites already use — so the suite set is split
into even, deterministic partitions that run concurrently. Add a
regression test that proves the shards always cover the full suite set
with no gaps or overlap.
### Alternatives considered
- **Raise `maxWorkers` for the server project** to parallelize within
one runner — rejected: the server suites share process-level state
(DB/port), which is exactly why `maxWorkers: 1` is pinned.
- **Two shards instead of three** — would leave the lane at ~3.6m, still
above the next bottleneck (Canary Dry Run, ~4.1m wouldn't be the gate).
Three lands the lane comfortably below it.
- **Do nothing / accept the slow lane** — rejected: it gates every PR.
### Roadmap alignment
Developer-experience / CI tooling. Not core product roadmap work; does
not overlap with planned features in `ROADMAP.md`.
## What Changed
- `scripts/run-vitest-stable.mjs`: the `general-server` general-test
group now accepts `--shard-index` / `--shard-count`. It enumerates the
full server test set (the whole `server/src` tree, minus the route/authz
suites that already run in their own serialized shards) and splits it
deterministically by modulo. The non-sharded local invocation (`pnpm
test:run:general --group general-server`) is unchanged.
- `.github/workflows/pr.yml`: the `general_tests` matrix runs
`general-server` as 3 parallel shards (1/3, 2/3, 3/3). Workspace groups
are unchanged. The `verify` gate already aggregates the whole matrix
result, so the required check name is unaffected.
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: a `node:test`
suite asserting the 3 shards form a complete, non-overlapping partition
of the general-server set, that no route/authz suite leaks into it, and
that shard flags are rejected for the parallel workspace groups. Wired
into the `policy` job.
## Verification
- New partition test passes locally: `node --test
./scripts/__tests__/run-vitest-stable-shard.test.mjs` (3/3).
- Confirmed the 3 shards form a complete, non-overlapping partition of
all 213 files (71/71/71).
- Ran a live thin shard (3 real server suites, including one outside
`__tests__`) — 23 tests passed, confirming positional-include execution
works end to end.
- This PR's own CI is the authoritative check: all three `General tests
(server (n/3))` jobs went green on the prior run, collectively covering
every suite the old single job ran.
## Risks
- Low risk. No product code changes — only test orchestration and CI
matrix. Shard partitioning is deterministic and is now covered by an
automated test that fails if the partition ever develops a gap or
overlap. Modulo-on-sorted-filenames balances duration reasonably,
matching the approach already proven by the serialized route shards.
## Model Used
- Claude (Anthropic), `claude-opus-4-8`, extended thinking + tool use
(agentic coding via Paperclip).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (N/A — no UI change)
- [x] I have updated relevant documentation to reflect my changes
(inline comments explain the sharding rationale)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending this PR's run)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending review)
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
Devin Foley
Dotta
Nicky Leach
github-actions[bot]
scotttong
