## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The server package is published to npm, but its native-runtime
driver code lives in `packages/paperclip-runner`, a private workspace
package that is never published
> - So the server build vendors the runner's compiled code by copying it
in directly, instead of taking it as a normal npm dependency
> - But `cp -R` only copies code, not `node_modules`, so every npm
package the runner imports has to be re-declared by hand in
`server/package.json` to stay resolvable once vendored
> - That hand mirroring step is silent and easy to forget: it missed
`smol-toml` in #13110, and CI stayed green while production crash-looped
3 seconds into every start (#13116)
> - This pull request keeps the proven `cp -R` vendor step exactly as it
was, and adds a build check that derives the required dependency set
from an esbuild scan of the vendored entry points, failing loudly and
precisely if any package the runner actually needs isn't declared in
`server/package.json`
> - The benefit is the dependency list is now verified against the real
module graph instead of hand-copied, so this exact class of bug cannot
pass a green build again -- without changing how the runner's code is
laid out on disk, which several of its modules depend on for unrelated
filesystem lookups
## Linked Issues or Issue Description
Refs: #13110 (introduced the `smol-toml` import that the vendor step
could not resolve), #13116 (the follow-up fix for a different oversight
in the same PR), #11813 (the same "vendored package installed outside
the monorepo dependency graph loses a runtime dependency" failure shape,
in the Kubernetes plugin installer instead of the server build)
No issue exists yet for this specific incident, so per CONTRIBUTING.md
option (B):
**What happened?**
`packages/paperclip-runner/package.json` added `smol-toml` as a runtime
dependency in #13110. `server/package.json`'s existing convention (see
`acpx`, `ajv`) requires mirroring every runtime dependency the vendored
runner imports into `server/package.json` too, because the server build
copies the runner's compiled `dist/` tree with `cp -R` -- code only, no
`node_modules`. That mirroring step was missed. CI never runs the
compiled server (`node dist/index.js`); it only builds it, type-checks
it, and boots the app in dev mode via `tsx` against source, which never
touches the vendored path. So the PR merged green, and the deployed
server crash-looped in production:
```
Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'smol-toml' imported from
/srv/paperclip/app/server/dist/vendor/paperclip-runner/drivers/codex/codex-startup-trust.js
```
**Expected behavior**
Any npm package the vendored runner code needs at runtime should either
be guaranteed present by construction, or the build should fail with a
clear, actionable error before the change ever reaches a PR -- not
silently pass CI and fail only once deployed.
**Steps to reproduce (the original incident)**
1. Add a new runtime dependency to
`packages/paperclip-runner/package.json` (e.g. a TOML parser) and use it
from a module reachable from the runner's `index.ts` export graph.
2. Do not add the same dependency to `server/package.json`.
3. Run `pnpm build` in `server/` -- it succeeds.
4. Run `node dist/index.js` -- it crashes with `ERR_MODULE_NOT_FOUND`
for the new package.
## What Changed
- **Revision note:** the first version of this PR replaced the `cp -R`
vendor step with an esbuild bundle of the runner's entry points.
Greptile's review correctly caught that this broke packaged
ACPX/OpenCode provider startup: several runner modules resolve sibling
build artifacts via `import.meta.url`-relative filesystem paths (not JS
imports) at whatever depth their source file sits at, and bundling
collapses/rearranges that layout. The current version keeps the file
layout untouched and only adds verification. See the second commit's
message for the full explanation.
- `server/scripts/verify-runner-vendor-dependencies.mjs`: a new build
step that runs esbuild with `write: false` (a pure module-graph scan --
nothing is written to disk) against the runner's two entry points server
actually imports (`index.js`, `testing.js`), with `packages: "external"`
so its metafile reports exactly which npm packages the code needs at
runtime. It fails with a precise, actionable error if any of them isn't
declared in `server/package.json`'s `dependencies`. This is deliberately
more precise than "mirror every dependency the runner declares": running
it against this repo's real manifests shows
`packages/paperclip-runner/package.json` declares dependencies
(`react-markdown`, the codex/opencode CLI packages, ...) that only its
unrelated `./react` and `./browser` export subpaths use -- server never
imports those, so a blanket mirror rule would demand dependencies server
doesn't actually need.
- `server/package.json`: added the new check into the `build` script
(right after the runner is built, before the expensive `tsc`/copy steps,
so it fails fast), and added `smol-toml` (`^1.4.2`, matching
`packages/paperclip-runner/package.json`) to `dependencies` -- the
actual missing piece from #13110. The vendor step (`cp -R
../packages/paperclip-runner/dist/. dist/vendor/paperclip-runner/`) is
unchanged from before this PR.
- Widened `server/vitest.config.ts`'s `include` to also run
`scripts/**/*.test.mjs`, and added
`server/scripts/verify-runner-vendor-dependencies.test.mjs` unit-testing
the pure dependency-diff function (`findMissingVendorDependencies`)
against the exact shape of the `smol-toml` incident, plus a case proving
an unreachable dependency (like `react-markdown`) is correctly never
flagged.
- Updated `server/src/__tests__/server-package-build-script.test.ts`'s
existing build-script assertions to match.
## Verification
- `node --check` on the new script -- syntax OK. `node -e` JSON-parsed
the edited `package.json` files after every edit.
- Unit-verified `findMissingVendorDependencies` directly against:
nothing missing, one missing (the `smol-toml` shape), and multiple
missing with stable sort order.
- Ran the actual check against this repo's real
`packages/paperclip-runner/package.json` and `server/package.json` (via
a standalone `node` invocation, since `pnpm build` needs a Rust
toolchain this sandbox doesn't have -- see below) to see its real
output. It correctly reported `smol-toml`, `acpx`, and `ajv` as already
satisfied, and did **not** flag `react-markdown`, `remark-gfm`,
`json-schema-to-ts`, `opencode-ai`, `@openai/codex`, or the
`@agentclientprotocol/*` packages -- confirming the "reachable from
index.js/testing.js" scoping works as intended and doesn't demand
dependencies server doesn't need.
- Built a fixture tree at a real filesystem location (not just
in-process) mimicking `packages/paperclip-runner`: a manifest declaring
both a reachable dependency (`smol-toml`, actually imported by the
fixture's `dist/index.js`/`testing.js`) and an unreachable one
(`react-markdown`, declared but never imported). Copied the real script
next to a fixture `server/package.json` and ran it as its own process
(`node server/scripts/verify-runner-vendor-dependencies.mjs`), twice:
- `smol-toml` missing from the fixture's server dependencies → the
script throws with the exact intended message and exits 1.
- `smol-toml` present, `react-markdown` absent → the script exits 0,
proving the unreachable dependency is correctly never flagged.
- Not verified locally: the real `packages/paperclip-runner` build, and
therefore the check running end-to-end against its true
`dist/index.js`/`dist/testing.js`. This sandbox has no Rust toolchain
(the runner's own build compiles a Cargo binary) and an incomplete
workspace install. CI's `Build` job (`.github/workflows/pr-trusted.yml`)
runs the real thing; I'll watch it on this PR.
## Risks
- The check's precision (scoping to what's reachable from
`index.js`/`testing.js`, rather than every declared runner dependency)
means a dependency that becomes reachable through some *other* export
subpath server starts importing later would need this check's
entry-point list updated too. That list is a 2-line array in the script
with a comment explaining why, and matches the only two paths server/src
actually imports today (verified by a repo-wide search).
- This only changes a build-time check; the actual vendored file layout
(`cp -R` of the runner's whole compiled tree) is byte-for-byte the same
as before this PR, so there's no behavioral change to the running server
beyond `smol-toml` now being present as intended.
- I could not exercise the real Rust-backed build locally (no Cargo in
this sandbox); see Verification. I am relying on CI's `Build` job to
confirm this end to end and will fix forward if it surfaces something
the fixture-based testing didn't.
## Model Used
Claude Sonnet 5 (`claude-sonnet-5`), via Claude Code. Standard
(non-extended) reasoning mode, with tool use (Bash, Read, Edit/Write,
`gh`) for repository exploration, local esbuild-based verification
against hand-built fixtures, and PR authoring. No extended thinking
mode.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass — see Verification: full
local verification was not possible (no Rust toolchain, incomplete
workspace install in this sandbox); watching CI's `Build` job on this PR
to confirm.
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes — no
user-facing docs describe this internal build step; none needed
updating.
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — pending, will monitor.
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
addressed the first review round; watching for re-review.
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner needs a narrow server trust boundary before an
adapter can start it.
> - The package has durable runner transport, but the server does not
host or authorize that transport.
> - Native persistence exists, but no writer connects PRP events to
those records.
> - A direct adapter must not enter this path by accident.
> - This pull request adds a hidden, run-bound PRP server coordinator.
> - The benefit is a recoverable server boundary that remains
unavailable to normal execution.
## Linked Issues or Issue Description
Refs #11962
Refs #12129
Refs #12169
**Subsystem affected**
Cross-cutting. The change affects the runner package and server
orchestration.
**Problem or motivation**
The server cannot authenticate runnerd, commit PRP events before ACK,
authorize semantic tools, or enter native finalization from a durable
runner result. The application must have this hidden boundary before a
guarded adapter can use the runner.
**Proposed solution**
Add an authenticated PRP WebSocket authority and register it only for
one exact persisted native Codex run. Bind each connection and event to
the company, issue, agent, run, runner, session, turn, item, and
verified runner identity. Commit each event before its cumulative ACK.
Project only authorized same-task read tools. Rebuild the accepted
result and finalization record from durable result and terminal events.
**Alternatives considered**
The server could expose a broad runner API key or route semantic calls
through existing adapter endpoints. Those options grant too much
authority and weaken replay recovery. The server could also add the
user-facing adapter in this pull request. That option would mix rollout
selection with the transport trust boundary and make legacy
compatibility harder to review.
**Roadmap alignment**
This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface.
## What Changed
- Add the durable PRP server authority with one-use bootstrap tickets,
reconnect leases, encrypted frames, bounded state, cumulative ACKs, and
idempotent commands.
- Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://`
URL from the configured Paperclip API URL.
- Register one authority only after the coordinator verifies the
complete native Codex run binding.
- Commit validated PRP events to `heartbeat_run_events` before ACK.
Reject source gaps and conflicting replays.
- Rebuild accepted results and finalization records from durable result
and terminal events. Enforce finalization owner leases and retry times.
- Project five same-task read operations. Recheck run, agent, task, and
company authority for each call.
- Keep the route hidden. No adapter selects this coordinator, and no
code starts runnerd.
- Vendor the compiled runner TypeScript runtime into the server package
while keeping the workspace package development-only for the server.
- Document the package, database writer, run-log payload, and credential
exclusions.
## Verification
- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All
TypeScript protocol checks and 69 Vitest tests pass, including
commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust
integration tests pass. Conformance and replay parity pass.
- Run the focused server WebSocket, coordinator, package-build, and
startup-wiring suites. All 26 tests pass, including a clean-checkout
reproduction with the runner `dist` directory absent.
- Run `pnpm -r typecheck`.
- Run `pnpm test:run`.
- Run `pnpm build`.
- Confirm that the diff contains 19 files. Confirm that it contains no
workflow or `pnpm-lock.yaml` change.
## Risks
- The server installs the WebSocket route at startup. An unregistered or
malformed run path fails closed and creates no native record.
- Bootstrap tickets are one use. The private state directory uses mode
`0700`, and the state file uses mode `0600`. The file stores derived
authentication verifiers and never stores raw tickets or lease tokens.
- The journal has explicit frame, command, event-window, and file-size
bounds. A bound violation closes the runner connection or rejects the
command.
- A runner event reaches the database before its ACK. A crash between
event commit and ACK causes a byte-equivalent replay, not a second
logical effect.
- The coordinator accepts only an existing queued or running native
Codex row with exact company, task, agent, runner, session, and
completion-contract ownership.
- Existing direct adapters do not call this service. They keep their
current execution, transcript, result, and finalization paths.
- The server has no production dependency on the private runner package.
Its build copies the compiled runtime into `server/dist`; the workspace
link is development-only. This adds no external package and does not
change the lockfile.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Execution workspaces give each run an isolated directory and a
selected base ref
> - A remote-only base ref can fail before `git worktree add` when the
ref is not local
> - A setup failure before adapter dispatch must block the run without
an agent-only retry
> - This pull request resolves both remote-tracking ref forms and bounds
recovery for the same unresolved ref
> - The benefit is correct workspace setup and no repeated pre-adapter
recovery loop
## Linked Issues or Issue Description
This PR has no existing public issue. It addresses a workspace setup
bug.
**What happened?**
A remote-only base ref could fail before `git worktree add`. A setup
failure before adapter dispatch could also queue an agent-only
missing-comment retry.
**Expected behavior**
Paperclip must resolve `fix/foo` and `origin/fix/foo` before it creates
a worktree. An unresolved ref must create a human-owned configuration
blocker. Paperclip must not queue an agent-only retry when the adapter
never starts.
**Steps to reproduce**
1. Configure an execution workspace with a base ref that exists only on
the remote.
2. Start a run that creates a fresh worktree.
3. Repeat the run with the same unresolved ref.
4. Observe one configuration blocker and no repeated agent-only recovery
action.
**Paperclip version or commit**
`7664e323189bc219d8cbe00433b2e82b682b0504`
**Deployment mode**
Built from source with `pnpm dev`.
**Agent adapter(s) involved**
Not adapter-specific. The failure occurs before adapter dispatch.
**Database mode**
Not database-related.
**Access context**
Both board and agent execution paths can use execution workspaces.
Related public pull request: `Refs #11123`.
## What Changed
- Resolve remote-only base refs with the authenticated fetch helper
before `git worktree add`.
- Support both unqualified refs and remote-tracking refs.
- Raise a `configuration_incomplete` blocker when the requested ref
remains unresolved.
- Suppress missing-comment retries when setup fails before adapter
dispatch.
- Add the requested ref to the recovery fingerprint to bound identical
recovery actions.
- Add focused tests and update the execution semantics document.
## Verification
- `tsc --noEmit` passed for the changed server code.
- Focused Vitest suites passed, including four base-ref tests,
fingerprint deduplication, and pre-adapter retry suppression.
- GitHub Actions must run the full pull request gate.
## Risks
Low risk. The change affects workspace setup before adapter dispatch.
Existing worktree reuse remains unchanged. An unresolved ref now creates
a clear configuration blocker instead of starting an adapter run.
## Model Used
OpenAI GPT-5; exact model ID `gpt-5`; agentic tool use and repository
review.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Managed worktree services run isolated Paperclip instances with
cloned databases.
> - A reachable service was reported as ready even when its database,
runtime identity, or login path was not usable.
> - The first candidate added verified database seeding and managed
repair in #11665.
> - This pull request consolidates that candidate with signed login
handoff and a complete readiness contract.
> - Post-QA fixes close five defects in repair identity, repair
responses, UI retry, seed journal handling, and seed-source trust.
> - The benefit is a workspace that either opens safely or reports one
accurate recovery action.
## Linked Issues or Issue Description
No public GitHub issue exists for this work, so the problem is described
here.
**What happened**
Managed workspace URLs could return HTTP 200 and report ready while
login failed. QA also found cases where repair used the wrong instance
identity, returned a generic error, left the UI stuck, rejected a safe
journal lag, or trusted a mutable workspace manifest.
**Expected behavior**
Opening a ready workspace signs the board user in to the correct
isolated instance. Provisioning and repair use a registered source and
report a structured recovery state.
**Actual behavior**
Entry depended on a password copied into the clone. Several failure
paths could publish stale readiness, hide the repair precondition, or
trust state that the workspace could modify.
**Additional context**
This pull request includes the commits first published in #11665. That
pull request keeps the original base head for review history. This
consolidated pull request is the merge candidate. Related open readiness
work includes #11575 and #11621.
## What Changed
- Adds a short-lived, signed, single-use login ticket. It binds the
user, workspace, instance, and runtime origin.
- Exchanges the ticket through Better Auth. It creates the session and
cookie through the supported adapter path.
- Adds protected workspace readiness fields for the database, clone
data, login handoff, seed phase, and runtime identity.
- Fails readiness closed when the guest has no company or
execution-workspace binding.
- Binds ticket issuance to the exact cloned user and active company
membership selected for the handoff.
- Verifies every current active board identity through the exact-user
handoff before publication or reuse.
- Gates managed runtime publication on the readiness contract and the
recorded worktree instance identity.
- Refreshes runtime work products from the live runtime row after a port
change.
- Adds one workspace access card with ready, degraded, repairing, and
failed states.
- Uses the runtime response identity for repair. It returns structured
repair precondition errors.
- Lets a valid source journal lag converge during provisioning.
- Binds seed and repair manifests to a source registered outside the
agent-writable worktree.
- Clears recovered UI errors so a successful retry can open the
workspace.
- Makes runtime tests register canonical sources and avoid ports owned
by live host listeners.
- Keeps Vitest on source suites when compiled `dist` trees exist.
- Isolates CLI and adapter tests from ambient AWS and runtime API
environment variables.
- Preserves a 404 response for cross-company workspace ID lookups before
runtime authorization.
- Makes concurrent single-flight coverage independent of
path-canonicalization scheduling order.
## Verification
The following checks passed on the integrated head:
```sh
pnpm -r typecheck
pnpm build
pnpm check:token-gates
pnpm --filter @paperclipai/db check:migrations
```
- The server source lane passed 420 files and 4,953 tests. Five tests
were skipped.
- The CLI lane passed 57 files and 385 tests.
- The database lane passed 26 files and 97 tests.
- The shared package passed 58 files and 506 tests.
- The adapter utility lane passed 640 tests. Four tests were skipped.
- The Claude adapter passed 220 tests. One test was skipped.
- The Codex adapter passed 323 tests.
- The OpenClaw adapter passed 13 tests.
- The OpenCode adapter passed 42 tests.
- The plugin SDK passed 45 tests.
- The workspace runtime suite passed 124 tests.
- The caller-scoped readiness and handoff suite passed 52 tests.
- The workspace provisioning shell suite passed 7 tests.
- The runtime exposure suite passed 17 tests while live host mappings
occupied fixed test ports.
- `git diff --check` passed and the worktree is clean.
The serialized route lane will run in GitHub CI with its normal shards.
No deployment or active-workspace migration was performed.
## Risks
- This is a medium-risk authentication and runtime-readiness change.
- The login ticket uses exact origin, workspace, instance, and user
binding. It has a short expiry and a one-time nonce.
- Runtime publication is stricter. A real readiness, identity, per-user
handoff, or control-plane database disagreement now blocks publication.
- This pull request supersedes #11665 as the merge candidate. Close
#11665 after this pull request merges.
- No new database migration is included. The lockfile and workflow files
are unchanged.
- Deployment and active-workspace migration are intentionally outside
this pull request.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
Claude Opus 5 (`claude-opus-5[1m]`), 1M context, extended thinking, tool
use, and code execution produced the main candidate. OpenAI GPT-5
(`gpt-5`) through Codex, with agentic reasoning, tool use, and code
execution, integrated the post-QA fixes and hardened the test gates. The
Codex context-window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is an open-source AI agent management platform; its test
suite spans a `server` package that mounts real embedded Postgres
databases in `beforeAll`/`afterAll` hooks
> - The `server` package CI shard runs all ~93 suites serially
(`maxWorkers=1`) on a loaded CI host; each suite boots and tears down
its own embedded Postgres in hook callbacks
> - vitest's default `hookTimeout` is 10 seconds; under load, graceful
embedded-Postgres shutdown occasionally crosses that threshold
> - This produces intermittent `Error: Hook timed out in 10000ms`
failures in `afterAll` hooks — not test assertion failures — and the
suites pass on re-run, making them textbook flaky tests
> - Inspecting `embedded-postgres@18.1.0-beta.16` shows that `stop()`
takes no argument (no fast-shutdown mode), SIGINTs postgres (already
PostgreSQL "fast shutdown"), and resolves only on the child's `exit`
event with no internal time bound
> - Two targeted fixes: (1) raise `hookTimeout` and `teardownTimeout` to
30 s in `server/vitest.config.ts` — one config change that eliminates
the flake for all ~93 suites at once; (2) wrap `stop()` in a 5 s bounded
`Promise.race` in the test helper so a slow shutdown can never hang the
hook regardless of OS scheduling variance
> - This PR changes only test-infra and test-config; no production-code
behavior changes
## Linked Issues or Issue Description
No public GitHub issue exists for this flake. Inline bug description
(bug report template):
**What happened?**
The `General tests (server (N/3))` CI shards intermittently fail with
`Error: Hook timed out in 10000ms` in `afterAll` hooks and pass on
re-run. Every test assertion passes; only the teardown hook exceeds
vitest's default timeout.
**Expected behavior**
CI passes reliably. Teardown timeouts should not be a source of flake.
**Steps to reproduce**
Run the server test suite repeatedly on a loaded host or in CI with
`maxWorkers=1` — the shard occasionally crosses 10 s in `afterAll`
during embedded-Postgres shutdown.
**Paperclip version**
`master`, any build that includes `server/vitest.config.ts` without an
explicit `hookTimeout`.
**Deployment mode**
Self-hosted (CI).
## What Changed
- **`server/vitest.config.ts`** — added `hookTimeout: 30000` and
`teardownTimeout: 30000`. Removes flake across all ~93 server suites at
once. 30 s gives generous headroom over observed worst-case teardown
while still catching a genuinely hung hook.
- **`packages/db/src/test-embedded-postgres.ts`** — added
`stopEmbeddedPostgresBounded()`, a 5 s `Promise.race` wrapper around
`stop()`. Applied at all three call sites inside `cleanup()`. Data dir
is still removed unconditionally; errors are still swallowed; the
null-instance guard is preserved. Existing behavior unchanged except the
shutdown can no longer block indefinitely.
## Verification
- `tsc --noEmit` clean on `packages/db` (built against worktree-local
`shared`)
- `packages/db` `client.test.ts` passes 14/14 — boots embedded Postgres
and exercises the bounded teardown via `cleanup()` in `afterEach`
- Standalone bounded-race semantics verified: hang resolves at the 5 s
bound; late or immediate `stop()` rejection swallowed; no unhandled
rejection; null-instance path safe
- CI: all 3 server shards + split-verify lane (Async-Verification Gate)
expected green after this PR
```bash
# Reproduce the teardown test locally:
cd packages/db && npx vitest run src/client.test.ts
# Type-check packages/db:
npx tsc --noEmit -p packages/db/tsconfig.json
```
## Risks
Low risk. No product-code changes — test-infra and test-config only. The
vitest timeout increase is additive (raises the ceiling; never lowers
it). The bounded race wrapper preserves prior teardown behavior exactly:
data dir always removed, errors always swallowed, stop is still
attempted. A worst-case outcome is that a genuinely hung `stop()` now
surfaces as a test timeout at 30 s instead of 10 s — still caught, just
later.
## Model Used
- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200 k tokens
- **Capabilities:** tool use, code execution, extended reasoning
- **Mode:** Paperclip agent heartbeat (autonomous execution with human
board oversight)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Test infrastructure across server, ui, packages/* runs on Vitest
> - Dependabot opened a narrow bump (3.2.4 → 3.2.6), but the wider
workspace is on 3.2.4 and the major-version bridge to v4 needs a
coordinated change set across configs and tests
> - Staying on 3.x indefinitely leaves us behind on Vitest 4 (perf,
pool, and config improvements) and forces repeated patch-only dependabot
churn
> - This pull request upgrades Vitest to 4.1.8 across the workspace,
updates `server/vitest.config.ts` and `scripts/run-vitest-stable.mjs`
for the new API, and adjusts two UI tests for the new assertion
semantics
> - The benefit is a single, coherent Vitest 4 upgrade that supersedes
#7570 and gets us on the supported major line
## What Changed
- Bump `vitest` from `3.2.4` to `4.1.8` across root, `server`, `ui`, and
all `packages/*` (including plugin examples and sandbox providers)
- Update `server/vitest.config.ts` for Vitest 4 config surface
- Update `scripts/run-vitest-stable.mjs` to match the new runner
behavior
- Adjust `ui/src/components/CommentThread.test.tsx` and
`ui/src/components/MarkdownEditor.test.tsx` for Vitest 4 matcher/timing
semantics
- Refresh `pnpm-lock.yaml`
## Verification
- `pnpm install` resolves cleanly with the new lockfile
- `pnpm -w -r test` (server, ui, packages) runs under Vitest 4.1.8
## Risks
- Major-version Vitest bump: behavioral changes in pools, fake timers,
and matcher strictness can surface flake. Test config and the two UI
tests were updated to match v4 semantics; broader test runs should be
watched on CI before merge.
- Supersedes dependabot PR #7570 (3.2.4 → 3.2.6); that PR should be
closed.
## Model Used
- Claude (Anthropic) — `claude-opus-4-7`, extended thinking, tool use
enabled
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
Closes#7570
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - The server route suite is a core confidence layer for auth, issue
context, and workspace runtime behavior
> - Some route tests were doing extra module/server isolation work that
made local runs slower and more fragile
> - The stable Vitest runner also needs to pass server-relative exclude
paths to avoid accidentally re-including serialized suites
> - This pull request tightens route test isolation and runner
serialization behavior
> - The benefit is more reliable targeted and stable-route test
execution without product behavior changes
## What Changed
- Updated `run-vitest-stable.mjs` to exclude serialized server tests
using server-relative paths.
- Forced the server Vitest config to use a single worker in addition to
isolated forks.
- Simplified agent permission route tests to create per-request test
servers without shared server lifecycle state.
- Stabilized issue goal context route mocks by using static mocked
services and a sequential suite.
- Re-registered workspace runtime route mocks before cache-busted route
imports.
## Verification
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/agent-permissions-routes.test.ts
server/src/__tests__/issues-goal-context-routes.test.ts
server/src/__tests__/workspace-runtime-routes-authz.test.ts --pool=forks
--poolOptions.forks.isolate=true`
- `node --check scripts/run-vitest-stable.mjs`
## Risks
- Low risk. This is test infrastructure only.
- The stable runner path fix changes which tests are excluded from the
non-serialized server batch, matching the server project root that
Vitest applies internally.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, GPT-5 coding agent, tool-enabled with
shell/GitHub/Paperclip API access. Context window was not reported by
the runtime.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - A fast-moving control plane needs stable local tests and repeatable
local maintenance tools so contributors can safely split and review work
> - Several route suites needed stronger isolation, Codex manual model
selection needed a faster-mode option, and local browser cleanup missed
Playwright's headless shell binary
> - Storybook static output also needed to be preserved as a generated
review artifact from the working branch
> - This pull request groups the test/local-dev maintenance pieces so
they can be reviewed separately from product runtime changes
> - The benefit is more predictable contributor verification and cleaner
local maintenance without mixing these changes into feature PRs
## What Changed
- Added stable Vitest runner support and serialized route/authz test
isolation.
- Fixed workspace runtime authz route mocks and stabilized
Claude/company-import related assertions.
- Allowed Codex fast mode for manually selected models.
- Broadened the agent browser cleanup script to detect
`chrome-headless-shell` as well as Chrome for Testing.
- Preserved generated Storybook static output from the source branch.
## Verification
- `pnpm exec vitest run
src/__tests__/workspace-runtime-routes-authz.test.ts
src/__tests__/claude-local-execute.test.ts --config vitest.config.ts`
from `server/` passed: 2 files, 19 tests.
- `pnpm exec vitest run src/server/codex-args.test.ts --config
vitest.config.ts` from `packages/adapters/codex-local/` passed: 1 file,
3 tests.
- `bash -n scripts/kill-agent-browsers.sh &&
scripts/kill-agent-browsers.sh --dry` passed; dry-run detected
`chrome-headless-shell` processes without killing them.
- `test -f ui/storybook-static/index.html && test -f
ui/storybook-static/assets/forms-editors.stories-Dry7qwx2.js` passed.
- `git diff --check public-gh/master..pap-2228-test-local-maintenance --
. ':(exclude)ui/storybook-static'` passed.
- `pnpm exec vitest run
cli/src/__tests__/company-import-export-e2e.test.ts --config
cli/vitest.config.ts` did not complete in the isolated split worktree
because `paperclipai run` exited during build prep with `TS2688: Cannot
find type definition file for 'react'`; this appears to be caused by the
worktree dependency symlink setup, not the code under test.
- Confirmed this PR does not include `pnpm-lock.yaml`.
## Risks
- Medium risk: the stable Vitest runner changes how route/authz tests
are scheduled.
- Generated `ui/storybook-static` files are large and contain minified
third-party output; `git diff --check` reports whitespace inside those
generated assets, so reviewers may choose to drop or regenerate that
artifact before merge.
- No database migrations.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex coding agent based on GPT-5, with shell, git, Paperclip
API, and GitHub CLI tool use in the local Paperclip workspace.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
Note: screenshot checklist item is not applicable to source UI behavior;
the included Storybook static output is generated artifact preservation
from the source branch.
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Express server with CRUD routes for agents, goals, issues, projects,
and activity log. Includes validation middleware, structured error
handling, request logging, and health check endpoint with tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Nicky Leach
Dotta
Devin Foley
Forgotten