Commit Graph

248 Commits

Author SHA1 Message Date
Dotta d2665ff6b4
fix(ui): align the mobile task chat composer with the thread (#11296)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The task thread is where people read work and guide agents.
> - The mobile composer should use the same content width as the thread.
> - The composer kept the desktop 80% width on mobile, so its edges did
not align with the thread.
> - Long assignee-aware placeholder text could also clip inside the
mobile editor.
> - Some extracted style tokens used legacy HSL wrappers around complete
semantic colors, which made those declarations invalid.
> - This pull request makes the composer full width on mobile, preserves
the narrower desktop layout, wraps the placeholder, and repairs the
invalid color compositions.
> - The benefit is a stable mobile composer that aligns with the task
thread and keeps its intended visual styles.

## Linked Issues or Issue Description

Related work: Refs #11263.

**What happened?**

At mobile widths, the task chat composer used the same 80% width as the
desktop composer. Its horizontal edges did not align with the full task
thread. A long assignee-aware placeholder could clip on one line. The
composer's extracted shadow also used a legacy `hsl(var(...))` wrapper
around complete semantic color values, so the browser could reject the
declaration.

**Expected behavior**

The composer must match the task thread width on mobile. It must stay
narrower on larger screens. Long placeholder text must wrap inside the
editor. Semantic color tokens must form valid shadows and gradients.

**Steps to reproduce**

1. Open a task with the chat-style thread on a mobile viewport.
2. Compare the composer edges with the task thread edges.
3. Select an assignee whose placeholder text wraps to two lines.
4. Inspect the computed composer shadow and the extracted semantic color
styles.

**Paperclip version or commit**

The change is based on `dc6fcd1ff1` from `master`.

**Deployment mode**

Local build from source. The behavior also applies to packaged web
builds.

## What Changed

- Made the task chat composer full width below the medium breakpoint and
kept the 80% desktop width.
- Matched the composer dock padding to the task thread padding.
- Allowed long composer placeholders to wrap and reserved enough mobile
editor height for two lines.
- Replaced invalid legacy HSL wrappers around full semantic colors in
extracted shadows, gradients, and approval styles.
- Added a token gate that prevents legacy `hsl(var(--token))` wrappers
from returning.
- Added focused regression tests for responsive width, padding,
placeholder wrapping, mobile height, and semantic shadow validity.

## Verification

- `pnpm check:token-gates` — all four gates pass.
- `pnpm --filter @paperclipai/ui exec vitest run
src/components/TaskChatThread.test.tsx
src/components/task-chat/TaskChatComposer.test.tsx
src/components/task-chat/TaskChatComposerStyles.test.ts` — 37 tests
pass.
- `pnpm --filter @paperclipai/ui typecheck` — passes.
- `pnpm --filter @paperclipai/ui build` — passes. The build prints
existing CSS optimizer and bundle-size warnings.

## Risks

- Low risk. The width change is limited to the mobile breakpoint. The
desktop 80% layout remains in place.
- The semantic token fixes can affect shadows and gradients that were
previously invalid. The new gate prevents the invalid wrapper pattern
from returning.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with `gpt-5.6-sol`. The context-window size is not
exposed in this environment. The model used reasoning, repository tools,
code execution, and GitHub tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-14 12:53:11 -04:00
dmndbrp-oss a8d118a779
Prefer public base URL for generated invite links (#7619)
Fixes #7623

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Company invites are part of the access subsystem and must produce
URLs that recipients can open from outside the host machine.
> - Paperclip already has public/auth base URL configuration for
deployments behind a public hostname, Tailscale, or a reverse proxy.
> - Invite URL composition was still deriving its origin from the
incoming request host, so loopback-bound servers emitted
`http://127.0.0.1:3100/invite/...`.
> - A loopback invite URL is not shareable with a remote human or agent,
even when the token itself is valid.
> - This pull request makes invite URL builders prefer the configured
public base URL and keep the existing request-host fallback when it is
unset.
> - The benefit is that copied invite links use the reachable deployment
origin without changing local-only behavior.

## Linked Issues or Issue Description

Fixes #7623

No duplicate or related PRs/issues were found in a GitHub search for
invite URL, loopback, public base URL, and `authPublicBaseUrl` terms.

## What Changed

- Added base URL resolution in `server/src/routes/access.ts` that strips
trailing slashes and prefers configured `authPublicBaseUrl` over the
request-derived host.
- Threaded `authPublicBaseUrl` through invite summary, invite onboarding
manifest, onboarding text, access routes, `createApp`, and server
startup wiring.
- Added `server/src/__tests__/invite-url-public-base-url.test.ts`
covering configured public-base precedence, unset fallback behavior, and
trailing-slash normalization.
- Registered the invite public-base URL test in the serialized Vitest
server runner.

## Verification

```bash
pnpm install --frozen-lockfile
pnpm exec vitest run server/src/__tests__/invite-url-public-base-url.test.ts
pnpm run test:run:serialized
```

Local results from the rebased PR branch:

- `pnpm install --frozen-lockfile` exited 0.
- Targeted invite URL test exited 0: 1 file, 3 tests passed.
- Serialized server suite exited 0: 106 serialized suites completed; the
new invite URL test passed inside that runner.

Manual check after deployment: set `PAPERCLIP_AUTH_PUBLIC_BASE_URL` or
equivalent public base URL config, create a company invite, and confirm
the returned/copied invite URL uses that public origin instead of
`127.0.0.1`.

## Risks

Low risk. The new public base URL parameter is optional and falls back
to existing request-derived behavior when unset. The main operational
risk is misconfigured public base URL input; the implementation only
trims trailing slashes and otherwise trusts the configured origin.

## Model Used

- Original implementation: Anthropic `claude-sonnet-4-6`, 200k context,
tool use and test execution.
- Conflict repair and verification: OpenAI Codex GPT-5.5, coding agent
with shell, git, GitHub CLI, and local test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Coder (Claude) <coder-claude@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Paperclip Coder (Claude) <lad-agent@paperclip.ing>
2026-08-12 16:44:28 -07:00
Devin Foley 71e9d6bb0b
feat(release): candidate-branch beta builds and the release checklist (#11209)
> Follow-up to #11208 (merged): rebased onto master and ready for
review.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release channels promote artifacts along canary → nightly → beta
→ stable, with the happy path being promotion of an existing build
> - When one or two targeted fixes are needed before a beta or stable,
the only options today are waiting for the next nightly or absorbing a
whole day of unrelated master changes
> - The channel model was designed with an escape hatch for exactly
this: short-lived candidate branches carrying only cherry-picked fixes
> - This pull request implements candidate-branch beta builds with full
verification, documents the stable fix path through the
soak-justification gate, and adds the release captain's checklist
> - The benefit is that a surgical fix can ship forward without either
delay or blast radius, with its provenance recorded

## Linked Issues or Issue Description

Refs #11008 — completes the fix-path half of the channel model
introduced there.

**Subsystem affected**

Release automation: `scripts/release.sh`,
`.github/workflows/release.yml`, `doc/RELEASING.md`, new
`doc/RELEASE-CHECKLIST.md`, tests.

**Problem or motivation**

Beta promotion only accepts commits that already shipped as a nightly,
and stable promotion expects a soaked beta. There is no supported way to
ship one or two cherry-picked fixes between lanes: an urgent fix must
wait for the nightly cycle or pull in every unrelated master change from
the day. The original channel design called for candidate branches to
cover this, and they were deferred from the initial implementation.

**Proposed solution**

Candidate-branch beta builds: cut `candidate/beta-<target>` from a
nightly's source commit, cherry-pick the fixes, and dispatch `channel:
beta` with the new `candidate_branch` input. Selection enforces the
naming convention, rejects heads that already shipped as a beta or
predate the candidate tooling, and records the cherry-picked commits in
the job summary. Because candidate heads never went through a canary or
nightly, publication is gated on a full `release-verify` run (promoted
nightlies keep skipping re-verification). The stable fix path
(`candidate/release-<target>` as `source_ref`) works through the
existing soak gate: the justification requirement is the deliberate,
recorded trade-off for shipping unsoaked bits, and is now documented as
such.

## What Changed

- `scripts/release.sh`: `--from-candidate` flag (beta only) waives the
shipped-a-nightly requirement while keeping the duplicate-beta guard
- `.github/workflows/release.yml`: `candidate_branch` dispatch input;
candidate mode in `select_beta` (naming validation, duplicate and
tooling-era rejection, cherry-pick recording); new
`verify_beta_candidate` job gating candidate publishes on full
verification
- `doc/RELEASING.md`: beta fix-path and stable fix-path sections
- `doc/RELEASE-CHECKLIST.md` (new): the release captain's checklist for
all four lanes as built
- Tests: dry-run fixture coverage for `--from-candidate` (waives the
nightly guard, keeps the duplicate guard, rejected outside beta) and
wiring tests for candidate validation plus the verification gate

## Verification

- `node --test` on the four affected suites: 42 pass in total (17 + 25
across the two runs), including the 5 new tests
- `bash -n` on `release.sh`; YAML parse of the workflow
- After merge: exercise the path end to end the first time a real
cherry-picked beta is needed — dispatch with a `candidate/beta-*` branch
and confirm the summary records the picks and verification runs

## Risks

- Candidate builds bypass the smoke-tested-nightly provenance by design;
the compensating controls are full verification before publish, the
post-publish beta smoke, the human `npm-beta` gate, and recorded
cherry-picks
- The stable fix path rides the existing justification mechanism rather
than adding a second bypass — one recorded escape hatch, not two

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-11 10:05:13 -07:00
Devin Foley 2da6a248c3
fix(release): surface recovery commands when a lane tag push is rejected (#11208)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's promotion lanes publish to npm, then push a
lane tag and dispatch the Docker image build at that tag
> - The first nightly of the beta-tooling merge published to npm and
then died at the tag push: GITHUB_TOKEN may not create refs pointing at
workflow-modifying commits from dispatch or scheduled runs
> - The failure was a bare `remote rejected` with no guidance, leaving
the release half-finished (npm live, no tag, no images) until an
operator reverse-engineered the recovery
> - This pull request makes every lane's tag push degrade into exact
recovery instructions in the job summary
> - The benefit is that a rare platform-permission rejection becomes a
two-minute runbook operation instead of a forensic exercise

## Linked Issues or Issue Description

Refs #11008 — the incident occurred promoting that change's own merge
commit, the first workflow-modifying commit to flow through the lanes it
introduced.

**Subsystem affected**

Release automation: `.github/workflows/release.yml`, `doc/RELEASING.md`,
workflow wiring tests.

**Problem or motivation**

Run 31445344811 published `2026.811.0-nightly.0` to npm, then failed
pushing `nightly/v2026.811.0-nightly.0`: `refusing to allow a GitHub App
to create or update workflow .github/workflows/release.yml without
workflows permission`. The tagged commit modifies workflow files, and
GITHUB_TOKEN may not create refs pointing at such commits from dispatch
or scheduled runs (push-event runs are exempt, which is why the canary
tag on the same commit succeeded). The job failed with no explanation
and the Docker dispatch never ran.

**Proposed solution**

Wrap the nightly, beta, and stable tag pushes: on rejection, write the
exact recovery commands into the job summary — create and push the tag
with maintainer credentials, dispatch `docker.yml` at the tag, and for
stable also run `create-github-release.sh` — then fail the job. Document
the cause and recovery in the failure playbooks and pin the three
recovery blocks with a wiring test.

## What Changed

- `.github/workflows/release.yml`: recovery-summary wrappers on the
nightly, beta, and stable tag-push steps
- `doc/RELEASING.md`: failure-playbook entry for the
workflows-permission rejection
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
asserting all three lanes carry the recovery summary

## Verification

- Wiring tests: 6 pass; YAML parse of the workflow
- The recovery commands are exactly the ones used to resolve the real
incident (tag push + `docker.yml` dispatch for
`nightly/v2026.811.0-nightly.0`)

## Risks

- Low. The happy path is unchanged (a successful push skips the
wrapper); the failure path trades a bare error for actionable output and
still fails the job, since the release state is genuinely incomplete

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-11 08:48:13 -07:00
Devin Foley d648becb90
refactor(ci): split workspaces-a into two Vitest native shards
Split the slow workspaces-a CI lane into two Vitest native shards and keep release verification in parity.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-08-10 17:32:44 -07:00
Devin Foley 6601014898
fix(release): reject promotion sources that predate their channel tooling (#11197)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem promotes builds along canary → nightly → beta
→ stable, and each publish job checks out the promotion's source commit
and runs that tree's release tooling
> - The first beta dispatch failed with `unexpected argument: beta`: the
selected nightly's source predated the beta channel, so its `release.sh`
did not know the argument
> - The failure was clean (argument parsing, nothing published) but
cryptic, and the same trap waits for any promotion of a source older
than its target channel's tooling
> - This pull request makes the selection jobs reject such sources with
an actionable error and documents the property
> - The benefit is that a bootstrapping or old-source promotion fails in
seconds with instructions, instead of mid-publish with a parser error

## Linked Issues or Issue Description

Refs #11008 — the guard hardens the beta promotion flow introduced
there, after its first dispatch surfaced the gap described below.

**Subsystem affected**

Release automation: `.github/workflows/release.yml`, `doc/RELEASING.md`,
workflow wiring tests.

**Problem or motivation**

Run 31444045044 (first beta dispatch) failed in `publish_beta` with
`unexpected argument: beta`. Promotions deliberately build from the
pinned source commit, which means they also run that commit's
`scripts/release.sh` — and a source that predates the target channel's
introduction cannot publish it. Nothing guards this today; the error
surfaces deep in the publish job with no explanation.

**Proposed solution**

Guard at selection time: `select_nightly` requires the source canary's
`release.sh` to know the nightly channel, and `select_beta` requires the
source nightly's `release.sh` to know the beta channel. Each guard
literally matches the channel case arm and fails closed with a clear
message naming the remedy (promote a newer source). Document the
tooling-era property in `RELEASING.md` and pin the guards with a wiring
test.

## What Changed

- `.github/workflows/release.yml`: tooling-era guards in
`select_nightly` and `select_beta`
- `doc/RELEASING.md`: documents that promotions run the source commit's
release tooling
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
pinning both guards

## Verification

- Wiring tests: 5 pass
- Guard expressions exercised against real commits: accepts the
beta-capable merge commit of the beta-channel change, rejects a pre-beta
commit
- YAML parse of the workflow
- After merge: the next beta dispatch selects a beta-capable nightly and
passes the guard

## Risks

- Low. Selection-time check only; the guards match the channel case arm
literally and fail closed (with the same actionable message) if that
line is ever reformatted

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. All changes model-authored under
human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 17:22:54 -07:00
Devin Foley 8f7b8b3fda
feat(release): add human-gated beta channel with stable soak enforcement (#11008)
> Follow-up to #11006 (merged): rebased onto master and ready for
review.

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem now publishes canary (every master push),
nightly (scheduled, smoke-gated, added in #11006), and stable (manual)
> - There is still no human-approved release-candidate lane between
nightly and stable, and nothing enforces that a stable actually soaked
anywhere before shipping
> - Betas need a real approval gate, and stables need a soak policy that
is data, not prose
> - This pull request adds the beta channel: a manual promotion of a
chosen nightly behind the `npm-beta` environment gate, re-smoked after
publish, plus a stable preflight that enforces a 3-day beta soak with a
written-justification bypass
> - The benefit is a complete canary → nightly → beta → stable train
where every stable shipped as a beta first, and emergencies leave a
written trace

## Linked Issues or Issue Description

**Subsystem affected**

Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

After #11006 the project has canary and nightly prerelease lanes, but no
release-candidate lane. Stable promotion has no enforced soak: any ref
can ship as stable directly. There is no approval boundary for a
broader-audience prerelease, and no structured way to record why an
emergency release skipped validation.

**Proposed solution**

Add a `beta` channel: a manual dispatch that promotes a chosen nightly's
source commit, publishes behind the `npm-beta` GitHub environment
(required reviewers are the gate), re-smokes the published beta, and
tags `beta/vX`. Enforce in the stable path that the source commit
shipped as a beta at least 3 days earlier (measured from the beta's npm
publish time), with a `skip_soak_justification` input as the recorded
emergency bypass.

**Alternatives considered**

Codifying the soak policy in docs only. Rejected: an unenforced policy
decays; the preflight makes the policy executable while the
justification input keeps the emergency path usable and auditable.

## What Changed

- `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel —
requires HEAD to carry a `nightly/v*` tag, publishes the package set as
`YYYY.MDD.P-beta.N` under dist-tag `beta`, tags
`beta/vYYYY.MDD.P-beta.N`
- `.github/workflows/release.yml`:
- `channel: beta` dispatch path: `select_beta` resolves the newest (or
an explicit `source_version`) nightly and fails loudly on selection
problems; `publish_beta` runs behind the `npm-beta` environment, pushes
the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release
smoke suite against the exact published beta version
- stable path: new `preflight_stable` job enforces the 3-day beta soak
from the beta's npm publish time; `skip_soak_justification` bypasses
with the reason echoed into the job summary; dry runs report without
blocking
- `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both
images, with exact version stamping
- `.github/workflows/release-smoke.yml`: `beta` added to the dispatch
choice list
- Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate,
and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta`
environment setup, including the warning to create the environment
before the first beta dispatch (GitHub auto-creates unprotected
environments on first reference)
- Tests: beta version-counting coverage in
`scripts/release-registry-versions.test.mjs`; beta identity and
nightly-tag guard coverage in
`scripts/__tests__/release-dry-run-notes.test.mjs`

## Verification

- `node --test` on the two touched suites: 17 pass, including the 3 new
beta tests
- `bash -n` on both shell scripts and YAML parse of all three workflows
- After merge, in order: create the `npm-beta` environment, dispatch
`channel: beta` with `dry_run: true` to preview, then a real promotion
of a published nightly through the approval gate, then a stable dry-run
against a young beta to see the soak gate report

## Risks

- If the `npm-beta` environment does not exist when the first beta
dispatch runs, GitHub creates it with no protection rules and the beta
publishes without approval. Mitigated by documentation and by creating
the environment before merge (operator step)
- Until the first beta exists, every stable dispatch requires
`skip_soak_justification`. This is deliberate — the first beta ships
immediately after this merges — but it is a behavior change to the
stable dispatch
- The soak clock reads the beta's npm publish time from the registry; a
registry outage makes the preflight fall back to requiring justification
(fail-closed)

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 16:52:59 -07:00
Devin Foley f94f6003c6
fix(release-smoke): pin the smoke container to the lan bind preset (#11189)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's nightly lane gates every nightly on the
release smoke suite, which boots the published artifact in a Docker
container and drives real onboarding
> - The gate kept failing even after the readiness budget fix (#11187),
and the new container-log dump revealed the server was healthy but
listening on 127.0.0.1 inside the container, unreachable through
Docker's port mapping
> - `onboard --yes` without an explicit `--bind` prefers trusted-local
quickstart defaults: it writes a loopback bind into the instance config
and ignores the deployment env vars the harness passes, and that config
outranks `HOST` at runtime
> - This pull request pins the smoke container to the `lan` bind preset
and adds a wiring test for it
> - The benefit is a working nightly gate, verified end to end against a
real published canary

## Linked Issues or Issue Description

**Subsystem affected**

Release smoke testing: `docker/Dockerfile.onboard-smoke`,
`scripts/__tests__/release-verify-workflow.test.mjs`.

**Problem or motivation**

Nightly run 31428558684 failed in smoke with the server unreachable at
the mapped port for the full 420 second budget. The container logs
(captured thanks to #11187) show a fully booted server with `Bind
loopback (127.0.0.1)`. The harness sets `HOST=0.0.0.0` and the
deployment env vars, but `onboard --yes` without `--bind` deliberately
prefers trusted-local defaults, writes `bind: loopback` into the
instance config, and the config outranks `HOST` at runtime. A loopback
listener inside a container is invisible to the port mapping, so the
health check can never pass. This behavior predates the current stable,
so the harness was silently broken against every recent version — it
only surfaced now because the nightly lane is the suite's first CI
consumer.

**Proposed solution**

Pass `--bind lan` in the smoke container command (the flag is supported
by `latest` and canary alike; it selects the all-interfaces preset and
keeps the env-driven authenticated deployment), and pin the flag with a
wiring test so it cannot regress silently.

## What Changed

- `docker/Dockerfile.onboard-smoke`: the onboard command is now `onboard
--yes --bind lan --data-dir ...`, with a comment explaining why the flag
is load-bearing
- `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test
asserting the smoke Dockerfile pins a non-loopback bind preset

## Verification

- Full local harness run against the real nightly candidate
`2026.810.0-canary.1`: container healthy, bind banner shows `lan
(0.0.0.0)`, authenticated bootstrap completed (admin created, bootstrap
invite accepted, board session verified), `/api/health` returns
`bootstrapStatus: ready`
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`: 4
pass
- After merge: dispatch `release.yml` with `channel: nightly` to run the
gate end to end in CI

## Risks

- Low. The change only affects the smoke container. `--bind lan` inside
a container exposes the port to the container network only; reachability
from outside still goes through Docker's explicit port mapping

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (CI log forensics, upstream source
tracing, local Docker reproduction and verification). All changes
model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 13:51:49 -07:00
Devin Foley 30f6999cbe
fix(release-smoke): configurable readiness timeout and diagnostics for slow containers (#11187)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's nightly lane (#11006) gates every nightly
publish on the release smoke suite, which boots the published artifact
in a Docker container
> - The suite's first CI execution failed at the health readiness check:
the harness hard-codes a 90 second budget, but a CI container
cold-installs paperclipai from npm and initializes embedded postgres
with no warm caches
> - When the timeout expired with the container still running, the
harness printed no container logs, so the failure gave no diagnostics
> - This pull request makes the readiness budget configurable, raises it
for CI, and dumps container logs on timeout
> - The benefit is that the nightly gate measures the artifact, not the
runner's cold caches, and a red smoke run is diagnosable from its logs

## Linked Issues or Issue Description

**Subsystem affected**

Release smoke testing: `scripts/docker-onboard-smoke.sh`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

Run 31426044332 (first forced nightly after #11006) failed in
`smoke_nightly` with `server did not become ready at
http://localhost:3232/api/health` after exactly 90 seconds. The
harness's readiness window is hard-coded to 90 attempts at 1 second.
Locally that works because the npm cache is warm; in CI the container
downloads the full package set and embedded postgres first. The timeout
path also printed no container logs when the container was still
running, so there was no way to see how far boot had progressed.

**Proposed solution**

Make the readiness budget an environment variable
(`SMOKE_READY_TIMEOUT_SECONDS`, default unchanged at 90 for local use),
set it to 420 in the CI workflow, and dump the last 150 container log
lines when the readiness check times out on a still-running container.

## What Changed

- `scripts/docker-onboard-smoke.sh`: `SMOKE_READY_TIMEOUT_SECONDS` env
var (default 90) replaces the hard-coded readiness budget; timeout with
a still-running container now prints the tail of `docker logs`
- `.github/workflows/release-smoke.yml`: sets
`SMOKE_READY_TIMEOUT_SECONDS=420` for CI runs

## Verification

- `bash -n` on the harness and YAML parse of the workflow
- The real proof is the next `channel: nightly` dispatch of
`release.yml`, which re-runs this suite in CI with the new budget

## Risks

- Low. The local default is unchanged; CI runs simply wait longer before
declaring failure, and a genuinely broken artifact still fails (with
logs now)

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. Diagnosis from CI run logs; patch
model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 13:19:33 -07:00
Devin Foley f9173782cd
feat(release): add smoke-gated nightly channel and lane-separated Docker tags (#11006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem publishes the `paperclipai` npm package set
and the Docker images on two lanes: canary on every master push, and
stable on manual promotion
> - There is no middle ground between those lanes. Users must track
every merge or wait weeks for a stable. Docker `:latest` also tracks
master, so Docker users have no stable image at all
> - A calm prerelease lane needs to exist, and it must never ship a
build that failed its checks
> - This pull request adds the nightly channel: a scheduled job that
selects the newest master commit with a green canary publish, runs the
full release smoke suite against that exact published canary, and only
then republishes it as the nightly. It also separates Docker tags by
lane, so `:latest` finally means stable
> - The benefit is that users can follow prereleases at a nightly
cadence with a smoke-tested guarantee, and Docker users get real
`:canary`, `:nightly`, and stable image tags

## Linked Issues or Issue Description

**Subsystem affected**

Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

The project publishes only `canary` (every master push) and `latest`
(manual stable). Users who want prereleases without per-merge churn have
no option. Docker has a second problem: master builds overwrite
`:latest`, and CI-published stables never produced Docker images,
because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger
in `docker.yml`. No stable-versioned image exists in ghcr today.

**Proposed solution**

Add a `nightly` channel. A scheduled job selects the newest
canary-tagged master commit, smoke-tests that exact published canary,
and republishes the same commit as `YYYY.MDD.P-nightly.N` under the
`nightly` dist-tag. Separate Docker tags by lane (`:canary` for master,
`:nightly` for nightly tags, `:latest` plus version tags for stable tags
only), and have the release jobs dispatch `docker.yml` at the new tag so
lane images actually build.

**Alternatives considered**

Moving the `nightly` dist-tag to the existing canary version without a
republish. Rejected: the version string would say `canary` while the
user is on nightly, which breaks at-a-glance lane identification in bug
reports and `--version` output.

## What Changed

- `scripts/release-lib.sh`: channel-parameterized
`next_prerelease_version` and `prerelease_tag_name` helpers (canary
helpers delegate to them), a `require_channel_tag_at_head` guard, and
the no-provenance retry for Sigstore transparency-log duplicates now
covers the `nightly` dist-tag as well as `canary`
- `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry
a `canary/v*` tag, publishes the full public package set as
`YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source
commit `nightly/vYYYY.MDD.P-nightly.N`
- `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) —
select candidate, smoke it via `release-smoke.yml`, publish on green
under the existing `npm-canary` environment, push the tag, dispatch
`docker.yml`. New `channel` dispatch input (default `stable`, so
existing stable dispatches are unchanged) with `nightly_source_version`
and `dry_run` support for forced runs. The stable path now also
dispatches `docker.yml` at the new `v*` tag
- `.github/workflows/docker.yml`: lane tag mapping for both image jobs —
master pushes publish `:canary` and no longer move `:latest`;
`nightly/v*` tags publish `:nightly`; only stable `v*` tags publish
`:latest` and the versioned tags. New `workflow_dispatch` trigger for
the release-job dispatches. Build-version stamping uses the exact
nightly version on nightly tag builds
- `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch
choice list
- `doc/CHANNELS.md` (new): user-facing guide to the channels
- `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping
table, and a nightly failure playbook
- `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses
`npm-canary` and needs no npm trusted-publisher changes
- Tests: channel-parameterized version helper coverage in
`scripts/release-registry-versions.test.mjs`, and nightly flow coverage
(publish identity, notes not required, canary-tag guard) in
`scripts/__tests__/release-dry-run-notes.test.mjs`

## Verification

- `node --test` on the release script suites: 68 pass, including 6 new
tests. The only failure, `acpx-patch-packaging.test.mjs`, needs
installed `node_modules` and fails identically on a pristine checkout of
master in the same environment
- `bash -n` on both shell scripts and YAML parse of all three workflows
- Live fail-path check: `./scripts/release.sh nightly --print-version`
from a master tip with no canary tag fails with `HEAD has no canary/v*
tag`
- Live success-path check: the same command from the
`canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0`
- Live selection check: the candidate-selection shell logic run against
the real repository selects the commit of `canary/v2026.806.0-canary.7`,
which matches the current npm `canary` dist-tag exactly
- After merge: dispatch `release.yml` with `channel: nightly` and
`dry_run: true` to preview, then a real forced run to validate end to
end before the first scheduled run

## Risks

- Docker `:latest` changes meaning from "latest master build" to "latest
stable release". This is deliberate and will be announced. Users who
want the old behavior pull `:canary`. Until the first stable release
after this change, `:latest` stays at its current (master-built) image
- The nightly is a rebuild of the same source commit, not the
byte-identical canary artifact that was smoked. The lockfile pins
dependencies, and the publish path's registry-visibility and
clean-prefix install gates still run on the nightly artifacts
- All npm publishing must stay inside `release.yml` because npm trusted
publishing pins that workflow file per package. The nightly jobs were
added to `release.yml` for exactly that reason; this constraint is now
documented in `RELEASING.md`
- The stable-lane Docker dispatch fails gracefully (a warning with
manual instructions) when the source ref predates `docker.yml`'s
`workflow_dispatch` trigger

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 12:16:55 -07:00
Dotta 03cfad7ceb
feat(apps): connect Notion through MCP OAuth (#11009)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps give agents governed access to external tools.
> - The Apps gallery lists Notion, but the server required manually
configured OAuth credentials.
> - Notion's hosted MCP server supports OAuth discovery and dynamic
client registration.
> - Notion also requires HTTPS or a loopback HTTP redirect URI.
> - This pull request adds a direct Notion MCP OAuth path with PKCE and
reusable dynamic clients.
> - It also adds the current Apps UI states for connect and
reauthorization.
> - The benefit is a secure Notion connection with no manual client
credential setup.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The Apps gallery, Apps connect route, OAuth token lifecycle, and managed
MCP gateway.

**Subsystem affected**

`server/`, `packages/shared/`, `scripts/`, and `ui/`.

**Current behavior**

The Notion gallery cards are disabled. The server uses the classic
Notion OAuth endpoints and requires operator-supplied client
credentials. It does not register an OAuth client from provider
metadata. Concurrent refreshes can also replay a rotating refresh token.

**Proposed behavior**

Enable the Notion Apps flow. Discover OAuth metadata from
`https://mcp.notion.com/mcp`. Register and reuse a public RFC 7591
client with PKCE. Require HTTPS or loopback HTTP callbacks. Serialize
refreshes, store each rotated refresh token before the new access token
can be used, and show a reconnect state for `invalid_grant`.

**Reason and benefit**

Operators can connect the built-in Notion MCP app without creating or
copying OAuth credentials. Paperclip keeps dynamic clients and rotating
tokens in the company secret store.

**Breaking changes**

None. Explicit environment client credentials still take priority.
Existing Slack and Linear OAuth endpoint hints remain unchanged. Other
OAuth apps remain disabled unless they are allowlisted.

**Additional context**

PR #10910 is a related, broader Connections v3 wizard replacement. This
PR is the focused current Apps flow. The MCP Tool Gateway and Connected
Apps items in `ROADMAP.md` cover this planned capability.

## What Changed

- Classify all 20 reviewed Notion MCP tools with provider-scoped read
and write defaults.
- Require approval for selected Notion mutations, including move,
duplicate, and convert actions that generic verb matching missed.
- Preserve company-scoped connection and catalog resolution for Notion
profiles and policies.
- Add RFC 7591 dynamic client registration with
`token_endpoint_auth_method=none` and mandatory PKCE.
- Store the dynamic client ID on the connection and store any returned
client secret in the company secret store.
- Reuse the registered client for later connects and keep explicit
environment credentials as the first choice.
- Discover protected-resource and authorization-server metadata from the
Notion MCP endpoint.
- Add `redirectConstraints: "https-or-loopback-http"` to the generated
Notion app definition and shared contract.
- Reject non-loopback plain HTTP callbacks before network access with a
TLS setup error.
- Serialize client registration and token refresh operations within the
server process.
- Store a rotated refresh token before publishing the refreshed access
token.
- Treat `invalid_grant` as terminal and move the connection to a clear
reauthorization state.
- Add focused coverage for registration reuse, callback constraints,
refresh rotation, and terminal grants.
- Enable the Notion Apps route and add connect, redirect, success,
error, and reconnect UI states.
- Keep non-allowlisted OAuth apps blocked and cover the UI policy with
regression tests.

## Verification

- The focused Notion policy integration test passed with embedded
PostgreSQL.
- The focused 20-tool classification test passed.
- The server typecheck passed on the governance head.
- `pnpm -r typecheck` passed on the rebased head.
- `pnpm --filter @paperclipai/server typecheck` passed after the
security follow-up.
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
-t \u0027DCR|refresh tokens|invalid_grant|abandoned lease\u0027` passed
10 focused security tests.
- `pnpm build` passed on the rebased head.
- `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts
-t 'OAuth|oauth'` passed 14 tests.
- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts`
passed 5 tests.
- The complete server group passed 3,686 tests with 4 skipped.
- The complete UI group passed 3,656 tests.
- The full local runner found one environment-only CLI failure because
this agent runtime injects static AWS credentials into a test that
expects `AWS_PROFILE` only. `env -u AWS_ACCESS_KEY_ID -u
AWS_SECRET_ACCESS_KEY pnpm exec vitest run
cli/src/__tests__/secrets.test.ts` passed all 8 tests.
- The prior UI verification passed 55 focused tests, `pnpm
check:token-gates`, the Storybook build, and review of six 1440 x 1000
screenshots.
- OAuth request sequence: protected-resource metadata `GET
https://mcp.notion.com/.well-known/oauth-protected-resource/mcp`;
authorization metadata `GET
https://mcp.notion.com/.well-known/oauth-authorization-server`; dynamic
registration `POST https://mcp.notion.com/register`; authorization `GET
https://mcp.notion.com/authorize`; token exchange and refresh `POST
https://mcp.notion.com/token`; MCP traffic `POST
https://mcp.notion.com/mcp`.
- The live metadata and registration probe confirmed that Notion accepts
HTTPS and loopback HTTP redirects. It rejects a plain HTTP private
hostname.
- A later QA task owns the full browser consent and managed gateway
tool-list dry run against a configured HTTPS deployment.

## Risks

- Notion can add tools. Unrecognized names use the generic classifier,
and new or changed risky tools stay quarantined after connection
activation.
- A deployment that uses a private non-loopback hostname must configure
HTTPS before it can connect Notion.
- Dynamic registration creates a provider-side client. Paperclip reuses
it because registration does not provide a standard delete operation.
- Refresh coordination uses a database CAS lease across service
instances. An unclean crash leaves an uncertain lease and requires
reconnect instead of risking refresh-token replay.
- The current Apps surface overlaps with PR #10910. Merge order can
require a small conflict resolution if that PR lands first.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex on a GPT-5 runtime. The exact deployment ID and context
window are not exposed. The runtime used reasoning, repository tools,
code execution, and network tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 22:18:08 -05:00
Devin Foley 00a24d7e8f
ci: split general-server tests into five shards with refreshed durations (#10925)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The PR workflow runs the server vitest suite across sharded runners
because the suite is pinned to one worker.
> - In successful PR run 30930345729 (2026-08-04), shard `server (3/4)`
took 311 seconds of wall time and was the slowest check in the run.
> - The suite has grown to about 946 seconds of serial vitest time, but
the duration manifest was last sampled on 2026-08-01 at about 882
seconds.
> - This pull request refreshes the per-suite duration manifest from
that run's logs and splits the lane into five shards.
> - The benefit is a shorter PR critical path: each shard carries about
196 seconds of suite time, level with the other lanes.

## Linked Issues or Issue Description

Refs #10663 (previous split of this lane into four shards).
Related: #10923 splits the separate serialized-suites lane into five
shards. Both PRs touch `.github/workflows/pr.yml` in different matrix
blocks; whichever merges second needs a trivial rebase.

**What existing behavior does this improve?**

The `general-server` vitest lane runs in four shards with a duration
manifest sampled on 2026-08-01.

**Current behavior**

In PR run 30930345729, shard 3/4 ran for 311 seconds (273 seconds in the
test step) and was the longest check in the run. The suite now totals
about 946 seconds of serial vitest time.

**Proposed behavior**

Run the same suite set in five shards, balanced with a per-suite
duration manifest refreshed from that run's shard logs (279 suites
measured by diffing consecutive completion timestamps).

**Reason and benefit**

The refreshed LPT partition balances at about 196 seconds of suite time
per shard (about 240 seconds per job), level with the other PR lanes. No
test coverage is lost.

**Breaking changes**

None. The change only alters the CI partition size and the duration
manifest.

## What Changed

- Bump the `general-server` shard matrix in `.github/workflows/pr.yml`
from four to five shards.
- Refresh `scripts/general-server-shard-durations.json` from the
2026-08-04 run's shard logs.
- Update `SHARD_COUNT` in
`scripts/__tests__/run-vitest-stable-shard.test.mjs` to five.

## Verification

- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9
pass, including the complete non-overlapping partition proof and the
duration-balance check.
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2
pass.
- `node --test scripts/__tests__/e2e-shard.test.mjs` — 7/7 pass.
- A 5-way dry-run partition covers all suites exactly once with equal
projected weights.

## Risks

- Low risk. The change only alters CI partition size and duration
weights; the suite set is unchanged.
- One more runner is used per PR run for this lane.
- Stale duration weights degrade gracefully: suites missing from the
manifest get the median weight.

## Model Used

- Claude (Anthropic), Claude Code CLI, model ID `claude-fable-5`,
extended thinking with tool use enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
(workflow comments explain the new shard math)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Claude <claude@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-05 16:44:29 -07:00
Devin Foley b6e58019f2
ci: split serialized tests into five shards (#10923)
## Thinking Path

> - Paperclip uses CI to keep control-plane changes safe and mergeable.
> - The PR workflow splits serialized server tests across isolated
runners.
> - A recent successful run spent 305 seconds in serialized shard 2/4.
> - That job was the slowest check in the run.
> - The four shards reported about 739 seconds of Vitest suite time.
> - This pull request adds a fifth serialized shard and keeps release
verification aligned.
> - The benefit is a shorter PR critical path with no loss of test
coverage.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The PR and release verification workflows run serialized server tests in
four shards.

**Current behavior**

Successful PR run 30876682788 spent 305 seconds in `Verify serialized
server suites (2/4)`. The test step used 256 seconds and made this job
the slowest check.

**Proposed behavior**

Run the same serialized suite set in five complete and non-overlapping
shards.

**Reason and benefit**

The measured suites reported about 739 seconds of total Vitest time.
Five runners reduce the expected average suite time from about 185
seconds to about 148 seconds before setup overhead.

**Breaking changes**

None. The change only alters CI partition size.

## What Changed

- Split serialized server tests into five shards in the PR workflow.
- Apply the same five-shard layout to release verification.
- Add a partition test that proves complete and non-overlapping
serialized coverage.
- Update release workflow coverage tests for five shards.

## Verification

- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs
scripts/__tests__/release-verify-workflow.test.mjs`
- `git diff --check`

## Risks

- Low risk. CI uses one additional runner for the serialized lane.
- Round-robin partition weights can still vary as suite timings change.

> This change does not overlap with planned core work in `ROADMAP.md`.
Related PR #10663 optimized the separate general-server lane.

## Model Used

- OpenAI Codex, GPT-5, agentic coding with reasoning, tool use, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com>
2026-08-05 16:29:49 -07:00
Joonyoung Park c54936e2e9
fix(openclaw-gateway): use per-agent claimedApiKeyPath in wake text (#4668)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The `openclaw-gateway` adapter wakes a remote agent over a WebSocket
gateway. It sends a wake prompt. That prompt tells the agent which
environment variables to set and which file holds its Paperclip API key.
> - Each agent stores its claimed key in its own JSON file. The adapter
already exposes a `claimedApiKeyPath` config field for this. The field
is documented in `src/index.ts`. It also has an input in the agent
settings UI.
> - `buildWakeText` ignored that field. It hardcoded the shared default
path into the wake prompt text.
> - Every agent therefore read the same key file at wake time. Agents
authenticated as the wrong identity. The first API call failed.
> - This pull request passes `ctx.config.claimedApiKeyPath` into
`buildWakeText`. It uses the existing `resolveClaimedApiKeyPath` helper.
That helper falls back to the documented default.
> - The benefit is that each agent reads its own claimed-key file. Each
agent authenticates as itself.

## Linked Issues or Issue Description

Fixes #10071
Fixes #4976
Fixes #3098
Fixes #8076

These four open issues report the same defect. Earlier duplicates are
already closed: Refs #2561, Refs #2592, Refs #930.

Related pull requests that address the same root problem (duplicate
search):

- #3396 — same core change, no tests
- #3370 — heavier approach, injects `PAPERCLIP_CLAIMED_API_KEY_PATH`
into the wake env and adds server onboarding defaults
- #5970 — renames the config field to `paperclipApiKeyPath`
- #8072 — same core change, bundled with an unrelated protocol-version
change
- #784 — adds shell quoting and preflight instructions
- #3296 — bundled with an unrelated Claude hello-probe fix

## What Changed

- `packages/adapters/openclaw-gateway/src/server/execute.ts`
- `buildWakeText` now accepts `claimedApiKeyPath` as a parameter. It no
longer hardcodes the path.
- The `execute` call site passes
`resolveClaimedApiKeyPath(ctx.config.claimedApiKeyPath)`. That helper
returns the documented default
`~/.openclaw/workspace/paperclip-claimed-api-key.json` when the agent
sets no override.
  - `resolveClaimedApiKeyPath` is now exported so tests can call it.
- `packages/adapters/openclaw-gateway/src/server/execute.test.ts` — adds
`resolveClaimedApiKeyPath` cases: a configured value, an empty string, a
whitespace-only string, `undefined`, `null`, and non-string input.
- `packages/adapters/openclaw-gateway/vitest.config.ts` (new) —
package-level vitest config. It matches the config used by sibling
adapters such as `opencode-local`.
- `vitest.config.ts` (root) — adds the adapter to the workspace project
list.
- `scripts/run-vitest-stable.mjs` — adds
`@paperclipai/adapter-openclaw-gateway` to `nonServerProjects`.
**Maintainer-added during rebase.** The CI test lanes do not run a bare
`vitest`. They call `run-vitest-stable.mjs`, which invokes vitest with
an explicit `--project` allowlist. Without this entry the CI lanes skip
this package, and the root project-list entry alone has no effect on CI.

## Verification

Run the package suite directly:

```
pnpm install --frozen-lockfile
pnpm exec vitest run --project @paperclipai/adapter-openclaw-gateway
```

The suite covers `resolveSessionKey`, `buildAgentParams`, and the new
`resolveClaimedApiKeyPath` cases. The first two already existed in this
file but never executed in CI before this change.

Typecheck the package:

```
pnpm --filter @paperclipai/adapter-openclaw-gateway typecheck
```

Behavioural check, which no automated test covers:

1. Set `claimedApiKeyPath` to a per-agent value such as
`~/.openclaw/workspace/paperclip-keys/<agent>.json` in the agent's
gateway adapter settings.
2. Trigger a wake for that agent.
3. Confirm the rendered wake text names that file. It must not name the
shared default.

Maintainer note: this branch was rebased onto current `master` by a
maintainer. The original branch was two months stale. Only two conflicts
occurred, both additive: the import line and the tail of
`execute.test.ts`, and the project list in the root `vitest.config.ts`.
The `execute.ts` change applied without conflict. CI and Greptile re-run
against the rebased head.

## Risks

- Low for existing deployments. `resolveClaimedApiKeyPath` preserves the
default path exactly. Any agent that never set `claimedApiKeyPath`
receives the same wake text as before.
- The behaviour changes only for agents that already set a per-agent
path. Those agents previously received the wrong instruction. They now
receive the correct one.
- No database, schema, or API surface changes.
- CI now runs this package's test file for the first time. That file
includes the pre-existing `resolveSessionKey` and `buildAgentParams`
tests, which were never executed before.
- Five other adapters (`cursor-cloud`, `cursor-local`, `gemini-local`,
`grok-local`, `pi-local`) sit in the root project list but remain absent
from the CI allowlist. This pull request does not change them. That gap
is tracked separately.

## Model Used

- Contributor's change: Anthropic Claude, model ID `claude-opus-4-7`,
approximately 200K context, extended thinking. Used for triage, patch
authoring, and the original description.
- Rebase, the `run-vitest-stable.mjs` entry, and this description:
Anthropic Claude, model ID `claude-opus-5`, tool use enabled. Run by a
Paperclip maintainer.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
— the branch name carries an internal ticket id. A fork branch cannot be
renamed without opening a new pull request, so this is left as-is. The
internal reference has been removed from the description.
- [ ] I have run tests locally and they pass — the contributor verified
the pre-rebase branch. The rebased head is verified by CI on this pull
request.
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes —
`claimedApiKeyPath` is already documented in `src/index.ts` and exposed
in the agent settings UI, so no documentation change is needed
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green — pending the post-rebase run
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups —
pending re-review of the rebased head
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Pieter (CTO) <pieter@openclaw.local>
Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>
2026-08-05 00:53:43 -05:00
Devin Foley f0b06d2de9
feat(claude): environment-aware test-environment probe and claude-local CI coverage (#10833)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The claude_local adapter runs Claude Code on sandbox execution
targets, and operators verify an agent's configuration with the
test-environment probe before running it
> - Real runs merge the selected environment's env vars (secret refs
included) under the agent's adapter config env, but the probe built its
config from the adapter config alone — so environment-level auth worked
in runs while the Test button reported missing auth, and a dropped
secret binding passed silently
> - The claude env-test hints also did not recognize
`CLAUDE_CODE_OAUTH_TOKEN` even though the CLI accepts it, and a hello
probe that hit the subscription usage limit reported a hard failure
although authentication worked
> - Separately, the claude-local package test suites were absent from
the CI project list, so two suites drifted broken without notice
> - This pull request makes the probe resolve the same layered env as a
real run, adds the missing auth hint, classifies usage-limit probe
results as a warning, repairs the drifted suites, and turns the
claude-local project on in CI
> - The benefit is a Test button that tells the truth about
environment-level configuration, and a test suite that actually gates
the claude-local adapter

## Linked Issues or Issue Description

No public issue exists; related open PRs: Refs #9488 (recognizes
CLAUDE_CODE_OAUTH_TOKEN in environment checks — overlaps with the
auth-hint portion of this PR via a differently named check; it does not
cover the environment-envVars probe merge, the usage-limit
classification, or the CI coverage), Refs #9933 (live credential
validation in environment checks — complementary, no file-level conflict
with the route change).

The underlying problem, following the enhancement template:

**Current behavior**

The test-environment route builds the probe config from the agent's
adapterConfig only. Real runs merge the selected environment's envVars
under the agent env, so environment-level env vars (including auth such
as `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` bound as environment
secrets) work in runs while "Test environment" cannot see them, and a
missing secret binding passes silently. The claude env-test hints do not
recognize `CLAUDE_CODE_OAUTH_TOKEN`. A hello probe that hits the
subscription usage limit reports a hard `claude_hello_probe_failed`. The
claude-local package test suites do not run in CI, and two of them are
stale.

**Proposed behavior**

The probe resolves the selected environment's envVars
(environment-consumer secret bindings included) and merges them under
the agent config env with the run-path precedence; missing bindings
surface as an explicit error check that fails the test. The env-test
emits a `claude_oauth_token_configured` info check when that variable is
set. Usage-limit probe results classify as a
`claude_hello_probe_usage_limited` warning because auth works and only
the usage window is spent. The claude-local suites run in CI. Docs state
the resulting facts.

**Reason and benefit**

The Test button should tell the truth: it previously contradicted run
behavior for environment-level configuration and hid broken secret
bindings. Enabling the package suites in CI prevents further silent
drift — two suites were already broken on master without anyone
noticing.

**Breaking changes**

None. Runs are unchanged. The probe route only adds env layers and
checks; setups without environment envVars behave exactly as before.

## What Changed

- `server/src/routes/agents.ts`: the test-environment route resolves the
selected environment's envVars (forbidden keys stripped,
environment-consumer secret context) and merges them under the agent
adapterConfig env, mirroring `resolveExecutionRunAdapterConfig`
precedence. Missing secret bindings are skipped, reported as an
`environment_env_binding_missing` error check, and fail the test —
matching the `ConfigurationIncompleteFailure` a real dispatch would
raise.
- `packages/adapters/claude-local/src/server/test.ts`: new
`claude_oauth_token_configured` info hint between the API-key warning
and the subscription fallback; hello-probe classification gains a
`claude_hello_probe_usage_limited` warning for provider-quota results
(previously a hard `claude_hello_probe_failed`).
- `scripts/run-vitest-stable.mjs`: add
`@paperclipai/adapter-claude-local` to `nonServerProjects` so CI runs
the package suites.
- `packages/adapters/claude-local/src/server/execute.remote.test.ts`:
assert both runtime asset syncs (skills and mcp-config); the suite
predated the mcp-config asset.
- `packages/adapters/claude-local/src/server/test.probe.test.ts`:
usage-limit fixture now expects the usage-limited warning; new fixture
covers the genuine transient path (529 overloaded); new tests cover the
token hint and API-key precedence.
- `server/src/__tests__/agent-test-environment-routes.test.ts`: new
tests for the env merge (agent wins on conflict, forbidden key
filtered), missing-binding reporting, and the no-execution-target
fallback path.
- `docs/adapters/claude-local.md`, `docs/adapters/overview.md`: state
the auth-input facts (API key or oauth token wins over stored logins;
snapshot-owns-auth applies when neither is configured) and describe the
environment-aware Test behavior.

## Verification

- `npx vitest run --project @paperclipai/adapter-claude-local` — 131
tests pass (both drifted suites repaired; they fail on master today).
- `npx vitest run
server/src/__tests__/agent-test-environment-routes.test.ts` — 7 tests
pass.
- `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs` —
passes with the added project.
- `pnpm typecheck` in `server/` and `packages/adapters/claude-local/` —
clean.

## Risks

- Low risk. The run path is untouched; the probe route change is
additive and inert when the environment has no envVars.
- The probe now performs environment-consumer secret resolution at test
time; access is authorized per binding exactly as at run time, and the
audit consumer is the environment (as before for adapter-config
resolution).
- Enabling the claude-local project in CI adds about 2 seconds of vitest
wall time to the general workspaces group and could surface future
regressions in that package — which is the point.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking enabled, agentic
tool use via Claude Code (CLI).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-04 10:31:14 -07:00
Devin Foley 97590ff8c4
feat(dev): add pnpm dev:mobile and dev:both for prebuilt UI preview (#10718)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The board UI is a React SPA served by the paperclip server; the
standard local dev flow is `pnpm dev`, which runs vite in dev mode with
HMR and an unbundled module graph
> - The unbundled dev bundle is hundreds of MB of JS across many
requests, which is fine on a local machine but unusable from a phone or
tablet on slow/lossy links (airplane wifi, mobile data, distant tailnet
peers)
> - Contributors who want to iterate on the board from a mobile device
today have no supported way to preview a small production-shaped bundle
without stopping the dev server and running a one-off `vite preview`
with manual proxy plumbing
> - This pull request adds `pnpm dev:mobile` — build the UI and serve
`ui/dist` via `vite preview` on port 3101, with `/api` proxied to the
running dev server on 3100 — plus `pnpm dev:both` to run both flavors
together
> - The benefit is a supported second flavor of the dev server for
phones/tablets that runs alongside the normal one, without touching the
primary `pnpm dev` flow

## Linked Issues or Issue Description

**Subsystem affected**

ui/ — React + Vite board UI

**Problem or motivation**

The vite dev server serves an unbundled module graph, which is fine on
localhost but unusable from a phone or tablet on a slow link.
Contributors testing responsive behavior on mobile devices have no
supported way to serve a small production-shaped SPA against the running
dev API. Running `vite preview` directly does not work either — the
server's board mutation guard checks that the browser's Origin matches
the request Host, and a preview on a second port would fail every
mutation.

**Proposed solution**

Add two root scripts:

- `pnpm dev:mobile` — build `ui/dist` and serve it via `vite preview` on
port 3101, with `/api` proxied to the API server on 3100.
- `pnpm dev:both` — run `pnpm dev` and `pnpm dev:mobile` together in a
single terminal with prefixed output and shared signal handling.

The vite preview config binds `0.0.0.0`, sets `allowedHosts: true` so it
accepts arbitrary hostnames (LAN, tailnet, ngrok, etc.), and the shared
`/api` proxy forwards the client's original Host header as
`x-forwarded-host`. The paperclip server's mutation guard already
prefers `x-forwarded-host` over `host` when computing trusted origins,
so the browser's Origin becomes trusted automatically.

**Alternatives considered**

- Bespoke node proxy script — works but duplicates what vite preview
already does.
- Loosen the mutation guard to accept arbitrary origins — reduces
security for the primary server for the sake of a dev-only workflow.
- Second server config that binds a second port from the paperclip
server itself — much larger change and mixes runtime concerns with a
dev-tooling convenience.

## What Changed

- New `pnpm dev:mobile` script — build UI then run `vite preview` on
port 3101.
- New `pnpm dev:both` script — run `pnpm dev` and `pnpm dev:mobile`
together via `scripts/dev-both.mjs`, which prefixes each child's output,
propagates SIGINT/SIGTERM, and exits when either child exits.
- `ui/vite.config.ts` — add a `preview` block (port 3101, host
`0.0.0.0`, `allowedHosts: true`, shared `/api` proxy).
- New `ui/src/lib/vite-api-proxy.ts` — extracts the `/api` proxy factory
shared by dev and preview, and forwards the client Host as
`x-forwarded-host` (plus `x-forwarded-proto`).
- New unit test `ui/src/lib/vite-api-proxy.test.ts` covering the
header-injection behavior and the pass-through when no Host is present.

## Verification

- `pnpm --filter @paperclipai/ui exec vitest run
src/lib/vite-api-proxy.test.ts` — 3 tests pass.
- `pnpm --filter @paperclipai/ui typecheck` — clean.
- `pnpm --filter @paperclipai/ui build` — clean.
- Manual: ran `vite preview` against an echo listener and confirmed the
request arrives with `x-forwarded-host` set to the client Host header
and `x-forwarded-proto: http`. Then ran `pnpm dev:mobile` against the
live dev server and verified board mutations (mark issue read, resolve
recovery action, run routine) succeed from a second-port browser session
that previously 403'd.

## Risks

Low risk. Changes are limited to dev tooling — no runtime code paths, no
server changes, no schema/migrations. The `apiProxy` refactor is a no-op
behaviorally for the existing dev server (same target, same `ws: true`);
the only new behavior is the two `x-forwarded-*` headers, and the server
side already prefers those headers when trusting origins. `dev:mobile`
and `dev:both` are additive; existing `pnpm dev` is untouched.

## Model Used

Claude Opus 4.7 (1M context), extended thinking, tool use (bash, file
edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-02 20:26:03 -07:00
Dotta dcac49a4fd
feat(workspaces): defer isolated setup until runtime start (#10653)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Isolated workspaces give each task a safe and reproducible checkout.
> - The existing setup cloned the development database before an agent
needed to run the app.
> - This made worktree creation slower and heavier for tasks that never
start a service.
> - Runtime services already use one server start path for heartbeat,
operator, and startup recovery flows.
> - This pull request moves heavy setup to that start path and keeps
worktree creation lean.
> - The benefit is faster isolated workspace creation with the same
reliable runtime setup when a service starts.

## Linked Issues or Issue Description

Related pull request: #10652 covers the initial deferred
database-seeding slice. This pull request supersedes it with end-to-end
runtime provisioning and safe cleanup.

**What existing behavior does this improve?**

This improves isolated worktree creation, runtime service startup, and
isolated instance cleanup.

**Subsystem affected**

Cross-cutting: CLI worktree setup, server runtime orchestration, shared
workspace contracts, and development scripts.

**Current behavior**

Paperclip seeds an isolated development database during worktree
creation. It can also leave an isolated instance directory after
workspace teardown. This work happens even when no runtime service
starts.

**Proposed behavior**

Paperclip creates the worktree with a lean eager setup. It runs an
idempotent runtime provision command before the first managed service
spawn. Concurrent starts share one provision attempt. Teardown removes
the isolated instance safely.

**Reason and benefit**

Many agent tasks only edit and test code. They do not need a running
Paperclip instance. Deferring the database seed reduces workspace
startup cost while preserving automatic setup for tasks that start the
app.

**Breaking changes**

None. The new runtime provision command is optional. Existing workspace
behavior is unchanged when it is absent.

## What Changed

- Split Paperclip worktree setup into a lean eager script and an
idempotent runtime provision script.
- Added `runtimeProvisionCommand` to project, issue, realized workspace,
and persisted workspace contracts.
- Added a per-workspace provision mutex before local service spawn for
heartbeat, operator, and startup recovery flows.
- Added a persisted `provisioning` service state and the
`workspace_runtime_provision` operation phase.
- Kept provision time outside the service readiness timeout and made
failed attempts visible and retryable.
- Reclaimed isolated instance data during safe workspace teardown.
- Serialized deferred database seeding across processes and bound
teardown to the instance root captured in persisted workspace metadata.
- Added tests for config flow, concurrency, retry, no-op behavior,
readiness timing, scripts, CLI commands, and cleanup.
- Documented the eager and runtime provisioning contracts.

## Verification

- `pnpm -r typecheck`
- `pnpm build`
- `pnpm test:run` (server: 3,201 passed; UI: 3,345 passed; the CLI phase
exposed one environment-sensitive AWS doctor assertion because the agent
runtime injects static AWS credentials)
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest
run cli/src/__tests__/secrets.test.ts -t 'passes AWS doctor checks when
non-secret provider config is present'`
- Focused runtime tests cover serialized provisioning, retry after
stderr failure, absent-command no-op behavior, operation logging,
persisted state order, and readiness timeout exclusion.
- Focused CLI and cleanup tests cover concurrent seed serialization,
stale-lock fail-closed behavior, persisted instance ownership, and
rewritten sibling pointers.

## Risks

- A faulty runtime provision script blocks service startup. Paperclip
records stderr, marks the service failed, and retries on the next start.
- Concurrent service requests share an in-process provision attempt,
while the seed command uses an atomic filesystem lock across processes.
A stale lock fails closed and requires an operator to verify no seed is
running before removing it.
- Isolated instance cleanup is destructive. The cleanup service
validates ownership and path containment before removal.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, `gpt-5.6-sol`, with agentic reasoning, tool use, and
code execution. The service does not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-02 10:37:10 -05:00
Devin Foley 8540ce2973
ci: shard general-server tests 4 ways (#10663)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request CI must give contributors fast and stable feedback.
> - The `general-server` Vitest lane runs many single-worker server
suites.
> - A recent completed PR run showed this lane as the slowest completed
check.
> - Three shards still left one runner with the largest share of work.
> - This pull request splits that lane into four duration-balanced
shards.
> - The benefit is a shorter critical path for the same server test
coverage.

## Linked Issues or Issue Description

No public GitHub issue exists for this CI maintenance change.

**Pre-submission checklist**

- I confirmed this improves existing behavior. It does not add a new
command, endpoint, or concept.
- I searched open public issues and pull requests for related CI
sharding work.

**What existing behavior does this improve?**

The pull request workflow's `general-server` Vitest lane.

**Subsystem affected**

Cross-cutting. This affects GitHub Actions CI and the Vitest shard
duration manifest.

**Current behavior**

The `general-server` lane uses three shards. The server suites now total
about 880 seconds of serial Vitest wall time. The slowest shard was
about 313 seconds in the measured run.

**Proposed behavior**

The `general-server` lane uses four shards. Each shard receives about
220 seconds of predicted suite weight from the refreshed duration
manifest.

**Reason and benefit**

The slowest PR check controls how soon a reviewer can trust the PR. Four
balanced shards reduce the slowest `general-server` shard while keeping
the same suite selection rules.

**Breaking changes**

None. This only changes CI partitioning and duration data for existing
test suites.

**Additional context**

Related public searches found no exact open issue or pull request for
this `general-server` sharding change.

## What Changed

- Split the `general-server` CI matrix from three shards to four shards.
- Refreshed `scripts/general-server-shard-durations.json` with wall-time
weights from a recent completed PR run.

## Verification

- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check origin/master...HEAD`
- Dry-ran the four `general-server` shards locally during
implementation. The partition covers 300 unique suites with about 220.56
seconds of predicted weight per shard.
- Ran a local sensitive-data scan before push. It found only test
filenames that contain words such as `secret` or `token`, not credential
values.

## Risks

Low risk. The main risk is that the duration manifest becomes stale as
suite costs move. Missing suites fall back to the median weight, so the
lane still runs if the manifest is incomplete.

## Model Used

OpenAI Codex, GPT-5, with tool use and local command execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-01 19:03:28 -07:00
Dotta 173d6d2a71
Reclaim isolated worktree instances during teardown (#10649)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip creates isolated instances for server-managed git
worktrees.
> - The worktree teardown path removes the git worktree but leaves its
isolated instance directory behind.
> - The leaked directory can retain an embedded PostgreSQL process and
database files.
> - Teardown must remove only the collision-resistant instance assigned
to that exact worktree path.
> - This pull request stops the verified embedded PostgreSQL process and
removes the guarded instance directory.
> - The benefit is complete worktree cleanup without risk to another,
default, or live Paperclip instance.

## Linked Issues or Issue Description

**What happened?**

Closing a server-managed git worktree removed the git worktree and
branch, but it left the isolated Paperclip instance directory behind. A
live embedded PostgreSQL process could also keep running against that
directory.

**Expected behavior**

Worktree teardown must stop the isolated embedded PostgreSQL process and
remove only the instance assigned to that exact worktree. It must refuse
mismatched instance IDs and all paths outside
`PAPERCLIP_WORKTREES_DIR/instances/`.

**Steps to reproduce**

1. Create a server-managed git worktree with a repo-local
`.paperclip/.env` file.
2. Start its isolated embedded PostgreSQL instance.
3. Close the execution workspace.
4. Observe that the git worktree is removed but the isolated instance
directory remains.

**Paperclip version or commit**

The bug reproduces on `master` before this change.

**Deployment mode**

Local development with a server-managed git worktree and embedded
PostgreSQL.

## What Changed

- Give server-managed worktrees collision-resistant instance IDs derived
from their resolved absolute paths.
- Capture the repo-local instance pointer before custom teardown
commands can remove it.
- Require the pointer's instance ID to match the exact worktree-derived
ID.
- Resolve and validate the instance path against the canonical managed
worktree instance root.
- Verify and stop the matching embedded PostgreSQL process before
directory removal, including process-exit races.
- Record successful and refused cleanup operations in the workspace
operation log.
- Add focused ownership, process-race, path-safety, and runtime
integration tests.
- Document automatic isolated-instance cleanup for server-managed
worktrees.

## Verification

- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/workspace-instance-cleanup.test.ts` — 9 tests passed.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/workspace-runtime.test.ts -t "records teardown and cleanup
operations when a recorder is provided"` — 1 test passed and 99 tests
skipped.
- `node scripts/__tests__/provision-worktree-self-heal.test.mjs` — 4
tests passed.
- `bash -n scripts/provision-worktree.sh` — passed.
- `pnpm --filter @paperclipai/server build` — passed.
- `git diff --check` — passed.

## Risks

The main risk is removal of the wrong instance directory. Provisioning
assigns a path-derived ID with a SHA-256 suffix, and cleanup requires
that exact ID in addition to a safe instance identifier, an absolute
configured home, a strict child path, canonical path checks, and a
second canonical path check immediately before removal. It refuses
legacy or mismatched IDs, symlink escapes, and all paths outside the
managed worktree instance root. Cleanup failures become visible warnings
and do not delete an unverified path.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5. The runtime does not expose the exact model
snapshot or context-window size. The agent used reasoning, repository
tools, GitHub tools, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Paperclip <paperclip@paperclip.ing>
2026-08-01 20:41:11 -05:00
Devin Foley 8444e5735c
ci: pass e2e shard specs without separator (#10640)
## Thinking Path

> - Paperclip uses pull request CI to test changes before merge.
> - The e2e PR lane runs Playwright specs in a shard matrix.
> - Each shard builds a list of spec files for its matrix entry.
> - The workflow passed that list after a literal `--` separator.
> - Playwright did not receive the list as file filters.
> - This pull request removes the separator and adds a guard test.
> - The benefit is that each e2e shard runs only its assigned specs.

## Linked Issues or Issue Description

Refs #10629.

**What happened?**

The e2e shard step used `pnpm run test:e2e -- $specs`. The shard spec
list was not applied as Playwright file filters.

**Expected behavior**

Each e2e shard should pass only its selected specs to Playwright.

**Steps to reproduce**

1. Inspect `.github/workflows/pr.yml` at the merge commit for #10629.
2. Find the `e2e_shards` command that invokes `pnpm run test:e2e`.
3. See the literal `--` before `$specs`.

**Paperclip version or commit**

`86767951`

**Deployment mode**

GitHub Actions PR CI.

## What Changed

- Removed the literal `--` from the e2e shard `pnpm run test:e2e $specs`
invocation.
- Added a regression test that checks the workflow passes `$specs`
without that separator.

## Verification

- `node --test scripts/__tests__/e2e-shard.test.mjs`

## Risks

Low risk. This changes one CI command and one workflow guard test. The
main risk is shell argument handling in the workflow, and the guard now
covers the expected command shape.

## Model Used

OpenAI GPT-5 through Codex. The run used shell and GitHub CLI tool
access. The runtime did not expose a context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-01 15:04:21 -07:00
Devin Foley 8676795188
ci: split e2e PR lane into three shards (#10629)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The pull request workflow protects changes with a Playwright e2e
lane.
> - That lane already uses a weighted file partition so slow specs do
not cluster by test count.
> - Recent green PR runs showed the two e2e shard jobs were slower than
the next slow required lane.
> - The largest spec is indivisible, so a third shard lets that spec run
alone and lets the rest split by duration.
> - This pull request changes only the PR e2e shard matrix and the guard
test.
> - The benefit is a shorter expected PR critical path while the
required `e2e` aggregate check name stays stable.

## Linked Issues or Issue Description

Refs #9923

**What existing behavior does this improve?**

The `pull_request` workflow Playwright e2e lane.

**Subsystem affected**

Cross-cutting: GitHub Actions CI and test scripts.

**Current behavior**

The PR workflow runs the weighted Playwright e2e partition across two
jobs. Recent green runs showed those jobs as the slowest required
checks.

**Proposed behavior**

The PR workflow runs the same e2e spec set across three weighted jobs.
The aggregate required check stays named `e2e`.

**Reason and benefit**

The third shard lets the slow smoke-lab spec run alone while the rest of
the catalog stays balanced. This should shorten the PR critical path.
The win is bounded by fixed per-job setup time.

**Breaking changes**

None. The required aggregate check contract is preserved.

## What Changed

- Change the PR e2e shard matrix from two entries to three entries.
- Update the shard guard test to expect three shards.
- Floor the balance bound at the largest single spec weight.
- Assert that the workflow does not define more shard indexes than
`SHARD_COUNT`.

## Verification

- `node --test ./scripts/__tests__/e2e-shard.test.mjs` passes with 6
tests.
- The recorded-weight partition is complete and non-overlapping: 168.0s,
116.5s, and 114.4s.
- I checked `ROADMAP.md` and found no overlapping roadmap-level core
feature.
- I searched public GitHub PRs and issues for related e2e shard work. I
found related PR #9923 and no open duplicate for this branch or change.

## Risks

- This adds one extra GitHub Actions runner to the PR e2e lane.
- The wall-clock win is bounded by fixed per-job setup.
- Behavior risk is low because the aggregate required check remains
named `e2e`.

## Model Used

OpenAI Codex, GPT-5, tool-enabled coding agent in this repository. The
runtime did not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Cody <noreply@paperclip.ing>
2026-08-01 10:10:22 -07:00
scotttong c185e64b77
feat(ui): chat-style task view behind an experimental flag (#10606)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators spend most of their time on the issue detail page. They
talk to the assigned agent there through comments.
> - The current page reads as a ticket form. The thread sits below
properties, the composer sits mid-page, and live agent activity renders
as dense transcript logs.
> - Talking to an agent is a conversation. A chat-first layout matches
that mental model better than a ticket form.
> - A layout change this large must not disrupt current users. It needs
a safe opt-in path and full parity with the existing thread features.
> - This pull request adds a chat-style task view behind a new
"Chat-Style Tasks" experiment toggle. The flag is off by default and the
existing page is unchanged when it is off.
> - The benefit is a focused, readable conversation with the agent: live
tool activity folds into compact summaries, the composer stays at the
bottom, and properties, plan, and artifacts move into header tabs.

## Linked Issues or Issue Description

Refs #49 (chat with agents is a much-wanted feature).

Related PRs found in the dedup search:
- #4489 — an earlier, closed attempt to promote the conversation to the
primary surface on issue detail. This PR is a fresh, flag-gated take on
the same goal.
- #8837 — an open PR that proposes a two-column task layout. It
restructures the same page but keeps the ticket paradigm; this PR is
orthogonal because it is opt-in and chat-first.

**Subsystem affected**

UI (issue detail page).

**Problem or motivation**

The issue detail page presents agent conversations as a ticket:
properties first, thread below, composer in the middle of the page, and
raw transcript noise during live runs. Users who mainly converse with
their agents must scroll past chrome to follow the conversation, and
live activity is hard to read.

**Proposed solution**

An opt-in chat-style view of the issue detail page, gated by a new
"Chat-Style Tasks" experiment toggle in Settings → Experimental. With
the flag on, the thread fills the center pane, the composer docks to the
bottom of the viewport, Properties / Plan / Artifacts become header
tabs, live turns show a status pill with the current tool action and
elapsed time, and settled turns collapse to a "Worked · N tools" summary
that expands into per-tool rows. With the flag off, nothing changes.

**Alternatives considered**

Restyling the existing layout in place (rejected: too disruptive without
an opt-out), and a separate chat page beside the issue page (rejected:
splits the task's single source of truth). A per-request lab page
(`/task-chat-lab`, dev-only) was kept for design iteration instead.

**Roadmap alignment**

ROADMAP.md "CEO Chat" wants lighter conversations that still resolve to
real work objects. This PR keeps the core task-and-comments model — it
only changes presentation, opt-in — so it does not duplicate that
planned work.

## What Changed

- New `enableTaskChatRedesign` instance setting, exposed as a
"Chat-Style Tasks" experiment card in Settings → Experimental (shared
feature catalog, validators, server instance-settings service, and UI
settings page).
- New `ui/src/components/task-chat/` component family: chat thread with
turn grouping, agent reply bubbles, live status pill, collapsible turn
summaries with per-tool rows, plan tab with a sticky CTA action bar,
inline interaction cards, per-request mode chips, and a bottom-docked
composer.
- A shared tool taxonomy (`tool-taxonomy.ts`) maps tool names to verbs
and icons; the status pill, tool rows, and the classic transcript view
all use it.
- A transcript adapter converts stored run logs into chat turns; it
dedupes tool-call updates by `toolUseId` so tool counts match the
expanded rows, and it keeps a tool row's first real name when later
generic updates arrive.
- Composer: posts on Cmd/Ctrl+Enter, supports image paste with
object-URL thumbnail previews (revoked on clear/unmount), and uploads
through the issue attachments route.
- `IssueDetail.tsx`: with the flag on, pane tabs move to the header bar,
the header is not sticky, and the chat fills the center; with the flag
off, the previous layout renders unchanged.
- Motion tokens for the new animations live in `ui/src/index.css` with a
`motion-tokens.ts` catalog and a test that keeps the two in sync (the
catalog now also covers the shared enter/exit/swap tokens that the
decision/quicklook block declares).
- A dev-only `/task-chat-lab` page with fixtures and a tweak panel for
motion tuning.

## Verification

- `pnpm typecheck` — clean across the workspace.
- `pnpm check:token-gates` — 3/3 CLEAN.
- `cd ui && pnpm vitest run` — 3,344 of 3,345 tests pass locally. The
one failure is `IssueProperties.test.tsx` monitor-row time formatting,
which is timezone-sensitive: it also fails on unmodified `origin/master`
in a non-UTC timezone and passes with `TZ=UTC`. It is not related to
this change.
- `cd server && pnpm vitest run
src/__tests__/instance-settings-service.test.ts` — 21/21 pass (covers
the new setting).
- Manual: start the dev server, open Settings → Experimental, enable
"Chat-Style Tasks", and open any issue. The thread fills the page, the
composer docks to the bottom, and Properties / Plan / Artifacts appear
as header tabs. Assign an agent and comment to watch a live run: the
status pill shows the current tool action with elapsed time, and the
finished turn folds into a "Worked · N tools" summary. Disable the
toggle and confirm the classic page is unchanged.
- Visual snapshot baselines are intentionally not updated: per
`doc/design/DECISION-SHEET.md`, "Per-change snapshot verification
demoted to dormant (Jul 13 2026)".

## Risks

- The flag-off path goes through the same `IssueDetail.tsx` file, so a
regression there would affect current users. Mitigation: the classic
markup renders through the same components as before behind explicit
flag conditionals, and the full UI suite passes.
- The transcript adapter interprets stored run-log formats, including
legacy entries without `toolUseId`. Malformed logs degrade to generic
tool rows rather than crashing.
- The new view changes no server behavior other than one additive
instance setting; it is additive and default-off. Overall risk with the
flag off is low.

## Model Used

- Claude (Anthropic), model id `claude-fable-5` (Claude Fable 5),
extended thinking enabled, agentic tool use (file editing, shell, test
execution) via Claude Code / Claude Agent SDK.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 02:26:47 -07:00
Dotta b163c6c473
fix(server): preserve hot restart intent across path upgrade (#10593)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server can preserve eligible agent runs during a controlled hot
restart.
> - A path change moved restart state from the Paperclip home root to
the instance root.
> - A staged update can therefore make the old server and the new server
read different intent files.
> - The old server then drains live runs, while the new server can start
without a shutdown snapshot.
> - This pull request adds a correlated compatibility handoff and
records the live preflight set.
> - It also verifies the target process instance on Linux, macOS, and
Windows.
> - The benefit is complete and safe run classification across the path
upgrade.

## Linked Issues or Issue Description

No public GitHub issue covers this defect.

**What happened?**

A staged hot restart can run an older server that reads
`hot-restart-intent.json` from the Paperclip home root and a new server
that writes the file under the instance root. The old server misses the
request and uses graceful drain. The new server later finds its marker
without a shutdown snapshot. Before this change, that state could
produce an empty loss list even when live runs existed before restart.

**Expected behavior**

The old server must receive the PID-targeted restart request at its
legacy path. The new server must correlate the legacy shutdown snapshot
with its instance-scoped request. Every run that was live during
preflight must appear as adopted, finalized while down, or lost. A
reused PID must not let a stale marker claim a different process
instance.

**Steps to reproduce**

1. Start a server version from before the instance-root marker change.
2. Keep one or more local-agent heartbeat runs active.
3. Stage a current build and request a hot restart from that build.
4. Observe that the old server reads only the home-root path while the
staged build writes only the instance-root path.
5. Observe graceful drain and a new-server intent that has no shutdown
snapshot.

**Paperclip version or commit**

The path transition entered `master` in #10045. The hot-restart adoption
flow came from #9647. This fix targets current `master` and
compatibility with the immediately preceding home-root behavior.

**Deployment mode**

Self-hosted server built from source with controlled service hot
restarts.

Related work: #9628 is the original broader hot-restart feature PR.
#10556 addresses embedded PostgreSQL lifecycle behavior and does not
address marker-path compatibility.

## What Changed

- Write an authoritative instance-scoped intent and a correlated legacy
home-root handoff marker.
- Merge a legacy shutdown snapshot only when immutable request identity
fields match.
- Prevent a non-default instance from consuming an uncorrelated
legacy-only marker.
- Record preflight running heartbeat IDs and reconcile snapshot
omissions from current database state.
- Serialize marker claims, snapshot writes, stale recovery, and matching
cleanup with recoverable per-path filesystem leases.
- Read process start identity on Linux, macOS, and Windows to
distinguish a reused PID from the original server.
- Require identity for new restart requests and fail closed when a
supported platform cannot provide it.
- Classify older markers by comparing the replacement server boot time
or operating-system process start time with the request time.
- Close the preflight database client explicitly and use a root-safe SQL
query.
- Add focused unit, platform-branch, database-backed, and CI regression
coverage.
- Document the compatibility handoff, process identity probes, and
instance-scoped report path.

## Verification

- `pnpm exec vitest run server/src/services/hot-restart.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts -t
"hot-restart|old-server legacy|preflight live|preflight run|spawn
identity before hot restart"` — 24 tests passed and 90 tests were
skipped across 2 files.
- `pnpm exec vitest run server/src/services/hot-restart.test.ts` — 17
tests passed.
- `pnpm exec vitest run
server/src/__tests__/issue-watchdogs-routes.test.ts -t "restarts a
stalled claimed run"` — 1 test passed and 10 tests were skipped.
- `pnpm exec vitest run
server/src/__tests__/agent-action-audit-routes.test.ts -t "allows an
agent with issue:delegate"` — 1 test passed and 7 tests were skipped.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `git diff --check` — passed.
- GitHub Actions — 26 of 26 checks passed at
`55a79cb029be8b1dc89926d9d89ccd2181266d5c`.
- Greptile — 5/5 at the same head with no unresolved current-head review
threads.

## Risks

- The legacy handoff path is shared across instances. Exclusive claims
and per-path leases prevent overwrite and match-before-delete races.
- Process identity uses platform commands as a fallback when the health
endpoint has no identity. Linux reads `/proc`, macOS and BSD use `ps`,
and Windows uses PowerShell.
- A supported-platform identity probe failure aborts the restart. This
fails closed instead of replacing an unknown live process.
- Older intent files do not contain process identity. The server
compares the replacement boot or process start time with the request
time when those values are available.
- A preflight database read can fail before the marker is written. The
command fails closed instead of claiming a restart whose live-run set is
unknown.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment model ID and
context-window size were not exposed by this runtime. Reasoning,
repository editing, shell execution, test execution, GitHub CLI, and
Paperclip API capabilities were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-31 21:55:59 -07:00
Dotta c9116686bd
test(installer): cover cross-version update migrations (#10587)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Managed updates can change both the application payload and its
database schema
> - Unit tests cannot prove that an older live install upgrades through
a real migration and remains recoverable
> - The managed-install work in #10045 needs a repeatable cross-version
system test
> - This pull request adds an isolated end-to-end harness for update,
migration, backup, service restart, and rollback behavior
> - The benefit is a direct proof that managed upgrades preserve the
existing database and service lifecycle across versions

## Linked Issues or Issue Description

Refs #10045

This test is a focused follow-up to the managed install integration.
Merge #10045 first so the tested install, update, service, backup, and
rollback commands are available.

## What Changed

- Added a cross-version managed-update E2E script.
- Installed an older Git ref, initialized its embedded PostgreSQL
database, and updated to a ref with one additional migration.
- Verified the pre-update backup, payload switch, service recovery,
migration result, database-cluster reuse, and rollback behavior.
- Isolated Paperclip state under a dedicated test home and cleaned up
the service and managed install on success or failure.
- Added regression tests for shell syntax, required-ref validation,
side-effect-free preflight failure, and complete failure cleanup.

## Verification

- `node --test scripts/__tests__/e2e-update-migrations.test.mjs`
- `bash -n scripts/e2e-update-migrations.sh`
- GitHub latest-head CI: build, typecheck, release registry, canary
dry-run, general tests, serialized suites, and both browser E2E shards
passed.
- Full harness execution needs an isolated macOS or Linux host with a
real launchd or systemd user service. It is intentionally not run on a
live Paperclip server host.

## Risks

- The script manages a real user service and downloads two Git refs. Run
it only on an isolated test host.
- The test needs #10045 because `origin/master` does not yet contain the
managed install lifecycle.
- The script uses a dedicated `PAPERCLIP_HOME`, refuses a pre-existing
shim or test home, and removes its service and install during cleanup.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex based on GPT-5. The runtime did not expose a more
specific deployment ID or context-window size. Reasoning, repository
access, shell execution, and GitHub tooling were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-31 18:53:34 -07:00
Dotta fc5a30805e
feat(cli): add managed install, update, and service lifecycle (#10045)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Operators need a predictable installation path that survives beyond
an ephemeral `npx` process
> - A durable installation needs an owned per-user payload store, stable
command shim, safe shell integration, and supported service lifecycle
> - Updates must preserve recoverability by backing up data, installing
side-by-side, verifying the new payload, and retaining rollback state
> - Bootstrap scripts and privileged service operations must fail closed
across download, filesystem, ownership, and consent boundaries
> - This pull request integrates managed install, update, rollback,
service, uninstall, doctor, bootstrap-installer, and runtime-serving
support into one workflow
> - The benefit is a recoverable, inspectable, and documented
installation lifecycle with explicit safety boundaries across Linux,
macOS, containers, WSL, npm, npx, and source checkouts

## Linked Issues or Issue Description

### Problem

Paperclip lacks a first-class durable installation and lifecycle
workflow. Operators currently have to assemble npm/npx installation,
PATH setup, background-service management, updates, rollback,
diagnostics, and uninstall behavior themselves. That makes upgrades
harder to recover, creates inconsistent behavior across platforms, and
leaves shell/download/service trust boundaries without one documented
implementation.

### Proposed Solution

Add a managed per-user install store and stable shim, a verified shell
bootstrap installer, service lifecycle commands, install-mode-aware
update/rollback behavior, doctor checks, and documentation. Managed
updates back up the database, install and smoke-test a side-by-side
payload, atomically switch `current`, and retain prior payloads. The
shell installer pins registry/download trust boundaries and requires
explicit consent for non-interactive privileged actions.

### Alternatives Considered

- Keep recommending `npx`: simple for evaluation, but ephemeral and
unsuitable for stable services, atomic updates, or rollback.
- Require global npm installation only: familiar, but cannot provide the
owned side-by-side payload store and retained rollback semantics.
- Split the capability across multiple PRs: rejected because install,
update, service, uninstall, bootstrap, and serving behavior share
contracts and security boundaries that need review together.

### Related Pull Requests

- Supersedes #10042 and #10044 with one integrated final diff.
- Incorporates and replaces the closed preparatory work in #10032 and
#10034.

## What Changed

- Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall,
service lifecycle, onboarding integration, and managed-install doctor
checks.
- Added a private managed payload store, verified manifest/marker
ownership, exclusive mutation locks, atomic manifest/current/shim
writes, retained previous payloads, and provenance validation.
- Added npm and GitHub-ref install sources with exact target resolution,
registry isolation, database backup, side-by-side verification, atomic
activation, service restart coordination, and failure rollback.
- Made managed-update backups report actionable service-start and
`--no-backup` recovery guidance for unreachable databases, while clean
never-onboarded instances skip an empty backup.
- Added systemd user and launchd service definitions, status/health/log
commands, single-instance coordination, stale-port recovery, and
explicit sudo/lingering consent handling.
- Added the `scripts/install.sh` bootstrap path with checked two-stage
downloads, pinned public npm registry usage, platform checks,
dry-run/non-interactive controls, and Docker fixtures.
- Added embedded Postgres/native bootstrap integration,
hot-restart/systemd-notify serving support, passive update notices,
configuration contracts, README/CLI/install documentation, and focused
regression tests.
- Security re-review should explicitly re-verify: (1)
`addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or
non-regular rc files, assert current-user ownership, preserve
restrictive modes, and replace atomically; (2) managed shim replacement
rejects unsafe parents, foreign-owned or multiply linked files, and uses
checked atomic replacement; (3) the shell installer and sudo path
preserve explicit consent and checked downloads; and (4) installed
service/runtime serving remains bound to the validated managed shim and
instance configuration.

## Verification

- `bash -n scripts/install.sh scripts/clean-install-git.sh
scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh`
- `pnpm exec vitest run cli/src/__tests__/install-store.test.ts
cli/src/__tests__/install-command.test.ts
cli/src/__tests__/managed-install-check.test.ts
cli/src/__tests__/onboard-service.test.ts
cli/src/__tests__/service-health-check.test.ts
cli/src/__tests__/service-manager.test.ts
cli/src/__tests__/update-command.test.ts
cli/src/__tests__/update-notice.test.ts
packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests
passed
- `pnpm --dir cli typecheck`
- `pnpm --dir cli build`
- Follow-up verification: `pnpm exec vitest run
cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli
typecheck`, `pnpm --dir cli build`, and `pnpm --filter
@paperclipai/server typecheck`.
- `pnpm -r typecheck`
- `pnpm build`
- Full `pnpm test:run` exercised all suites; an injected static AWS
credential changed one unrelated doctor expectation, which passed when
those credentials were removed. A second run cleared that case and
exposed stale pre-existing adapter-utils `dist` output; rebuilding
`@paperclipai/adapter-utils` made the isolated test pass. The updated PR
CI is the authoritative clean-workspace full-suite run.

## Risks

- Installer/update code writes executable shims, symlinks, shell rc
blocks, service definitions, and managed payloads; ownership,
regular-file, symlink, hard-link, marker, and path-containment checks
fail closed before destructive changes.
- The bootstrap installer executes downloaded tooling; downloads are
staged and checked before execution, npm traffic is pinned to the public
registry, and non-interactive privileged behavior requires explicit
consent.
- Linux lingering may invoke `sudo`; the command is surfaced and
confirmed before execution, and unsupported service managers fall back
to foreground-run guidance.
- Database migrations remain forward-only; payload rollback does not
reverse migrations, so managed updates create a backup before activation
unless explicitly disabled.
- Service restart and runtime serving touch process/port ownership;
lifecycle locks, health/version checks, and stable-shim service
definitions reduce split-brain and stale-process risk.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with
reasoning, repository/API access, shell execution, and test tooling. The
runtime did not expose a reliable context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:52:23 -07:00
Dotta 79eff0aea1
fix(scripts): self-heal isolated workspace provisioning when the base CLI is broken (#10574)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents run tasks in isolated execution workspaces that are
provisioned as git worktrees by `scripts/provision-worktree.sh`
> - The script runs the base workspace's CLI (`cli/src/index.ts` via the
base `tsx` install) to seed each new worktree, and it only checked that
those files exist
> - pnpm links each package's `node_modules` into a hash-versioned
virtual store; a lockfile change followed by a partial or filtered
install prunes old hashed dirs without relinking every package, leaving
dangling symlinks
> - A CLI with dangling symlinks fails ESM resolution
(`ERR_MODULE_NOT_FOUND`) at boot, so provisioning aborts with
`setup_failed` — deterministically, on every retry, with no self-heal
path
> - This pull request makes provisioning health-check the CLI by
actually booting it, repair the base install when the check fails, and
degrade to the no-CLI fallback config instead of failing the run
> - The benefit is that a class of permanent `setup_failed` loops
becomes self-healing, and workspace provisioning survives a broken base
CLI

## Linked Issues or Issue Description

No public GitHub issue exists; the underlying bug is described here per
`bug_report.yml`. Related PR: #10578 self-heals the sibling
workspace-validation failure loop uncovered by the same incident
diagnosis.

**What happened?**
Isolated-workspace runs failed at provision time with `setup_failed`.
Every retry failed identically. One observed incident burned 4 runs
across two adapters before the task was stranded.

**Expected behavior**
Provisioning either succeeds or degrades gracefully; a broken base CLI
install repairs itself instead of permanently blocking all new
worktrees.

**Steps to reproduce**
In the base workspace, cause a lockfile-affecting dependency bump plus a
partial/filtered `pnpm install` so a package symlink (e.g.
`cli/node_modules/drizzle-orm`) dangles into a pruned virtual-store dir.
Start any isolated-workspace run. Provision fails with
`ERR_MODULE_NOT_FOUND` and the run ends `setup_failed`; retries never
recover.

**Paperclip version or commit**
master as of the branch point of this PR.

**Deployment mode**
Local trusted deployment with git-worktree isolated workspaces.

## What Changed

- `base_cli_healthy` now boots the base CLI (`--help`) instead of only
testing file existence, which exercises the top-level import graph.
- New `repair_base_workspace_install`: when the health check fails, run
a non-interactive `pnpm install --prod=false --force --frozen-lockfile`
in the base workspace. `--force` guarantees relinking when pnpm's
up-to-date heuristics would skip dangling symlinks; `--frozen-lockfile`
keeps the repair from mutating the shared lockfile.
- The repair install is serialized with `flock` on a lock file inside
the resolved git dir (`git rev-parse --absolute-git-dir`), so locking
also covers base workspaces that are linked worktrees, where `.git` is a
file.
- If every CLI candidate is unusable (including a base CLI the repair
could not fix), provisioning falls back to the existing no-CLI fallback
config writer (loudly, on stderr) instead of failing the run. A CLI that
runs and fails `worktree init` still fails provisioning with its real
exit code — that deliberate fail-closed policy is unchanged and covered
by an existing server regression test.
- Fixed a latent bug: `run_isolated_worktree_init` returned 0
unconditionally after the init subshell, so callers treated a failed
init as success. Exit codes now propagate.

## Verification

- Reproduced the incident state (dangling `cli/node_modules/drizzle-orm`
symlink); the base CLI failed with the exact `ERR_MODULE_NOT_FOUND` seen
in the incident run logs.
- Ran the patched script against a fresh scratch worktree: health check
failed → locked repair install ran (~26 s warm) → symlink relinked →
`worktree init` completed → exit 0 with `.paperclip/config.json` and
`.env` written.
- Happy path (healthy base CLI): provisioning behavior unchanged, exit
0.
- Verified `git rev-parse --absolute-git-dir` resolves a real directory
for both a normal checkout and a linked worktree.
- New hermetic tests: `node --test
./scripts/__tests__/provision-worktree-self-heal.test.mjs` (4 tests:
healthy CLI used, broken CLI degrades, locked repair end-to-end with a
fake pnpm, init failure propagates). Not yet wired into a CI workflow.
- `server`: the existing `realizeExecutionWorkspace` fail-closed
regression test ("fails instead of writing an unseeded fallback config
when worktree init errors after CLI detection succeeds") passes against
the new script.
- `bash -n scripts/provision-worktree.sh` is clean.

## Risks

- Low risk overall: the script only adds recovery paths; the happy path
is unchanged.
- The repair install runs in the shared base workspace. It is bounded by
`--frozen-lockfile` (no lockfile mutation) and serialized by `flock`,
but it can add ~30 s to the first provision after a base install breaks.
- If the repair cannot fix the CLI and no other CLI candidate exists,
runs now continue with an unseeded fallback config instead of failing;
that is intentional, and the fallback path already existed. Genuine
`worktree init` failures from a working CLI still fail the run.

## Model Used

Claude Fable 5 (Anthropic, model ID `claude-fable-5`), extended
thinking, agentic tool use (Claude Code harness).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-31 16:18:52 -07:00
Devin Foley dd1a7f5290
Ensure app-home ownership before the privilege drop, not only on remap (#10530)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Docker image persists all instance state (project checkouts,
worktrees, run logs, uploads) under `PAPERCLIP_HOME`, and deployments
mount a volume there for durability
> - The entrypoint starts as root and drops privileges to the `node`
user, but it fixes `PAPERCLIP_HOME` ownership only when it remaps the
user's UID/GID
> - A freshly mounted volume arrives root-owned and shadows the image's
build-time `chown`, so a default-UID boot drops privileges onto an
unwritable home and the server crashes on its first `mkdir`
> - This pull request makes the entrypoint probe the home's ownership
and chown whenever it does not match the runtime user, before the
privilege drop
> - The benefit is that the image works out of the box on any
platform-managed volume, with the common already-correct boot staying
chown-free

## Linked Issues or Issue Description

No public issue exists — describing the bug inline (per the bug report
template).

**What happened?**
Running the image with a freshly created volume mounted at `/paperclip`
(a Docker named volume, a Kubernetes PV, or any platform-managed volume)
and the default `USER_UID`/`USER_GID` crashes on boot: `Error: EACCES:
permission denied, mkdir '/paperclip/instances/default/logs'`.

**Expected behavior**
The container boots and initializes its instance tree on the mounted
volume, exactly as it does when `/paperclip` is the image's own
(build-time chowned) directory.

**Steps to reproduce**
1. `docker volume create paperclip-data`
2. `docker run -v paperclip-data:/paperclip
ghcr.io/paperclipai/paperclip:<any current tag>`
3. Observe the EACCES crash on the first `mkdir` under `/paperclip`.

**Root cause**
`scripts/docker-entrypoint.sh` chowns `/paperclip` only inside its
UID/GID remap branch (`changed=1`). A fresh volume mount is root-owned
and shadows the image's build-time `chown node:node /paperclip`; with
the default 1000:1000 no remap happens, so no chown happens, and `gosu
node` drops onto an unwritable home.

**Paperclip version or commit:** reproduces on `master` and any
published image.
**Deployment mode:** any; observed on managed-cloud volume mounts and
reproducible with plain Docker named volumes.
**Installation method:** Docker image (`ghcr.io/paperclipai/paperclip`).

**Related PRs (dedup search):** no open or merged PR touches the
entrypoint ownership logic; the entrypoint's privilege-handling tests
were added previously and this extends them. No duplicate found.

## What Changed

- `scripts/docker-entrypoint.sh`: the remap-conditional `chown` is
replaced by an ownership probe — after any UID/GID remap, the entrypoint
stats `PAPERCLIP_HOME` (default `/paperclip`) and runs `chown -R
node:node` only when the owner does not match the runtime user, before
`exec gosu node`. Covers fresh root-owned mounts and trees written under
a previous UID mapping; the already-correct boot performs no chown. The
unprivileged (non-root start) branch is unchanged.
- `server/src/__tests__/docker-entrypoint.test.ts`: `stat` stub added to
the harness; new cases for the fresh root-owned mount with default
UID/GID and for `PAPERCLIP_HOME`-relative probing; the remap case now
models the post-remap ownership mismatch.

## Verification

- `pnpm vitest run server/src/__tests__/docker-entrypoint.test.ts` — 7
passed (5 existing behaviors unchanged, 2 new).
- Live on a managed deployment: a container that crash-looped with the
EACCES above boots cleanly once the home is chowned before the drop (the
same effect this entrypoint change produces; forced there by a UID remap
as an interim workaround).

## Risks

- Low. Behavior changes only for boots where `PAPERCLIP_HOME` exists
with mismatched ownership — exactly the boots that crash today. `chown
-R` on a large previously-mismatched tree adds one-time boot latency;
correctly-owned homes skip it entirely. Kubernetes restricted /
OpenShift non-root starts keep the existing exec-directly path
untouched.

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic; Claude Code CLI with
extended thinking and tool use; tests executed locally via Vitest).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (no duplicates; extends the existing entrypoint privilege
tests)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-07-30 21:34:45 -07:00
Nicky Leach b517b887ad
fix(acpx): decouple host proxy spawn cwd from in-sandbox remoteCwd (#10122) 2026-07-23 16:48:52 -07:00
Devin Foley ad74fb5450
Add a feature catalog build artifact derived from the experimental settings schema (#10055)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Instances expose ~23 experimental feature settings, all declared in
one shared zod schema and toggled per instance
> - Deployment tooling and hosting control planes have no
machine-readable list of those feature keys for a given release — the
schema is only reachable from code that imports the package
> - Any external system that references feature keys therefore does so
as free text, and typos drift silently
> - This pull request derives a versioned `feature-catalog.json` build
artifact from the schema, with a compiler-checked metadata map so the
schema stays the single source of truth
> - The benefit is a stable contract external tooling can validate
feature-key references against, with zero runtime behavior change

## Linked Issues or Issue Description

No public issue exists; `feature_request` template fields:

**Problem or motivation:**
External deployment tooling cannot enumerate or validate an instance's
feature keys per release; free-text references fail silently when keys
are renamed or removed.

**Proposed solution:**
A metadata map keyed by the settings schema's own keys (compiler flags
drift) plus a build step emitting `feature-catalog.json` (keys, tiers,
defaults, `catalogVersion`) as a release artifact.

**Alternatives considered:**
A hand-maintained catalog file (drifts from the schema); serving the
schema from a runtime API (requires a running instance at validation
time — a build artifact works offline and pins to a release).

**Roadmap alignment:**
Supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`.

## What Changed

Adds a metadata map (title, description, tier, cloud/self-hosted
defaults) keyed by the keys of `instanceExperimentalSettingsSchema`, so
the schema stays the single source of truth and the compiler flags any
drift. A new build step (`build:feature-catalog --version <v>`) emits
`feature-catalog.json` — all 23 feature keys, their tiers, and a
`catalogVersion` — as a release artifact that managed-hosting control
planes can validate feature-flag writes against. No runtime behavior
changes.

- New `packages/shared/src/feature-catalog.ts`: per-flag metadata map
keyed by a type derived from the settings schema
(adding/removing/renaming a flag without updating the map is a compile
error), plus `featureCatalogArtifactSchema` and
`buildFeatureCatalogArtifact`/`renderFeatureCatalogArtifact` for the
artifact
- New `scripts/generate-feature-catalog.ts` wired as `pnpm
build:feature-catalog --version <v>`
- `scripts/create-github-release.sh` generates the artifact and uploads
it as a GitHub Release asset (with a dry-run preview line)
- Tests in `packages/shared/src/feature-catalog.test.ts`

## Verification

- `vitest run packages/shared/src/feature-catalog.test.ts` — 9 tests:
schema-key coverage, drift detection, artifact shape
- `pnpm --filter @paperclipai/shared typecheck`
- Artifact generation run end-to-end: `pnpm build:feature-catalog
--version 0.0.0-test` emits 23 keys with `catalogVersion`

## Risks

Low risk — no runtime behavior changes; the change is metadata, a build
script, and a release-artifact emission step only.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 18:12:56 -07:00
Dotta 204c416478
fix(release): publish bundled packages with trusted npm staging (#10047)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip publishes its CLI, server, adapters, and shared packages
through automated canary and stable release workflows.
> - `@paperclipai/adapter-utils` bundles the patched `acpx` runtime, so
it must use npm 11 for OIDC trusted publishing.
> - The prior staging directory contained pnpm's `.pnpm` symlink forest,
which crashes npm 11's directory-pack step on GitHub runners and
produces a consumer-broken bundled dependency tree.
> - This pull request rebuilds staged production dependencies as a
physical npm tree, reapplies repository patches, and publishes that
clean directory directly with npm 11 trusted publishing.
> - The benefit is a release path that retains GitHub Actions OIDC
trusted publishing while shipping a working patched acpx runtime to
consumers.

## Linked Issues or Issue Description

Refs: #9980, #10030, #10041

No public GitHub issue exists for this release failure.

### What happened?

Canary and stable publishing began routing `@paperclipai/adapter-utils`
through npm after it declared `bundleDependencies: ["acpx"]`. Publishing
the pnpm-deployed directory with npm 11 crashes during npm's
directory-pack phase on GitHub runners with `Exit handler never
called!`. The same staged shape also produces a broken consumer artifact
because acpx cannot resolve transitive runtime dependencies after
installation.

### Expected behavior

Bundled packages publish directly from a self-contained staging
directory through npm 11 OIDC trusted publishing, and consumers receive
a working patched acpx runtime with its transitive dependencies.

### Steps to reproduce

1. Stage `packages/adapter-utils` using the old `pnpm deploy`-only
shape.
2. Publish that directory with npm 11 on a GitHub runner.
3. npm crashes before registry/OIDC activity while walking the `.pnpm`
symlink forest.
4. Install an artifact packed from that old shape into a fresh npm
project and run acpx; its runtime dependency resolution fails.

### Deployment mode

GitHub Actions canary/stable release workflow.

### Relevant logs or output

```text
npm error Exit handler never called!
```

## What Changed

- After `pnpm deploy`, remove the staged pnpm `node_modules` tree and
run `npm install --omit=dev --ignore-scripts --no-audit --no-fund` to
create a physical hoisted production tree.
- Apply every root `pnpm.patchedDependencies` patch whose package is
declared in the staged package's bundled dependencies, failing staging
if any patch cannot apply.
- Assert the staged acpx runtime contains the required `onAgentStderr`
patch marker.
- Publish the clean staging directory directly with pinned npm 11.18.0,
retaining GitHub Actions OIDC trusted publishing, verbose diagnostics,
and the duplicate-transparency-log retry without provenance.
- Keep pinned npm 10.9.7 packing only for local/dry-run payload
verification; registry publishing does not use a tarball argument.
- Add focused coverage for npm-tree staging, patch application, direct
directory publish arguments, and bundled tlog retries.

## Verification

- `bash -n scripts/release-lib.sh scripts/release.sh`
- `node --test scripts/release-lib.test.mjs
scripts/acpx-patch-packaging.test.mjs` — 12/12 passed.
- `pnpm test:release-registry` — 68/68 passed.
- Real staging smoke: `node scripts/prepare-bundled-package.mjs
packages/adapter-utils <stage>` produced a real `node_modules/acpx`
directory, no `.pnpm` directory, and the `onAgentStderr` patch marker.
- Real npm 11 directory-publish smoke: `npx --yes npm@11.18.0 publish
--dry-run --tag canary --access public --loglevel verbose` packed 26
bundled dependencies and reached the expected existing-version registry
rejection without `Exit handler never called!`.
- The merge-triggered `publish_canary` workflow remains the live OIDC
trusted-publishing verification.

## Risks

- The live GitHub Actions trusted-publishing path can only be fully
proven by the merge-triggered canary run; npm debug-log upload remains
available if it fails.
- Bundling acpx continues to freeze platform-specific transitive
artifacts such as esbuild binaries from the Linux release runner. This
is a pre-existing consequence of the bundling decision in #9980 and is
not expanded here.
- Rebuilding dependencies with npm depends on the exact bundled
dependency versions in the staged manifest; staging fails hard if
repository patches no longer apply.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with
repository tool use and code execution. The harness did not expose a
model context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 17:01:11 -05:00
Dotta 4e8cd757eb
fix(release): add npm publish crash diagnostics (#10041)
## Thinking Path

> - Paperclip publishes a coordinated set of packages through its
release workflows
> - Bundled packages use a pinned npm CLI so trusted publishing works
consistently
> - The canary publisher now crashes deterministically inside npm before
useful output reaches the workflow log
> - The npm debug log that contains the underlying failure disappears
with the hosted runner
> - This pull request upgrades the pinned publish CLI and preserves both
verbose HTTP activity and npm debug logs on failure
> - The benefit is that the plausible HTTP-layer fix ships immediately,
while any remaining CI-only failure becomes diagnosable

## Linked Issues or Issue Description

- **Problem:** The canary release workflow fails on the first bundled
package with `npm error Exit handler never called!` and no preceding
diagnostic output.
- **Expected behavior:** Bundled packages publish through trusted
publishing, or the workflow retains enough npm diagnostics to identify
the actual failure.
- **Reproduction:** Run the canary release workflow in GitHub Actions;
the failure reproduced on both attempts of run 29948506814.
- **Version/commit:** Current `master` after #10024 and #10030.
- **Deployment mode:** GitHub-hosted release workflow using Node.js 24
and npm trusted publishing.
- Related: #10024, #10030.

## What Changed

- Bumped the bundled publish CLI from npm 11.16.0 to npm 11.18.0.
- Added `--loglevel verbose` to bundled npm publish invocations.
- Dumped the last 300 lines of every npm debug log after failed canary
or stable publishes, with common registry credential forms redacted.
- Updated release assertions to pin npm 11.18.0 and verify verbose
logging.

## Verification

- `pnpm test:release-registry` — 66 tests passed.
- `bash -n scripts/release-lib.sh`.
- Parsed `.github/workflows/release.yml` with Python/PyYAML.
- Smoke-tested npm log redaction with representative Authorization,
`_authToken`, and token environment values.
- Smoke-tested npm debug-log redaction against Authorization,
`_authToken`, and `npm_token` examples.
- `git diff --check origin/master...HEAD`.
- The merge-triggered canary workflow remains the live
trusted-publishing verification.

## Risks

- Low code risk: changes are isolated to the release publisher and its
workflow diagnostics.
- npm 11.18.0 could expose a different registry/runtime regression;
failure-time debug log dumping makes that actionable.
- Verbose npm output increases release log volume but does not change
package contents or dist-tags; common credential forms are redacted
before debug logs are printed.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.3 Codex, tool-enabled coding agent with repository
and shell execution; context window size is not exposed in this
environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 14:48:54 -05:00
Dotta 47c38777d8
fix(release): use trusted publishing npm for bundled packages (#10030)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip publishes canary and stable packages through a shared
release script
> - GitHub Actions authenticates those publishes through npm trusted
publishing and an OIDC identity token
> - Bundled-dependency packages recently moved from pnpm publish to a
pinned npm CLI to preserve their bundled files
> - That pin selected npm 10, which cannot use trusted publishing, so
the first bundled package failed with `ENEEDAUTH`
> - This pull request keeps bundled-package packing on npm 10 while
routing actual publishing through the trusted-publishing-capable npm 11
version
> - The benefit is bundled packages keep their required npm packaging
behavior while canary and stable releases authenticate successfully

## Linked Issues or Issue Description

**What happened**

The canary release job failed while publishing
`@paperclipai/adapter-utils` with `ENEEDAUTH`. The package has bundled
dependencies, so the release helper selected pinned `npm@10.9.7`; the
workflow provides OIDC trusted publishing rather than an npm token, and
npm 10 cannot use that authentication path. Because this is the first
package attempted, the release exited before trying the remaining
packages.

**Expected behavior**

Bundled-dependency packages should publish with an npm CLI that both
preserves bundled dependencies and supports GitHub Actions trusted
publishing.

**Steps to reproduce**

Run the canary release workflow from master after PR #9980. The
`publish_canary` job reaches `@paperclipai/adapter-utils`, invokes `npx
npm@10.9.7 publish`, and fails with `ENEEDAUTH`.

**Deployment mode**

GitHub Actions canary and stable npm release workflows.

Refs #9980.

## What Changed

- Kept bundled-package dry-run packing on npm `10.9.7`, which
successfully produces the staged tarball.
- Routed bundled-package publishing through npm `11.16.0`, which
supports GitHub Actions trusted publishing.
- Split the pack and publish helpers so future npm changes cannot
silently couple the two compatibility requirements.
- Updated focused release and ACPX packaging tests to enforce both
versions and call paths.

## Verification

- `pnpm test:release-registry` — 67 passed locally.
- Initial all-npm-11 PR head: Canary Dry Run reproduced an npm-internal
crash during bundled `pack`.
- Current head `5b3961ed13dd26ed2d6b1096ea23fd91b32e4353`: Canary Dry
Run passed with split npm pack/publish helpers.
- All PR checks passed, including build, typecheck + release registry,
server/workspace suites, both e2e shards, security gates, and Greptile.
- Greptile reviewed the current head at 5/5 confidence with no blocking
issues.

## Risks

- Low risk: the change only separates the npm CLI used for
bundled-package packing from the CLI used for publishing.
- The versions remain explicitly pinned because npm 11.16.0 currently
crashes on the bundled pack payload, while npm 10.9.7 cannot perform
trusted publishing.
- Focused tests assert both pins and both helper call paths, and the
full Canary Dry Run passes on the current head.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with
repository, shell, GitHub CLI, and code-execution tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 13:19:15 -05:00
Dotta d31a28828b
fix(acpx): support Windows agent spawning (#9980)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Local Claude, Codex, Gemini, and custom ACP adapters run through the
shared embedded ACPX engine
> - That engine wrapped every local agent command in a generated Bash
script to inject environment variables and filter child stderr
> - Windows cannot directly spawn that Bash wrapper, and npm/pnpm ACP
binaries are exposed through `.cmd` shims there
> - ACPX 0.12 already supports per-session child environment variables,
so the wrapper is unnecessary
> - This pull request registers agent commands directly, injects env
through ACPX session options, captures child stderr in-process, and adds
a real Node ACP spawn smoke on Ubuntu and Windows
> - The benefit is one cross-platform spawn path with a reusable smoke
test instead of parallel shell-wrapper implementations

## Linked Issues or Issue Description

Fixes #9941.
Refs #9428 and #9771.

**What happened**

ACPX-backed local agents failed to start on Windows because Paperclip
registered a generated POSIX `.sh` wrapper as the agent command. Windows
also needs the `.cmd` npm/pnpm shim when resolving built-in ACP
binaries, and symlink creation can fail with `EPERM` for seeded
auth/skill files.

**Expected behavior**

The same ACPX engine path should spawn a real ACP agent on Windows and
Linux, forward Paperclip/runtime env without mutating `process.env`,
preserve filtered/unfiltered child stderr behavior, and fall back to
copies where Windows symlinks are unavailable.

**Steps to reproduce**

Run a local ACPX adapter on Windows with the prior wrapper path. ACPX
attempts to spawn the generated `.sh` file and the agent never
initializes.

**Deployment mode**

Local Paperclip adapters using
`packages/adapter-utils/src/acpx-engine/`.

## What Changed

- Removed generated Bash agent/env wrappers and registered local
commands directly with ACPX.
- Passed the resolved child environment through ACPX
`sessionOptions.env`, including resume retry paths.
- Added a minimal `acpx@0.12.0` package patch exposing child stderr
callbacks and allowing documented uppercase env-map keys in persisted
session options.
- Moved stderr tee/filter behavior in-process: raw stderr remains in the
per-run file while benign `nes/close` noise is omitted from live stderr.
- Preferred `.cmd` ancestor binaries on Windows and added `EPERM` copy
fallbacks for Codex auth seeding and Gemini skill materialization.
- Added a real Node ACP echo-agent spawn smoke that can run directly on
any supported platform.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts
packages/adapter-utils/src/acpx-engine/spawn-smoke.test.ts` — 57 passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — passed.
- `node --test scripts/acpx-patch-packaging.test.mjs
scripts/release-lib.test.mjs` — 10 passed.
- Full canary release dry run under Node 24.18.0 / npm 11.16.0 — passed
in an isolated scratch clone.
- `git diff --check` — passed during implementation verification.
- One-time GitHub Actions proof: [Ubuntu ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774579),
[Windows ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774558),
and [Canary Dry
Run](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774497)
passed on head `f345ac69f2`; the dedicated smoke jobs are intentionally
not retained in the recurring PR workflow.

## Risks

- The ACPX stderr callback and env persistence exemption are carried as
a pnpm dependency patch until ACPX exposes/fixes those behaviors
upstream.
- Child stderr is synchronously appended to preserve ordering and
failure diagnostics; unusually high-volume agent stderr could briefly
block the Node event loop.
- The Windows-specific `.cmd` resolution and symlink `EPERM` branches
are proven by the standalone smoke test and the linked one-time
`windows-latest` run rather than a permanent CI gate.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5.4 via Codex CLI, medium reasoning, repository/tool
execution enabled; context-window size is not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 10:22:29 -05:00
Dotta d23fbf8ae4
feat(connections): add AppDefinition Wave 1 catalog (#9981)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Connections is the subsystem that defines which external apps and
MCP-style integrations operators can browse, configure, and run
> - The v3 schema core in #9958 added stable connection identities, auth
metadata, and grant-aware contracts, but the app catalog still used the
older gallery shape
> - The product needs a richer, typed AppDefinition catalog so browsing
and setup can render provider-specific auth and configuration
requirements consistently
> - This pull request moves the Wave 1 app catalog onto generated
AppDefinition data and carries that shape through shared types, server
lookup paths, and app connection UI
> - The benefit is that follow-up runtime and wizard work can build
against one catalog contract instead of local-only mock/gallery data

## Linked Issues or Issue Description

Refs #9958.

No public GitHub issue exists for this branch. This is the catalog layer
for the Connections v3 stack after the schema-core foundation in #9958.

## What Changed

- Adds generated AppDefinition data for the Wave 1 catalog and ingestion
reporting.
- Replaces the legacy tool app gallery exports with
AppDefinition-centered shared contracts, validators, and tests.
- Updates server tool-access lookup behavior to use the AppDefinition
catalog.
- Updates app connection UI surfaces and tests to consume
AppDefinition-backed catalog data.
- Documents the catalog ingestion workflow in the connector playbook.

## Verification

- `pnpm run preflight:workspace-links`
- `pnpm exec vitest run packages/shared/src/app-definitions.test.ts
packages/shared/src/app-definitions-url.test.ts
ui/src/pages/apps/AppsConnect.test.tsx
server/src/__tests__/tool-access-service.test.ts`

## Risks

- Medium: this changes the catalog contract used by shared, server, and
UI app connection surfaces.
- Catalog data quality matters because generated definitions now drive
browse/setup display.
- Follow-up runtime and wizard PRs must rebase on this branch or on
master after this lands.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 Codex coding agent with repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-21 15:57:12 -05:00
Devin Foley 1944c86153
fix(ci): preserve required e2e check for sharded runs (#9923)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The pull request workflow is the main merge gate for changes to that
app.
> - The Playwright e2e lane is expensive because every spec shares one
isolated server and runs serially.
> - Splitting that lane across runners shortens the critical path, but
the public required-check contract still needs a check named exactly
`e2e`.
> - This pull request shards the real e2e work while preserving a fast
aggregate `e2e` job for branch protection.
> - The benefit is a faster PR workflow without making otherwise-good
PRs unmergeable because a legacy required check disappeared.

## Linked Issues or Issue Description

No public GitHub issue exists for this CI follow-up.

Related prior CI work:

- Refs #8360
- Refs #9168
- Refs #9516

Bug report:

### What happened?

Sharding the PR e2e lane directly at the workflow job level changes the
emitted check names to shard-specific names, while existing branch
protection expects a check named exactly `e2e`.

### Expected behavior

The PR workflow should be able to run e2e specs across multiple runners
while still emitting a stable aggregate check named `e2e`.

### Steps to reproduce

1. Open a PR against `master`.
2. Run the PR workflow with the e2e lane split only as a matrix job.
3. Observe that the shard checks complete, but a required check named
exactly `e2e` never appears.

### Paperclip version or commit

Current `master`.

### Deployment mode

GitHub Actions pull request workflow.

## What Changed

- Added `scripts/e2e-shard.mjs`, which partitions default Playwright e2e
specs by recorded per-spec duration.
- Added `scripts/e2e-shard-durations.json` with measured e2e spec
durations so the slow smoke-lab spec does not dominate one runner.
- Split the PR workflow e2e lane into two `e2e_shards` matrix jobs and
added a fast aggregate job named exactly `e2e`.
- Added `scripts/__tests__/e2e-shard.test.mjs` to lock the shard
partition, ignored-spec sync, manifest coverage, and aggregate
required-check contract.

## Verification

- `node --test scripts/__tests__/e2e-shard.test.mjs`
- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check upstream/master..HEAD`
- Searched GitHub for duplicate or related e2e-shard / required-check
PRs and issues before opening this PR; no direct duplicate was found.

## Risks

Low risk. The main risk is that the duration manifest can drift as specs
are added or runtimes change; missing specs fall back to the median
known duration, and the focused shard test catches empty, overlapping,
or badly imbalanced partitions.

## Model Used

OpenAI GPT-5 via Codex CLI coding agent, with shell/tool execution and
repository inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-20 16:40:19 -07:00
Dotta 992389480a
fix(server): restore hot-restart run adoption (#9647)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The local heartbeat/runtime subsystem starts long-running local
agent processes and records their run state.
> - Operators sometimes need to rebuild and restart the Paperclip server
while local agent processes are still alive.
> - A normal restart should remain conservative, but a guarded
production hot restart needs an explicit marker, startup reconciliation,
and an inspectable report.
> - The broader hot-restart PR is currently merge-conflicted, so this
pull request lands the minimal server-side recovery path on current
`master`.
> - The benefit is that deploy operators can restart from a current
branch without reverting production changes and without marking adopted
live runs as `process_lost`.

## Linked Issues or Issue Description

No public GitHub issue exists for this deploy-safety fix.

Bug fix:

- What happened: the current deployable `master` branch did not include
the hot-restart marker CLI, startup adoption report path, or health
version proof needed by guarded service restarts.
- Expected behavior: a deploy operator can write a one-shot marker
before restarting, the old server snapshots eligible running child
processes, the new server reports adopted/finalized/lost runs, and
adopted live runs are not reaped as `process_lost`.
- Steps to reproduce: restart a server with running local child-process
heartbeat runs without the marker/adoption path; startup orphan reaping
has no adoption metadata and treats live detached children as lost.
- Paperclip version/commit: fixed on top of `master` at `b606869a6`.
- Deployment mode: production/local-service style deployments that
rebuild and restart the primary `paperclip.service`.
- Related PR: Refs #9628. This PR intentionally lands a smaller
deploy-safe subset because #9628 is currently merge-conflicted.
- Duplicate search: searched public PRs/issues for `hot restart` and
`process_lost adoption`; #9628 is the directly related prior
implementation.

## What Changed

- Added `scripts/request-hot-restart.ts` to write a one-shot hot-restart
intent marker under `PAPERCLIP_HOME`.
- Added `server/src/services/hot-restart.ts` for intent/report path
resolution, parsing, atomic writes, shutdown snapshots, and marker
cleanup.
- Wired server shutdown/startup so explicit hot restarts snapshot active
runs, skip the normal heartbeat drain, reconcile live child processes on
boot, and write `hot-restart-report.json`.
- Preserved adopted run metadata so normal orphan reaping does not
regress adopted live runs to `process_lost`.
- Added `serverVersion` health proof alongside existing `version`, plus
docs and regression coverage.

## Verification

- `pnpm vitest run server/src/__tests__/health.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts` — 2 files
passed, 100 tests passed.
- `pnpm --filter @paperclipai/server typecheck`
- `env PAPERCLIP_HOME="$PAPERCLIP_RUN_SCRATCH_DIR/hot-restart-cli-smoke"
pnpm --filter @paperclipai/server exec tsx
../scripts/request-hot-restart.ts --server-pid 12345`
- Branch ancestry checked after `git fetch origin master`:
`origin/master` was `b606869a6`, and `HEAD..origin/master` was empty.

## Risks

- Medium risk: process adoption depends on PID/PGID metadata and the
service manager leaving child processes alive for the guarded restart.
- Normal restarts remain conservative, but an incorrect marker PID
intentionally falls back to graceful drain instead of adoption.
- The PR is server-only and does not include the broader
UI/experimental-setting work from #9628.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 via Codex coding agent in a Paperclip execution
workspace; tool use and shell/code execution enabled; context window not
surfaced by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-16 02:46:09 -05:00
Dotta 3db2e6bdd2
feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 8/8 and focuses on end-to-end coverage,
operator docs, evals, and release notes
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: The complete stack needs discoverable browser scenarios,
operator guidance, threat modeling, eval coverage, and a parity proof
before merge.
- Proposed solution: Adds MCP user-story and Smoke Lab e2e suites,
docs/evals/release notes, the skill update, and the root e2e driver
script registration.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/07-ui-apps-activation`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for flag audit and e2e/browser acceptance;
Greptile on every PR.

## What Changed

- Adds MCP user-story and Smoke Lab e2e suites, docs/evals/release
notes, the skill update, and the root e2e driver script registration.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck`
- `node --check scripts/e2e-mcp-user-stories.mjs`
- `pnpm exec playwright test --config tests/e2e/playwright.config.ts
--list` — 43 tests discovered
- `git diff pap10341-split/08-e2e-docs
6b40e3876d9297105d4ec306e47e46d351c86172` — empty (0 bytes)

## Risks

- Browser suites depend on runtime services and environment setup; this
PR validates discovery locally while QA owns full flag-on/flag-off
execution.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556#9557#9558#9559#9560#9561#9562#9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 15:48:57 -05:00
Dotta 9c8adee48b
feat(mcp) [split 5/8]: integrate adapters and deployment runtime (#9560)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 5/8 and focuses on remaining adapters,
CLI, plugin examples, and deployment packaging
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: The backend runtime needs packaging, CLI propagation,
worktree provisioning, release manifests, and remaining adapter/plugin
consumers.
- Proposed solution: Adds the remaining runtime/deployment integration
after compile-required contracts and concrete MCP injection moved into
lower server levels.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is
`pap10341-split/04-server-runtime-wiring`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for CLI, packaging, and worktree behavior;
Greptile on every PR.

## What Changed

- Adds the remaining runtime/deployment integration after
compile-required contracts and concrete MCP injection moved into lower
server levels.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck`
- Focused CLI Vitest run — 4 files, 49 tests passed

## Risks

- Packaging omissions could make the feature work in source but fail in
Docker, worktrees, or release assembly.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556#9557#9558#9559#9560#9561#9562#9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 15:15:43 -05:00
Dotta 7b35de65aa
feat(mcp) [split 1/8]: add fixture demo servers (#9556)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 1/8 and focuses on fixture and demo MCP
servers
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: Developers need deterministic local MCP fixtures and visible
demo servers without pulling in the governed production runtime.
- Proposed solution: Adds the Google Sheets and KV demo MCP packages,
fixture catalog/servers, smoke harness, guide, and the root
smoke/typecheck registration hunks.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `master`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for fixture and smoke coverage; Greptile on every
PR.

## What Changed

- Adds the Google Sheets and KV demo MCP packages, fixture
catalog/servers, smoke harness, guide, and the root smoke/typecheck
registration hunks.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck`
- `pnpm --filter @paperclipai/google-sheets-mcp-server test` — 27 tests
passed
- `pnpm --filter @paperclipai/kv-demo-mcp-server test` — 12 tests passed

## Risks

- The new packages add dependencies that are intentionally not committed
to `pnpm-lock.yaml`, per repository policy.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556#9557#9558#9559#9560#9561#9562#9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 12:56:21 -05:00
Nicky Leach 90f85a7d11
Add telemetry proposal extractor (#9544)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - It emits telemetry events to understand product usage — registered
event names are gated by a generated `PAPERCLIP_EVENTS` registry so the
client only enqueues known, schema-approved events
> - When product teams want to instrument a new behaviour, they must
first register the event name — but schema registration is a
commit-and-release cycle, which creates friction in fast-moving product
iterations
> - A proposal lane is needed: let developers mark a `track()` call with
a typed `@ts-expect-error` proposal marker so the event name can be
reviewed and tracked in CI before the schema is formally registered
> - The existing client had no guard against unregistered event names,
so any call with an out-of-registry name (or a prototype-inherited key)
would silently enter the queue, state, and network flush path
> - This PR adds an `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard
at the entry point of `track()` to swallow unregistered calls before any
side effects, adds `scripts/extract-proposed-events.mjs` to scan source
for proposal markers and emit a v2 JSON manifest with provenance and
rationale, and documents the complete proposal workflow
> - The benefit is that new instrumentation can be proposed and reviewed
in code without touching the registered schema, and tooling can surface
missing rationale before events graduate to stable

## Linked Issues or Issue Description

No existing GitHub issue covers this change. This PR introduces a new
feature.

**Feature motivation:** Paperclip's telemetry schema is intentionally
stable — registered event names are code-generated and gated. Product
engineers who want to instrument a new behaviour today must land a
schema change first, creating a two-step process that slows iteration. A
proposal lane lets developers write the instrumentation call ahead of
schema registration, protected by a compile-time `@ts-expect-error`
marker that an extractor script can surface for review. This PR
implements both the client-side safety gate and the extraction tooling.

Refs: #9518 (closed predecessor — docs-only; this PR supersedes it with
the full implementation)

## What Changed

- Added `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the top of
`TelemetryClient.track()`: unregistered event names (including
prototype-inherited keys) are now swallowed before any state, queue, or
network operation
- Added `scripts/extract-proposed-events.mjs`: scans TypeScript source
for `@ts-expect-error -- proposed-telemetry(<issue>): <rationale>`
markers; emits a v2 JSON manifest per proposed event including name,
rationale, provenance (repo-relative file + line), and a
`rationale_missing` flag for CI enforcement
- Added `scripts/extract-proposed-events.test.mjs`: test suite covering
marker parsing, multi-line markers, path validation, out-of-repo
rejection, and the v2 schema output contract
- Added `doc/TELEMETRY_WORKFLOW.md`: documents the proposal workflow,
the canonical multi-line marker example, rationale requirements, and how
to graduate a proposed event to stable schema
- Updated `packages/shared/src/telemetry/README.md`: added "Proposed
Events" section to the Telemetry Data Contract per the contributing
guide requirement for telemetry changes

## Verification

Run all of the following from the repo root:

```sh
# Extractor unit tests
node --test scripts/extract-proposed-events.test.mjs

# Telemetry client + types tests
pnpm exec vitest run --config vitest.config.ts \
  src/telemetry/client.test.ts src/telemetry/client-types.test.ts \
  --reporter=verbose
# (run from packages/shared)

# Type-check
pnpm --filter @paperclipai/shared typecheck

# Smoke-run the extractor in local-test mode
node scripts/extract-proposed-events.mjs --ref local-test
```

All four commands pass locally.

## Risks

- **Silent drop on unregistered events:** The `Object.hasOwn` guard
fails closed — any event name not in `PAPERCLIP_EVENTS` is silently
dropped. If the generated registry is missing an event that was
previously tracked, those calls will be silently lost. Mitigation: the
extractor script surfaces proposed events that need registration; the
TypeScript type system already enforces `TelemetryEventName ⊆
PAPERCLIP_EVENTS` at compile time.
- **Extractor is read-only:** `extract-proposed-events.mjs` reads source
and emits JSON; it does not modify any files. No runtime or schema risk.
- Overall risk: **low**. The guard is additive and defensive; the
extractor and docs are additive only.

## Model Used

- Provider: Anthropic
- Model ID: `claude-sonnet-4-6`
- Context window: 200 K tokens
- Capabilities: tool use, extended context, code generation

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 23:47:27 -07:00
Dotta 6a36ae47fe
Fix stable release dry-run notes gate (#9334)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release subsystem uses GitHub Actions dry-run previews so
maintainers can validate stable and canary release behavior before
publishing.
> - Stable release publishing needs a release notes gate so real
`latest` publishes never happen without authored notes.
> - The same gate was also blocking stable dry-run previews, which made
release QA fail before publish-sensitive work could be previewed.
> - This pull request narrows the notes requirement to real non-dry-run
stable publishes.
> - The benefit is that stable dry-run dispatches can preview the
release without same-day notes, while real stable publishes remain
protected.

## Linked Issues or Issue Description

No public GitHub issue exists for this release-QA blocker, so the bug is
described inline below.

### What happened?

Stable dry-run release previews fail when the same-day release notes
file is absent. The release script runs the stable notes-file gate
before release preview work even when `--dry-run` is set.

### Expected behavior

`./scripts/release.sh stable --dry-run` should preview the stable
release without requiring `releases/vYYYY.MDD.P.md`. Real non-dry-run
stable publishes must still fail before build/publish work starts when
the notes file is missing.

### Steps to reproduce

1. Check out current `master` before this fix.
2. Ensure the computed same-day stable release notes file does not exist
under `releases/`.
3. Run `./scripts/release.sh stable --skip-verify --dry-run`.
4. Observe that the script exits with `stable release notes file is
required` instead of reaching the release preview plan.

### Paperclip version or commit

Reproduced on `master` at `9a1d4b7983dfd50e8eb40ee9770e44999d405f60`.

### Deployment mode

Built from source / GitHub Actions release workflow.

## What Changed

- Narrowed the stable release notes gate to `channel=stable` and
`dry_run=false`.
- Updated the release script usage note to say the notes file is
required for non-dry-run stable releases.
- Added a targeted Node test covering dry-run allowed behavior and
non-dry-run blocked behavior.
- Stubbed release fixture registry-version checks so the test isolates
the notes gate without hitting npm.

## Verification

- `node --test scripts/__tests__/release-dry-run-notes.test.mjs` passed
with 2/2 subtests.
- `bash -n scripts/release.sh` exited 0.

## Risks

Low risk. The behavior change only relaxes the notes-file gate for
stable dry-runs. The new test verifies real non-dry-run stable publish
still fails before build/publish work starts when notes are missing.

## Model Used

OpenAI Codex, GPT-5-based coding agent, with shell/tool execution in a
local repository workspace.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 21:11:07 -05:00
Devin Foley ce7dedf33d
perf(ci): balance general-server test shards by recorded suite duration (#9516)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its PR CI runs the general-server vitest lane pinned to
`maxWorkers=1` and sharded across 3 runners (introduced in #8360)
> - Suites were assigned to shards round-robin by sorted file index, so
shard test time was unbalanced: a recent PR run split 73s / 153s / 115s,
and the heaviest shard made "General tests (server 2/3)" the slowest
check in the whole workflow at 314s wall
> - The slowest shard sets the lane's wall time, so unbalanced
partitions waste the other two runners and stretch the PR critical path
> - This pull request replaces the round-robin assignment with a
deterministic longest-processing-time partition weighted by a checked-in
per-suite duration manifest
> - The benefit is near-even shard weights (projected 113s / 113s / 113s
with the current manifest), taking roughly 40s off the PR critical path
with no reduction in coverage

## Linked Issues or Issue Description

- Refs #8360 (introduced the 3-way general-server sharding this PR
rebalances)
- No public issue exists. Problem: the general-server test lane's
round-robin shard assignment ignores per-suite duration, so one shard
can carry multiple 30s+ suites while another finishes in half the time;
the slowest shard alone determines the check's wall time.

## What Changed

- `scripts/general-server-shard.mjs` (new): manifest loader and
deterministic LPT (longest-processing-time) partitioner; suites missing
from the manifest get the median recorded weight, and a missing or
malformed manifest degrades to uniform weights so the lane never fails
on stale data
- `scripts/general-server-shard-durations.json` (new): per-suite
duration manifest sampled from a real PR run (240 suites); the
`$comment` field documents how to regenerate it
- `scripts/run-vitest-stable.mjs`: both shard-selection sites (run and
`--dry-run`) now use the balanced partition instead of index round-robin
- `scripts/__tests__/run-vitest-stable-shard.test.mjs`: 6 new tests
covering skew-balance vs round-robin, determinism, median fallback for
unlisted suites, malformed-manifest degradation, manifest coverage of
the current suite set, and real-partition balance
- `server/src/__tests__/heartbeat-issue-rewake-throttle.test.ts`:
hardened the `afterEach` sweep — post-run bookkeeping (run-event
records, follow-up wake scheduling) can still insert rows briefly after
a run reaches a terminal status, and a late insert landing between the
`agent_wakeup_requests` and `agents` deletes failed teardown with a
foreign-key violation on the first CI attempt of this PR; the sweep now
retries so a late background write cannot take down the shard
- `release-verify.yml` shares the same runner script and inherits the
balancing with no workflow change

## Verification

- `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs` — 9/9
pass (run against current master)
- `npx vitest run src/__tests__/heartbeat-issue-rewake-throttle.test.ts`
— 6/6 pass against embedded Postgres with the hardened teardown
- `node --test scripts/__tests__/release-verify-workflow.test.mjs` — 2/2
pass
- `node scripts/run-vitest-stable.mjs --dry-run` with each shard flag
shows every suite assigned exactly once across the 3 shards, with
projected weights ~113s each

## Risks

- Low risk: partition changes which runner executes which suite, not
what runs; a completeness test asserts every suite is assigned to
exactly one shard
- The duration manifest will drift as suites are added/changed; unlisted
suites get the median weight and a coverage test flags when the manifest
covers less than half the suite set, so drift degrades balance
gracefully rather than breaking the lane

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic), extended thinking
enabled, agentic tool use (file edits, shell, test execution) via Claude
Code

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude (Paperclip SWE) <noreply@paperclip.ing>
2026-07-13 12:25:54 -07:00
Nicky Leach 8775bde4ce
Add runtime asset build-gap guard
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server package ships runtime asset trees used by built-in agents
and onboarding templates.
> - A prior server build omitted those source asset trees from `dist`,
allowing a built artifact to differ from runtime expectations.
> - The copy step is now present, but the existing build-gap gate only
checked TypeScript coverage for packages whose build skips `tsc`.
> - This pull request extends that standing gate so source asset files
under the server runtime asset trees must exist at the matching `dist`
paths after build.
> - The benefit is that future server asset additions fail loudly in CI
instead of silently shipping an incomplete `dist`.

## Linked Issues or Issue Description

### What happened?

After a server build, runtime asset files under
`server/src/built-ins/**` and `server/src/onboarding-assets/**` could be
missing from `dist/` with no build failure. The existing build-gap gate
only checked TypeScript coverage for packages that skip `tsc`; it did
not verify that non-TypeScript source assets were copied to `dist`. A
server build that forgot the `cp -R` step, or that added a new asset
tree without updating the copy command, would produce an incomplete
`dist` without any CI signal.

### Expected behavior

After `pnpm --filter @paperclipai/server build`, every
non-TypeScript/non-JavaScript source file under `server/src/built-ins/`
and `server/src/onboarding-assets/` must exist at the matching path
under `server/dist/`. If any file is missing, the build-gap gate must
exit non-zero with a diagnostic listing the missing files and the
command to fix them.

### Steps to reproduce

1. Remove a copied runtime asset: `rm
server/dist/built-ins/agents/reflection-coach/AGENTS.md`
2. Run the guard: `node scripts/run-typecheck-build-gaps.mjs
--runtime-assets-only`
3. Before this fix: the command exits 0 and the missing file goes
undetected.

### Paperclip version or commit

Reproduced on `master` at `c36f1a4af` (`@paperclipai/server` 0.3.1).

### Deployment mode

Not deployment-specific — the build-gap check runs in CI on any
checkout.

## What Changed

- Extended `scripts/run-typecheck-build-gaps.mjs` with a source-derived
server runtime asset parity check for non-`.ts`/non-`.js` files under
`server/src/built-ins/**` and `server/src/onboarding-assets/**`.
- Added a guard-only mode, `--runtime-assets-only`, for focused
pass/fail verification after a server build.
- Wired `pnpm run typecheck:build-gaps` to prepare plugin SDK build
deps, build the server package, then run the existing build-gap gate
plus the new asset check.

## Verification

Pass path:

```text
$ pnpm --filter @paperclipai/plugin-sdk ensure-build-deps
> @paperclipai/plugin-sdk@1.0.0 ensure-build-deps .../packages/plugins/sdk
> node ../../../scripts/ensure-plugin-build-deps.mjs

$ pnpm --filter @paperclipai/server build
> @paperclipai/server@0.3.1 build .../server
> tsc && mkdir -p dist/onboarding-assets dist/built-ins && cp -R src/onboarding-assets/. dist/onboarding-assets/ && cp -R src/built-ins/. dist/built-ins/

$ node scripts/run-typecheck-build-gaps.mjs --runtime-assets-only
[typecheck:build-gaps] server runtime assets present in dist: 7 file(s)
```

Regression simulation (guard catches the missing file):

```text
$ rm server/dist/built-ins/agents/reflection-coach/AGENTS.md
$ node scripts/run-typecheck-build-gaps.mjs --runtime-assets-only
[typecheck:build-gaps] Missing server runtime asset(s) in dist:
  - source: server/src/built-ins/agents/reflection-coach/AGENTS.md
    expected dist: server/dist/built-ins/agents/reflection-coach/AGENTS.md
Run pnpm --filter @paperclipai/server build and ensure source runtime asset trees are copied into dist.
```

Standing gate (full end-to-end):

```text
$ pnpm run typecheck:build-gaps
[typecheck:build-gaps] typechecking 4 workspace(s): paperclipai, @paperclipai/plugin-authoring-smoke-example, @paperclipai/plugin-llm-wiki, @paperclipai/ui
[typecheck:build-gaps] server runtime assets present in dist: 7 file(s)
```

## Risks

Low risk. The check only reads source and dist files during the
build-gap gate. The main tradeoff is that the gate now builds
`@paperclipai/server` so a clean checkout has generated `dist` content
to validate.

## Model Used

OpenAI Codex, GPT-5 based coding agent with repository tool use and
shell execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 08:35:30 -07:00
Dotta 36ec79c196
feat: add attention queue and Decisions surface (#9380)
## Thinking Path

> - Paperclip is the control plane for autonomous AI companies, where
operators need a reliable way to find and act on work awaiting their
input.
> - The attention and issue-thread interaction subsystems expose those
decision points across server APIs and the board UI.
> - The previous navigation and interaction presentation left these
actions fragmented and did not offer a controlled rollout for the
Decisions surface.
> - This branch adds the attention feed, richer interaction cards,
grouping, dismiss/snooze behavior, and a gated Decisions sidebar entry.
> - It also keeps experimental settings and API contracts synchronized,
with an idempotent migration for the new dismissal state.
> - This pull request delivers the complete, tested attention/Decisions
experience as one reviewable unit.

## Linked Issues or Issue Description

- Adds an operator-focused attention queue and Decisions experience:
grouped decision cards, semantic interaction actions, dismiss/snooze
handling, resilient interaction states, and an experimental flag to
control the Decisions navigation entry.


## Feature Context

### Problem or Motivation

Operators currently have to hunt across approvals, interactions, failed
runs, and budget alerts to find decisions that need their action.

### Proposed Solution

Provide a gated Decisions attention queue that groups actionable items,
supports direct resolution, and preserves operator control through
dismiss and snooze actions.

### Alternatives Considered

Keep separate, source-specific views only; this leaves cross-cutting
operator decisions fragmented and harder to prioritize.

### Roadmap Alignment

This improves the V1 control-plane operator workflow by making pending
governed actions discoverable in one company-scoped surface.

## What Changed

- Added server attention-feed services, routes, interaction handling,
dismiss/snooze support, and an idempotent `0145` inbox-dismissal
migration.
- Added shared attention, inbox-dismissal, and experimental-settings
contracts.
- Added Decisions/attention UI, interaction-card states, sidebar
badge/navigation integration, grouping, keyboard support, and Storybook
coverage.
- Added tests for attention behavior, thread interactions, settings
normalization, dismissals, and API behavior.
- Removed generated screenshots from the final PR diff and rebased the
branch onto current `master`.

## Verification

- `pnpm check:token-gates` — passed.
- `pnpm exec vitest run
packages/shared/src/issue-thread-interactions.test.ts
server/src/__tests__/attention-service.test.ts
server/src/__tests__/inbox-dismissals.test.ts
server/src/__tests__/issue-thread-interactions-service.test.ts
server/src/__tests__/issue-thread-interaction-routes.test.ts
ui/src/lib/attention.test.ts
ui/src/components/AttentionQueueRow.test.tsx
ui/src/components/IssueThreadInteractionCard.test.tsx
ui/src/pages/InstanceExperimentalSettings.test.tsx` — passed: 158 tests
across 9 focused files.
- GitHub Actions for `ad636f560`: build and typecheck/release-registry
have passed; remaining general-server and Greptile checks are in
progress.

## Risks

- Moderate: this is a cross-layer attention/interaction feature with a
new migration and navigation behavior.
- The `enableDecisions` experimental setting defaults to off, limiting
rollout impact.
- Existing dismissal data is backfilled to `dismiss`; the migration is
idempotent and uses guarded constraint creation.

> ROADMAP.md was checked; no duplicate planned core feature was
identified. Related open pull requests were searched before opening this
PR.

## Model Used

- OpenAI GPT-5.5 via Codex CLI, with tool use and local code execution.
Context-window size unavailable in this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My public PR branch name describes the change and contains no
internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally; focused tests pass and the remaining
unrelated AWS test failure is documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 17:09:57 -05:00
Dotta cec0fc249a
[codex] Parallelize release verify workflow (#9168)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Releases publish the same app and package set that operators
install, so release verification should keep full release-strength
coverage.
> - The release workflow currently verifies stable and canary releases
with one serial job that typechecks, runs all tests, and builds.
> - The PR workflow already proves the test surface can be split into
grouped general suites and serialized shards without changing coverage.
> - This pull request extracts the release verify work into a reusable
workflow and fans out the independent lanes.
> - The benefit is faster stable and canary release verification while
preserving the existing publish and preview gates.

## Linked Issues or Issue Description

No public GitHub issue exists for this CI improvement.

**Subsystem affected**

Cross-cutting (multiple of the above)

**Problem or motivation**

Release verification spends most of its wall time in a single serial
test step even though the same stable test surface is already
partitioned for PR CI. Stable dispatches and master-push canaries
therefore wait on one long runner after setup, typecheck, tests, and
build run sequentially.

**Proposed solution**

Add a reusable release verification workflow with parallel typecheck,
grouped general tests, serialized test shards, and build lanes. Have
both stable and canary release verification call it with the ref they
need to verify.

**Alternatives considered**

Keeping the serial `pnpm test:run` job preserves the old shape but keeps
stable and canary releases waiting on one long runner. Skipping
verification when a source SHA already has green CI would be faster, but
adds stale-check and lookup risk beyond this change.

**Roadmap alignment**

No overlapping item found in `ROADMAP.md`; this is release CI
maintenance.

**Additional context**

The new workflow keeps the release-strength full `pnpm -r typecheck`,
uses the existing stable test grouping/sharding entry points, and leaves
publish/preview jobs unchanged.

## What Changed

- Added `.github/workflows/release-verify.yml` as a `workflow_call`
workflow accepting a `ref` input.
- Split release verification into parallel `typecheck`, `general_tests`,
`serialized_tests`, and `build` jobs with 20-minute lane timeouts.
- Mirrored the PR workflow's stable test partition: `general-server`
shards 1-3, `general-workspaces-a`, `general-workspaces-b`, and four
serialized shards.
- Replaced `release.yml` `verify_canary` and `verify_stable` job bodies
with calls to the reusable workflow while leaving publish and preview
jobs unchanged.
- Added a Node test that guards the release workflow delegation and
split verify surface.

## Verification

- `actionlint 1.7.12 .github/workflows/release.yml
.github/workflows/release-verify.yml`
- `node ./scripts/release-package-map.mjs check`
- `node --test ./scripts/__tests__/release-verify-workflow.test.mjs
./scripts/__tests__/run-vitest-stable-shard.test.mjs`
- `git diff --check`

## Risks

- Release verification now starts more jobs per release event,
increasing total runner setup/install minutes. This matches the existing
PR CI tradeoff and should reduce release wall time substantially.
- The called workflow checks out the requested ref shallowly. That is
intentional for verify lanes; publish and preview jobs still retain
their existing full-history checkouts.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-class coding agent in local tool-use mode with shell
execution, repository editing, GitHub connector access, and medium
reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-09 19:28:00 -05:00
Dotta b13eb5b2b5
Skill Studio: three-pane skill IDE with sandboxed test runs (#9241)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Skills Manager gives operators a reusable skill layer, but
iteration still required manual edits, ad hoc prompts, and indirect run
inspection.
> - Skill authors need a focused workflow for editing skill files,
saving representative test inputs, and running those inputs through an
agent without exposing harness tasks as normal company work.
> - The backend therefore needs durable test inputs, reusable run
templates, hidden harness issues, scoped run execution, retention
metadata, and read-containment rules around hidden work.
> - The frontend needs a three-pane Studio that keeps skill files, saved
inputs/templates, and run output/history visible together while
preserving the existing design system and token rules.
> - This pull request ships that Skill Studio surface end to end:
database migrations, shared contracts, server APIs/services, hidden
harness execution behavior, UI routes/components, and focused tests.
> - The benefit is faster and safer skill iteration, with inspectable
outputs and fewer ways for internal harness work to leak into normal
task lists, costs, or adjacent read APIs.

## Linked Issues or Issue Description

No public GitHub issue exists for this feature. Feature request summary:

- Problem: Skill authors need to edit and test company skills in one
place instead of switching between the skill detail page, task creation,
run output, and manual prompt history.
- Proposed solution: Add a Skill Studio workbench with saved inputs,
reusable templates, hidden sandboxed test runs, live run status, output
inspection, run history, rerun/delete controls, and frontmatter-aware
editing.
- Expected users: Paperclip operators and agent-company maintainers who
create, fork, import, and tune skills.
- Related public PRs: Supersedes #9205, which was replaced so the public
PR branch name follows contributor policy.
- Duplicate search: searched public GitHub issues and PRs for "Skill
Studio"; no other active public issue or PR directly covers this
feature.

## What Changed

- Added database migrations for Skill Studio test inputs, test runs,
test run retention, and reusable run templates.
- Added shared Skill Studio types, validators, route helpers,
frontmatter utilities, and status handling.
- Added server services and routes for saved inputs, test runs,
templates, reruns, terminal-run deletion, hidden harness issue
execution, and run-detail hydration.
- Strengthened hidden-issue read containment across issue-adjacent
routes and cost rollups used by skill test harness work.
- Added the Skill Studio UI with skill file editing, frontmatter
editing, saved inputs, templates, run creation/cancel/rerun/delete
flows, output rendering, history, route support, and responsive pane
behavior.
- Added focused backend, shared, and UI tests for the new APIs, routing
logic, editor/run behavior, hidden-issue containment, and migration
safety.
- Rebased onto current `master`, removed the generated lockfile diff
from the PR, and verified no workflow files are changed.

## Verification

- [x] `pnpm --filter @paperclipai/db check:migrations`
- [x] `pnpm check:token-gates`
- [x] `pnpm exec vitest run
server/src/__tests__/company-skills-service.test.ts
server/src/__tests__/company-skills-routes.test.ts
server/src/__tests__/company-skill-test-runs-service.test.ts
ui/src/lib/skill-studio.test.ts ui/src/pages/SkillStudio.test.tsx` — 5
files, 132 tests passed
- [x] Greptile review on the latest PR head
- [x] GitHub PR checks on the latest PR head

## Risks

- Medium risk because this is a broad feature touching database schema,
server orchestration, issue visibility, and a large UI surface.
- Hidden harness issue containment is security-sensitive; this PR
includes regression coverage for adjacent read paths and cost rollups.
- The new migrations are additive and use idempotent guards where
applicable, but deployed databases that previously tested draft
migration numbers should still be checked carefully.
- The UI depends on a new resizable panels package in `ui/package.json`;
the lockfile is intentionally left to repository automation.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5 coding agent with shell, git, and GitHub CLI tool
use. Earlier feature commits include assistance from other Paperclip
coding agents; this PR preparation, rebase, cleanup commit, and PR body
were completed by OpenAI Codex in a Paperclip worktree.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 13:08:56 -05:00
Devin Foley eedc7ddef2
Make ACP the default engine for local adapters (#9238)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Adapter packages are the bridge between the control plane and local
agent harnesses such as Claude Code, Codex, and Gemini CLI.
> - ACP support was concentrated in a separate `acpx_local` adapter,
which made ACP feel like a separate agent choice instead of an execution
capability of the harness adapters.
> - Claude, Codex, and Gemini now have ACP-capable harnesses, so the
native adapter should own ACP selection, fallback, config, transcript
parsing, and environment diagnostics.
> - The standalone ACPX adapter still needs a compatibility path for
existing rows, but it should not be offered as an active adapter for new
agents.
> - This pull request moves the shared ACP runtime into
`@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to
prefer ACP when prerequisites are available, and retires `acpx_local` to
a tombstone.
> - The benefit is one adapter per harness, richer ACP transcripts by
default where possible, and a migration path for existing Claude/Codex
ACPX agents.

## Linked Issues or Issue Description

Closes #5932 — the broken default `acpx_local` Claude path is replaced
by native `claude_local` ACP support, existing Claude/Codex ACPX rows
migrate to native adapters, and new agents no longer choose the
standalone ACPX adapter.

Refs #4893 — original merged ACPX local adapter runtime that this PR
replaces with native per-harness ACP engines.
Refs #6590 — prior ACPX-Claude seamlessness work folded into the new
native Claude ACP path.
Refs #197 — related open generic ACP/Kiro adapter work; this PR does not
close it because Kiro/custom generic ACP remains a separate adapter
decision.
Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR
retires the built-in standalone adapter but does not add a native Kimi
adapter.
Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime
guidance into the shared/native ACP engine path instead of the old
standalone adapter.
Refs #8881 — related `acpx_local` POSIX shell failure from the old
`acpx` pin; this PR updates ACP dependencies but does not claim
custom/OMP ACP support as a first-class native adapter.
Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes
the old runtime path obsolete for new agents but keeps it as a
non-closing reference.

Problem description:

- The standalone `acpx_local` adapter duplicates Claude/Codex agent
choices that already have first-class local adapters.
- ACP should be an execution engine capability of each harness adapter
when the underlying harness supports ACP.
- Existing `acpx_local` agents should either migrate to native harness
adapters or fail with an explicit retirement message instead of silently
falling back to the process adapter.

## What Changed

- Added `@paperclipai/acpx-engine` as the shared ACP execution,
session-codec, CLI formatter, and UI parser package.
- Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select
ACP by default when prerequisites pass, with `engine=cli` opt-out and
`engine=acp` strict mode.
- Added ACP config schema/UI fields, environment checks, session-codec
preservation, transcript parsing, and adapter capability metadata for
the native adapters.
- Retired `acpx_local` to a server tombstone, removed its
UI/package/runtime image surface, and added a migration for existing
Claude/Codex ACPX agents.
- Updated package manifests, lockfile, release tooling, docs, Kubernetes
sandbox defaults, and tests.

## Verification

- `corepack pnpm --filter @paperclipai/acpx-engine typecheck`
- `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck`
- `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck`
- `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck`
- `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run`
- `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest
run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts
src/ui/build-config.test.ts`
- `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest
run src/server/acp.test.ts src/ui/build-config.test.ts`
- `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest
run src/server/acp.test.ts src/ui/build-config.test.ts
src/ui/parse-stdout.test.ts`
- `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps &&
corepack pnpm --filter @paperclipai/server exec tsc --noEmit`
- `corepack pnpm --filter @paperclipai/server exec vitest run
src/__tests__/adapter-routes.test.ts
src/__tests__/adapter-session-codecs.test.ts
src/__tests__/adapter-models.test.ts`
- `corepack pnpm --filter @paperclipai/ui typecheck`
- `corepack pnpm --filter @paperclipai/ui exec vitest run
src/adapters/metadata.test.ts
src/adapters/adapter-display-registry.test.ts
src/components/AgentConfigForm.test.ts
src/components/AgentConfigForm.render.test.tsx
src/components/transcript/RunTranscriptView.test.tsx`
- `node --test scripts/bootstrap-npm-package.test.mjs
scripts/release-package-map.test.mjs
scripts/verify-release-registry-state.test.mjs`

Note: the server typecheck script calls `pnpm` internally; this dev
shell exposes pnpm through Corepack only, so I ran the two script steps
manually with `corepack pnpm`.

## Risks

- Migration changes existing `acpx_local` Claude/Codex agents to native
adapter types and clears old ACPX task sessions/runtime state.
- Custom ACP commands remain on the retired tombstone and will need a
separate future adapter/plugin path.
- ACP auto-selection depends on local Node and ACP server command
prerequisites; remote and unsupported environments fall back to CLI
unless `engine=acp` is explicit.
- `@paperclipai/acpx-engine` is a new public package and needs npm
trusted-publishing bootstrap before release automation can publish it.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 via Codex coding agent. Exact hosted model build and
context-window size are not exposed in this runtime. Tool use included
shell execution, repository editing, GitHub CLI operations, and local
test/typecheck execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-08 19:05:03 -07:00
Devin Foley e68ee09809
perf(release): batch npm registry version queries (#9202)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release subsystem publishes the public workspace packages and
also powers release-related CI validation.
> - The release flow currently asks npm for package versions one package
at a time in multiple places.
> - That serial registry latency slows the PR Canary Dry Run path and
real release invocations even though the checks are independent.
> - This pull request batches npm registry version lookups with bounded
concurrency and reuses the result for version calculation.
> - The benefit is shorter non-build release-script time while
preserving the fresh target-version existence check before publishing.

## Linked Issues or Issue Description

- No public GitHub issue exists for this release-script performance
cleanup.

### Problem or motivation

Release validation spends avoidable time on repeated serial `npm view`
calls across the public package set. The slow path affects PR release
validation and real release invocations because version discovery waits
on independent registry reads one at a time.

### Proposed solution

Fetch package version maps concurrently with bounded parallelism, reuse
that map for stable/canary version calculation, and keep a fresh
parallel absence check for the target publish version.

### Alternatives considered

Keeping the existing serial shell loop is simpler, but it preserves the
CI latency cost. Caching the final target-version existence check was
rejected because release publish safety should still query npm freshly
before publishing.

### Roadmap alignment

This is a small release-tooling performance improvement. It does not
duplicate any planned core product work found in `ROADMAP.md`.

## What Changed

- Added `scripts/release-registry-versions.mjs` to fetch npm package
version maps and assert target-version absence with bounded parallelism.
- Updated `scripts/release.sh` to prefetch package versions once and to
batch the final target-version absence check.
- Updated `next_stable_version` and `next_canary_version` to use the
prefetched version map when present, with the existing per-package npm
fallback preserved.
- Added release-registry helper coverage and included it in `pnpm run
test:release-registry`.
- Hardened the release publish helper tests so their fake `pnpm`/`npm`
fixture PATH is preserved under non-login shell execution.

## Verification

- `node --test scripts/release-registry-versions.test.mjs`
- `pnpm run test:release-registry`
- `bash -n scripts/release.sh scripts/release-lib.sh`
- `git diff --check`
- Safety scan before push: searched changed files for common
key/token/password patterns and PII markers; only benign script-name
text matched (`secrets:migrate-inline-env`).
- Remote PR checks on the latest head passed, including `Typecheck +
Release Registry`, `Canary Dry Run`, build, tests, e2e, policy, security
scans, and commitperclip review.
- Greptile reviewed the latest head with Confidence Score 5/5 and no
blocking issues.

## Risks

- Low risk. The release version helpers keep their original npm fallback
when no prefetched version map is supplied.
- The existence check remains fresh and uncached before publish, but now
reports all matching package/version pairs from a parallel check.
- If npm has transient failures during the prefetch step, missing or
failed packages still map to an empty version list, matching the old
helper behavior.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agent using GPT-5, with shell/tool execution in
the local repository.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude <noreply@paperclip.ing>
2026-07-08 08:04:39 -07:00
Dotta d3919713bc
[codex] Document Storybook visual baseline platform lock (#9216)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Storybook visual baselines protect UI surfaces from unintended
visual drift.
> - Pixel-perfect screenshot baselines are sensitive to OS, font
rasterization, and browser environment.
> - The suite already stores external baseline artifacts and has an
opt-in CI path.
> - Local runs on non-matching platforms can report false-positive diffs
unless the platform lock is explicit.
> - This pull request documents the Linux/Ubuntu baseline constraint and
makes the local static server port explicit.
> - The benefit is clearer visual-review guidance and more predictable
Playwright web server startup.

## Linked Issues or Issue Description

No public GitHub issue exists.

### What happened?

The Storybook visual baseline suite requires a matching Linux capture
environment for pixel-exact comparisons, but the docs did not clearly
warn local users that non-Linux environments can produce false-positive
diffs. The Playwright web server command also relied on the static
server's default port instead of passing the configured port explicitly.

### Expected behavior

Developers should see clear Linux/Ubuntu baseline guidance before
running the visual suite locally, and Playwright should start the
Storybook static server on the same explicit port that the test config
expects.

### Steps to reproduce

1. Review the Storybook visual docs before this PR.
2. Run or inspect the Storybook visual Playwright config.
3. Notice the missing platform guidance and implicit static server port
coupling.

### Paperclip version or commit

Reproducible on `master` before this branch.

### Deployment mode

Local dev (pnpm dev) / built from source.

## What Changed

- Documents the Linux/Ubuntu-only baseline limitation in the developer
docs and visual-suite README.
- Adds `--port` parsing and validation to the Storybook static server
helper.
- Adds regression coverage for `--port` followed by another flag.
- Passes the Playwright web server port explicitly from the Storybook
visual config.

## Verification

- Passed: `node --check scripts/serve-storybook-static.mjs`
- Passed: `node --test
scripts/__tests__/serve-storybook-static.test.mjs`
- Passed: `node --test
scripts/__tests__/storybook-visual-baseline.test.mjs`
- Greptile: 5/5 with no unresolved review threads after commit
`94a649755a2ae7c4a34a3e8a1f16ec4d26d738fd`.
- Not run: full `pnpm test:storybook-visual`, because it builds
Storybook and runs the browser visual suite; this PR only changes docs
plus server port plumbing.

## Risks

Low risk. The server still defaults to port 6106 when no explicit port
is provided, and invalid port values now fail fast with a clear error
before the Playwright server waits for an unreachable URL.

## Model Used

OpenAI GPT-5 Codex coding agent with local command execution and
repository editing tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-08 07:23:02 -05:00
scotttong c07e650cd7
feat(ui): single-source design tokens, visual regression suite, and theme retune (#9134)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its UI is the operator's daily surface: task lists, boards, budgets,
agent status — all built on shadcn components and Tailwind
> - Visual values (colors, spacing, type sizes, radii) were hardcoded at
~1,600 call sites: the same "small gray label" was 9/10/11px depending
on the file, charts disagreed with chips about status colors, two
toggle-switch implementations coexisted in two greens, and there was no
visual regression coverage
> - This made the UI drift-prone and made any restyle a
hundreds-of-files project, which discourages design iteration
> - This pull request extracts visual values into a single token layer
in `ui/src/index.css`, adds a Storybook visual regression suite backed
by external immutable baseline archives, and then applies a deliberate
retune reviewed change-by-change on screenshot diffs
> - The benefit is that Paperclip's look becomes a config surface:
retheming is a token edit reviewed as a snapshot diff, drift is blocked
by a token gate, and future UI PRs can prove exactly what changed
visually without committing hundreds of PNGs

## Linked Issues or Issue Description

No existing public issue covers this work (searched "design tokens",
"visual regression", "design system" across issues and PRs). Related in
spirit: Refs #8982 (theming a hardcoded panel — a one-off instance of
the same problem class this PR addresses systematically).

**Problem (feature-request form):** UI visual values are hardcoded per
call site with no source of truth and no regression coverage;
consistency depends on reviewer memory, and restyling requires mass file
edits.
**Proposed solution (this PR):** a single token layer + enforcement gate
+ externally stored visual snapshot suite, then an intentional restyle
on top of that foundation.

## What Changed

- **Token extraction (zero visual change, machine-verified during
development):** committed codemods (`scripts/codemod-*.mjs`) moved
~1,600 hardcoded color/type/spacing/radius/shadow/misc values into named
tokens in a non-inline `:root` block of `ui/src/index.css`.
- **Visual regression suite:** `pnpm test:storybook-visual` covers 255
stories × light/dark = 510 Playwright screenshots at `maxDiffPixels: 0`,
plus new primitive-coverage stories and deterministic-render fixes.
- **External visual baselines:** committed PNG snapshots were removed.
`tests/storybook-visual/baseline-manifest.json` pins an immutable
archive URL/hash/size/count, and `scripts/storybook-visual-baseline.mjs`
handles `download`, `verify`, `pack`, and trusted maintainer `upload`
flows.
- **Opt-in visual CI artifacts:** added a `Storybook Visual` workflow
that runs on manual dispatch or PRs labeled `storybook-visual`,
downloads/verifies the baseline, runs Playwright, and uploads Playwright
report/test-result artifacts for review. Normal PR runs do not mutate
baseline objects.
- **Token gate:** `pnpm check:token-gates` — zero hex literals, zero
arbitrary bracket values, zero raw font-sizes in `ui/src/components/**`
and `ui/src/pages/**`, with a documented inline allowlist for legitimate
opt-outs.
- **Theme retune (intentional, snapshot-reviewed):** new base theme
values; radius ladder derived from a single `--radius` knob; micro-type
cluster collapsed to a named ladder (`--text-nano/micro/compact` +
Tailwind `text-xs`/`text-sm`); letter-spacing collapsed to named steps.
- **One status-color vocabulary:** charts, quota/budget bar fills,
RUNNING/live chips, and liveness indicators all use the canonical
`--status-*` hues. Light-mode legibility fixes for red alert surfaces
that used dark-tuned text classes.
- **One switch:** `ToggleSwitch` restyled to the registry capsule form,
second hand-rolled implementation removed, and all call sites unified.
- **Docs:** `DESIGN.md` is the design contract; `doc/design/` holds
audit reports, decision logs, and updated guidance for external baseline
review/update workflows.
- Dead code removed (`agentStatusBadge` duplicate map), byte-identical
contrast constants consolidated, semantic renames
(`--project-seed`/`--project-none`, `--liveness-blue`).

## Verification

- `pnpm check:token-gates` — 3/3 gates CLEAN during the design-system
run
- `pnpm typecheck` && `pnpm --filter @paperclipai/ui build` — green
during the design-system run
- `node --test scripts/__tests__/storybook-visual-baseline.test.mjs` —
pass after external-baseline rework
- `pnpm exec tsc --noEmit --pretty false --module NodeNext
--moduleResolution NodeNext --target ES2022 --types
node,@playwright/test tests/storybook-visual/playwright.config.ts
tests/storybook-visual/storybook-visual.spec.ts` — pass after
external-baseline rework
- `git diff --check origin/pr/9134..HEAD` — pass after external-baseline
rework
- `find tests/storybook-visual -type f -name '*.png' -print | wc -l` —
`0`
- `node scripts/storybook-visual-baseline.mjs verify` — intentionally
fails closed until the first trusted maintainer publishes the baseline
archive and updates `baseline-manifest.json`

## Risks

- **Large but shallow:** the PR still touches many UI files due to
mechanical token extraction and retune work, but committed PNG snapshot
churn has been removed from the branch.
- **Baseline publication required before the visual suite can pass in
clean clones:** the manifest currently has placeholder archive metadata.
A trusted maintainer must publish the first immutable archive, then
update `baseline-manifest.json`.
- **Rendering platform variance:** the external baseline should be
captured in the documented Linux/Chromium environment. Future CI runs
verify against the pinned archive and fail closed on checksum/count
mismatch.
- **Visual CI is opt-in while stabilizing:** add the `storybook-visual`
label or dispatch the workflow manually to produce downloadable
Playwright report/test-result artifacts.
- **Scheduled follow-ups, deliberately out of scope:** Tailwind palette
classes map to semantic tokens in a dedicated pass; card/pill component
consolidation; ESLint ratchet. Tracked in
`doc/design/DECISION-SHEET.md`.

## Model Used

Claude Fable 5 (Anthropic, `claude-fable-5`, Mythos-class tier) with
extended thinking, running in Claude Code with tool use; mechanical
phases delegated to Claude Sonnet subagents. Follow-up external-baseline
rework assisted by OpenAI Codex (`gpt-5` coding agent with repository,
terminal, and GitHub tool use). All bulk rewrites executed via
deterministic, idempotent scripts committed in `scripts/`; intentional
visual changes were human-reviewed on screenshot contact sheets.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run targeted local verification and documented the
intentional baseline-publication failure above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green *(pending new CI run after this
rework)*
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
*(pending review)*
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code) and
OpenAI Codex

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Dotta <bippadotta@protonmail.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-07 16:22:16 -05:00