Commit Graph

7 Commits

Author SHA1 Message Date
Dotta 0cc796b7bd
Build isolated preview artifacts for exact-source deployments (#13041)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Managed deployments need a cloud image and a database migration
package.
> - Branch commits can lack both artifacts until a normal release runs.
> - Operators need to test an exact commit without advancing release
aliases.
> - This pull request adds a preview build mode to the existing release
workflow.
> - Builds use an immutable source SHA and publish isolated, reusable
artifacts.

## Linked Issues or Issue Description

**Subsystem affected**

Release automation, cloud Docker images, and shared/database npm
packages.

**Problem or motivation**

An operator cannot deploy an unpublished branch with new migrations
using only
the normal release artifacts. Publishing it through a normal lane would
also
advance shared release aliases.

**Proposed solution**

Dispatch the trusted release workflow on master with a full source SHA
and a
request UUID. Build missing SHA images and, when needed, deterministic
preview
shared/DB packages. Publish packages under the preview dist-tag with
exact
workspace pins. Reuse matching artifacts on retries.

**Roadmap alignment**

This extends release tooling for operator validation. It does not add a
core
product feature or duplicate a planned product capability. Related PR
searches
found no duplicate preview deployment workflow.

## What Changed

- Add the preview channel, request correlation, artifact checks, and
result artifact.
- Compile source packages in a separate job from the npm publisher. The
publisher
  uses trusted master code and disables package lifecycle scripts.
- Publish only SHA cloud image tags. Preserve release aliases. Use
full-SHA tags and no shared build cache.
- Verify full source identity for reused packages and images. Both image
and npm publishers
use isolated jobs and the externally master-restricted npm-canary
environment. Fail on registry
  authentication errors, outages, or artifact identity mismatches.
- Let bundled-package preparation use patches from the requested source
checkout.
- Document publishing configuration, artifact contracts, and deployment
order.

## Verification

- Passed `pnpm -r typecheck` and `pnpm build`.
- Passed `pnpm test:release-registry`: 107 tests, including eight
preview tests.
- Passed `actionlint -shellcheck= .github/workflows/release.yml`.
- Built real shared and DB preview tarballs from an isolated exact-SHA
checkout.
Verified package source identity and all 244 SQL files and journal
entries.
- Verified the full revision behind an existing published SHA cloud
image.
- `pnpm test:run` exposed missing local embedded PostgreSQL library
symlinks.
The package's postinstall repair restored initdb; all 12 previously
affected
suites passed on rerun (95 tests). Additional local matrix reruns are in
progress.
The complete PR CI matrix is green, including general/serialized tests,
e2e,
typecheck, build, release registry, canary dry run, and the required
verify gate.
- Live preview publication and staging deployment require this workflow
on master
and the compatible control-plane backend. They have not run yet. No
production
  deployment was performed.

## Risks

Preview npm versions are immutable public artifacts. Both packages must
retain
their trusted publisher for release.yml in environment npm-canary.
Source builds
must remain separated from privileged npm publishing. The deploying
control plane
must verify source identity, integrity, and migration compatibility
before use.

Normal release jobs retain their existing conditions. Roll back by
stopping preview
dispatches and reverting the workflow/tooling. Published preview
versions remain
isolated from normal release tags.

## Model Used

OpenAI GPT-6 through Codex, with repository tools, code execution, and
test runs.
The session does not expose a more specific model version or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with the available version and
capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs
- [x] I have described the issue in-PR following the feature template
- [x] I have not referenced internal or instance-local issues or links
- [x] My branch name describes the change and contains no internal
ticket identifier
- [ ] I have run the full tests locally and they pass
- [x] I have added tests for the new behavior
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open recommendations or follow-ups
- [x] I will address review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-08 09:21:58 -05:00
Dotta 08af15bd76
fix(release): omit dev dependencies from bundle staging (#12584)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release system publishes the server package with selected
runtime dependencies inside its tarball.
> - The staging step writes a temporary package manifest before it runs
npm install.
> - That manifest kept private development dependencies that npm still
tried to resolve with `--omit=dev`.
> - The canary release then stopped because the matching private runner
version was not published yet.
> - This pull request removes development dependencies from only the
temporary install manifest.
> - The benefit is that bundled package staging installs only the
runtime dependencies that the tarball includes.

## Linked Issues or Issue Description

Refs #12582

**What happened?**

The canary release failed while it prepared `@paperclipai/server`. npm
tried to resolve `@paperclipai/paperclip-runner@2026.831.0-canary.8`
from the temporary staging manifest. The runner package was not
published at that version, so npm returned `ETARGET`. See the [failed
release
job](https://github.com/paperclipai/paperclip/actions/runs/33395418107/job/99504474818).

**Expected behavior**

Bundled package staging must install only dependencies that the
published tarball bundles. Private development dependencies must not
affect the staging install.

**Steps to reproduce**

1. Prepare a bundled package with a public bundled runtime dependency.
2. Add an unpublished package version to `devDependencies`.
3. Run `scripts/prepare-bundled-package.mjs`.
4. Observe that npm resolves the development dependency even when the
command uses `--omit=dev`.

**Paperclip version or commit**

`5a988df600ebda30e446496862bf83c76d6d53d6`

## What Changed

- Remove `devDependencies` from the temporary manifest used for bundled
package installation.
- Keep the final publish manifest unchanged.
- Add unit and staging regression checks for the unpublished development
dependency case.

## Verification

- `pnpm install --frozen-lockfile`
- `node --test scripts/acpx-patch-packaging.test.mjs` (12 passed)
- `pnpm test:release-registry` (98 passed)
- `git diff --check`
- The full test suite and build were not run. This change has focused
release-packaging coverage.

## Risks

- Low risk. The change affects only the temporary manifest used to
install bundled runtime dependencies.
- The script restores the complete publish manifest before it creates
the package tarball.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with `gpt-5.6-sol`. The model used agentic reasoning,
tool use, and code execution. The context window size is not exposed in
this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-31 08:52:40 -05:00
Dotta 8610e7934e
fix(release): bundle vendored runner ACPX runtime (#12582)
## Thinking Path

> - Paperclip is the open source app that people use to manage AI agents
for work.
> - The npm release includes the Paperclip server and a vendored runner.
> - The vendored runner imports ACPX when it starts a Codex agent.
> - The server package did not include the ACPX version that the runner
needs.
> - A fresh canary install therefore stopped with `ERR_MODULE_NOT_FOUND`
after onboarding.
> - This pull request bundles the patched ACPX runtime with the server
package.
> - The benefit is that a fresh npm install can load the vendored
runner.

## Linked Issues or Issue Description

No public issue exists for this bug. A GitHub search found no duplicate
or related pull request.

**What happened?**

A fresh `npx paperclipai@canary onboard` command completed onboarding.
The server then failed to start. Node could not resolve `acpx` from the
vendored Paperclip runner.

**Expected behavior**

The server should start after onboarding from a fresh npm cache and a
temporary data directory.

**Steps to reproduce**

1. Run `npx paperclipai@canary onboard --data-dir "$(mktemp -d
/tmp/paperclip-canary.XXXXXX)"`.
2. Select Quickstart.
3. Start Paperclip.
4. Observe `ERR_MODULE_NOT_FOUND` for `acpx`.

**Paperclip version or commit**

`paperclipai@2026.831.0-canary.6`

**Deployment mode**

Other: local trusted Quickstart through `npx`.

**Installation method**

npm through `npx`.

**Agent adapter(s) involved**

Codex.

**Database mode**

Embedded PGlite.

**Access context**

Board operator during onboarding.

**Node.js version**

Node.js 26.4.0.

**Operating system**

macOS.

**Relevant logs or output**

```shell
Cannot find package 'acpx' imported from .../node_modules/@paperclipai/server/dist/vendor/paperclip-runner/drivers/acpx/codex-runtime-adapter.js
```

**Relevant config (if applicable)**

No custom configuration was required.

**Additional context**

The published adapter utilities contain a nested `acpx@0.12.0`. Node
cannot resolve that nested package from the sibling vendored runner.
Installing `acpx@0.13.1` at the clean package root makes the failing
runner import succeed.

**Privacy checklist**

The log excerpt contains no user path, token, company name, or other
private value.

## What Changed

- Added `acpx@0.13.1` as a bundled server runtime dependency.
- Added a version-specific patch check for the ACPX versions used by the
server and adapter utilities.
- Added release-package coverage for the server ACPX bundle.

## Verification

- `pnpm test:release-registry` passed 98 tests.
- `node --test scripts/acpx-patch-packaging.test.mjs` passed 12 tests.
- `pnpm exec vitest run
server/src/__tests__/server-package-build-script.test.ts` passed 4
tests.
- `node --test scripts/release-package-map.test.mjs` passed 12 tests.
- `pnpm -r typecheck` passed.
- A clean extracted server tarball contained the patched `acpx@0.13.1`
runtime.
- The previously failing vendored runner module imported from that clean
tarball.
- The repository-wide test suite was stopped before completion at the
maintainer's request because it takes too long for this urgent packaging
fix.

## Risks

- Risk is low.
- The server tarball grows because it now contains ACPX and its
production dependencies.
- The release stager now uses a version-specific marker to verify the
ACPX patch.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with `gpt-5.6-sol`.
- The model used reasoning mode, tool use, and code execution.
- The context window size was not disclosed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-31 08:08:27 -05:00
Dotta 9ca24bba3c
feat(runner): pin the Codex ACPX runtime (#12400)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The package-local host boundary is ready for a concrete ACP
implementation, but the first production profile is Codex only.
> - ACPX must not inherit the server process environment or choose an
executable by pathname after admission.
> - Codex must not re-enable ambient apps, memory, skills, MCP
configuration, or instructions inside its isolated home.
> - This pull request pins only the two required production packages and
applies narrowly tested host patches.
> - The benefit is a minimal dependency boundary that follows the
repository's CI-owned lockfile process.

## Linked Issues or Issue Description

**Agent or provider**

Codex through `acpx@0.13.1` and `@agentclientprotocol/codex-acp@1.6.2`.

**Why this adapter is useful**

The injected runtime host needs a concrete ACP session manager and the
exact reviewed Codex ACP server. Upstream ACPX does not yet expose a
host-owned spawn callback, and upstream Codex ACP does not yet apply
Paperclip's isolated instruction, MCP, app, memory, and skill boundary.
Both behaviors are required before the dependency can execute inside the
runner.

**How the agent is invoked**

The next pull request will adapt these pinned packages to the private
runtime host. ACPX receives a host-owned callback that consumes the
already verified executable lease. Codex receives only the isolated
environment, explicit base instructions, explicit MCP servers, and the
skills rooted in its private `CODEX_HOME`. This pull request alone does
not spawn either package or register an adapter.

**Additional context**

This pull request is stacked on #12399. It adds no Pi, Claude, AWS, SDK,
lab, browser, or UI dependency. It intentionally does not commit
`pnpm-lock.yaml`: the repository policy job regenerates a manifest-only
PR lockfile artifact for downstream frozen installs, and the lockfile
bot updates master separately.

## What Changed

- Pin `acpx` to `0.13.1` and the Codex ACP server to `1.6.2` in the
runner package.
- Register both patches in the pnpm 9 root configuration and newer-pnpm
workspace configuration.
- Preserve the existing embedded-Postgres and ACPX 0.12 patch entries
used by other packages.
- Patch ACPX to evaluate an allowlisted environment at child-spawn time
and keep spawn cwd out of provider-visible session identity.
- Patch ACPX to accept a host-owned spawn callback with the resolved
arguments and options, allowing the verified command lease to own
execution.
- Patch Codex ACP to retain runner-owned MCP server identity in
permission requests.
- Patch Codex ACP to pass explicit Paperclip base instructions on both
start and resume.
- In isolated mode, disable ambient apps, memory, and existing MCP
configuration; load skills only from `CODEX_HOME`; and configure only
requested servers.
- Add a package contract test that enforces exact versions, Codex-only
dependency scope, both pnpm patch registries, and every required patch
hook.

## Verification

- Both patch files dry-apply successfully to fresh published tarballs
for `acpx@0.13.1` and `@agentclientprotocol/codex-acp@1.6.2`.
- A local no-lockfile install applied both patches; their runtime
markers and exact installed versions were inspected.
- Runner TypeScript typecheck — passed against the patched packages.
- Runner package tests — passed: 16 Node protocol/package tests and 426
Vitest tests.
- `pnpm -r typecheck` — passed for all applicable workspaces.
- `pnpm build` — passed, including runner binary, server, UI, and
workspace packages.
- `git diff --check` — passed.
- The diff contains 6 files and does not change `pnpm-lock.yaml`, a
GitHub workflow, server selection, or UI behavior.

## Risks

The primary risk is drift between published package contents and
checked-in compiled patches. Exact versions are pinned, both patches are
exercised by package-contract gates, and CI performs the authoritative
regenerated-lockfile frozen install. The spawn callback does not grant a
new executable path: the following adapter must consume the opaque
verified command lease. Codex isolation changes activate only when
`PAPERCLIP_ACPX_ISOLATED_CONTEXT=1`, so existing direct Codex adapters
are unaffected.

## Model Used

OpenAI Codex with GPT-5 and repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked an existing public item or described the
issue in this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal task
identifier
- [x] I have run the affected tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have documented the dependency, patch, isolation, and lockfile
boundaries
- [ ] All applicable GitHub Actions are green
- [ ] Greptile is 5/5 with every actionable comment resolved
- [x] I will address all review findings before requesting merge
2026-08-30 18:11:25 -05:00
Dotta fc5a30805e
feat(cli): add managed install, update, and service lifecycle (#10045)
## Thinking Path

> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Operators need a predictable installation path that survives beyond
an ephemeral `npx` process
> - A durable installation needs an owned per-user payload store, stable
command shim, safe shell integration, and supported service lifecycle
> - Updates must preserve recoverability by backing up data, installing
side-by-side, verifying the new payload, and retaining rollback state
> - Bootstrap scripts and privileged service operations must fail closed
across download, filesystem, ownership, and consent boundaries
> - This pull request integrates managed install, update, rollback,
service, uninstall, doctor, bootstrap-installer, and runtime-serving
support into one workflow
> - The benefit is a recoverable, inspectable, and documented
installation lifecycle with explicit safety boundaries across Linux,
macOS, containers, WSL, npm, npx, and source checkouts

## Linked Issues or Issue Description

### Problem

Paperclip lacks a first-class durable installation and lifecycle
workflow. Operators currently have to assemble npm/npx installation,
PATH setup, background-service management, updates, rollback,
diagnostics, and uninstall behavior themselves. That makes upgrades
harder to recover, creates inconsistent behavior across platforms, and
leaves shell/download/service trust boundaries without one documented
implementation.

### Proposed Solution

Add a managed per-user install store and stable shim, a verified shell
bootstrap installer, service lifecycle commands, install-mode-aware
update/rollback behavior, doctor checks, and documentation. Managed
updates back up the database, install and smoke-test a side-by-side
payload, atomically switch `current`, and retain prior payloads. The
shell installer pins registry/download trust boundaries and requires
explicit consent for non-interactive privileged actions.

### Alternatives Considered

- Keep recommending `npx`: simple for evaluation, but ephemeral and
unsuitable for stable services, atomic updates, or rollback.
- Require global npm installation only: familiar, but cannot provide the
owned side-by-side payload store and retained rollback semantics.
- Split the capability across multiple PRs: rejected because install,
update, service, uninstall, bootstrap, and serving behavior share
contracts and security boundaries that need review together.

### Related Pull Requests

- Supersedes #10042 and #10044 with one integrated final diff.
- Incorporates and replaces the closed preparatory work in #10032 and
#10034.

## What Changed

- Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall,
service lifecycle, onboarding integration, and managed-install doctor
checks.
- Added a private managed payload store, verified manifest/marker
ownership, exclusive mutation locks, atomic manifest/current/shim
writes, retained previous payloads, and provenance validation.
- Added npm and GitHub-ref install sources with exact target resolution,
registry isolation, database backup, side-by-side verification, atomic
activation, service restart coordination, and failure rollback.
- Made managed-update backups report actionable service-start and
`--no-backup` recovery guidance for unreachable databases, while clean
never-onboarded instances skip an empty backup.
- Added systemd user and launchd service definitions, status/health/log
commands, single-instance coordination, stale-port recovery, and
explicit sudo/lingering consent handling.
- Added the `scripts/install.sh` bootstrap path with checked two-stage
downloads, pinned public npm registry usage, platform checks,
dry-run/non-interactive controls, and Docker fixtures.
- Added embedded Postgres/native bootstrap integration,
hot-restart/systemd-notify serving support, passive update notices,
configuration contracts, README/CLI/install documentation, and focused
regression tests.
- Security re-review should explicitly re-verify: (1)
`addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or
non-regular rc files, assert current-user ownership, preserve
restrictive modes, and replace atomically; (2) managed shim replacement
rejects unsafe parents, foreign-owned or multiply linked files, and uses
checked atomic replacement; (3) the shell installer and sudo path
preserve explicit consent and checked downloads; and (4) installed
service/runtime serving remains bound to the validated managed shim and
instance configuration.

## Verification

- `bash -n scripts/install.sh scripts/clean-install-git.sh
scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh`
- `pnpm exec vitest run cli/src/__tests__/install-store.test.ts
cli/src/__tests__/install-command.test.ts
cli/src/__tests__/managed-install-check.test.ts
cli/src/__tests__/onboard-service.test.ts
cli/src/__tests__/service-health-check.test.ts
cli/src/__tests__/service-manager.test.ts
cli/src/__tests__/update-command.test.ts
cli/src/__tests__/update-notice.test.ts
packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests
passed
- `pnpm --dir cli typecheck`
- `pnpm --dir cli build`
- Follow-up verification: `pnpm exec vitest run
cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli
typecheck`, `pnpm --dir cli build`, and `pnpm --filter
@paperclipai/server typecheck`.
- `pnpm -r typecheck`
- `pnpm build`
- Full `pnpm test:run` exercised all suites; an injected static AWS
credential changed one unrelated doctor expectation, which passed when
those credentials were removed. A second run cleared that case and
exposed stale pre-existing adapter-utils `dist` output; rebuilding
`@paperclipai/adapter-utils` made the isolated test pass. The updated PR
CI is the authoritative clean-workspace full-suite run.

## Risks

- Installer/update code writes executable shims, symlinks, shell rc
blocks, service definitions, and managed payloads; ownership,
regular-file, symlink, hard-link, marker, and path-containment checks
fail closed before destructive changes.
- The bootstrap installer executes downloaded tooling; downloads are
staged and checked before execution, npm traffic is pinned to the public
registry, and non-interactive privileged behavior requires explicit
consent.
- Linux lingering may invoke `sudo`; the command is surfaced and
confirmed before execution, and unsupported service managers fall back
to foreground-run guidance.
- Database migrations remain forward-only; payload rollback does not
reverse migrations, so managed updates create a backup before activation
unless explicitly disabled.
- Service restart and runtime serving touch process/port ownership;
lifecycle locks, health/version checks, and stable-shim service
definitions reduce split-brain and stale-process risk.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with
reasoning, repository/API access, shell execution, and test tooling. The
runtime did not expose a reliable context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 18:52:23 -07:00
Dotta 204c416478
fix(release): publish bundled packages with trusted npm staging (#10047)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip publishes its CLI, server, adapters, and shared packages
through automated canary and stable release workflows.
> - `@paperclipai/adapter-utils` bundles the patched `acpx` runtime, so
it must use npm 11 for OIDC trusted publishing.
> - The prior staging directory contained pnpm's `.pnpm` symlink forest,
which crashes npm 11's directory-pack step on GitHub runners and
produces a consumer-broken bundled dependency tree.
> - This pull request rebuilds staged production dependencies as a
physical npm tree, reapplies repository patches, and publishes that
clean directory directly with npm 11 trusted publishing.
> - The benefit is a release path that retains GitHub Actions OIDC
trusted publishing while shipping a working patched acpx runtime to
consumers.

## Linked Issues or Issue Description

Refs: #9980, #10030, #10041

No public GitHub issue exists for this release failure.

### What happened?

Canary and stable publishing began routing `@paperclipai/adapter-utils`
through npm after it declared `bundleDependencies: ["acpx"]`. Publishing
the pnpm-deployed directory with npm 11 crashes during npm's
directory-pack phase on GitHub runners with `Exit handler never
called!`. The same staged shape also produces a broken consumer artifact
because acpx cannot resolve transitive runtime dependencies after
installation.

### Expected behavior

Bundled packages publish directly from a self-contained staging
directory through npm 11 OIDC trusted publishing, and consumers receive
a working patched acpx runtime with its transitive dependencies.

### Steps to reproduce

1. Stage `packages/adapter-utils` using the old `pnpm deploy`-only
shape.
2. Publish that directory with npm 11 on a GitHub runner.
3. npm crashes before registry/OIDC activity while walking the `.pnpm`
symlink forest.
4. Install an artifact packed from that old shape into a fresh npm
project and run acpx; its runtime dependency resolution fails.

### Deployment mode

GitHub Actions canary/stable release workflow.

### Relevant logs or output

```text
npm error Exit handler never called!
```

## What Changed

- After `pnpm deploy`, remove the staged pnpm `node_modules` tree and
run `npm install --omit=dev --ignore-scripts --no-audit --no-fund` to
create a physical hoisted production tree.
- Apply every root `pnpm.patchedDependencies` patch whose package is
declared in the staged package's bundled dependencies, failing staging
if any patch cannot apply.
- Assert the staged acpx runtime contains the required `onAgentStderr`
patch marker.
- Publish the clean staging directory directly with pinned npm 11.18.0,
retaining GitHub Actions OIDC trusted publishing, verbose diagnostics,
and the duplicate-transparency-log retry without provenance.
- Keep pinned npm 10.9.7 packing only for local/dry-run payload
verification; registry publishing does not use a tarball argument.
- Add focused coverage for npm-tree staging, patch application, direct
directory publish arguments, and bundled tlog retries.

## Verification

- `bash -n scripts/release-lib.sh scripts/release.sh`
- `node --test scripts/release-lib.test.mjs
scripts/acpx-patch-packaging.test.mjs` — 12/12 passed.
- `pnpm test:release-registry` — 68/68 passed.
- Real staging smoke: `node scripts/prepare-bundled-package.mjs
packages/adapter-utils <stage>` produced a real `node_modules/acpx`
directory, no `.pnpm` directory, and the `onAgentStderr` patch marker.
- Real npm 11 directory-publish smoke: `npx --yes npm@11.18.0 publish
--dry-run --tag canary --access public --loglevel verbose` packed 26
bundled dependencies and reached the expected existing-version registry
rejection without `Exit handler never called!`.
- The merge-triggered `publish_canary` workflow remains the live OIDC
trusted-publishing verification.

## Risks

- The live GitHub Actions trusted-publishing path can only be fully
proven by the merge-triggered canary run; npm debug-log upload remains
available if it fails.
- Bundling acpx continues to freeze platform-specific transitive
artifacts such as esbuild binaries from the Linux release runner. This
is a pre-existing consequence of the bundling decision in #9980 and is
not expanded here.
- Rebuilding dependencies with npm depends on the exact bundled
dependency versions in the staged manifest; staging fails hard if
repository patches no longer apply.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with
repository tool use and code execution. The harness did not expose a
model context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 17:01:11 -05:00
Dotta d31a28828b
fix(acpx): support Windows agent spawning (#9980)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Local Claude, Codex, Gemini, and custom ACP adapters run through the
shared embedded ACPX engine
> - That engine wrapped every local agent command in a generated Bash
script to inject environment variables and filter child stderr
> - Windows cannot directly spawn that Bash wrapper, and npm/pnpm ACP
binaries are exposed through `.cmd` shims there
> - ACPX 0.12 already supports per-session child environment variables,
so the wrapper is unnecessary
> - This pull request registers agent commands directly, injects env
through ACPX session options, captures child stderr in-process, and adds
a real Node ACP spawn smoke on Ubuntu and Windows
> - The benefit is one cross-platform spawn path with a reusable smoke
test instead of parallel shell-wrapper implementations

## Linked Issues or Issue Description

Fixes #9941.
Refs #9428 and #9771.

**What happened**

ACPX-backed local agents failed to start on Windows because Paperclip
registered a generated POSIX `.sh` wrapper as the agent command. Windows
also needs the `.cmd` npm/pnpm shim when resolving built-in ACP
binaries, and symlink creation can fail with `EPERM` for seeded
auth/skill files.

**Expected behavior**

The same ACPX engine path should spawn a real ACP agent on Windows and
Linux, forward Paperclip/runtime env without mutating `process.env`,
preserve filtered/unfiltered child stderr behavior, and fall back to
copies where Windows symlinks are unavailable.

**Steps to reproduce**

Run a local ACPX adapter on Windows with the prior wrapper path. ACPX
attempts to spawn the generated `.sh` file and the agent never
initializes.

**Deployment mode**

Local Paperclip adapters using
`packages/adapter-utils/src/acpx-engine/`.

## What Changed

- Removed generated Bash agent/env wrappers and registered local
commands directly with ACPX.
- Passed the resolved child environment through ACPX
`sessionOptions.env`, including resume retry paths.
- Added a minimal `acpx@0.12.0` package patch exposing child stderr
callbacks and allowing documented uppercase env-map keys in persisted
session options.
- Moved stderr tee/filter behavior in-process: raw stderr remains in the
per-run file while benign `nes/close` noise is omitted from live stderr.
- Preferred `.cmd` ancestor binaries on Windows and added `EPERM` copy
fallbacks for Codex auth seeding and Gemini skill materialization.
- Added a real Node ACP echo-agent spawn smoke that can run directly on
any supported platform.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts
packages/adapter-utils/src/acpx-engine/spawn-smoke.test.ts` — 57 passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — passed.
- `node --test scripts/acpx-patch-packaging.test.mjs
scripts/release-lib.test.mjs` — 10 passed.
- Full canary release dry run under Node 24.18.0 / npm 11.16.0 — passed
in an isolated scratch clone.
- `git diff --check` — passed during implementation verification.
- One-time GitHub Actions proof: [Ubuntu ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774579),
[Windows ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774558),
and [Canary Dry
Run](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774497)
passed on head `f345ac69f2`; the dedicated smoke jobs are intentionally
not retained in the recurring PR workflow.

## Risks

- The ACPX stderr callback and env persistence exemption are carried as
a pnpm dependency patch until ACPX exposes/fixes those behaviors
upstream.
- Child stderr is synchronously appended to preserve ordering and
failure diagnostics; unusually high-volume agent stderr could briefly
block the Node event loop.
- The Windows-specific `.cmd` resolution and symlink `EPERM` branches
are proven by the standalone smoke test and the linked one-time
`windows-latest` run rather than a permanent CI gate.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5.4 via Codex CLI, medium reasoning, repository/tool
execution enabled; context-window size is not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 10:22:29 -05:00