## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release system builds the workspace before it prepares npm
packages.
> - Bundled packages then move the built files and runtime dependencies
into a standalone staging directory.
> - npm pack and npm publish still run package lifecycle scripts in that
directory.
> - The server `prepack` script requires the source workspace and cannot
run from the standalone directory.
> - This pull request disables lifecycle scripts only for npm operations
on prepared bundled packages.
> - The benefit is that release packaging uses the artifacts that the
release already built.
## Linked Issues or Issue Description
Refs #12584
**What happened?**
The canary dry run passed bundled dependency installation and then
failed while it packed `@paperclipai/server`. npm ran the server
`prepack` script in the standalone staging directory. That script called
`pnpm run prepare:ui-dist && pnpm run build`, which requires files from
the source workspace. See the [failed canary dry-run
job](https://github.com/paperclipai/paperclip/actions/runs/33399041712/job/99510762364).
**Expected behavior**
Bundled package packing and publishing must use the artifacts that the
release already built. They must not run workspace-only package
lifecycle scripts from the standalone staging directory.
**Steps to reproduce**
1. Build the Paperclip workspace.
2. Prepare the bundled server package in a temporary directory.
3. Run npm pack from that directory.
4. Observe that npm runs the server `prepack` script outside the source
workspace.
**Paperclip version or commit**
`08af15bd7629790a618e8787c11490c96a1b619a`
## What Changed
- Add `--ignore-scripts` to npm pack for prepared bundled packages.
- Add `--ignore-scripts` to both normal and no-provenance npm publish
attempts for prepared bundled packages.
- Update release helper tests to require this behavior.
## Verification
- `node --test scripts/acpx-patch-packaging.test.mjs
scripts/release-lib.test.mjs` (22 passed)
- `pnpm test:release-registry` (98 passed)
- `git diff --check`
- The full test suite and build were not run locally. GitHub runs the
canary dry run and full matrix.
## Risks
- Low risk. The flag applies only to bundled packages that the release
prepares after the workspace build.
- Normal pnpm package publishing is unchanged.
- Package lifecycle scripts remain in the published manifest.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with `gpt-5.6-sol`. The model used agentic reasoning,
tool use, and code execution. The context window size is not exposed in
this environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
> Follow-up to #11006 (merged): rebased onto master and ready for
review.
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem now publishes canary (every master push),
nightly (scheduled, smoke-gated, added in #11006), and stable (manual)
> - There is still no human-approved release-candidate lane between
nightly and stable, and nothing enforces that a stable actually soaked
anywhere before shipping
> - Betas need a real approval gate, and stables need a soak policy that
is data, not prose
> - This pull request adds the beta channel: a manual promotion of a
chosen nightly behind the `npm-beta` environment gate, re-smoked after
publish, plus a stable preflight that enforces a 3-day beta soak with a
written-justification bypass
> - The benefit is a complete canary → nightly → beta → stable train
where every stable shipped as a beta first, and emergencies leave a
written trace
## Linked Issues or Issue Description
**Subsystem affected**
Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.
**Problem or motivation**
After #11006 the project has canary and nightly prerelease lanes, but no
release-candidate lane. Stable promotion has no enforced soak: any ref
can ship as stable directly. There is no approval boundary for a
broader-audience prerelease, and no structured way to record why an
emergency release skipped validation.
**Proposed solution**
Add a `beta` channel: a manual dispatch that promotes a chosen nightly's
source commit, publishes behind the `npm-beta` GitHub environment
(required reviewers are the gate), re-smokes the published beta, and
tags `beta/vX`. Enforce in the stable path that the source commit
shipped as a beta at least 3 days earlier (measured from the beta's npm
publish time), with a `skip_soak_justification` input as the recorded
emergency bypass.
**Alternatives considered**
Codifying the soak policy in docs only. Rejected: an unenforced policy
decays; the preflight makes the policy executable while the
justification input keeps the emergency path usable and auditable.
## What Changed
- `scripts/release.sh` + `scripts/release-lib.sh`: `beta` channel —
requires HEAD to carry a `nightly/v*` tag, publishes the package set as
`YYYY.MDD.P-beta.N` under dist-tag `beta`, tags
`beta/vYYYY.MDD.P-beta.N`
- `.github/workflows/release.yml`:
- `channel: beta` dispatch path: `select_beta` resolves the newest (or
an explicit `source_version`) nightly and fails loudly on selection
problems; `publish_beta` runs behind the `npm-beta` environment, pushes
the tag, and dispatches `docker.yml`; `smoke_beta` re-runs the release
smoke suite against the exact published beta version
- stable path: new `preflight_stable` job enforces the 3-day beta soak
from the beta's npm publish time; `skip_soak_justification` bypasses
with the reason echoed into the job summary; dry runs report without
blocking
- `.github/workflows/docker.yml`: `beta/v*` tags publish `:beta` on both
images, with exact version stamping
- `.github/workflows/release-smoke.yml`: `beta` added to the dispatch
choice list
- Docs: `CHANNELS.md` beta entries; `RELEASING.md` beta lane, soak gate,
and failure playbook; `RELEASE-AUTOMATION-SETUP.md` `npm-beta`
environment setup, including the warning to create the environment
before the first beta dispatch (GitHub auto-creates unprotected
environments on first reference)
- Tests: beta version-counting coverage in
`scripts/release-registry-versions.test.mjs`; beta identity and
nightly-tag guard coverage in
`scripts/__tests__/release-dry-run-notes.test.mjs`
## Verification
- `node --test` on the two touched suites: 17 pass, including the 3 new
beta tests
- `bash -n` on both shell scripts and YAML parse of all three workflows
- After merge, in order: create the `npm-beta` environment, dispatch
`channel: beta` with `dry_run: true` to preview, then a real promotion
of a published nightly through the approval gate, then a stable dry-run
against a young beta to see the soak gate report
## Risks
- If the `npm-beta` environment does not exist when the first beta
dispatch runs, GitHub creates it with no protection rules and the beta
publishes without approval. Mitigated by documentation and by creating
the environment before merge (operator step)
- Until the first beta exists, every stable dispatch requires
`skip_soak_justification`. This is deliberate — the first beta ships
immediately after this merges — but it is a behavior change to the
stable dispatch
- The soak clock reads the beta's npm publish time from the registry; a
registry outage makes the preflight fall back to requiring justification
(fail-closed)
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem publishes the `paperclipai` npm package set
and the Docker images on two lanes: canary on every master push, and
stable on manual promotion
> - There is no middle ground between those lanes. Users must track
every merge or wait weeks for a stable. Docker `:latest` also tracks
master, so Docker users have no stable image at all
> - A calm prerelease lane needs to exist, and it must never ship a
build that failed its checks
> - This pull request adds the nightly channel: a scheduled job that
selects the newest master commit with a green canary publish, runs the
full release smoke suite against that exact published canary, and only
then republishes it as the nightly. It also separates Docker tags by
lane, so `:latest` finally means stable
> - The benefit is that users can follow prereleases at a nightly
cadence with a smoke-tested guarantee, and Docker users get real
`:canary`, `:nightly`, and stable image tags
## Linked Issues or Issue Description
**Subsystem affected**
Release automation: `scripts/release.sh`, `scripts/release-lib.sh`,
`.github/workflows/release.yml`, `.github/workflows/docker.yml`,
`.github/workflows/release-smoke.yml`.
**Problem or motivation**
The project publishes only `canary` (every master push) and `latest`
(manual stable). Users who want prereleases without per-merge churn have
no option. Docker has a second problem: master builds overwrite
`:latest`, and CI-published stables never produced Docker images,
because tags pushed with `GITHUB_TOKEN` do not fire the `v*` tag trigger
in `docker.yml`. No stable-versioned image exists in ghcr today.
**Proposed solution**
Add a `nightly` channel. A scheduled job selects the newest
canary-tagged master commit, smoke-tests that exact published canary,
and republishes the same commit as `YYYY.MDD.P-nightly.N` under the
`nightly` dist-tag. Separate Docker tags by lane (`:canary` for master,
`:nightly` for nightly tags, `:latest` plus version tags for stable tags
only), and have the release jobs dispatch `docker.yml` at the new tag so
lane images actually build.
**Alternatives considered**
Moving the `nightly` dist-tag to the existing canary version without a
republish. Rejected: the version string would say `canary` while the
user is on nightly, which breaks at-a-glance lane identification in bug
reports and `--version` output.
## What Changed
- `scripts/release-lib.sh`: channel-parameterized
`next_prerelease_version` and `prerelease_tag_name` helpers (canary
helpers delegate to them), a `require_channel_tag_at_head` guard, and
the no-provenance retry for Sigstore transparency-log duplicates now
covers the `nightly` dist-tag as well as `canary`
- `scripts/release.sh`: new `nightly` channel. It requires HEAD to carry
a `canary/v*` tag, publishes the full public package set as
`YYYY.MDD.P-nightly.N` under dist-tag `nightly`, and tags the source
commit `nightly/vYYYY.MDD.P-nightly.N`
- `.github/workflows/release.yml`: scheduled nightly chain (09:00 UTC) —
select candidate, smoke it via `release-smoke.yml`, publish on green
under the existing `npm-canary` environment, push the tag, dispatch
`docker.yml`. New `channel` dispatch input (default `stable`, so
existing stable dispatches are unchanged) with `nightly_source_version`
and `dry_run` support for forced runs. The stable path now also
dispatches `docker.yml` at the new `v*` tag
- `.github/workflows/docker.yml`: lane tag mapping for both image jobs —
master pushes publish `:canary` and no longer move `:latest`;
`nightly/v*` tags publish `:nightly`; only stable `v*` tags publish
`:latest` and the versioned tags. New `workflow_dispatch` trigger for
the release-job dispatches. Build-version stamping uses the exact
nightly version on nightly tag builds
- `.github/workflows/release-smoke.yml`: `nightly` added to the dispatch
choice list
- `doc/CHANNELS.md` (new): user-facing guide to the channels
- `doc/RELEASING.md`: nightly lane documentation, Docker tag mapping
table, and a nightly failure playbook
- `doc/RELEASE-AUTOMATION-SETUP.md`: note that nightly reuses
`npm-canary` and needs no npm trusted-publisher changes
- Tests: channel-parameterized version helper coverage in
`scripts/release-registry-versions.test.mjs`, and nightly flow coverage
(publish identity, notes not required, canary-tag guard) in
`scripts/__tests__/release-dry-run-notes.test.mjs`
## Verification
- `node --test` on the release script suites: 68 pass, including 6 new
tests. The only failure, `acpx-patch-packaging.test.mjs`, needs
installed `node_modules` and fails identically on a pristine checkout of
master in the same environment
- `bash -n` on both shell scripts and YAML parse of all three workflows
- Live fail-path check: `./scripts/release.sh nightly --print-version`
from a master tip with no canary tag fails with `HEAD has no canary/v*
tag`
- Live success-path check: the same command from the
`canary/v2026.806.0-canary.7` commit prints `2026.806.0-nightly.0`
- Live selection check: the candidate-selection shell logic run against
the real repository selects the commit of `canary/v2026.806.0-canary.7`,
which matches the current npm `canary` dist-tag exactly
- After merge: dispatch `release.yml` with `channel: nightly` and
`dry_run: true` to preview, then a real forced run to validate end to
end before the first scheduled run
## Risks
- Docker `:latest` changes meaning from "latest master build" to "latest
stable release". This is deliberate and will be announced. Users who
want the old behavior pull `:canary`. Until the first stable release
after this change, `:latest` stays at its current (master-built) image
- The nightly is a rebuild of the same source commit, not the
byte-identical canary artifact that was smoked. The lockfile pins
dependencies, and the publish path's registry-visibility and
clean-prefix install gates still run on the nightly artifacts
- All npm publishing must stay inside `release.yml` because npm trusted
publishing pins that workflow file per package. The nightly jobs were
added to `release.yml` for exactly that reason; this constraint is now
documented in `RELEASING.md`
- The stable-lane Docker dispatch fails gracefully (a warning with
manual instructions) when the source ref predates `docker.yml`'s
`workflow_dispatch` trigger
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use (repository exploration, local test
execution, live registry and git verification). All code, tests, and
docs in this PR were model-authored under human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source control plane people use to manage
AI-agent companies
> - Operators need a predictable installation path that survives beyond
an ephemeral `npx` process
> - A durable installation needs an owned per-user payload store, stable
command shim, safe shell integration, and supported service lifecycle
> - Updates must preserve recoverability by backing up data, installing
side-by-side, verifying the new payload, and retaining rollback state
> - Bootstrap scripts and privileged service operations must fail closed
across download, filesystem, ownership, and consent boundaries
> - This pull request integrates managed install, update, rollback,
service, uninstall, doctor, bootstrap-installer, and runtime-serving
support into one workflow
> - The benefit is a recoverable, inspectable, and documented
installation lifecycle with explicit safety boundaries across Linux,
macOS, containers, WSL, npm, npx, and source checkouts
## Linked Issues or Issue Description
### Problem
Paperclip lacks a first-class durable installation and lifecycle
workflow. Operators currently have to assemble npm/npx installation,
PATH setup, background-service management, updates, rollback,
diagnostics, and uninstall behavior themselves. That makes upgrades
harder to recover, creates inconsistent behavior across platforms, and
leaves shell/download/service trust boundaries without one documented
implementation.
### Proposed Solution
Add a managed per-user install store and stable shim, a verified shell
bootstrap installer, service lifecycle commands, install-mode-aware
update/rollback behavior, doctor checks, and documentation. Managed
updates back up the database, install and smoke-test a side-by-side
payload, atomically switch `current`, and retain prior payloads. The
shell installer pins registry/download trust boundaries and requires
explicit consent for non-interactive privileged actions.
### Alternatives Considered
- Keep recommending `npx`: simple for evaluation, but ephemeral and
unsuitable for stable services, atomic updates, or rollback.
- Require global npm installation only: familiar, but cannot provide the
owned side-by-side payload store and retained rollback semantics.
- Split the capability across multiple PRs: rejected because install,
update, service, uninstall, bootstrap, and serving behavior share
contracts and security boundaries that need review together.
### Related Pull Requests
- Supersedes #10042 and #10044 with one integrated final diff.
- Incorporates and replaces the closed preparatory work in #10032 and
#10034.
## What Changed
- Added `paperclipai install`, `update`/`upgrade`, rollback, uninstall,
service lifecycle, onboarding integration, and managed-install doctor
checks.
- Added a private managed payload store, verified manifest/marker
ownership, exclusive mutation locks, atomic manifest/current/shim
writes, retained previous payloads, and provenance validation.
- Added npm and GitHub-ref install sources with exact target resolution,
registry isolation, database backup, side-by-side verification, atomic
activation, service restart coordination, and failure rollback.
- Made managed-update backups report actionable service-start and
`--no-backup` recovery guidance for unreachable databases, while clean
never-onboarded instances skip an empty backup.
- Added systemd user and launchd service definitions, status/health/log
commands, single-instance coordination, stale-port recovery, and
explicit sudo/lingering consent handling.
- Added the `scripts/install.sh` bootstrap path with checked two-stage
downloads, pinned public npm registry usage, platform checks,
dry-run/non-interactive controls, and Docker fixtures.
- Added embedded Postgres/native bootstrap integration,
hot-restart/systemd-notify serving support, passive update notices,
configuration contracts, README/CLI/install documentation, and focused
regression tests.
- Security re-review should explicitly re-verify: (1)
`addManagedPathBlock`/`removeManagedPathBlock` reject symlinked or
non-regular rc files, assert current-user ownership, preserve
restrictive modes, and replace atomically; (2) managed shim replacement
rejects unsafe parents, foreign-owned or multiply linked files, and uses
checked atomic replacement; (3) the shell installer and sudo path
preserve explicit consent and checked downloads; and (4) installed
service/runtime serving remains bound to the validated managed shim and
instance configuration.
## Verification
- `bash -n scripts/install.sh scripts/clean-install-git.sh
scripts/clean-install-npm.sh scripts/test-install-sh-docker.sh`
- `pnpm exec vitest run cli/src/__tests__/install-store.test.ts
cli/src/__tests__/install-command.test.ts
cli/src/__tests__/managed-install-check.test.ts
cli/src/__tests__/onboard-service.test.ts
cli/src/__tests__/service-health-check.test.ts
cli/src/__tests__/service-manager.test.ts
cli/src/__tests__/update-command.test.ts
cli/src/__tests__/update-notice.test.ts
packages/db/src/embedded-postgres-native.test.ts` — 9 files, 66 tests
passed
- `pnpm --dir cli typecheck`
- `pnpm --dir cli build`
- Follow-up verification: `pnpm exec vitest run
cli/src/__tests__/update-command.test.ts` (14/14), `pnpm --dir cli
typecheck`, `pnpm --dir cli build`, and `pnpm --filter
@paperclipai/server typecheck`.
- `pnpm -r typecheck`
- `pnpm build`
- Full `pnpm test:run` exercised all suites; an injected static AWS
credential changed one unrelated doctor expectation, which passed when
those credentials were removed. A second run cleared that case and
exposed stale pre-existing adapter-utils `dist` output; rebuilding
`@paperclipai/adapter-utils` made the isolated test pass. The updated PR
CI is the authoritative clean-workspace full-suite run.
## Risks
- Installer/update code writes executable shims, symlinks, shell rc
blocks, service definitions, and managed payloads; ownership,
regular-file, symlink, hard-link, marker, and path-containment checks
fail closed before destructive changes.
- The bootstrap installer executes downloaded tooling; downloads are
staged and checked before execution, npm traffic is pinned to the public
registry, and non-interactive privileged behavior requires explicit
consent.
- Linux lingering may invoke `sudo`; the command is surfaced and
confirmed before execution, and unsupported service managers fall back
to foreground-run guidance.
- Database migrations remain forward-only; payload rollback does not
reverse migrations, so managed updates create a backup before activation
unless explicitly disabled.
- Service restart and runtime serving touch process/port ownership;
lifecycle locks, health/version checks, and stable-shim service
definitions reduce split-brain and stale-process risk.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex coding agents using GPT-5.5 and GPT-5.6-sol, with
reasoning, repository/API access, shell execution, and test tooling. The
runtime did not expose a reliable context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Thinking Path
> - Paperclip publishes a coordinated set of packages through its
release workflows
> - Bundled packages use a pinned npm CLI so trusted publishing works
consistently
> - The canary publisher now crashes deterministically inside npm before
useful output reaches the workflow log
> - The npm debug log that contains the underlying failure disappears
with the hosted runner
> - This pull request upgrades the pinned publish CLI and preserves both
verbose HTTP activity and npm debug logs on failure
> - The benefit is that the plausible HTTP-layer fix ships immediately,
while any remaining CI-only failure becomes diagnosable
## Linked Issues or Issue Description
- **Problem:** The canary release workflow fails on the first bundled
package with `npm error Exit handler never called!` and no preceding
diagnostic output.
- **Expected behavior:** Bundled packages publish through trusted
publishing, or the workflow retains enough npm diagnostics to identify
the actual failure.
- **Reproduction:** Run the canary release workflow in GitHub Actions;
the failure reproduced on both attempts of run 29948506814.
- **Version/commit:** Current `master` after #10024 and #10030.
- **Deployment mode:** GitHub-hosted release workflow using Node.js 24
and npm trusted publishing.
- Related: #10024, #10030.
## What Changed
- Bumped the bundled publish CLI from npm 11.16.0 to npm 11.18.0.
- Added `--loglevel verbose` to bundled npm publish invocations.
- Dumped the last 300 lines of every npm debug log after failed canary
or stable publishes, with common registry credential forms redacted.
- Updated release assertions to pin npm 11.18.0 and verify verbose
logging.
## Verification
- `pnpm test:release-registry` — 66 tests passed.
- `bash -n scripts/release-lib.sh`.
- Parsed `.github/workflows/release.yml` with Python/PyYAML.
- Smoke-tested npm log redaction with representative Authorization,
`_authToken`, and token environment values.
- Smoke-tested npm debug-log redaction against Authorization,
`_authToken`, and `npm_token` examples.
- `git diff --check origin/master...HEAD`.
- The merge-triggered canary workflow remains the live
trusted-publishing verification.
## Risks
- Low code risk: changes are isolated to the release publisher and its
workflow diagnostics.
- npm 11.18.0 could expose a different registry/runtime regression;
failure-time debug log dumping makes that actionable.
- Verbose npm output increases release log volume but does not change
package contents or dist-tags; common credential forms are redacted
before debug logs are printed.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, GPT-5.3 Codex, tool-enabled coding agent with repository
and shell execution; context window size is not exposed in this
environment.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip publishes canary and stable packages through a shared
release script
> - GitHub Actions authenticates those publishes through npm trusted
publishing and an OIDC identity token
> - Bundled-dependency packages recently moved from pnpm publish to a
pinned npm CLI to preserve their bundled files
> - That pin selected npm 10, which cannot use trusted publishing, so
the first bundled package failed with `ENEEDAUTH`
> - This pull request keeps bundled-package packing on npm 10 while
routing actual publishing through the trusted-publishing-capable npm 11
version
> - The benefit is bundled packages keep their required npm packaging
behavior while canary and stable releases authenticate successfully
## Linked Issues or Issue Description
**What happened**
The canary release job failed while publishing
`@paperclipai/adapter-utils` with `ENEEDAUTH`. The package has bundled
dependencies, so the release helper selected pinned `npm@10.9.7`; the
workflow provides OIDC trusted publishing rather than an npm token, and
npm 10 cannot use that authentication path. Because this is the first
package attempted, the release exited before trying the remaining
packages.
**Expected behavior**
Bundled-dependency packages should publish with an npm CLI that both
preserves bundled dependencies and supports GitHub Actions trusted
publishing.
**Steps to reproduce**
Run the canary release workflow from master after PR #9980. The
`publish_canary` job reaches `@paperclipai/adapter-utils`, invokes `npx
npm@10.9.7 publish`, and fails with `ENEEDAUTH`.
**Deployment mode**
GitHub Actions canary and stable npm release workflows.
Refs #9980.
## What Changed
- Kept bundled-package dry-run packing on npm `10.9.7`, which
successfully produces the staged tarball.
- Routed bundled-package publishing through npm `11.16.0`, which
supports GitHub Actions trusted publishing.
- Split the pack and publish helpers so future npm changes cannot
silently couple the two compatibility requirements.
- Updated focused release and ACPX packaging tests to enforce both
versions and call paths.
## Verification
- `pnpm test:release-registry` — 67 passed locally.
- Initial all-npm-11 PR head: Canary Dry Run reproduced an npm-internal
crash during bundled `pack`.
- Current head `5b3961ed13dd26ed2d6b1096ea23fd91b32e4353`: Canary Dry
Run passed with split npm pack/publish helpers.
- All PR checks passed, including build, typecheck + release registry,
server/workspace suites, both e2e shards, security gates, and Greptile.
- Greptile reviewed the current head at 5/5 confidence with no blocking
issues.
## Risks
- Low risk: the change only separates the npm CLI used for
bundled-package packing from the CLI used for publishing.
- The versions remain explicitly pinned because npm 11.16.0 currently
crashes on the bundled pack payload, while npm 10.9.7 cannot perform
trusted publishing.
- Focused tests assert both pins and both helper call paths, and the
full Canary Dry Run passes on the current head.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, with
repository, shell, GitHub CLI, and code-execution tools.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Local Claude, Codex, Gemini, and custom ACP adapters run through the
shared embedded ACPX engine
> - That engine wrapped every local agent command in a generated Bash
script to inject environment variables and filter child stderr
> - Windows cannot directly spawn that Bash wrapper, and npm/pnpm ACP
binaries are exposed through `.cmd` shims there
> - ACPX 0.12 already supports per-session child environment variables,
so the wrapper is unnecessary
> - This pull request registers agent commands directly, injects env
through ACPX session options, captures child stderr in-process, and adds
a real Node ACP spawn smoke on Ubuntu and Windows
> - The benefit is one cross-platform spawn path with a reusable smoke
test instead of parallel shell-wrapper implementations
## Linked Issues or Issue Description
Fixes#9941.
Refs #9428 and #9771.
**What happened**
ACPX-backed local agents failed to start on Windows because Paperclip
registered a generated POSIX `.sh` wrapper as the agent command. Windows
also needs the `.cmd` npm/pnpm shim when resolving built-in ACP
binaries, and symlink creation can fail with `EPERM` for seeded
auth/skill files.
**Expected behavior**
The same ACPX engine path should spawn a real ACP agent on Windows and
Linux, forward Paperclip/runtime env without mutating `process.env`,
preserve filtered/unfiltered child stderr behavior, and fall back to
copies where Windows symlinks are unavailable.
**Steps to reproduce**
Run a local ACPX adapter on Windows with the prior wrapper path. ACPX
attempts to spawn the generated `.sh` file and the agent never
initializes.
**Deployment mode**
Local Paperclip adapters using
`packages/adapter-utils/src/acpx-engine/`.
## What Changed
- Removed generated Bash agent/env wrappers and registered local
commands directly with ACPX.
- Passed the resolved child environment through ACPX
`sessionOptions.env`, including resume retry paths.
- Added a minimal `acpx@0.12.0` package patch exposing child stderr
callbacks and allowing documented uppercase env-map keys in persisted
session options.
- Moved stderr tee/filter behavior in-process: raw stderr remains in the
per-run file while benign `nes/close` noise is omitted from live stderr.
- Preferred `.cmd` ancestor binaries on Windows and added `EPERM` copy
fallbacks for Codex auth seeding and Gemini skill materialization.
- Added a real Node ACP echo-agent spawn smoke that can run directly on
any supported platform.
## Verification
- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/execute.test.ts
packages/adapter-utils/src/acpx-engine/spawn-smoke.test.ts` — 57 passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — passed.
- `node --test scripts/acpx-patch-packaging.test.mjs
scripts/release-lib.test.mjs` — 10 passed.
- Full canary release dry run under Node 24.18.0 / npm 11.16.0 — passed
in an isolated scratch clone.
- `git diff --check` — passed during implementation verification.
- One-time GitHub Actions proof: [Ubuntu ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774579),
[Windows ACPX spawn
smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774558),
and [Canary Dry
Run](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774497)
passed on head `f345ac69f2`; the dedicated smoke jobs are intentionally
not retained in the recurring PR workflow.
## Risks
- The ACPX stderr callback and env persistence exemption are carried as
a pnpm dependency patch until ACPX exposes/fixes those behaviors
upstream.
- Child stderr is synchronously appended to preserve ordering and
failure diagnostics; unusually high-volume agent stderr could briefly
block the Node event loop.
- The Windows-specific `.cmd` resolution and symlink `EPERM` branches
are proven by the standalone smoke test and the linked one-time
`windows-latest` run rather than a permanent CI gate.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI GPT-5.4 via Codex CLI, medium reasoning, repository/tool
execution enabled; context-window size is not exposed in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The release subsystem publishes the public workspace packages and
also powers release-related CI validation.
> - The release flow currently asks npm for package versions one package
at a time in multiple places.
> - That serial registry latency slows the PR Canary Dry Run path and
real release invocations even though the checks are independent.
> - This pull request batches npm registry version lookups with bounded
concurrency and reuses the result for version calculation.
> - The benefit is shorter non-build release-script time while
preserving the fresh target-version existence check before publishing.
## Linked Issues or Issue Description
- No public GitHub issue exists for this release-script performance
cleanup.
### Problem or motivation
Release validation spends avoidable time on repeated serial `npm view`
calls across the public package set. The slow path affects PR release
validation and real release invocations because version discovery waits
on independent registry reads one at a time.
### Proposed solution
Fetch package version maps concurrently with bounded parallelism, reuse
that map for stable/canary version calculation, and keep a fresh
parallel absence check for the target publish version.
### Alternatives considered
Keeping the existing serial shell loop is simpler, but it preserves the
CI latency cost. Caching the final target-version existence check was
rejected because release publish safety should still query npm freshly
before publishing.
### Roadmap alignment
This is a small release-tooling performance improvement. It does not
duplicate any planned core product work found in `ROADMAP.md`.
## What Changed
- Added `scripts/release-registry-versions.mjs` to fetch npm package
version maps and assert target-version absence with bounded parallelism.
- Updated `scripts/release.sh` to prefetch package versions once and to
batch the final target-version absence check.
- Updated `next_stable_version` and `next_canary_version` to use the
prefetched version map when present, with the existing per-package npm
fallback preserved.
- Added release-registry helper coverage and included it in `pnpm run
test:release-registry`.
- Hardened the release publish helper tests so their fake `pnpm`/`npm`
fixture PATH is preserved under non-login shell execution.
## Verification
- `node --test scripts/release-registry-versions.test.mjs`
- `pnpm run test:release-registry`
- `bash -n scripts/release.sh scripts/release-lib.sh`
- `git diff --check`
- Safety scan before push: searched changed files for common
key/token/password patterns and PII markers; only benign script-name
text matched (`secrets:migrate-inline-env`).
- Remote PR checks on the latest head passed, including `Typecheck +
Release Registry`, `Canary Dry Run`, build, tests, e2e, policy, security
scans, and commitperclip review.
- Greptile reviewed the latest head with Confidence Score 5/5 and no
blocking issues.
## Risks
- Low risk. The release version helpers keep their original npm fallback
when no prefetched version map is supplied.
- The existence check remains fresh and uncached before publish, but now
reports all matching package/version pairs from a parallel check.
- If npm has transient failures during the prefetch step, missing or
failed packages still map to an empty version list, matching the old
helper behavior.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex coding agent using GPT-5, with shell/tool execution in
the local repository.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The release workflow publishes canary npm packages on every push to
`master`
> - The failing canary job built successfully and published several
packages before npm failed on `@paperclipai/mcp-server`
> - The concrete failure was npm trusted-publishing provenance returning
`TLOG_CREATE_ENTRY_ERROR` because an equivalent Sigstore
transparency-log entry already existed
> - The package version was not visible on npm afterward, so the release
script could not safely treat that error as success by itself
> - This pull request adds a narrow recovery path for that npm
provenance failure and keeps the existing registry verification as the
final source of truth
> - The benefit is that transient duplicate transparency-log failures do
not break canary publication when a package can be republished without
provenance or is already visible on npm
## Linked Issues or Issue Description
Bug fix, no public GitHub issue found in duplicate search.
- What happened: the Release workflow canary publish failed in
`publish_canary` after npm returned `TLOG_CREATE_ENTRY_ERROR` while
publishing `@paperclipai/mcp-server@2026.609.0-canary.2`.
- Expected behavior: canary publishing should either recover from npm's
duplicate transparency-log failure when the package can still be
published, or fail later in registry verification if the package never
appears.
- Steps to reproduce: inspect
https://github.com/paperclipai/paperclip/actions/runs/27230012891/job/80411422155
from push `05cb18cf28074a6d1074c7575c5a44133146e368`.
- Deployment mode: GitHub Actions Release workflow, npm trusted
publishing.
- Duplicate search: no open PRs or issues found for `canary publish TLOG
provenance release` or the failing run/job IDs.
## What Changed
- Added `publish_package_to_npm` in `scripts/release-lib.sh` to wrap
canary/stable package publishing.
- Detects npm's duplicate Sigstore transparency-log error and checks
whether the package version is already visible on npm.
- Retries that exact package once with `--provenance=false` when npm hit
the duplicate tlog error but the version is not visible yet.
- Keeps unrelated publish failures as hard failures.
- Added shell-helper tests with fake `pnpm` and `npm` commands, and
included them in `pnpm test:release-registry`.
## Verification
- `node --test scripts/release-lib.test.mjs`
- `pnpm test:release-registry`
- Confirmed `pnpm publish --dry-run --no-git-checks --tag canary
--access public --provenance=false` is accepted by pnpm 9.15.4.
## Risks
- Low risk: the recovery only triggers when npm output contains both
`TLOG_CREATE_ENTRY_ERROR` and the duplicate transparency-log message.
- Publishing without provenance is a fallback for canary continuity; if
npm still does not expose the package, the existing registry
verification step still fails the release.
- The same helper is used by stable publishing too, but only for this
exact npm provenance failure path.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
This is a release reliability bug fix. I checked `ROADMAP.md`; it does
not duplicate planned core product work.
## Model Used
OpenAI Codex coding agent, GPT-5-class model, tool-enabled local shell
and GitHub CLI workflow, medium reasoning mode.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Its release automation publishes canary packages to npm and then
validates the published registry state before considering the release
healthy
> - The failing canary run `25139465018` showed that npm can expose a
newly published version through version-specific endpoints before the
root package document has fully converged
> - That made a successful canary publish look like a failed release
because the verifier trusted stale root metadata too early
> - This pull request hardens the registry verification path by
preferring version-specific manifest checks, retrying
convergence-sensitive failures, and distinguishing permanent failures
from propagation lag
> - While validating that change in CI, a separate teardown race in
`heartbeat-stale-queue-invalidation.test.ts` surfaced and was hardened
so the PR could pass reliably
> - The benefit is that transient npm propagation lag no longer fails a
successful canary publish, while genuine registry-state and
dependency-integrity failures still stop the release flow promptly
## What Changed
- Hardened `scripts/verify-release-registry-state.mjs` so it prefers
version-specific manifest resolution over stale root metadata, adds
bounded registry-fetch timeouts, and classifies failures as retriable vs
non-retriable.
- Updated `scripts/release-lib.sh` and `scripts/release.sh` so
post-publish registry verification retries only convergence-sensitive
failures and reports immediate permanent failures clearly.
- Expanded `scripts/verify-release-registry-state.test.mjs` with
regression coverage for stale root metadata, fetch timeout behavior,
peer dependency range handling, non-retriable canary-latest cases, and
related verifier edge cases.
- Hardened
`server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts`
teardown to tolerate the late-comment foreign-key race that CI exposed
while validating this branch.
## Verification
- `pnpm run test:release-registry`
- `node --check scripts/verify-release-registry-state.mjs`
- `bash -n scripts/release.sh && bash -n scripts/release-lib.sh`
- PR checks passed on head `5c422600fc12acac61f6b7c267a4dc915df622b1`:
`policy`, `verify`, `e2e`, `security/snyk`, and `Greptile Review`
## Risks
- Low risk. The main behavioral changes are limited to release
automation and verifier retry semantics, plus a test-only teardown
hardening for a CI race.
> I checked [`ROADMAP.md`](ROADMAP.md). This is a narrow release bugfix
and does not overlap planned core feature work.
## Model Used
- OpenAI Codex via Paperclip `codex_local` with tool use and local code
execution enabled. This agent session runs on a GPT-5-class coding
model; the exact backend model ID/context window is not exposed by the
local adapter runtime.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I have addressed all Greptile and reviewer comments before
requesting merge
Dotta
Devin Foley
dotta
Dotta