paperclip/scripts
Devin Foley ef37203a48
perf(ci): build standalone public packages concurrently (#8567)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - CI runs a Canary Dry Run job that exercises `release.sh`, which
builds the standalone sandbox-provider packages for publish
> - That step (`scripts/build-standalone-public-packages.mjs`) built the
7 provider plugins serially — each doing `rm -rf dist && tsc` — making
it the dominant cost (~49s) inside the slowest PR check (~4.9m wall)
after the general-server lane was already sharded
> - The packages are independent (their own `node_modules` via
`--ignore-workspace`, their own `dist`), so the serial build is pure
latency with no correctness benefit
> - This pull request builds them with a bounded-concurrency pool sized
to the runner CPU count (overridable via
`STANDALONE_BUILD_CONCURRENCY`), buffering each package's output and
flushing it as one block so parallel logs stay readable, and aggregating
failures by original index
> - The benefit is a faster Canary Dry Run / PR feedback loop without
changing what gets built or published

## Linked Issues or Issue Description

No public GitHub issue exists. Inline feature/perf description:

### Problem or motivation

`build-standalone-public-packages.mjs` builds standalone provider
packages serially, making it the largest single cost inside the slowest
PR check.

### Proposed solution

Run independent per-package builds through a bounded-concurrency worker
pool sized to runner CPU count, with an env override and readable
buffered logs.

### Alternatives considered

Keep the serial build for simpler logs, but that preserves the avoidable
CI latency.

### Roadmap alignment

This is CI maintenance and does not overlap planned core roadmap work.

## What Changed

- `scripts/build-standalone-public-packages.mjs`: replaced the serial
per-package build loop with a bounded-concurrency pool (default = runner
CPU count, override via `STANDALONE_BUILD_CONCURRENCY`); per-package
stdout/stderr is buffered and flushed as a single block; failures are
aggregated by original package index so one failure neither aborts the
others mid-flight nor obscures which package broke.
- `scripts/__tests__/build-standalone-concurrency.test.mjs`: new
`node:test` unit suite covering the pool (limit respected, all items
run, ordered failure aggregation, env-override resolution).
- `.github/workflows/pr.yml`: wired the new unit test into the policy
job.

## Verification

- `node --test
./scripts/__tests__/build-standalone-concurrency.test.mjs` → 6/6 pass
- `node ./scripts/release-package-map.mjs check` → OK (29 enabled for CI
publish)
- `git diff --check origin/master..HEAD` → clean

## Risks

- Low risk. Build inputs/outputs are unchanged; only scheduling differs.
The concurrency is bounded by CPU count and overridable; output is
buffered per package so logs remain attributable. If a package fails,
all failures are still reported with their package index.

## Model Used

- Claude (Anthropic), `claude-opus-4-8`, extended thinking with tool
use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-23 17:52:12 -07:00
..
__tests__ perf(ci): build standalone public packages concurrently (#8567) 2026-06-23 17:52:12 -07:00
smoke [codex] Move maintainer task skills under .agents (#7658) 2026-06-06 10:47:59 -05:00
backfill-issue-reference-mentions.ts Add first-class issue references (#4214) 2026-04-21 10:02:52 -05:00
backup-db.sh Add paperclipai db:backup CLI command 2026-03-05 06:02:12 -06:00
bootstrap-npm-package.mjs fix(ci): gate new release packages on npm bootstrap (#5146) 2026-05-03 19:31:28 -07:00
bootstrap-npm-package.test.mjs [codex] Add workspace diff viewer plugin (#6071) 2026-05-18 08:50:06 -05:00
build-npm.sh fix: validate canary release path in CI 2026-03-17 15:35:59 -05:00
build-standalone-public-packages.mjs perf(ci): build standalone public packages concurrently (#8567) 2026-06-23 17:52:12 -07:00
capture-acpx-skills-screenshots.mjs Add ACPX local adapter runtime (#4893) 2026-04-30 19:57:05 -05:00
capture-pap-2351-binding-picker.mjs Add secrets provider vaults and remote import (#5429) 2026-05-09 18:22:17 -05:00
check-docker-deps-stage.mjs Add E2B sandbox provider plugin (#4452) 2026-04-25 11:01:11 -07:00
check-forbidden-tokens.mjs Redact current user in comments and token checks 2026-03-11 22:17:21 -05:00
check-no-git-push.mjs PAPA-430: workspace finalize gates + no-remote-git enforcement (#6969) 2026-05-29 08:25:29 -07:00
check-no-git-push.test.mjs PAPA-430: workspace finalize gates + no-remote-git enforcement (#6969) 2026-05-29 08:25:29 -07:00
check-release-package-bootstrap.mjs fix(ci): gate new release packages on npm bootstrap (#5146) 2026-05-03 19:31:28 -07:00
check-release-package-bootstrap.test.mjs fix(ci): gate new release packages on npm bootstrap (#5146) 2026-05-03 19:31:28 -07:00
clean-onboard-git.sh http clone 2026-03-09 10:03:45 -05:00
clean-onboard-npm.sh scripts 2026-03-03 14:00:10 -06:00
clean-onboard-ref.sh feat: add committed-ref onboarding smoke script 2026-03-09 09:59:43 -05:00
create-github-release.sh fix: use origin for github release creation in actions 2026-03-18 09:10:00 -05:00
dev-runner-output.mjs Guard dev health JSON parsing 2026-04-06 21:23:33 -05:00
dev-runner-output.ts fix(runtime): handle empty dev runner responses 2026-04-06 21:56:13 -05:00
dev-runner-paths.mjs [codex] Add backup endpoint and dev runtime hardening (#4087) 2026-04-20 06:08:55 -05:00
dev-runner-snapshot.mjs Address dev runner snapshot review feedback 2026-06-01 22:03:51 +00:00
dev-runner.mjs [codex] Add skills CLI and catalog management (#6782) 2026-05-28 07:33:51 -10:00
dev-runner.ts Add dev runner snapshot race regression test 2026-06-01 21:55:30 +00:00
dev-service-profile.ts Introduce bind presets for deployment setup 2026-04-11 07:09:07 -05:00
dev-service.ts Add idempotent local dev service management 2026-03-29 10:55:26 -05:00
discord-daily-digest.sh [codex] Harden heartbeat scheduling and runtime controls (#4223) 2026-04-21 12:24:11 -05:00
docker-build-test.sh chore(docker): improve base image and organize docker files 2026-04-01 11:36:27 +00:00
docker-entrypoint.sh fix: skip gosu when already running as target user (#2908) 2026-06-12 10:33:52 -07:00
docker-onboard-smoke.sh chore(docker): improve base image and organize docker files 2026-04-01 11:36:27 +00:00
ensure-plugin-build-deps.mjs [codex] Add skills CLI and catalog management (#6782) 2026-05-28 07:33:51 -10:00
ensure-workspace-package-links.ts [codex] fix worktree dev dependency ergonomics (#3743) 2026-04-15 09:47:29 -05:00
generate-company-assets.ts Address Greptile review on portability PR 2026-03-23 17:23:59 -05:00
generate-npm-package-json.mjs fix: add npm provenance package metadata 2026-03-17 16:01:48 -05:00
generate-org-chart-images.ts Add standalone Playwright-based org chart image generator 2026-03-20 05:55:33 -05:00
generate-org-chart-satori-comparison.ts feat: multi-style pure SVG org chart renderer (no Playwright needed) 2026-03-20 06:33:29 -05:00
generate-plugin-package-json.mjs Add E2B sandbox provider plugin (#4452) 2026-04-25 11:01:11 -07:00
generate-ui-package-json.mjs Publish @paperclipai/ui from release automation 2026-03-26 11:13:11 -05:00
kill-agent-browsers.sh [codex] Stabilize tests and local maintenance assets (#4423) 2026-04-24 15:11:42 -05:00
kill-dev.sh kill chrome test servers too 2026-04-09 06:14:12 -05:00
kill-vitest.sh [codex] Harden heartbeat scheduling and runtime controls (#4223) 2026-04-21 12:24:11 -05:00
kill-workspaces.sh Add workspace runtime kill script 2026-06-05 03:28:07 +00:00
link-plugin-dev-sdk.mjs fix(plugins): move dev SDK linking out of plugin postinstall scripts (#8255) 2026-06-18 07:45:53 -07:00
link-plugin-dev-sdk.test.js fix(plugins): move dev SDK linking out of plugin postinstall scripts (#8255) 2026-06-18 07:45:53 -07:00
measure-issue-chat-long-thread.mjs Improve issue thread scale and markdown polish (#4861) 2026-04-30 13:18:01 -05:00
migrate-inline-env-secrets.ts refactor: rename packages to @paperclipai and CLI binary to paperclipai 2026-03-03 08:45:26 -06:00
paperclip-commit-metrics.ts feat: implement multi-user access and invite flows (#3784) 2026-04-17 09:44:19 -05:00
paperclip-issue-update.sh chore: improve worktree tooling and security docs 2026-04-10 22:26:30 -05:00
prepare-server-ui-dist.sh ci: speed up PR verify workflow (#6137) 2026-05-16 11:28:25 -07:00
provision-worktree.sh External object references across issue surfaces (#8512) 2026-06-23 08:27:19 -05:00
release-lib.sh [codex] Recover duplicate npm provenance canary publishes (#7839) 2026-06-09 15:35:07 -05:00
release-lib.test.mjs [codex] Recover duplicate npm provenance canary publishes (#7839) 2026-06-09 15:35:07 -05:00
release-package-manifest.json fix(release): publish all remaining @paperclipai workspace packages from CI + guard unpublishable edges (#8365) 2026-06-19 19:11:20 -07:00
release-package-map.mjs fix(release): publish all remaining @paperclipai workspace packages from CI + guard unpublishable edges (#8365) 2026-06-19 19:11:20 -07:00
release-package-map.test.mjs fix(release): publish all remaining @paperclipai workspace packages from CI + guard unpublishable edges (#8365) 2026-06-19 19:11:20 -07:00
release.sh [codex] Recover duplicate npm provenance canary publishes (#7839) 2026-06-09 15:35:07 -05:00
rollback-latest.sh chore: switch release calver to mdd patch 2026-03-18 07:57:36 -05:00
run-typecheck-build-gaps.mjs ci: speed up PR verify workflow (#6137) 2026-05-16 11:28:25 -07:00
run-vitest-stable.mjs perf(ci): shard the general-server test lane across 3 runners (#8360) 2026-06-19 19:22:17 -07:00
screenshot-blocked-inbox.mjs [codex] Add blocked inbox attention view (#5603) 2026-05-13 16:41:36 -05:00
screenshot-file-viewer.mjs Add workspace file viewer and artifact links (#7681) 2026-06-09 17:17:43 -05:00
screenshot-one.mjs Add workspace file viewer and artifact links (#7681) 2026-06-09 17:17:43 -05:00
screenshot-pap2373.mjs Cancel stale queued heartbeats when issue graph changes (PAP-2314) (#4534) 2026-04-26 21:17:38 -05:00
screenshot-recovery-card.cjs [codex] Add source-scoped recovery actions (#5599) 2026-05-12 09:37:15 -05:00
screenshot-subissues.mjs Present ordered sub-issues as a workflow checklist (#4523) 2026-04-26 07:36:49 -05:00
screenshot.cjs feat: add authenticated screenshot utility (#2622) 2026-04-03 10:51:26 -07:00
verify-release-registry-state.mjs fix: harden release registry verification against npm lag (#4816) 2026-05-09 22:18:12 -07:00
verify-release-registry-state.test.mjs fix: harden release registry verification against npm lag (#4816) 2026-05-09 22:18:12 -07:00