Commit Graph

173 Commits

Author SHA1 Message Date
Devin Foley c5c80e1feb
ci(release-verify): split server tests five ways like pr-trusted (#13185)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Every master push publishes a canary through release.yml, gated by
release-verify.yml — the fleet's staging deploys and the
nightly/beta/stable chain all start from those canaries
> - release-verify splits the server test suite across three shards with
a 20-minute job cap, while pr-trusted splits the same suite across five
> - The server suite grew on 2026-09-10 and the three shards moved to
17-19 minutes; that evening every push-triggered canary run was
cancelled by the 20-minute cap mid-verify, and no canary published after
18:50 UTC
> - This pull request mirrors pr-trusted's five-way server split in
release-verify, putting shards back at the 10-15 minute range with real
headroom
> - The benefit is a canary lane that reports test verdicts instead of
dying on an infrastructure cap

## Linked Issues or Issue Description

**What happened?**

Push-triggered Release runs stopped publishing canaries on 2026-09-10.
Runs at 19:34, 22:30, and 22:37 UTC were all cancelled by "The job has
exceeded the maximum execution time of 20m0s" on a `verify_canary /
General tests (server (N/3))` shard. No canary published after 18:50
UTC, which also starves the staging fleet's continuous deploys.

**Expected behavior**

release-verify's server shards finish well inside the 20-minute cap and
runs conclude with a test verdict, as pr-trusted's five-way split of the
same suite does (10-15 minutes per shard).

**Steps to reproduce**

1. Compare server shard durations in the `verify_canary` job across
2026-09-10: 11-14 minutes in the morning, 17-19 minutes from 15:06 UTC,
over 20 minutes by evening.
2. Observe runs 34521169020, 34537798488, and 34538332689 cancelled at
the cap.

**Paperclip version or commit**

`master` at `d1ba17eec` (current tip; its canary run was one of the
cancelled ones).

## What Changed

- `release-verify.yml`: the `general-server` matrix goes from three
shards to five, byte-for-byte the shape `pr-trusted.yml` already runs,
with a comment recording why.

## Verification

- The identical five-way split runs green on every pr-trusted run (10-15
minutes per shard today, including on PRs merged this evening).
- The suite's own growth (slower chat-connector tests) is being
addressed separately; this PR only removes the artificial cliff.

## Risks

- Low risk: two more runners per verify run; no test content changes. If
shard durations regress further, the cap fires again — which is the
correct signal once shards have honest headroom.

## Model Used

- Claude (Anthropic), model ID `claude-fable-5` (Claude Fable 5),
extended thinking, tool use via Claude Code CLI.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-10 17:47:00 -07:00
Nicky Leach 2a05b5ed34
ci: split runner verification from build (#13142)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip uses GitHub Actions to verify changes before release.
> - The Paperclip Runner has a separate verification boundary.
> - The build job currently runs this verification before the workspace
build.
> - This pull request moves runner verification into its own parallel
job.
> - The benefit is clearer CI results and less wait time for independent
work.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The trusted PR and release verification workflows run Paperclip Runner
verification inside the Build job.

**Subsystem affected**

Cross-cutting (GitHub Actions CI workflows).

**Current behavior**

The Build job runs `pnpm --filter @paperclipai/paperclip-runner
check:all` before it builds the workspace. A runner verification failure
appears as a Build failure. The workspace build cannot run in parallel
with runner verification.

**Proposed behavior**

Each workflow has a `Verify Paperclip Runner` job with the same
checkout, dependency install, and command. The Build job only builds its
required outputs. Both jobs run after the same gate and policy jobs.

**Reason and benefit**

The runner command is an independent verification boundary. A dedicated
job gives it a clear status and allows it to run in parallel with Build.

**Breaking changes**

None. The same runner verification command still runs in both workflows.

## What Changed

- Added a dedicated `Verify Paperclip Runner` job to the trusted PR
workflow.
- Added a dedicated `Verify Paperclip Runner` job to the release
verification workflow.
- Kept the Build jobs independent and retained their existing build
commands.
- Updated the trusted-workflow policy test for the additional
dependency-install job.

## Verification

- Ran `git diff --check`.
- Ran `node --test ./scripts/__tests__/e2e-shard.test.mjs`.
- Ran `pnpm exec prettier --check .github/workflows/pr-trusted.yml
.github/workflows/release-verify.yml`.
- Confirmed both jobs retain their prior runner, dependency, and policy
prerequisites.

## Risks

Low risk. The runner verification job repeats the existing setup. It
adds one parallel GitHub Actions runner to each affected workflow.

## Model Used

OpenAI Codex, GPT-5.6, 128k context window, reasoning and tool-use
capabilities.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-10 01:18:06 -07:00
Tonio 5488a79eb5
ci(docker): build each architecture on a native runner instead of QEMU (#12821)
## Thinking Path

A fleet rollout failed with `image_manifest_not_found` for a commit that
had merged and gone green. Tracing that back: the fleet resolves
releases against the `-cloud` image, that image comes from `docker.yml`,
and `docker.yml`'s runs on master have been reading `cancelled` for a
long stretch. The cloud half was fine; the production half was hanging
and taking the run down with it — and, because a run holds the
concurrency slot for its whole duration, starving later commits of a
build at all.

## Linked Issues or Issue Description

No tracking issue — described inline, per CONTRIBUTING.md.

**What's wrong.** `build-and-push` builds `linux/amd64,linux/arm64` on
an x86 runner, so arm64 runs under QEMU. It hangs there —
deterministically, in the same step:

```
#111 [linux/arm64 build  8/10] RUN pnpm --filter @paperclipai/server build
```

…then emits nothing until `timeout-minutes: 60` kills it. Three
consecutive runs on 2026-09-04, silent for **38, 43 and 45 minutes**
respectively. The amd64 leg reached `production 5/5` minutes earlier in
every one.

**Why it stayed hidden.** A timed-out job is reported by GitHub as
**cancelled, not failed**. The run conclusion reads "cancelled", which
looks like supersession rather than breakage, so the production image
quietly stopped publishing.

**The knock-on.** A run that burns the full hour holds the top-level
concurrency slot for that hour. `cancel-in-progress: false` keeps
exactly one pending slot, so merges arriving faster than one an hour
supersede each other while queued. Sampling the last ten master commits,
**five produced no image at all** — their Docker runs have zero job
records because they never started.

**Expected.** Both architectures publish, and a commit merged during a
busy period still gets an image.

## What Changed

`build-and-push` becomes a two-leg matrix, each on a runner of its own
architecture:

| platform | runner |
|---|---|
| `linux/amd64` | `ubuntu-latest` |
| `linux/arm64` | `ubuntu-24.04-arm` |

Each leg pushes **by digest** (`push-by-digest=true`, untagged), and a
new `merge-and-push` job names the digests into one manifest list with
the real lane tags. Nothing is publicly tagged until the merge, so a
half-published multi-arch image is never a pullable state.

Two supporting changes:

- **Per-arch BuildKit cache refs** (`:buildcache-amd64` /
`:buildcache-arm64`). Separate runners sharing one ref would overwrite
each other on every build.
- **The PID-1 orphan-reaping check moves to the merge job**, since that
is where a tagged, pullable image first exists. It still runs against
the pushed image rather than a local build, for the same reason as
before.

**arm64 is kept, not dropped.** The cloud variant is amd64-only and can
be — managed hosts are amd64. This is the self-hosted image and ARM
hosts consume it, so dropping arm64 would break them. GitHub-hosted
arm64 runners are free for public repositories, which this is.

`build-and-push-cloud` is untouched. It was already `platforms:
linux/amd64` and has been succeeding in ~14 minutes throughout — that is
why `-cloud` images exist at all.

## Verification

Parsed the workflow and asserted its shape (jobs, matrix, `needs`, step
order, that the cloud job is unchanged). The artifact actions are pinned
by SHA with version comments, matching the repo's dominant convention —
`upload-artifact` v7 and `download-artifact` v8, the same pins used
across the other workflows; v8 is required for the `pattern` /
`merge-multiple` inputs the merge job uses.

**This PR's CI does not exercise the change.** `docker.yml` triggers on
master and tag pushes, never on pull requests — deliberately, since it
publishes release images. The first real run is after merge, so the
check is: the next master push produces a `Docker` run whose
`build-and-push (amd64)`, `build-and-push (arm64)` and `merge-and-push`
jobs all succeed, and whose conclusion is `success` rather than
`cancelled`.

## Risks

- **Not testable before merge**, per above. If the matrix is wrong the
next master push fails loudly rather than silently — which is already
better than the current state, where the failure mode is an invisible
"cancelled".
- **First use of `ubuntu-24.04-arm` in this repo.** No other workflow
uses an ARM runner. They are free for public repos, but if the label is
unavailable the arm64 leg will fail to schedule and the merge will not
run — no image, same as today, and visible.
- **Digest-push changes the publish shape.** Between the legs finishing
and the merge running, digests exist untagged in ghcr. Anything watching
for tags sees no intermediate state; anything enumerating untagged
manifests will see more of them.
- **Cache refs change name**, so the first build after this lands is
cold on both legs and will be slower than steady state.
- **Does not fix the underlying QEMU hang** — it avoids it. If arm64
ever has to build under emulation again, the same stall is presumably
still there.
- No application code, schema, server or persistence change.

## Model Used

Anthropic Claude — Opus 5, model ID `claude-opus-5`, run through Claude
Code.

Extended thinking enabled. Tool use throughout: GitHub Actions API to
correlate run/job outcomes and read build logs, `git` for ancestry
checks, and a YAML parser to validate the rewritten workflow's
structure.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

A workflow change has no unit test to add, and `docker.yml` cannot run
on a PR; the verification section states what to check on the first
master run instead.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Devin Foley <devin@paperclip.ing>
2026-09-09 15:56:28 -07:00
Devin Foley 01ad858492
ci: raise the multi-arch Docker publish timeout to 120 minutes (#13114)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Docker workflow publishes the server images that all deployments
pull, including the `sha-*` images that downstream consumers deploy
> - The `build-and-push` job builds for linux/amd64 and QEMU-emulated
linux/arm64, and that build now takes more than its 60 minute job
timeout
> - Every run dies at the timeout, and each doomed hour-long run holds
the per-ref concurrency slot, so queued master pushes supersede each
other and no image publishes at all
> - This pull request raises the multi-arch job timeout to 120 minutes
> - The benefit is that image publishing works again, with headroom for
the build to grow

## Linked Issues or Issue Description

Related: #12821 replaces the QEMU-emulated arm64 build with native
runners — that is the durable fix for the build duration itself. This PR
is the immediate unblock so images publish again while #12821 lands.

No existing issue for the outage. Description follows the bug report
template:

**What happened?**

The `docker.yml` `build-and-push` job hits its 60 minute
`timeout-minutes` cap on every run. The last fully successful
`docker.yml` run was September 2. Since then almost every run ends
`cancelled`: the multi-arch build is killed at the timeout, and runs
queued behind it are superseded by newer master pushes before they can
start. The amd64-only `build-and-push-cloud` job often still succeeds
inside those cancelled runs, which masked the breakage.

**Expected behavior**

Every master push and canary tag dispatch publishes its `sha-*`
production and cloud images, and the `promote_canary_channel` job runs.

**Steps to reproduce**

Look at the runs of the Docker workflow on master: `gh run list
--workflow docker.yml --branch master`. Nearly every run since September
5 ends `cancelled` or `failure`. Open a cancelled run: the
`build-and-push` job runs for 61+ minutes and its "Build and push" step
ends `cancelled` at the job timeout. The last runs that succeeded
(September 2) took 39 to 54 minutes for the same job.

## What Changed

- Raise `timeout-minutes` on the `build-and-push` job from 60 to 120,
with a comment that explains why. The amd64-only `build-and-push-cloud`
job keeps its 60 minute cap.

## Verification

- `actionlint .github/workflows/docker.yml` reports no issues in this
change (only pre-existing info-level shellcheck notes in untouched
steps).
- Compared job durations across the last successful runs (39-54 minutes)
and the recent timeout kills (61+ minutes) to confirm the cap is the
failure cause.
- After merge, the next master push should produce a `docker.yml` run
that completes with both build jobs green.

## Risks

Low risk. The change only gives the existing build more time. A
genuinely hung build now occupies a runner for up to 120 minutes instead
of 60. The slow arm64 emulated build itself is worth a separate look
(native arm runners or splitting the platforms), but that is a larger
change than this outage fix.

## Model Used

Claude (Anthropic) — Fable 5 (`claude-fable-5`), extended thinking,
agentic tool use via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (no code paths changed;
workflow linted with actionlint)
- [x] I have added or updated tests where applicable (not applicable for
a CI timeout value)
- [x] I have updated relevant documentation to reflect my changes (the
workflow comment documents the rationale)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-09 14:34:45 -07:00
Devin Foley 8f099c3f83
ci: dispatch a Docker build for every canary tag (#12950)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Every master merge publishes an npm canary, and downstream managed
deployments consume the matching `sha-<short>-cloud` Docker image.
> - The canary image relies on the master-push `docker.yml` run, whose
single pending concurrency slot is superseded by every newer push.
> - On a busy day the image build never survives: five consecutive
canaries shipped npm packages with no cloud image on 2026-09-06, and
downstream deploys starved for ~18 hours while master kept moving.
> - The nightly and beta lanes already solve exactly this by dispatching
`docker.yml` at their tag ref after pushing it.
> - This pull request wires the canary lane into the same mechanism, so
every canary tag gets an image build no master push can supersede.
> - The benefit is that a published canary always has its images, and
downstream release resolution stops walking back to day-old builds
during busy merge windows.

## Linked Issues or Issue Description

Refs #12769 / #12855 (the earlier image-publishing incident in the same
pipeline, different failure mode).

**What happened?**

`docker.yml` serialises per ref with one pending slot (`concurrency:
docker-${{ github.ref }}`, `cancel-in-progress: false`). Master pushes
arriving faster than the ~50-minute build supersede the pending build
indefinitely. On 2026-09-06, canaries `2026.906.0-canary.1` through `.3`
(and the commits between) published to npm with no `sha-<short>-cloud`
image on GHCR — the Docker run list shows `cancelled, cancelled,
cancelled` for their commits. Downstream managed rollouts correctly
refused to deploy image-less canaries and pinned at `canary.0` for ~18
hours.

**Expected behavior**

Every published canary has its Docker images. A busy merge window must
not be able to prevent image publication for tagged releases.

**Steps to reproduce**

1. Merge to master more often than the Docker build takes to complete,
for several hours.
2. Observe npm canaries advancing while every `Docker` run for their
commits completes `cancelled`.
3. Observe `ghcr.io/paperclipai/paperclip:sha-<short>-cloud` returning
404 for each of those canaries.

**Paperclip version or commit**

Master at `83987210` (diagnosis time); the starved canaries were
`2026.906.0-canary.1`–`.3`.

**Deployment mode**

GitHub Actions release + image pipeline; consumed by managed cloud
deployments.

## What Changed

- `.github/workflows/release.yml`, `publish_canary` job:
- New step after "Push canary tag": dispatch `docker.yml` at
`refs/tags/canary/v<version>` — identical to the nightly and beta lanes'
existing step, including the step-summary note. The dispatched run keys
its concurrency off the tag ref, so master pushes cannot supersede it,
and `docker.yml`'s `type=sha` mapping publishes `sha-<short>` /
`sha-<short>-cloud` for any ref.
  - The job gains `actions: write`, mirroring `publish_nightly`.
- No `docker.yml` changes: it already accepts `workflow_dispatch` for
exactly this pattern.
- No dry-run guard needed: `publish_canary` runs only on `push` events,
so the `dry_run` dispatch input cannot reach it.

## Verification

- YAML lints clean; the step is a line-for-line mirror of the proven
nightly-lane step (tag source swapped for the job's existing
`canary_tag` output).
- The concurrency claim is docker.yml's own documented behavior: groups
are per-ref, and a tag ref is distinct from `refs/heads/master`.
- Not run: a live canary publish — the next master merge after this
lands exercises it end to end; the worst case (double build for a canary
whose master-push run also survives) is benign, since both runs push
identical content-addressed tags.

## Risks

- Low. Roughly one additional Docker build per canary on busy days (on
quiet days the master-push run and the dispatched run both execute —
duplicate work, identical images, no conflict since the sha tags are
content-equivalent for the same commit).
- `actions: write` on `publish_canary` matches the grant
`publish_nightly` already carries for the same purpose.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Fable 5 (Anthropic, model id `claude-fable-5`), extended
thinking, agentic tool use in Claude Code: GHCR manifest probing,
Actions run-list forensics, and workflow-lane comparison.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (workflow-only change; YAML
linted; no test harness targets release.yml)
- [x] I have added or updated tests where applicable (not applicable —
CI wiring mirroring an existing proven lane)
- [x] I have updated relevant documentation to reflect my changes (the
step's inline comment documents the invariant)
- [x] I have considered and documented the risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-08 16:56:35 -07:00
Dotta 0cc796b7bd
Build isolated preview artifacts for exact-source deployments (#13041)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Managed deployments need a cloud image and a database migration
package.
> - Branch commits can lack both artifacts until a normal release runs.
> - Operators need to test an exact commit without advancing release
aliases.
> - This pull request adds a preview build mode to the existing release
workflow.
> - Builds use an immutable source SHA and publish isolated, reusable
artifacts.

## Linked Issues or Issue Description

**Subsystem affected**

Release automation, cloud Docker images, and shared/database npm
packages.

**Problem or motivation**

An operator cannot deploy an unpublished branch with new migrations
using only
the normal release artifacts. Publishing it through a normal lane would
also
advance shared release aliases.

**Proposed solution**

Dispatch the trusted release workflow on master with a full source SHA
and a
request UUID. Build missing SHA images and, when needed, deterministic
preview
shared/DB packages. Publish packages under the preview dist-tag with
exact
workspace pins. Reuse matching artifacts on retries.

**Roadmap alignment**

This extends release tooling for operator validation. It does not add a
core
product feature or duplicate a planned product capability. Related PR
searches
found no duplicate preview deployment workflow.

## What Changed

- Add the preview channel, request correlation, artifact checks, and
result artifact.
- Compile source packages in a separate job from the npm publisher. The
publisher
  uses trusted master code and disables package lifecycle scripts.
- Publish only SHA cloud image tags. Preserve release aliases. Use
full-SHA tags and no shared build cache.
- Verify full source identity for reused packages and images. Both image
and npm publishers
use isolated jobs and the externally master-restricted npm-canary
environment. Fail on registry
  authentication errors, outages, or artifact identity mismatches.
- Let bundled-package preparation use patches from the requested source
checkout.
- Document publishing configuration, artifact contracts, and deployment
order.

## Verification

- Passed `pnpm -r typecheck` and `pnpm build`.
- Passed `pnpm test:release-registry`: 107 tests, including eight
preview tests.
- Passed `actionlint -shellcheck= .github/workflows/release.yml`.
- Built real shared and DB preview tarballs from an isolated exact-SHA
checkout.
Verified package source identity and all 244 SQL files and journal
entries.
- Verified the full revision behind an existing published SHA cloud
image.
- `pnpm test:run` exposed missing local embedded PostgreSQL library
symlinks.
The package's postinstall repair restored initdb; all 12 previously
affected
suites passed on rerun (95 tests). Additional local matrix reruns are in
progress.
The complete PR CI matrix is green, including general/serialized tests,
e2e,
typecheck, build, release registry, canary dry run, and the required
verify gate.
- Live preview publication and staging deployment require this workflow
on master
and the compatible control-plane backend. They have not run yet. No
production
  deployment was performed.

## Risks

Preview npm versions are immutable public artifacts. Both packages must
retain
their trusted publisher for release.yml in environment npm-canary.
Source builds
must remain separated from privileged npm publishing. The deploying
control plane
must verify source identity, integrity, and migration compatibility
before use.

Normal release jobs retain their existing conditions. Roll back by
stopping preview
dispatches and reverting the workflow/tooling. Published preview
versions remain
isolated from normal release tags.

## Model Used

OpenAI GPT-6 through Codex, with repository tools, code execution, and
test runs.
The session does not expose a more specific model version or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with the available version and
capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs
- [x] I have described the issue in-PR following the feature template
- [x] I have not referenced internal or instance-local issues or links
- [x] My branch name describes the change and contains no internal
ticket identifier
- [ ] I have run the full tests locally and they pass
- [x] I have added tests for the new behavior
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open recommendations or follow-ups
- [x] I will address review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-08 09:21:58 -05:00
Dotta 54a99d8840
fix(evals): make the chat viewer the default published Evalbook (#12952)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Direct Runner evals retain evidence across model configurations.
> - Evalbook already has a grid and a read-only Runner Lab chat viewer.
> - Public projection stripped the view and selected a second plain
result page.
> - This change uses the existing viewer for public and private results.
> - The data access differs, but the presentation does not.

## Linked Issues or Issue Description

Refs #12931, #12945. Related open runtime-contract PR #11634 does not
contain this report-only change.

**What happened?**

The public direct-eval campaign opened plain result pages. The
access-controlled artifact used the chat viewer. Users could not follow
the same recorded interaction from the published grid.

**Expected behavior**

Every newly generated Runner Evalbook opens the existing chat viewer.
The grid and durable run history remain. Public evidence has explicit
redactions.

**Steps to reproduce**

Open campaign gha-34062394019-1 from the direct-eval history. Click a
result, then compare its plain page with the corresponding Actions
artifact.

**Paperclip version or commit**

Reproduced at 83987210d6.

**Deployment mode**

Static GitHub Actions artifacts and S3/CloudFront publication.

Companion site-theme renderer:
https://github.com/paperclipai/paperclip-evals/pull/19. This removes the
Python light theme and links the same built stylesheet.

## What Changed

- Add a closed public chat projection. Require mock isolation evidence
before publishing recorded text. Scrub private references and withhold
tool payloads, reasoning and provider state.
- Validate public HTML against the exact trusted viewer shell and asset
bytes. Validate the public DTO and local links. Keep CSP restrictions on
outbound requests and forms.
- Make the workflow render both data projections with the canonical
viewer. Pass a viewer-only artifact to the trusted publisher. Reject an
old renderer pin before paid execution.
- Fix report-only start position, missing-state inspector, read-only
controls and redaction labels. Tool evidence links select and highlight
the Evidence tab even when reopened. Runner execution and the full-stack
E2E workflow are unchanged.
- Add a no-model report refresh command. Preserve original campaign
identity, measurements and immutable history; label report revisions.
- Document the single presentation and public/private evidence boundary.
- Use one Runner Lab stylesheet and local fonts for the grid, Latest,
test design, inventory, server gate and S3 history index. Keep static
styles scoped away from live chat.
- Emit exact published report/history URLs to the Actions summary and
job outputs only after successful upload. Set the deployment link from
that output.
- Switch public Eval and Evidence panes without rendering both at once.

- Add all-run history with like-for-like pass-rate and cost timelines,
regression/recovery lists, exact commit links, source refs and Actions
links.
- Record all-attempt costs including retries. Keep provider list cost
separate from estimates. Label missing coverage and historical
final-only costs.
- Retain all run records beyond 200, backfill a separate derived
analytics projection, and exclude report refreshes from measurements.

## Verification

- Focused report/publishing/projection/workflow/adapter tests: 29
passed.
- Workflow Evalbook adapter tests: three passed.
- Viewer unit tests: eight passed; Vite viewer build passed.
- Companion renderer tests: 59 total, 57 passed, two inventory tests
skipped because the expected sibling checkout is absent.
- Re-rendered all 375 retained attempts from the completed campaign and
validated the public bundle. Zero new model calls; 356/358 selected
cells still pass.
- Browser walkthrough: grid to failed chat; prompt, named tool calls,
correct blocked status, visible assertions, no loading spinner or
composer. Public payload redactions are explicit.
- Full typecheck and build passed locally. test:run ended with 17 failed
files and 19 failed tests in unrelated server/worktree areas (3772 tests
passed). Latest-head CI is the final merge gate. The browser sources
also typecheck with a temporary TypeScript-7-compatible path
configuration; the checked-in browser config still uses removed baseUrl
options and is unchanged here.
- Real Chromium verification passed for passing, failing and
missing-recording attempts in both the full and public bundles. It
checks tool expansion, navigation, reload, read-only controls, narrow
viewport visibility and the public no-network boundary. Future
publications run it automatically.

- Fresh live proof: three gpt-5.4-mini native Codex cases passed on the
first attempt (get-task-context, create-child-task,
workflow-context-document-progress); estimated total $0.00632625.
Generated the full canonical report and verified all three file://
pages, all seven DevTools tabs, and evidence cross-links in Chromium.
Private screenshots remain local.
- Follow-up 99697c2c5: viewer build, eight unit tests, browser token
gate and browser-source typecheck (existing TS7 configuration
workaround) passed. Added repeated tool-to-evidence navigation to the
publication browser gate.

- Site-theme follow-up: 26 publishing/security tests, 59 Python tests
(57 passed, two existing skips), eight viewer unit tests and the viewer
build passed. Chromium verified shared colors and
grid/test-design/Latest navigation.
- Re-rendered all 375 retained attempts with the shared site theme,
without new provider calls.

- Successful hosted publication: [refreshed
Evalbook](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-34062394019-1-report-site-theme-v3/index.html)
and [themed
history](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/index.html).
Real browser verified grid → test design → chat → All results and
history → run.

- Full maintained live suite completed on AWS: [Actions run
34074939112](https://github.com/paperclipai/paperclip/actions/runs/34074939112),
Paperclip 856813ba3a, evals
34e1846c06a39e641182dadce5de7ea739f657f1. All 358 cells across 11
configurations ran; 355 passed (99.2%), two behavior failures and one
infrastructure failure. Nine configurations are entirely green. All 360
retained attempts were rendered using the new design and published as an
immutable, zero-provider-call report refresh: [full
Evalbook](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-34074939112-1-report-history-v1/index.html).
The trusted master workflow ran the models; this PR's viewer rendered
their results afterward.
- Remaining live failures: native Codex mini did not emit the expected
discovery event for lazy-unauthorized-undisclosed (no mutation
occurred); GLM 5.3 scheduled a wake before approval in
workflow-governed-wait; GLM 5.3 timed out on create-task-document on
both attempts. No scoring rules were relaxed.
- Retry-inclusive recorded estimated model cost is at least $7.067692,
with usage for 357/360 attempts. Provider-reported list cost is a
separate alternative (at least $14.002806), not an additive cost.
Missing usage is unknown, not zero; AWS compute is excluded.
- [Production
history](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/index.html)
now contains nine model runs and two separately labeled report
refreshes. Matching full-suite comparison reports three newly failing
cells and two recoveries versus the previous run. This is observed
run-to-run variation, not proof of a deterministic code regression. Live
HTTP checks verify the newest report link, shared theme, exact source
SHAs and cost analytics.
- The new full report passed Chromium checks for passing, failing and
missing-recording chat pages, tool-to-Evidence links, grid/design/Latest
navigation, reload, read-only controls and narrow layouts. The Mac was
locked during final hosted verification; the new hosted history was
checked by HTTP and generated-page browser checks, not a fresh
interactive desktop walkthrough.
- History follow-up: 31 publishing/security/metrics tests pass. Viewer
build passes. Chromium checks desktop and narrow history pages with no
document-level horizontal overflow.

### Visual verification

Generated from the scrubbed completed campaign; no private provider
identities or raw tool payloads are included. Full private pages were
also browser-tested, but their private metadata is not published as
screenshots.

![Passing
replay](https://raw.githubusercontent.com/paperclipai/paperclip/c660fec08bd0492e844b1198ddcf0e6f387dd597/packages/paperclip-runner/docs/images/evalbook-chat/passed.png)
![Failed
replay](https://raw.githubusercontent.com/paperclipai/paperclip/c660fec08bd0492e844b1198ddcf0e6f387dd597/packages/paperclip-runner/docs/images/evalbook-chat/failed.png)
![Missing
recording](https://raw.githubusercontent.com/paperclipai/paperclip/c660fec08bd0492e844b1198ddcf0e6f387dd597/packages/paperclip-runner/docs/images/evalbook-chat/missing-recording.png)

Latest shared-theme proof:

![Pass-rate and cost
history](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/proofs/cost-history-sept6/history.png)

![Overview
grid](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/proofs/site-theme-sept6/index.png)
![Test
design](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/proofs/site-theme-sept6/test-design.png)
![Recorded
chat](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/proofs/site-theme-sept6/passed.png)

## Risks

- Public chat text is newly visible, but only for the isolated mock
boundary. The producer excludes raw payloads and the publisher fails
closed on unknown fields, secrets, shell changes and asset
substitutions.
- Requires the companion canonical renderer revision and an updated
RUNNER_PROTOCOL_EVALS_SHA after merge. Old pins fail before paid
execution.
- Existing published campaigns remain immutable. A report refresh is a
separate history entry, not a new model qualification.
- Successfully published the immutable site-theme refresh with the
configured report-bucket SSO profile. Original run records and
qualification pointers are preserved.

## Model Used

OpenAI Codex, GPT-5-based coding agent with reasoning, shell and browser
tools. Exact deployment model ID and context-window size are not exposed
in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green on
cad99dbf04
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
on cad99dbf04
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 07:47:17 -05:00
Dotta fee8d8dc39
fix(runner): repair direct live provider bootstrap (#12932)
## Thinking Path

> - Paperclip runs AI agents through qualified provider backends.
> - The direct live eval workflow builds one immutable Runner runtime
for every matrix cell.
> - The workflow reinstalled the packed Runner with npm.
> - That install discarded pnpm patches and selected provider
dependencies outside the qualified lock.
> - The first pnpm deployment model also placed its virtual-store marker
at the wrong level; a real deployment keeps `.pnpm` beside the scoped
Runner package.
> - AgentCore enforced the current context-aware harness but the direct
eval CLI did not supply the production v3 runtime context that harness
requires.
> - This pull request preserves the qualified dependency graph, resolves
the real deployment layout, and makes direct evals exercise the
production runtime-context contract.
> - The benefit is that live eval cells reach their provider turn with
the same artifacts and context contract that Paperclip qualified.

## Linked Issues or Issue Description

Refs: #12931

**What happened?**

The full direct live eval campaign failed every ACPX cell during
`session.open`. The portable runtime had an incorrect dependency root.
Its npm install also discarded the qualified ACP server patches.
AgentCore cells first failed because Runner enforced
`aws-agentcore-harness-v1` while the provisioned stack and eval profile
use `aws-agentcore-harness-context-v2`; after aligning that revision,
the direct eval CLI still omitted the required v3 runtime context.

**Expected behavior**

The direct eval runtime must preserve the frozen pnpm dependency graph
and patched provider bytes. Runner, server validation, OpenAPI, and the
deployed AgentCore stack must use one qualification revision. Direct
eval attempts must supply the same immutable native runtime-context
contract as production.

**Steps to reproduce**

1. Dispatch `Runner Direct Live Protocol Evals` from `master`.
2. Select an ACPX Claude, ACPX Codex, or AgentCore roster.
3. Observe a pre-turn provider bootstrap failure.

**Paperclip version or commit**

`d96452db059338b329b458ba8fe359fef72f1363`

**Deployment mode**

GitHub Actions on the RunsOn Linux x64 fleet.

## What Changed

- Build the reusable direct-eval runtime with `pnpm deploy --prod`.
- Resolve ACPX dependencies from the actual scoped-package layout of a
self-contained pnpm deployment.
- Align AgentCore configuration and qualification checks on
`aws-agentcore-harness-context-v2`.
- Materialize a minimal immutable v3 runtime context for each isolated
direct eval attempt.
- Add workflow, package-authority, runtime-context, Rust, and server
regression coverage.
- Document the qualified packaging, runtime-context, and AgentCore
revision contracts.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/runnerd-codex-transport.test.ts` (70 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/cli/eval-session-contract.test.ts` (14 tests)
- Focused Runner contract tests (36 tests)
- Focused server profile tests (47 tests)
- Focused Rust managed-provider and native-selector tests (19 tests)
- `node --test
packages/paperclip-runner/scripts/runner-protocol-eval-workflow-security.test.mjs`
- `actionlint .github/workflows/runner-protocol-live-evals.yml`
- A local `pnpm deploy --prod` produced both qualified ACP server
digests.
- A Linux reproduction of the first follow-up smoke identified the real
deployment root and the missing AgentCore runtime context.

## Risks

The AgentCore revision change rejects profiles that still use the
obsolete v1 value. This is intentional because the provisioned
context-aware harness and current eval profile use v2. Direct eval
prompts now receive the same fixed runtime-context preamble as
production, so behavior scores may move; that is the intended
qualification surface. The workflow package layout changes, but tests
assert the new entrypoint and dependency root. This change does not
modify the browser full-stack E2E workflow.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5.6. The context-window size is not exposed in
this session. The model used extended reasoning, repository tools, code
execution, Docker-based Linux reproduction, and GitHub Actions
diagnostics.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (for example, `docs/...` or
`fix/...`) and contains no internal Paperclip ticket id or
instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-06 15:45:07 -05:00
Dotta d96452db05
fix(runner): restore Vite 6 viewer compatibility (#12929)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner includes an issue-thread viewer for direct
Evalbook reports.
> - The runner package uses Vite 6.4.3.
> - Dependabot changed the React plugin from version 4.7.0 to version
6.1.1.
> - React plugin 6.1.1 requires Vite 7 or Vite 8 package internals.
> - The live evaluation workflow found this mismatch when it built the
viewer on a clean Linux worker.
> - This pull request restores the compatible plugin and adds the viewer
build to pull request CI.
> - The benefit is that CI detects this class of report-viewer build
failure before a paid evaluation campaign starts.

## Linked Issues or Issue Description

**What happened?**

The direct live evaluation workflow failed before model execution. The
`build:issue-thread` command could not load `@vitejs/plugin-react@6.1.1`
with Vite 6.4.3. The plugin imported the unavailable `vite/internal`
package path.

**Expected behavior**

The Evalbook issue-thread viewer must build from a clean frozen-lockfile
installation before the live evaluation matrix starts.

**Steps to reproduce**

1. Check out commit `165ca56a22adb60e5fda56045442d9c8498116a8`.
2. Run `pnpm install --frozen-lockfile --ignore-scripts`.
3. Run `pnpm --filter @paperclipai/paperclip-runner build:issue-thread`.
4. Observe the `ERR_PACKAGE_PATH_NOT_EXPORTED` error for
`vite/internal`.

**Paperclip version or commit**

`165ca56a22adb60e5fda56045442d9c8498116a8`

**Deployment mode**

Built from source in GitHub Actions on Ubuntu.

## What Changed

- Restore `@vitejs/plugin-react` 4.7.0 in the Vite 6 runner package.
- Follow the repository policy: trusted PR CI regenerates and verifies
the lockfile artifact, and the master refresh workflow commits the
lock-only update after merge.
- Build the Runner Evalbook viewer in pull request CI.

## Verification

- `ci / policy`: regenerated the dependency lock artifact successfully
- `pnpm --filter @paperclipai/paperclip-runner build:issue-thread`
- `actionlint .github/workflows/pr-trusted.yml
.github/workflows/runner-protocol-live-evals.yml`
- `node --test
packages/paperclip-runner/scripts/runner-protocol-eval-workflow-security.test.mjs`
- `git diff --check`

## Risks

Low risk. This change restores the previous React plugin major version
for one package. The selected version declares support for Vite 6. Pull
request CI now builds the affected viewer directly.

> This bug fix does not add or change a roadmap feature.

## Model Used

- OpenAI Codex with GPT-5. Tool use and code execution were enabled. The
Codex app managed the context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-06 09:16:21 -05:00
Dotta 9ecd93a54d
test(e2e): link runner campaign summaries (#12927)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip uses a paid full-stack campaign to verify runner behavior
across providers and environments.
> - The campaign already creates an interactive report, workflow logs,
and retained evidence artifacts.
> - The merge job summary shows result totals but does not link to those
resources.
> - Reviewers must search several workflow jobs and artifacts to find
the executed cells.
> - This pull request adds direct and safe links to the exact campaign,
each cell, the workflow logs, and the artifacts.
> - The benefit is that a reviewer can inspect a result from the Actions
summary with one click.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the `Merge and enforce campaign result` summary in the
`Runner Full-Stack E2E` workflow.

**Subsystem affected**

The runner E2E report generator and its GitHub Actions workflow are
affected.

**Current behavior**

The summary lists each selected cell and its result. It does not link to
the published campaign report, the workflow logs, or the evidence
artifacts.

**Proposed behavior**

The summary includes a `View results` section. It links to the exact
immutable campaign report, the workflow logs, and the artifacts. Each
cell name links to its stable section in the campaign report.

**Reason and benefit**

The current summary does not show reviewers where to inspect the run.
Direct links make the result evidence discoverable without manual URL
construction or artifact searches.

**Breaking changes**

None. This change only adds links and stable HTML anchors to existing
report output.

**Additional context**

Related: #12904. The cited successful campaign is [run
34026735033](https://github.com/paperclipai/paperclip/actions/runs/34026735033).

## What Changed

- Add a safe URL builder for public campaign, workflow, and artifact
links.
- Add a `View results` section to the GitHub Actions campaign summary.
- Link each summary table cell to its exact section in the immutable
campaign report.
- Add stable execution anchors to the generated dashboard.
- Reject non-HTTPS, credential-bearing, malformed, and ambiguous link
destinations.
- Document the new links and their retention or publication timing.

## Verification

- `pnpm test:e2e:runner:unit` — 116 tests passed.
- `pnpm test:e2e:runner:typecheck` — passed.
- `pnpm typecheck` — passed, including migration safety.
- `pnpm build` — passed.
- `pnpm exec prettier --check ...` for all changed files — passed.
- `git diff --check origin/master...HEAD` — passed.
- The full local server suite also ran. One unrelated macOS
workspace-runtime file passed 157 tests and failed 4 existing path and
port assumptions. Two failures compare `/var` with `/private/var`. Two
failures cannot reserve a port outside a hard-coded range. This PR does
not change that file or its dependencies.

## Risks

- The immutable campaign link becomes available after the history
publisher completes. The workflow and artifact links remain available
while publication runs.
- The artifact link requires GitHub access and follows the existing
30-day retention period.
- Invalid configured URLs are omitted instead of being rendered into the
summary.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex desktop agent with GPT-5. The runtime does not expose the
context-window size. The agent used repository inspection, agentic
reasoning, code execution, and GitHub CLI tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g.
`docs/no-internal-issue-references`, `fix/sandbox-secret-resolution`,
`feat/adapter-retry-backoff`) and contains no internal Paperclip ticket
id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-06 09:09:51 -05:00
Dotta 165ca56a22
fix(runner): scope live eval tokens to eval repo (#12911)
## Thinking Path

> - The merged direct live eval workflow must read the private
`paperclip-evals` repository at an exact commit.
> - Its first hosted dispatch failed before provider execution because
the GitHub App token was minted from the `paperclip` repository
installation.
> - GitHub returned 404 while resolving the private eval commit, proving
that token did not have the required repository scope.
> - Minting each short-lived token from the exact private eval
repository installation supplies only the cross-repository read boundary
the workflow needs.
> - A workflow regression now verifies every eval-token block keeps that
exact scope.

## Linked Issues or Issue Description

The first default-branch run of Runner Direct Live Protocol Evals failed
in its immutable eval-commit verification step with HTTP 404. No
provider jobs ran and no provider spend occurred.

**What existing behavior does this improve?**

It allows the protected direct live eval workflow to verify and check
out the private `paperclipai/paperclip-evals` repository.

**Current behavior**

All four eval-token blocks set `GH_REPO` to `paperclipai/paperclip`,
selecting a token installation that cannot read the private eval
repository.

**Proposed behavior**

Set `GH_REPO` to the exact `paperclipai/paperclip-evals` repository in
authorization, catalog, matrix, and report jobs.

**Reason and benefit**

The app mints a short-lived token from the correct repository
installation while the main repository continues to use its ordinary
read-only workflow token.

**Breaking changes**

None.

## What Changed

- Scoped all four private-eval installation tokens to
`paperclipai/paperclip-evals`.
- Added a regression requiring that exact scope in every token block.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `node --test .github/scripts/tests/get-bot-token.test.mjs` — 3 passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml` —
passed.
- `git diff --check` — passed.

## Risks

- The workflow reads a private repository. The token is still
short-lived, repository-specific, masked immediately, and used only by
the protected default-branch workflow.
- This changes no provider execution, Runner behavior, report content,
S3 publishing, or browser E2E behavior.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, repository inspection, code
editing, GitHub Actions diagnostics, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation where applicable
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-06 08:25:46 -05:00
Dotta af8439a70b
feat(runner): restore direct live eval campaigns and reports (#12909)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Runner executes agents through native and managed provider
drivers.
> - The direct live eval layer had drifted from the current Runner
contracts.
> - The old local workflow did not provide a complete parallel campaign
or durable report history.
> - The Runner also needed current native OpenCode and OpenRouter
qualification.
> - This pull request restores the direct campaign, corrects the runtime
gaps that the campaign found, and adds safe hosted Evalbook history.
> - The benefit is repeatable model comparison against an immutable
Runner and eval source revision.

## Linked Issues or Issue Description

Refs #11297
Refs #11634

**What existing behavior does this improve?**

This improves the direct live `paperclip-runner` eval workflow, provider
execution contract, and static Evalbook reporting path.

**Current behavior**

The direct evals do not have one maintained full campaign on current
`master`. OpenCode has no qualified multi-model OpenRouter roster.
Parallel provider bursts can compact committed events before the
transport observes them. Local reports do not have a separate safe S3
history index.

**Proposed behavior**

Run one immutable roster-plus-case matrix. Use the shared paid AWS
runner fleet. Keep raw artifacts access-controlled. Publish a sanitized
canonical Evalbook report under the separate `runner-protocol-evals` S3
prefix. Keep immutable campaign directories plus root history, latest,
and latest-green pointers.

**Reason and benefit**

Maintainers can compare native Codex, native OpenCode, ACPX, Claude
Managed, and AWS AgentCore behavior over time. They can inspect failures
without mixing this direct protocol layer with browser full-stack E2E.

**Breaking changes**

None. The new workflow and S3 prefix are additive. The existing Runner
full-stack E2E workflow and report remain separate.

## What Changed

- Added a trusted two-shard direct live workflow for up to 393
roster-plus-case cells.
- Reused the numeric actor allowlist, protected paid environment, and
RunsOn fleet controls from Runner full-stack E2E.
- Added immutable Runner and eval revision resolution, exact credential
boundaries, bounded retries, and cost ceilings.
- Added a public report projection that removes sessions, transcripts,
tool payloads, state, traces, raw failures, remote profile identities,
and credential-shaped values.
- Added additive S3 history under `runner-protocol-evals`, with
immutable campaigns and mutable root index pointers.
- Added native OpenCode model injection and current OpenRouter pricing
contracts.
- Fixed direct eval completion, workflow execution, semantic discovery,
warm-attach state reset, executable binding, and event-burst handling.
- Kept Runner browser full-stack E2E behavior and publication separate.
- Documented local and hosted direct eval operation.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `pnpm --filter @paperclipai/paperclip-runner build:typescript` —
passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml
.github/workflows/runner-full-stack-e2e.yml` — passed.
- Local current matrix at the revision in
[paperclip-evals#17](https://github.com/paperclipai/paperclip-evals/pull/17)
— 323 cells across 10 enabled configurations completed.
- Final local current matrix — 269 passed, 11 behavior failures, and 43
expected macOS-only ACPX platform failures.
- Targeted Runner checks — 13/13 eval-session tests, 15/15
publisher/security tests, and package typecheck passed; complete PR CI
is green, including all browser E2E shards.

## Risks

- Paid live campaigns can consume provider budget. Actor authorization,
exact per-cell ceilings, protected environments, and explicit schedule
enablement bound this risk.
- Public reports can leak provider data. The workflow publishes only a
separately projected report and validates every file before upload.
- The new workflow cannot publish until it is present on the default
branch. This pull request does not change the existing
`runner-full-stack-e2e` publication path.
- The campaign is large. It uses two GitHub matrices and caps combined
concurrency at the shared fleet limit.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, code editing, browser
inspection, repository tools, and live provider execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 20:18:11 -05:00
Dotta 1dceee9a4e
fix(runner): persist warm Daytona workspaces (#12901)
## Thinking Path

> - Paperclip manages AI agent work and the execution state for each
task.
> - Remote agents run in sandbox environments such as Daytona.
> - Daytona keeps files while a sandbox is stopped, but deletion removes
those files.
> - Runner Codex did not copy successful remote workspace changes back
to the host workspace.
> - A warm sandbox could therefore hide data loss until Daytona replaced
or deleted the sandbox.
> - This pull request makes the host workspace durable after every
successful turn and keeps verified reusable sandboxes warm.
> - The benefit is reliable multi-turn work across warm reuse, restart,
stop, and sandbox replacement.

## Linked Issues or Issue Description

**What happened?**

A successful native Codex turn in Daytona could leave workspace changes
only in the remote sandbox. A later warm turn appeared to work because
it reused that filesystem. A replacement sandbox could start from stale
host data and lose the successful changes.

**Expected behavior**

Paperclip must merge each successful remote turn into the authoritative
host workspace before it completes the run. A verified warm lease may
reuse its remote files. A replacement lease must reconstruct the exact
durable workspace seed.

**Steps to reproduce**

1. Run Codex in a reusable Daytona environment.
2. Write a file during one successful turn.
3. Replace the Daytona sandbox before the next turn.
4. Observe that the next turn can start without the prior file on the
unpatched code.

Related remote workspace foundation: #10070.

## What Changed

- Added explicit `host_current`, `durable_seed`, and `adopt_remote`
workspace preparation modes.
- Added atomic, versioned native workspace descriptors and seed archives
under `PAPERCLIP_HOME`.
- Added real native sandbox export and three-way host merge before
terminal result completion.
- Added workspace-only recovery after a proposed result. Recovery does
not submit another provider turn or consume the provider retry budget.
- Added fail-closed handling when a sandbox with unexported changes is
gone.
- Kept healthy reusable Daytona sandboxes started for legacy Codex and
Runner Codex.
- Kept the Runner Codex process and provider session across verified
warm turns.
- Added the paid `daytona-warm-continuity` browser suite. It contains
exactly the legacy Codex and Runner Codex cells. Each cell performs
three measured turns.
- Documented `pnpm test:e2e:runner -- --suite daytona-warm-continuity`.
No package script was added.
- Added no database migration. The metadata format is backward
compatible and idempotent.

## Verification

- `pnpm typecheck`
- `pnpm test:e2e:runner:unit` — 114 passed
- Native workspace, finalizer, session, and environment tests — 232
passed
- Daytona provider tests — 150 passed
- Workspace staging and merge tests — 98 passed
- Runner transport tests — 63 passed
- Legacy Codex restore tests — 5 passed
- Rust format and compile checks pass through root typecheck
- The paid Daytona suite was not run locally because the required
Daytona, OpenAI, and immutable image credentials are not present.

## Risks

- The main risk is an incorrect workspace identity or merge after a
crash. Durable descriptors bind the run, workspace, lease, provider
lease, local root, remote root, and baseline digest. Ambiguous evidence
fails closed.
- The host merge may conflict with concurrent host edits. The existing
three-way merge and exclusion rules handle this case and surface
failures.
- A deleted sandbox cannot recover unexported bytes. Paperclip now
blocks with `workspace_sync_out_unrecoverable` instead of reporting
success or rerunning the provider.
- There is no database migration. Descriptor writes and recovery are
atomic and idempotent.

## Model Used

OpenAI Codex with GPT-5. The run used agentic reasoning, repository
inspection, code execution, test execution, Git, and GitHub CLI tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 13:00:57 -05:00
Nicky Leach 3ed5b7c5c8
refactor(server): extract the active-run output watchdog into a feature module (#12853)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server recovery service monitors active runs and applies
watchdog decisions
> - The watchdog rules and database operations lived in one large
recovery service
> - This structure made the rules harder to test and made company
scoping harder to inspect
> - This pull request moves the watchdog into domain, application, and
adapter layers
> - The benefit is a smaller recovery service, pure policy tests, and
clear company-scoped ports

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The active-run output watchdog that detects silence, suppression, and
terminal evidence.

**Current behavior**

The recovery service contains the watchdog policy, use cases, database
operations, and process control in one file.

**Proposed behavior**

A feature module separates pure policy, use cases and ports, and
Postgres and process adapters. The recovery service delegates its public
watchdog methods to this module.

**Reason and benefit**

The separation makes policy decisions easy to test. Company identifiers
on every reader and writer port make tenant scope clear. Smaller service
methods reduce change risk.

**Breaking changes**

None. The recovery service keeps its public methods and call sites.

Related public watchdog work includes
[#7043](https://github.com/paperclipai/paperclip/pull/7043) and
[#7770](https://github.com/paperclipai/paperclip/pull/7770).

## What Changed

- Add the `server/src/modules/active-run-watchdog/` feature module with
domain, application, and adapter layers.
- Move watchdog policy, use cases, Postgres access, and local process
control into the module.
- Keep the recovery service public methods and delegate them to the
module.
- Add 53 pure module test cases and retain 8 Postgres integration cases.
- Add company scoping and transaction rollback coverage.

## Verification

- Run `vitest run --config vitest.config.ts src/modules` and confirm 3
files and 53 cases pass.
- Run `vitest run --config vitest.config.ts
src/__tests__/heartbeat-active-run-output-watchdog.test.ts` and confirm
1 file and 8 cases pass.
- Run the full server suite in pull request CI.
- Compare the type-check result with a fresh baseline on the same
checkout.

## Risks

The main risk is a behavior change in recovery decisions during the move
across layers. The pure policy tests cover the moved rules. The
integration tests cover database behavior, company scope, and
transaction rollback. Pull request CI runs the full server suite.

## Model Used

OpenAI Codex, GPT-5, runtime-managed context window, tool use, code
execution, and repository review support.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-05 09:15:59 -07:00
Dotta 87832c48fd
feat(runner-e2e): publish declared screenshots (#12895)
## Thinking Path

> - Paperclip uses runner end-to-end reports to compare agent profiles
and execution environments
> - The report dashboard shows each reviewed final-state screenshot as a
thumbnail and gallery item
> - The public history publisher removed all per-attempt images before
it regenerated the dashboard
> - Therefore the public dashboard had the new layout but could not show
the screenshots from the run
> - The publisher needs a narrow rule that keeps only screenshots from
the exact live fixture issue route
> - This pull request keeps those trusted PNG files in every future
public S3 and GitHub Pages report
> - The benefit is that each future report can show its screenshot
gallery without exposing logs, traces, videos, archives, arbitrary
images, or generated report trees

## Linked Issues or Issue Description

**What happened?**

The runner E2E job captured final-state screenshots in its private
artifact. The public S3 and GitHub Pages publication step removed those
screenshots before it regenerated the dashboard. As a result, the public
report showed the new dashboard controls but no screenshot thumbnails or
gallery items.

**Expected behavior**

Each future public runner E2E report must include reviewed PNG
screenshots from the live fixture issue. Other captures and active or
unsafe evidence must stay private.

**Steps to reproduce**

1. Run the runner full-stack E2E workflow on `master` before this
change.
2. Open the private `runner-e2e-report-*` artifact and confirm that it
contains per-attempt PNG screenshots.
3. Open the public campaign URL and confirm that the dashboard has no
screenshot gallery items.

**Paperclip version or commit**

The issue was reproduced on commit `64d8929`, after the report design
change in PR #12889.

**Deployment mode**

GitHub Actions with the public S3 and GitHub Pages report publishers.

Related design work: Refs #12889.

## What Changed

- Mark screenshots from the exact server-created live fixture issue
route with `public-runner-fixture`.
- Keep marked PNG files in both the S3 history bundle and the GitHub
Pages bundle.
- Keep captures from other issue routes, sensitive routes, and external
origins private.
- Bind public files to the normalized execution ID, attempt, and safe
PNG base name.
- Validate every retained image with the existing PNG signature and 12
MiB size checks.
- Skip missing-artifact sentinel results with attempt `0` when they have
no public screenshots.
- Continue to remove unmarked images, videos, traces, archives,
generated HTML reports, and other private evidence.
- Update publisher tests, workflow checks, report copy, and the
public-evidence security documentation.

## Verification

- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/history.test.ts tests/runner-e2e/report.test.ts`
- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/workflow-security.test.ts -t "uses environment-scoped
OIDC"`
- `pnpm test:e2e:runner:typecheck`
- `PAPERCLIP_PLAYWRIGHT_CHANNEL=chrome pnpm exec playwright test
--config tests/e2e/playwright.config.ts
tests/e2e/runner-e2e-dashboard.spec.ts`
- `pnpm -r typecheck`
- `pnpm build`
- Regenerated the dashboard from retained evidence for Actions run
`33968240659` without a paid matrix rerun. The public-stage proof
contained 121 screenshot gallery items and thumbnail frames, with zero
generated HTML report files. The trusted-fixture marker and route gate
have separate focused tests.
- All pull request CI checks pass on commit `ccd2b49e1`.

## Risks

- This change intentionally makes marked fixture screenshots public at
the campaign URL. A screenshot can show data that a raw-byte secret scan
cannot detect.
- The capture helper marks a screenshot only on the exact loopback issue
route for the fixture that the harness created. A different issue,
sensitive page, or external origin stays private.
- The publisher also requires the marker, a safe normalized path, a
valid PNG signature, and the size limit.
- The change does not publish videos, logs, traces, archives, arbitrary
images, or generated browser report trees.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, model `gpt-5.6-sol`, with high reasoning, repository
tool use, shell execution, browser inspection, and GitHub CLI access.
The working context was the Codex desktop task context.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-05 09:49:31 -05:00
Dotta 8430bd897f
ci: reuse trusted cache for Daytona images (#12862)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The full-stack runner campaign checks local and Daytona runner
behavior.
> - A Daytona image content miss starts a cold multi-stage Docker build.
> - Stable dependency and agent CLI layers take most of the image build
time.
> - Development targets must not write shared cache state.
> - This pull request adds a registry cache with a default-branch write
gate.
> - It also puts volatile source inputs after stable install layers.
> - The benefit is a shorter Daytona image build without weaker secret
isolation.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the Daytona runner image stage in the full-stack E2E
workflow.

**Subsystem affected**

The GitHub Actions runner E2E workflow and its Daytona Docker image are
affected.

**Current behavior**

Each new Daytona image content ID starts with an empty BuildKit cache. A
runner source change also invalidates dependency and agent CLI install
layers because volatile inputs occur before those layers.

**Proposed behavior**

All authorized campaigns can read one GHCR BuildKit cache. Only a
campaign whose target ref is the repository default branch can update
that cache. The Dockerfile installs dependencies and agent CLIs before
it consumes volatile runner source or revision metadata.

**Reason and benefit**

The paid runner matrix spends several minutes building the image before
any selected cell can start. Cache reuse removes repeated stable setup
work and makes focused Daytona iterations faster.

**Breaking changes**

None. The immutable content tag, digest inspection, Cosign signature,
image labels, pinned base images, and provider credential boundary stay
unchanged.

## What Changed

- Read a registry-backed BuildKit cache for Daytona image content
misses.
- Export the cache only when the resolved target ref is the default
branch.
- Keep provider credentials outside the image build and cache.
- Install provider-pack dependencies before runner source is copied.
- Keep expensive agent CLI installs before source revision metadata.
- Add workflow and Docker layer-order contract checks.

## Verification

- `prettier --write .github/workflows/runner-full-stack-e2e.yml
tests/runner-e2e/daytona-image.test.ts
tests/runner-e2e/workflow-security.test.ts`
- `actionlint .github/workflows/runner-full-stack-e2e.yml`
- `git diff --check`
- I did not run a test suite or Docker image build locally. The
requested iteration policy reserves those checks for GitHub Actions.

## Risks

Low risk. BuildKit can use a cache record only when its content key
matches the build instruction and input. Development targets have
read-only cache access. The cache contains public source and build
outputs, but it does not receive provider credentials or the GitHub
token as Docker build inputs.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 06:31:27 -05:00
Dotta bcc6fe7a44
fix(runner): restore multi-turn remote sessions (#12840)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The runner executes agent turns on local and remote providers.
> - A remote per-turn session must save its state before Paperclip
releases its sandbox.
> - The session runtime returned after 100 milliseconds while the remote
checkpoint still ran.
> - The next turn also checked the local state path instead of the
verified remote backup.
> - This pull request waits for the bounded remote close and accepts
only a verified suspended backup.
> - The benefit is reliable multi-turn execution without weaker identity
checks.

## Linked Issues or Issue Description

**What happened?**

A successful remote agent turn released its sandbox before the runner
saved the verified continuation backup. The next turn failed with
`runner_state_identity_mismatch`.

**Expected behavior**

Paperclip must finish the bounded remote checkpoint before it releases
the sandbox. A later turn must validate and restore the digest-matched
suspended backup.

**Steps to reproduce**

1. Run a native ACPX Claude Plan test in a non-reusable Daytona sandbox.
2. Reject the first plan to start a second turn.
3. Observe that the second turn fails before provider execution.

**Paperclip version or commit**

The failure reproduced at `13775a90b078ff64872f50961ea1b83d575e7bc6`.

**Deployment mode**

GitHub Actions with a Daytona sandbox.

## What Changed

- Wait for the internally bounded remote runner close and checkpoint
before the host returns.
- Preserve the existing short cleanup bound for other providers.
- Validate remote continuation lifecycle from a complete digest-verified
backup when local runner state is absent.
- Keep corrupt, non-suspended, mismatched, and unverified state
fail-closed.
- Make native Plan completion and accepted-Plan wake prompts
deterministic.

## Verification

- A prior 45-cell local campaign passed 44 cells. The only failure was
the OpenCode Plan prompt variance fixed here.
- A focused OpenCode local Plan rerun passed.
- ACPX Claude Daytona message and question cells passed.
- Focused regressions cover delayed checkpoint close and verified remote
backup lifecycle.
- GitHub Build and the focused ACPX Claude Daytona Plan cell will
validate this exact head.

## Risks

Remote runnerd sessions now wait for their internally bounded
close/checkpoint path before returning; generic provider cleanup retains
the existing 100 millisecond bound. Durable run success still cannot be
reversed. The environment release guard still blocks sandbox destruction
when no verified backup stamp exists.

## Model Used

OpenAI Codex, GPT-5.6, extended reasoning, with code execution and
GitHub Actions inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal task
id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [ ] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 06:25:06 -05:00
Devin Foley 4b0e324c63
ci: activate the Docker context integrity gate for PRs (#12860)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request CI runs through `pr.yml`, which pins the reusable
`pr-trusted.yml` workflow by commit SHA, so `pr-trusted.yml` changes
take effect only when the pin moves.
> - PRs #12855 and #12858 added the `docker_context_integrity` job and
wired it into the `verify` aggregate, but the pin still points at a
commit from before them.
> - Until the pin moves, a pull request that strips a committed Docker
build input still merges green and breaks every post-merge image build.
> - This pull request bumps the pin to the #12858 merge commit, the
standard second step of every `pr-trusted.yml` change.
> - The benefit is that the Docker context integrity gate now blocks
merges, which closes out the 2026-09-04 image-publishing incident end to
end.

## Linked Issues or Issue Description

Refs #12855 and #12858 (the gate this activates) and #12769 (the
incident that motivated it).

**What happened?**

The `docker_context_integrity` job exists on master but does not run on
pull requests, because `pr.yml` pins `pr-trusted.yml` at `a0a78ee6`,
which predates it.

**Expected behavior**

Pull requests run the gate, and the `verify` required check fails when a
change strips a committed Docker build input.

**Steps to reproduce**

1. Open any pull request before this change: the ci run shows no "Docker
context integrity" job.
2. After this change, the job runs on every full-CI pull request and
`verify` requires its result.

**Paperclip version or commit**

Pin moves from `a0a78ee60946a5f79f85b2bd0584fc766fae43bb` to `03609aa6`
(the #12858 merge commit).

**Deployment mode**

GitHub Actions pull request CI.

## What Changed

- `.github/workflows/pr.yml`: the `pr-trusted.yml` pin moves to the
#12858 merge commit. One line.

## Verification

- The pinned commit is master's current tip and contains the job, the
`verify` wiring, and both probe revisions; its own CI (on #12855 and
#12858) is fully green.
- This PR's ci run itself executes the newly pinned workflow, so the
gate's first live run is visible on this very pull request.

## Risks

- Low. Identical mechanism to every previous pin bump. If the new lane
misbehaves on some runner, reverting this one line restores the previous
pin.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Fable 5 (Anthropic, model id `claude-fable-5`), extended
thinking, agentic tool use in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (one-line pin bump; the
pinned workflow's tests ran green on #12855/#12858)
- [x] I have added or updated tests where applicable (covered by the pin
tests updated in #12855)
- [x] I have updated relevant documentation to reflect my changes (not
applicable to a pin bump)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 17:22:15 -07:00
Devin Foley bb920fb859
ci: keep the Docker build context complete and guard it on every PR (#12855)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub Actions builds the Docker images that ship Paperclip, and
downstream deployments consume the `-cloud` image variant on every
master merge.
> - PR #12769 slimmed the Docker build context with a broad
`.dockerignore` block for `packages/paperclip-runner`, and the block
also removed three files the image build itself reads.
> - The image build re-runs the runner's generated-file drift checks, so
it found no committed capability contract in the context and failed on
every master commit after the merge.
> - PR CI never runs those checks against the Docker context, so the
pull request stayed green and the breakage only appeared post-merge, on
every image build.
> - This pull request restores the three files with narrow
`.dockerignore` exceptions and adds a PR CI job that runs the drift
checks against the exact Docker build context.
> - The benefit is that image publishing works again now, and the next
context-slimming regression fails the pull request instead of every
post-merge image build.

## Linked Issues or Issue Description

Refs #12769 (the context-slimming change that exposed this) and #12608
(which committed the generated contract outputs the image build checks).

**What happened?**

Every `Docker` workflow run on master failed from 2026-09-04 12:58Z
onward, in both the `build-and-push` and `build-and-push-cloud` jobs.
The failing step reported `Generated contract drift:
generated/capability/capability-contract.md` from
`check:capability-contract` inside `pnpm --filter @paperclipai/server
build`. The committed contract file is current — regeneration on a full
checkout is a no-op. The file was simply absent from the build context:
the new `packages/paperclip-runner/**/*.md` ignore rule strips the
committed drift-check outputs
(`generated/capability/capability-contract.md`,
`generated/capability/downstream-handoff.md`), and the
`packages/paperclip-runner/docs` rule also strips
`docs/capability-contract.md`, which `check:capability-inventory` reads
next in the chain. No cloud image published for eight hours, which
stalled every downstream deployment that consumes the canary images.

**Expected behavior**

The Docker build context must contain every file the image build reads,
and a change that removes one must fail the pull request that introduces
it, not every image build after the merge.

**Steps to reproduce**

1. Check out master at any commit from `af3023f1` onward.
2. Run `docker buildx build -f .github/docker-context-checks.Dockerfile
.` (the probe added by this PR), or start the real `Docker` workflow
build.
3. Observe `Generated contract drift:
generated/capability/capability-contract.md` — while `node
packages/paperclip-runner/scripts/generate-capability-contract.mjs
--check` passes on the same checkout outside Docker.

**Paperclip version or commit**

`d593463ab` (master tip at diagnosis time; first failing commit
`af3023f1`).

**Deployment mode**

GitHub Actions image builds (`docker.yml`), consumed by managed cloud
deployments.

## What Changed

- `.dockerignore`: narrow exceptions (last match wins) re-include the
committed drift-check outputs
(`!packages/paperclip-runner/generated/**`) and the inventory check's
documentation input
(`!packages/paperclip-runner/docs/capability-contract.md`). Every other
exclusion from #12769 stays: no crate declares an explicit `[[test]]`
target, so cargo builds without the `tests` directories, and the image
build chain never runs the excluded smoke scripts.
- `.github/docker-context-checks.Dockerfile` (new): a small probe that
COPYs the real build context — identical `.dockerignore` semantics — and
runs the dependency-independent drift checks inside it
(`generate-capability-contract.mjs --check`,
`check-capability-inventory.mjs`). ajv installs in an isolated directory
for schema validation only; codegen checks such as
`generate-protocol-schema-module` stay out because their emitted bytes
vary with the ajv release and would raise false drift alarms outside the
locked dependency tree.
- `.github/workflows/pr-trusted.yml`: new `docker_context_integrity` job
builds the probe on every full-CI pull request, and the existing
`verify` aggregate now requires its result, so the guard gates merges
through the same required check as the other lanes.
- Activation note: `pr.yml` pins `pr-trusted.yml` by commit SHA, so the
new job starts gating pull requests after the usual follow-up `ci:
activate ...` pin bump once this merges. The `.dockerignore` fix needs
no activation — `docker.yml` reads it directly, so image builds recover
on the first master commit after this merges.

## Verification

- `docker buildx build -f .github/docker-context-checks.Dockerfile .` on
master (before the `.dockerignore` fix): fails with the exact production
error, `Generated contract drift:
generated/capability/capability-contract.md`.
- Same command with the `.dockerignore` exceptions applied: passes,
which also proves BuildKit honors the `!` exceptions, including the file
inside the excluded `docs` directory.
- `node scripts/generate-capability-contract.mjs --check` on a full
checkout: passes both before and after, which confirms the committed
contract was never stale — only missing from the context.
- Static sweep of every script in the image build chain (`build`,
`build:typescript` and their `check:*` steps) against the ignore rules:
the three restored files are the only build inputs the #12769 block
strips.
- YAML for `pr-trusted.yml` lints clean.

## Risks

- Low. The `.dockerignore` exceptions only re-add three committed files
to the build context; image contents do not change otherwise.
- The probe job adds one context transfer and two Node scripts per
full-CI pull request run (about one to two minutes, no dependency
install beyond one isolated ajv package).
- The `verify` aggregate now also requires the new job, mirroring the
existing pattern for the other lanes; on non-full-CI runs the job skips
and `verify` asserts the skip, unchanged from how the other lanes
behave.
- The new job only takes effect for pull requests after a follow-up pin
bump in `pr.yml` (same two-step flow as every `pr-trusted.yml` change).

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Claude Fable 5 (Anthropic, model id `claude-fable-5`), extended
thinking, agentic tool use in Claude Code: GitHub Actions log forensics
to isolate the failing check, static analysis of the build-chain scripts
against the ignore rules, and local docker buildx runs to reproduce the
failure and verify the fix.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (the docker probe, both
failing-before and passing-after; the drift checks themselves on a full
checkout)
- [x] I have added or updated tests where applicable (the probe IS the
regression test for this class)
- [x] I have updated relevant documentation to reflect my changes
(inline comments in `.dockerignore` and the probe explain the invariant)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 15:35:10 -07:00
Dotta 4ef6155aae
ci: harden paid runner browser and lock repair (#12829)
## Thinking Path

Paid cells now reuse the AWS image's system Chrome, but Playwright video
recording still resolves its revision-pinned FFmpeg helper from the
Playwright cache. Run 33875618534 proved Chrome qualification succeeds
and then failed before provider startup because that helper was absent.
The same run also exposed that generic lock repair can churn unrelated
package platform metadata, so the automated repair paths need
resolution-only regeneration rather than lockfile-only metadata refresh.

## What Changed

- install Playwright FFmpeg only on the AWS/system-Chrome path
- retry the small helper installation up to three times before provider
secrets are exposed
- keep the GitHub-hosted Chromium fallback unchanged
- bind static coverage to the exact FFmpeg step block and its pre-secret
ordering
- add pnpm `--resolution-only` to all four automated lock-repair paths
while retaining full transitive resolution
- require resolution-only repair in the shared workflow regression

The actual generated lockfile correction remains bot-owned by PR #12828
and is intentionally not committed here.

## Verification

- `node --test
.github/scripts/tests/lockfile-refresh-workflows.test.mjs`
- `actionlint -ignore SC2012` on all modified workflows
- focused Prettier checks
- `git diff --check`
- prior run 33875618534: system Chrome 151 qualified; missing Playwright
FFmpeg was the sole cell startup failure

## Risks

Low. The new network operation is limited to Playwright's pinned FFmpeg
payload, happens before paid credentials are exposed, and leaves the
hosted-runner path unchanged. Resolution-only is still a full
dependency-resolution pass, unlike lockfile-only, while avoiding
unrelated current-platform metadata churn.

## Model Used

GPT-5
2026-09-04 08:58:44 -05:00
Dotta af3023f1e3
fix(runner): repair paid provider startup paths (#12769)
## Thinking Path

> - Paperclip manages AI agents that perform work.
> - Paperclip Runner connects durable task runs to local provider
processes.
> - The full-stack paid matrix exposed failures after the runner
integrity repair.
> - Verified JavaScript entrypoints lost their relative module graph
when Linux executed them through descriptor paths.
> - Returned provider startup errors also remained pending and became
indeterminate after recovery.
> - Sparse Codex tool lifecycle events lost the `write_document`
identity before task transcript projection.
> - This pull request repairs those three boundaries and makes the
structured-question fixture deterministic.
> - The benefit is repeatable provider startup, exact failure replay,
and correct inline Plan placement.

## Linked Issues or Issue Description

Refs #12721 and #12700.

**What happened?**

The paid runner matrix failed ACPX and OpenCode startup before provider
session creation. The runner journal then replaced the original startup
error with an indeterminate recovery result. Native Codex saved a Plan
but rendered it only as a fallback card. A legacy Claude waiting reply
could also echo the reserved terminal marker before the answer arrived.

**Expected behavior**

Verified JavaScript providers must start from immutable
descriptor-backed artifacts. Returned startup failures must persist as
terminal failed command results. Native tool lifecycle updates must
preserve the `write_document` boundary. Pre-answer fixture output must
not contain the reserved terminal marker.

**Steps to reproduce**

1. Run the local provider cells in the Runner Full-Stack E2E workflow.
2. Observe ACPX and OpenCode fail during `session.open` before provider
execution.
3. Observe recovery report `execution_indeterminate` instead of the
original startup error.
4. Run the native Codex Plan cell and observe the fallback Plan card
after the tool activity row.
5. Run the legacy Claude structured-question resume cell and observe an
early marker echo in waiting prose.

**Paperclip version or commit**

`0f9452101740835ce0b1488a204bf48acd5bafc3`

**Deployment mode**

Local development with the paid GitHub Actions acceptance workflow.

## What Changed

- Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM
entrypoints before hashing and verified descriptor launch.
- Anchor ACPX dynamic provider package resolution at a
controller-derived provider-pack root and keep that root out of the
provider child environment.
- Persist executor-returned startup errors as redacted durable failed
command results while retaining indeterminate recovery for true process
death.
- Coalesce sparse native tool items by stable ID so a late
`write_document` name, input, and result reach the transcript boundary
once.
- Forbid the structured-question fixture from spelling or announcing its
reserved terminal marker before the user answers.

## Verification

- Rust and TypeScript regression tests cover durable failed replay, true
crash ambiguity, bundle closure, package-root derivation, environment
filtering, exact Codex tool lifecycle coalescing, and prompt
determinism.
- Local execution is intentionally limited to formatters and static diff
checks. GitHub Actions will run tests, type checks, builds, and security
checks.
- After ordinary CI is green, scoped paid cells will validate one ACPX
launch, one OpenCode launch, native Codex Plan projection, and legacy
Claude structured resume before a complete matrix rerun.
- Prior failing matrix:
https://github.com/paperclipai/paperclip/actions/runs/33682434315

## Risks

- Bundling changes the bytes covered by provider launch hashes.
Provider-pack generation already hashes the final built files.
- ACPX still loads qualified provider packages dynamically. The
controller supplies a normalized package root, while existing version,
digest, path, and descriptor checks remain active.
- Durable `failed` is terminal. Replays return the same redacted result
and do not execute the provider effect twice.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, parallel subagents, and GitHub Actions
coordination. The exact deployed snapshot and context-window size are
not exposed to this task.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked related public work or described the bug in
this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] No documentation change is required for this runtime repair
- [x] I have considered and documented the risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 07:58:44 -05:00
Dotta 89bf6a33c2
ci: activate Node-first pnpm setup for PRs (#12810)
## Thinking Path

> - Paperclip validates every change through an immutable reusable PR
workflow.
> - That caller still pinned a revision that ran pnpm setup before Node
setup.
> - The implementation fix in #12808 is therefore present on master but
inactive for ordinary PR CI.
> - Advancing the immutable caller pin activates the already tested
Node-first workflow.
> - A focused contract prevents the caller from silently returning to
the old revision.
> - The benefit is a faster PR feedback loop without changing product
code or secret boundaries.

## Linked Issues or Issue Description

Refs #12808

## What Changed

- Pin ordinary PR CI to trusted workflow revision
`a0a78ee60946a5f79f85b2bd0584fc766fae43bb`.
- Assert that the reusable workflow call is canonical, unique, and
SHA-pinned to that audited revision.

## Verification

- Focused workflow security test: 8/8.
- Prettier passed.
- Actionlint passed.
- `git diff --check` passed.

## Risks

Low risk. The change only advances an immutable reusable-workflow pin to
a revision whose full ordinary CI and security checks passed. Product
code and credentials are unchanged.

## Model Used

OpenAI Codex GPT-5 with agentic reasoning and repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used
- [x] I have linked the related public PR
- [x] I have not referenced internal issue links
- [x] My branch name describes the change
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have considered and documented risks
2026-09-03 21:28:49 -05:00
Dotta a0a78ee609
ci: bootstrap Node before pnpm setup (#12808)
## Thinking Path
The trusted workflows currently invoke `pnpm/action-setup@v6` before
installing the repository Node version. On hosts whose ambient Node is
older than 22.13, the action downloads standalone `@pnpm/exe`, which has
repeatedly taken several minutes. Supplying Node 24 first lets the same
pinned pnpm action use its normal Node-backed path.

## What Changed
- install Node 24 before every trusted `pnpm/action-setup` invocation
- preserve the existing pnpm-store cache setup, pinned actions,
telemetry suppression, conditions, and secret boundaries
- enforce ordering, modern Node, condition parity, and cache counts in
the workflow security contract

## Verification
- workflow security: 7/7
- Prettier
- actionlint (excluding one pre-existing SC2129 in an untouched Daytona
shell block)
- `git diff --check`

## Risks
Low. Product code, providers, paid-runner selection, credentials, and
pnpm version are unchanged. Jobs that restore pnpm cache run
`setup-node` a second time after pnpm becomes available; the first setup
is deliberately cache-free.

## Model Used
Codex (GPT-5)
2026-09-03 21:18:08 -05:00
Dotta 18ea965442
ci(runner): stamp paid target provenance (#12805)
## Thinking Path
Trusted workflow-dispatch runs execute an authorized target SHA, but
GitHub context still describes the default-branch workflow revision.
Retained paid results and artifact names were therefore labeling
target-branch executions as master. The workflow must explicitly pass
its authorized target coordinates to target code and trusted reporting.

## What Changed
- emit the canonical authorized target ref alongside the immutable
target SHA
- pass those coordinates to paid cells and the trusted report
- name shared build/provider artifacts with the target SHA rather than
workflow SHA
- add workflow-security coverage for all trusted provenance wiring

## Verification
- focused workflow-security tests: 6/6 passed
- Prettier and git diff checks passed
- run 33823252706 independently proved the pre-fix defect: functionally
green target cells were retained as master SHA 0ad180b85 instead of
feature SHA 33c7646d3

## Risks
The execution checkout and secret boundary were already pinned
correctly; this changes retained attribution and artifact labels only.
Target-side report code on PR #12769 consumes these trusted environment
values and overwrites untrusted cell metadata.

## Model Used
Codex (GPT-5)
2026-09-03 21:15:47 -05:00
Dotta 0ad180b85f
ci(runner): skip bootstrap registry telemetry (#12797)
## Thinking Path
Every trusted PR and paid-workflow job invokes the pinned pnpm setup
action. Its internal npm install is currently waiting four to seven
minutes on npm audit telemetry before any Paperclip or provider code
runs. Audit, funding, and update notifications are not integrity
controls for this action; its committed lockfile still verifies
installed package bytes.

## What Changed
- disable npm audit, funding, and update-notifier telemetry narrowly on
all seven pinned setup steps in each of the trusted PR and full-stack
workflows
- add a workflow security contract proving every setup invocation
remains covered and the overrides do not leak elsewhere

## Verification
- focused workflow security tests: 6/6 passed
- Prettier and git diff checks passed
- observed unhealthy setup: 4-7+ minutes; historical healthy setup:
about four seconds

## Risks
This skips npm vulnerability-report telemetry for the setup action
bootstrap only. Repository dependency checks, lockfile integrity,
provider-secret authorization, and target-lock verification remain
unchanged.

## Model Used
Codex (GPT-5)
2026-09-03 19:47:46 -05:00
Dotta 03faa644fb
ci(runner): inspect Daytona image metadata remotely (#12795)
## Thinking Path
The reused Daytona image path already verifies the signed immutable
digest. It then downloads every filesystem layer only to read OCI config
fields. Buildx can retrieve the same config from that immutable digest
without pulling the layers. The assertions can therefore stay intact
while removing the expensive transfer.

## What Changed
- inspect the signed immutable Daytona image config through Buildx after
GHCR logout
- preserve digest, source revision, content ID, platform, user, and
provider-pack assertions
- extend the workflow contract test for the metadata-only path

## Verification
- Daytona image and workflow security tests: 10 passed
- Prettier and git diff checks passed
- observed full pull/prune cost: about 4m55s; metadata inspection: about
one second

## Risks
The current image has one runnable linux/amd64 platform plus its
attestation. A future genuinely multi-platform image would need explicit
linux/amd64 selection.

## Model Used
Codex (GPT-5)
2026-09-03 18:10:38 -05:00
Dotta d3c04d8932
fix(runner-e2e): prepare frozen Daytona plugin dependencies (#12791)
## Thinking Path

> - Paperclip manages AI agents and their provider runtimes.
> - The paid runner workflow installs target dependencies with lifecycle
scripts disabled.
> - The bundled Daytona plugin depends on an audited repo-local plugin
SDK link.
> - The lifecycle-safe install path did not create that link.
> - This pull request restores only the trusted Daytona preparation step
before provider secrets are exposed.
> - The benefit is a working Daytona canary without enabling dependency
lifecycle scripts.

## Linked Issues or Issue Description

**What happened?**

The Daytona paid canary stopped before lease or provider startup. The
trusted paid job disabled root lifecycle scripts, so the repo-local
plugin SDK link was absent. The plugin install returned a missing
runtime dependency error for @paperclipai/plugin-sdk.

**Expected behavior**

The trusted workflow must prepare the bundled Daytona plugin without
running untrusted dependency lifecycle scripts. The paid cell must start
only after its runtime dependencies and entrypoints pass validation.

**Steps to reproduce**

1. Dispatch the runner full-stack paid workflow for
core-compatibility.runner-acpx-claude.daytona.message-marker.
2. Let the trusted job install root dependencies with lifecycle scripts
disabled.
3. Observe the Daytona plugin installation fail before a lease or
provider process starts.

**Paperclip version or commit**

Feature head 781ac7e08c. The failed canary
is Actions run 33803959325.

**Deployment mode**

GitHub Actions paid runner validation.

**Agent adapter(s) involved**

ACPX Claude through the bundled Daytona plugin.

**Additional context**

This is a small trusted-workflow prerequisite for public PR #12769. Old
green run 33118525827 created the SDK link through root postinstall.
This change keeps lifecycle scripts disabled and restores only the
audited prerequisite.

## What Changed

- Install standalone Daytona dependencies with lifecycle scripts
disabled.
- Run the audited repo-local plugin SDK linker before provider secrets
are exposed.
- Build the bundled Daytona plugin and verify its runtime dependency
plus both entrypoints.
- Add a security regression for ordering, scope, and secret isolation.

## Verification

- Five focused workflow-security tests passed.
- Seven focused linker tests passed.
- The exact Daytona preparation command completed locally in nine
seconds.
- Prettier and diff whitespace checks passed.

## Risks

Risk is low and limited to Daytona paid cells. The setup still disables
dependency lifecycle scripts. The trusted step runs before provider
credentials enter the job. Any missing or mismatched path fails closed
before provider startup.

## Model Used

OpenAI GPT-5 Codex with repository tools and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR with the bug template labels
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] Focused local tests pass
- [x] I added tests for the change
- [x] I updated the relevant trusted-workflow security regression
- [x] I documented the risks above
2026-09-03 16:31:22 -05:00
Dotta 313d6ca115
fix(runner): materialize pinned OpenCode binary (#12782)
## Thinking Path

> - Paperclip manages AI agents and their provider runtimes.
> - Paid runner validation installs target dependencies with lifecycle
scripts disabled.
> - OpenCode leaves a sentinel executable until its package lifecycle
script runs.
> - Running arbitrary lifecycle code would weaken the paid-secret
boundary.
> - This pull request materializes one exact pinned binary before
secrets are exposed.
> - The benefit is working OpenCode validation without trusting
dependency install scripts.

## Linked Issues or Issue Description

**What happened?**

Every local OpenCode paid cell stopped before provider startup because
`pnpm install --ignore-scripts` correctly retained
`opencode-ai/bin/opencode.exe` as a sentinel.

**Expected behavior**

The trusted workflow must make the exact lockfile-pinned OpenCode
executable available without running package lifecycle scripts.

**Steps to reproduce**

Run a local legacy or native OpenCode paid cell from the trusted
workflow after the target dependency install. The provider health check
reports that the OpenCode postinstall script was not run.

**Paperclip version or commit**

Default branch commit `865b4854fb44d3689f1c0ff17e3e715d52aaea73`.

## What Changed

- Materialize only `opencode-linux-x64-baseline@1.18.17` into the
matching `opencode-ai@1.18.17` package.
- Verify package identity, version, regular-file type, SHA-256 equality,
executable permissions, and runtime `--version`.
- Invoke the helper for local OpenCode and breadth cells and for remote
provider-pack assembly.
- Retain `pnpm install --ignore-scripts`.
- Add helper and trusted-workflow security regressions.

## Verification

- Helper syntax checks passed.
- Helper unit tests passed: 2/2.
- Workflow-security tests passed: 5/5.
- Prettier, actionlint, and diff whitespace checks passed.

## Risks

Risk is low and contained to paid runner setup. The helper supports only
Linux x64, fails closed on package or version drift, and runs before
provider credentials enter the job.

## Model Used

OpenAI GPT-5 Codex with repository tools and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change.
- [x] I have specified the model used.
- [x] I have checked ROADMAP.md and confirmed this does not duplicate
planned core work.
- [x] I have searched GitHub for duplicate or related PRs and found
none.
- [x] I have described the issue in this PR with the bug template
labels.
- [x] I have not referenced internal or instance-local issues.
- [x] My branch name describes the change.
- [x] Focused local tests pass.
- [x] I added tests for the change.
- [x] I updated the runner E2E security documentation.
- [x] I documented the risks above.
2026-09-03 14:12:17 -05:00
Dotta 865b4854fb
ci(runner): build paid artifacts once per campaign (#12777)
## Thinking Path

> - Paid cells repeated the same TypeScript and Rust builds even when
one campaign selected dozens of cells.
> - The trusted workflow can compile once without provider credentials
and distribute run-scoped, digest-verified artifacts.
> - The paid cell can then disable install lifecycle scripts, verify
each artifact before extraction, and expose provider credentials only to
the final test step.
> - Local JS-backed providers also need the setup-node interpreter
permission-qualified before Rust verifies the launch artifact.

## Linked Issues or Issue Description

Run 33786122875 proved target-lock setup and catalog selection, then
failed before provider creation because trusted master did not yet
qualify the setup-node interpreter. The same workflow also rebuilt
TypeScript and Rust inside every matrix cell.

## What Changed

- Build runner TypeScript and native binaries once per campaign in a
credential-free job.
- Build the remote provider pack once only when selected Daytona cells
require it.
- Upload run-scoped bundles with SHA-256 manifests and verify before
extraction in each paid cell.
- Remove repeated TypeScript, provider-pack, and Rust builds from paid
cells.
- Qualify the local provider Node interpreter before verified launch.
- Propagate the resolved target lockfile through all five target-code
jobs.
- Keep local-only selection off Daytona and exclude Xiaomi from the
67-cell catalog.

## Risks

A shared build artifact could fan out a bad payload to many cells. The
producing jobs receive no provider credentials, use the exact authorized
target SHA and resolved lockfile, and publish run-scoped artifacts.
Every consuming job verifies SHA-256 before extraction. Paid dependency
setup keeps lifecycle scripts disabled and provider credentials remain
scoped to the final test step.

## Verification

- Focused runner workflow-security, catalog, and Daytona-image tests:
25/25 passed.
- Prettier passed.
- Actionlint passed with only the two pre-existing SC2129 style notices
ignored.
- Git diff check passed.

## Model Used

OpenAI Codex, GPT-5.

## Checklist

- [x] Build jobs are credential-free.
- [x] Paid installs disable lifecycle scripts.
- [x] Artifacts are run-scoped and digest-verified before extraction.
- [x] Trusted report and history jobs remain isolated from target
artifacts.
- [x] No Daytona or Xiaomi paid run was started for this change.
2026-09-03 13:12:58 -05:00
Dotta 6e50ca9d0a
ci(runner): prepare target lockfile once for paid validation (#12774)
## Thinking Path

> - The trusted target-branch runner workflow checks out PR code before
paid tests.
> - PR policy intentionally forbids manual lockfile commits.
> - Some runner changes legitimately alter pnpm patch hashes.
> - Frozen installs therefore fail before test selection.
> - Resolve one script-disabled lockfile from the authorized immutable
target SHA and distribute it by exact artifact ID and digest.
> - Keep provider credentials and trusted reporting outside this
resolution job.

## Linked Issues or Issue Description

Target-branch paid runner campaigns currently fail frozen install when a
PR changes pnpm patch content, even though ordinary PR CI regenerates
the lockfile.

## What Changed

- Added one credential-free target-lock job that resolves the authorized
immutable target SHA with lifecycle scripts disabled.
- Uploaded the resolved lockfile with its SHA-256 and restored it by
exact artifact ID before every target-code frozen install.
- Left trusted reporting and history jobs on the workflow SHA.
- Changed the disabled-AWS fallback from unavailable ubuntu-latest-m to
ubuntu-latest.

## Risks

The workflow evaluates pnpm lockfile resolution from authorized target
code. That job receives no provider credentials, disables lifecycle
scripts, rejects unrelated workspace mutations, and exposes only a
digest-verified lockfile artifact. Paid-secret jobs consume only that
lockfile after exact artifact-ID and SHA-256 validation.

## Verification

- Runner workflow-security focused tests pass.
- actionlint passes.
- Prettier and git diff checks pass.

## Model Used

OpenAI Codex, GPT-5.

## Checklist

- [x] Change is narrowly scoped to paid runner orchestration.
- [x] Target lock resolution has no provider credentials and disables
lifecycle scripts.
- [x] Downloaded artifacts are selected by exact artifact ID and
verified by SHA-256.
- [x] Trusted reporting and history jobs remain on the workflow SHA.
2026-09-03 12:39:18 -05:00
dependabot[bot] 39898ab22f
chore(deps): bump paperclipai/paperclip/.github/workflows/pr-trusted.yml from 39b8ee2960 to f038633bf5 (#12562)
Bumps
[paperclipai/paperclip/.github/workflows/pr-trusted.yml](https://github.com/paperclipai/paperclip)
from 39b8ee2960 to
f038633bf5.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/paperclipai/paperclip/blob/master/doc/RELEASE-AUTOMATION-SETUP.md">paperclipai/paperclip/.github/workflows/pr-trusted.yml's
changelog</a>.</em></p>
<blockquote>
<h1>Release Automation Setup</h1>
<p>This document covers the GitHub and npm setup required for the
current Paperclip release model:</p>
<ul>
<li>automatic canaries from <code>master</code></li>
<li>manual stable promotion from a chosen source ref</li>
<li>npm trusted publishing via GitHub OIDC</li>
<li>protected release infrastructure in a public repository</li>
</ul>
<p>Repo-side files that depend on this setup:</p>
<ul>
<li><code>.github/workflows/release.yml</code></li>
<li><code>.github/CODEOWNERS</code></li>
</ul>
<p>Note:</p>
<ul>
<li>the release workflows intentionally use <code>pnpm install
--no-frozen-lockfile</code></li>
<li>this matches the repo's current policy where
<code>pnpm-lock.yaml</code> is refreshed by GitHub automation after
manifest changes land on <code>master</code></li>
<li>the publish jobs then restore <code>pnpm-lock.yaml</code> before
running <code>scripts/release.sh</code>, so the release script still
sees a clean worktree</li>
</ul>
<h2>1. Merge the Repo Changes First</h2>
<p>Before touching GitHub or npm settings, merge the release automation
code so the referenced workflow filenames already exist on the default
branch.</p>
<p>Required files:</p>
<ul>
<li><code>.github/workflows/release.yml</code></li>
<li><code>.github/CODEOWNERS</code></li>
</ul>
<h2>2. Configure npm Trusted Publishing</h2>
<p>Do this for every public package that Paperclip publishes.</p>
<p>At minimum that includes:</p>
<ul>
<li><code>paperclipai</code></li>
<li><code>@paperclipai/server</code></li>
<li><code>@paperclipai/ui</code></li>
<li>public packages under <code>packages/</code></li>
</ul>
<h3>2.1. In npm, open each package settings page</h3>
<p>For each package:</p>
<ol>
<li>open npm as an owner of the package</li>
<li>go to the package settings / publishing access area</li>
<li>add a trusted publisher for the GitHub repository
<code>paperclipai/paperclip</code></li>
</ol>
<h3>2.2. Add one trusted publisher entry per package</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f038633bf5"><code>f038633</code></a>
feat(runner): reduce ACPX provider state (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12417">#12417</a>)</li>
<li><a
href="7bb6cebeae"><code>7bb6ceb</code></a>
feat(runner): normalize ACPX provider events (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12416">#12416</a>)</li>
<li><a
href="fe2ddfad2b"><code>fe2ddfa</code></a>
feat(runner): validate ACPX event payloads (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12415">#12415</a>)</li>
<li><a
href="3db24d9366"><code>3db24d9</code></a>
feat(runner): bind ACPX event scope (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12414">#12414</a>)</li>
<li><a
href="75708fec6d"><code>75708fe</code></a>
feat(runner): add ACPX sidecar transport (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12412">#12412</a>)</li>
<li><a
href="9ad8dbffa0"><code>9ad8dbf</code></a>
feat(runner): add Codex ACPX sidecar (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12410">#12410</a>)</li>
<li><a
href="b93ad538b6"><code>b93ad53</code></a>
test(runner): add question adapter conformance (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12409">#12409</a>)</li>
<li><a
href="4fe3189f02"><code>4fe3189</code></a>
feat(runner): bridge Codex ACPX questions (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12408">#12408</a>)</li>
<li><a
href="96421b0663"><code>96421b0</code></a>
feat(runner): recover settled Codex ACPX sessions (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12407">#12407</a>)</li>
<li><a
href="30ef14edd4"><code>30ef14e</code></a>
feat(runner): wire the Codex ACPX backend (<a
href="https://redirect.github.com/paperclipai/paperclip/issues/12406">#12406</a>)</li>
<li>Additional commits viewable in <a
href="39b8ee2960...f038633bf5">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 10:23:42 -07:00
github-actions[bot] fa16f88d6b
chore(lockfile): refresh pnpm-lock.yaml (#12771)
Use full dependency resolution in automated lockfile repair paths, add regression coverage, and refresh the stale Rollup snapshot.

Co-Authored-By: Dotta <cryppadotta@users.noreply.github.com>
Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: lockfile-bot <lockfile-bot@users.noreply.github.com>
2026-09-03 11:08:34 -05:00
Dotta 98c569b2df
ci(runner): allow trusted branch targets (#12768)
## Thinking Path

> - Paperclip uses paid runner tests to qualify agent execution.
> - The runner workflow controls provider secrets and AWS runner access.
> - The trusted workflow must stay on the protected default branch.
> - The code under test often exists on a branch before merge.
> - CODEOWNERS need a safe way to select that branch.
> - This pull request separates workflow authority from the code under
test.
> - The benefit is pre-merge AWS testing without target-controlled
workflow code.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The manual Runner Full-Stack E2E workflow can test only the default
branch.

**Subsystem affected**

GitHub Actions and the paid runner E2E security boundary.

**Current behavior**

A CODEOWNER must merge runner changes before the trusted AWS workflow
can test them.
Selecting another branch as the workflow ref is rejected.

**Proposed behavior**

A CODEOWNER starts the workflow from `master` and supplies a
same-repository branch in `target_branch`.
The authorization job resolves the branch to one commit SHA.
Catalog, image, and paid test jobs check out that SHA after
authorization.
Report sanitization and AWS publication use the trusted workflow SHA.

**Reason and benefit**

This permits paid pre-merge qualification on AWS.
It keeps the workflow definition, report sanitizer, history publisher,
environment deployment, and runner-group permission on `master`.

**Breaking changes**

None.
The new input is optional.
An omitted input still tests the default branch.

## What Changed

- Add the optional `target_branch` workflow input.
- Resolve only a branch in `paperclipai/paperclip` to an immutable SHA.
- Pin catalog, image, paid test, and Daytona provenance to the target
SHA.
- Pin report sanitization and AWS history publication to the trusted
workflow SHA.
- Disable persisted checkout credentials in every job.
- Key cancellation by the selected target branch.
- Add policy regression coverage and operator documentation.

## Verification

- `pnpm test:e2e:runner:unit` passes with 65 tests.
- `actionlint -ignore SC2129
.github/workflows/runner-full-stack-e2e.yml` passes.
- Prettier checks pass for all changed files.
- `git diff --check` passes.

## Risks

A CODEOWNER can authorize selected branch code to receive a cell-scoped
provider credential.
This is the intended trust decision.
The workflow rejects fork refs and target-controlled workflow
definitions.
The trusted workflow SHA owns report sanitization and AWS history
publication.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5.
The exact serving snapshot and context-window size are not exposed.
The model used tool-enabled reasoning and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-03 10:38:56 -05:00
Dotta 1b74561fea
ci(runner): route paid matrix to AWS fleet (#12765)
## Thinking Path

> - Paperclip manages AI agents that perform work.
> - The paid runner matrix verifies complete runner behavior with real
providers.
> - Each matrix job currently repeats work on GitHub-hosted runners.
> - Paperclip has an ephemeral AWS runner fleet for trusted workflows.
> - The paid workflow needs a reviewed and fail-closed route to that
fleet.
> - This pull request adds that route and keeps the existing hosted
runner as the disabled-state fallback.
> - The benefit is faster paid campaigns with the same actor,
environment, and secret boundaries.

## Linked Issues or Issue Description

**What happened?**

The Runner Full-Stack E2E workflow always uses `ubuntu-latest-m`. It
limits the matrix to 57 parallel jobs. The repository AWS fleet can run
100 ephemeral jobs, but the paid workflow cannot select it.

**Expected behavior**

An explicit repository flag must select the reviewed AWS fleet label. A
missing or invalid flag must keep the existing hosted runner. The
workflow must authorize the stable actor identity before it routes any
paid job.

**Steps to reproduce**

1. Dispatch the Runner Full-Stack E2E workflow from `master`.
2. Inspect a paid matrix job.
3. Observe that the job requests `ubuntu-latest-m` even when the AWS
fleet should be used.

**Paperclip version or commit**

`da0947d3582ac7779d6bf11851c9938eca6c5c8c`

**Deployment mode**

GitHub Actions paid runner campaign.

## What Changed

- Add a fail-closed `RUNNER_E2E_AWS_ENABLED` switch.
- Select only the reviewed AWS fleet label or the existing hosted label.
- Permit up to 100 parallel jobs in AWS mode.
- Keep the hosted-runner limit at 57.
- Reauthorize paid execution before checkout and provider access.
- Stop paid checkouts from storing GitHub credentials.
- Cancel superseded validation-ref campaigns while preserving `master`
audit runs.
- Add workflow policy checks and operator documentation.

## Verification

- `git diff --check`
- `actionlint -ignore SC2129
.github/workflows/runner-full-stack-e2e.yml`
- The organization runner group permits this workflow only from
`refs/heads/master`.
- The repository AWS switch remains disabled until this pull request is
merged and a one-cell probe succeeds.

## Risks

- A wrong fleet policy can leave jobs queued. The disabled state keeps
the existing hosted runner.
- The AWS fleet uses paid compute. The workflow validates a configured
maximum of 100 jobs.
- The runner group, actor allowlist, and paid environment remain
separate enforcement layers.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, GitHub API coordination, and static
workflow analysis. The exact deployed model identifier and
context-window size are not exposed to this task.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-03 08:50:36 -05:00
Dotta 0f94521017
fix(runner): restore local session and task integrity (#12721)
## Thinking Path

> - Paperclip is the control plane for agents that perform work.
> - Paperclip Runner connects durable provider sessions to individual
task runs through PRP.
> - Provider continuity and per-run authority are different lifetimes.
> - The existing implementation mixed those lifetimes and lost event
metadata between provider frames, runnerd, persistence, API
sanitization, and the task thread.
> - That caused failed continuation, missing progress and Plans,
duplicate replies, hidden failures, and unsafe recovery.
> - This repair gives every heartbeat fresh authority, preserves
qualified provider-session continuity, and restores one lossless
presentation path without changing direct adapters.

## Linked Issues or Issue Description

**What happened?**

A second native heartbeat could reuse tickets, leases, command receipts,
sequence state, and run identity from the first heartbeat. Provider
phase and item identity could be lost before the UI read them. Redaction
could corrupt protocol discriminators while still missing malformed
credential tails. The task thread could fold progress into the final
response, hide failures, or show more than one final answer. Native
Codex also exposed approval modes that do not yet have a durable
approval bridge.

**Expected behavior**

Each heartbeat uses a new PRP authority epoch. Codex and OpenCode
preserve exact qualified provider sessions; ACPX emits an explicit
continuity event when its qualified process-replacement policy is used.
Every accepted provider event is presented, classified as internal, or
surfaced as unsupported. The task page shows chronological progress,
reasoning summaries, activity, Plans, interactions, terminal failures,
and exactly one final reply. Direct adapters retain their existing path.

**Steps to reproduce**

1. Enable the unified experimental Paperclip Runner setting.
2. Create a local native Codex, OpenCode, ACPX Claude, or ACPX Codex
agent.
3. Run response, Plan, structured-question/resume, restart,
cancellation, and failure scenarios.
4. Reload the task while active, waiting, failed, and settled.
5. On the old implementation, observe stale run authority, missing
classifications, incomplete output, or duplicated/folded replies.

**Paperclip version or commit**

The repair is based directly on `master` at
`87d05e194b643810d16d20612115acd01d735d43`.

**Deployment mode**

Local development with the embedded database.

Related work: Refs #12616, #12646, #12666, #12685, and #12700.

## What Changed

- Rotates PRP control-plane, outbox, ticket, lease, command, receipt,
and sequence authority for each heartbeat while carrying forward only a
validated provider-session identity.
- Reads `control-plane-state.json`, validates both durable schemas and
lifecycle values, resumes coherent current runs, archives qualified
settled authority, and quarantines malformed or mismatched scoped state
without moving ambiguous live legacy state.
- Preserves Codex provider phase and stable item identities so
commentary remains progress and only `final_answer` becomes final.
- Adds raw OpenCode HTTP/SSE boundary coverage and canonical reasoning
lifecycle mapping.
- Makes ACPX normalization lossless for visible reasoning, tool
lifecycle metadata, stable bounded identities, Plan revisions,
structured requests, failures, and qualified process replacement. Only
the compatible terminal assistant message is promoted as final.
- Applies schema-aware redaction before generic JWT-shaped detection and
scans every diagnostic string leaf. Malformed raw/escaped quoted
credential tails are redacted in both server and durable Rust state.
- Restores snapshot-style chronological task presentation, expandable
tool activity, inline Plan cards, visible waiting/resume/cancel/failure
states, and exactly one final answer.
- Makes `never` the only qualified native Codex permission mode and
rejects unsupported persisted native modes with remediation. OpenCode
and ACPX policies remain intact.
- Keeps the unified experimental Runner setting as the only enablement
flag. Onboarding and direct Codex, Claude, and OpenCode stay on their
legacy execution/finalization paths.
- Adds cross-language goldens, authority/recovery/fault coverage, exact
response/count assertions, and native plus legacy acceptance scenarios.

## Verification

- Pull-request GitHub Actions run Rust formatting/tests, TypeScript
checks, server/UI tests, builds, protocol drift checks, browser E2E, and
security scans.
- A separate workflow-only validation ref is pinned directly on this PR
head and runs the 35-cell paid local matrix: three core scenarios plus
structured-question resume and restart/resume for native Codex, native
OpenCode, ACPX Claude, ACPX Codex, and direct Codex/Claude/OpenCode.
Run: https://github.com/paperclipai/paperclip/actions/runs/33682434315
- Acceptance requires exact single visible replies, monotonic sequences,
matching envelope discriminators, one semantic terminal, one run
terminal, no unresolved interaction, no duplicate mutation, no secret
leakage, provider continuity, and zero native rows for direct adapters.
- Per maintainer direction, tests are running in GitHub Actions rather
than on the slower local host. Only formatters and static diff checks
were run locally.

## Risks

- Recovery from old or partial filesystem state is sensitive. The repair
fails closed, preserves active or unverifiable authority, and
quarantines only state whose scoped ownership is safe to move.
- Provider event formats can change. Closed validators and boundary
goldens turn new or malformed events into visible diagnostics instead of
silent drops.
- Shared task presentation could affect direct adapters. Runtime-fact
gating plus the direct-adapter matrix protect the existing path.
- Managed and remote providers are not qualified here. Shared code
continues to compile and fail safely, but live qualification is
deferred.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5. The exact deployed snapshot and
context-window size are not exposed to this task. It used agentic
reasoning, repository inspection, code editing, Git, parallel subagents,
and GitHub Actions.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] The paid local-provider matrix is green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-02 16:11:26 -05:00
Dotta 5716fe907e
test(runner): add full-stack acceptance and eval gates (#12700)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.

**Problem or motivation**

The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.

**Proposed solution**

Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.

**Alternatives considered**

We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.

**Roadmap alignment**

This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.

Related stack:

- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.

## What Changed

- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
  and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.

## Verification

Passed locally:

- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.

I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.

## Risks

The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.

Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.

Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.

The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.

This pull request does not enable the experimental native runner for
existing instances.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-02 08:55:08 -05:00
Dotta dda4dff645
fix(onboarding): restore browser launch and gate canaries (#12667)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The onboarding command starts the local server and opens the
first-run wizard.
> - Interactive onboarding stopped opening the browser by default.
> - Organization creation could also succeed in the API while the wizard
stayed on the name step.
> - The npm canary workflow did not prove that the published package
could complete this path.
> - This pull request repairs the startup and organization transitions
and adds an exact-version canary smoke gate.
> - The benefit is a quickstart that works for users and is tested after
each canary publish.

## Linked Issues or Issue Description

Related: #12557 covers a separate final-route onboarding handoff.

**What happened?**

Interactive `paperclipai onboard` runs did not open the onboarding page.
The organization API request could succeed while a same-company context
update caused the wizard to stay on the organization step. The canary
release lane did not test the exact published npm package through this
path.

**Expected behavior**

Interactive onboarding must open the browser once. A successful
organization request must advance to the first-agent step when the
surrounding context adopts the same organization. Each published canary
must install in a clean environment and reach the model connection step.

**Steps to reproduce**

1. Run `npx paperclipai@canary onboard --data-dir "$(mktemp -d
/tmp/paperclip-canary.XXXXXX)"` in an interactive terminal.
2. Enter an organization name while the company context refreshes from
the create response.
3. Observe that the browser does not open or that the wizard can remain
on the organization step after the API creates it.
4. Inspect the canary release lane and observe that no post-publish
onboarding test runs against the exact npm version.

**Paperclip version or commit**

The issue reproduced with `2026.901.0-canary.8` and the source state
before this pull request.

**Deployment mode**

Local trusted quickstart with embedded PostgreSQL. The install source
can be npm or a source checkout.

## What Changed

- Open the browser once for interactive foreground onboarding.
- Preserve explicit browser opt-outs and restore the prior environment
value after startup.
- Accept a same-company context update after organization creation and
reject a different-company takeover with an explicit error.
- Export the exact canary version from the publish job.
- Install and test that exact npm version in a clean Playwright smoke
job through the "Connect a model" step.
- Upload server logs, traces, screenshots, and the Playwright report
when the canary smoke fails.
- Document the interactive default and headless opt-outs.

## Verification

- `pnpm exec vitest run cli/src/__tests__/onboard.test.ts
ui/src/components/OnboardingWizard.step.test.tsx --reporter=dot` passes
with 37 tests.
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
passes with 9 tests.
- `pnpm exec playwright test --config tests/e2e/playwright.config.ts
tests/e2e/onboarding.spec.ts` passes with 2 tests.
- `PAPERCLIPAI_VERSION=2026.901.0-canary.8 pnpm run
test:canary-onboarding-smoke` passes against the published npm package.
- `pnpm check:token-gates` passes.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- A fresh interactive source run opens the browser and reaches "Connect
a model" after organization and agent naming.

## Risks

- Low risk. Automatic browser opening only applies to interactive
foreground onboarding.
- `PAPERCLIP_NO_BROWSER=1` and `PAPERCLIP_OPEN_ON_LISTEN=false` keep
headless runs silent.
- A different organization context still blocks the pending create
transition.
- The canary package is immutable before the smoke runs. A smoke failure
leaves the package published but makes the release workflow red.
- This change does not modify REST APIs, database schemas, or shared
data types.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex based on GPT-5. The runtime does not expose the exact
deployment snapshot or context-window size. The model used reasoning,
browser automation, repository tools, shell commands, code editing, and
test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-01 10:10:30 -05:00
Dotta e6ad2ea3d9
ci: activate stacked pull request optimization (#12509)
## Thinking Path

> - Paperclip uses GitHub Actions to verify pull requests before merge.
> - The pull request workflow calls a trusted reusable workflow at an
immutable commit SHA.
> - Pull request #12507 added stack-aware CI scope to that reusable
workflow.
> - The caller still points to the prior workflow commit, so the new
behavior is not active.
> - This pull request advances the caller to the merged commit from pull
request #12507.
> - The benefit is that native stack middle layers stop starting
redundant full CI matrices.

## Linked Issues or Issue Description

Refs #12507

**What existing behavior does this improve?**

The pull request workflow still calls the trusted CI definition that
predates stack-aware scope selection.

**Subsystem affected**

GitHub Actions pull request verification.

**Current behavior**

Every pull request layer in a native stack starts the complete test,
build, canary, and E2E matrix.

**Proposed behavior**

Call the trusted workflow from merged master commit
`39b8ee2960541d14b380f95365deecba6723d9bd`. That workflow runs full CI
for ordinary, top, and lowest-unmerged pull requests. It keeps policy
and required aggregate checks on middle layers.

**Reason and benefit**

This completes the two-step immutable workflow rollout from pull request
#12507. Large stacks will use fewer runners and will spend less time
waiting for duplicate jobs.

**Breaking changes**

Middle native stack layers no longer run the complete CI matrix.
Required aggregate checks remain present and fail closed if the stack
scope is missing or invalid.

## What Changed

- Pin `.github/workflows/pr.yml` to merged master commit
`39b8ee2960541d14b380f95365deecba6723d9bd`.
- Activate the stack-aware trusted workflow that merged in pull request
#12507.

## Verification

- `node --test scripts/__tests__/e2e-shard.test.mjs` — 11 tests passed.
- `actionlint .github/workflows/pr.yml .github/workflows/pr-trusted.yml`
— passed.
- `git diff --check origin/master...HEAD` — passed.
- Confirm that the caller SHA equals the merge commit for pull request
#12507.

## Risks

- The caller is immutable and points to a commit that exists on
`master`.
- Ordinary pull requests and merge-relevant stack layers still run full
CI.
- A rollback can restore the prior immutable SHA in one line.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment suffix and context
window are not exposed. The model used reasoning, repository tools, code
execution, Git, and GitHub API access.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 10:29:34 -05:00
Dotta 39b8ee2960
ci: optimize checks for stacked pull requests (#12507)
## Thinking Path

> - Paperclip uses GitHub Actions to protect changes before they enter
`master`.
> - GitHub evaluates every pull request in a native stack against the
stack base.
> - The current workflow therefore starts the complete CI matrix for
every layer in a stack.
> - A large stack can queue many copies of the same integrated
verification and delay every pull request.
> - GitHub provides stack position and base metadata so workflows can
select merge-relevant layers.
> - This pull request keeps policy and required check names on every
layer, but runs full CI only for ordinary pull requests, the top layer,
and the lowest unmerged layer.
> - The benefit is much lower CI load without weakening the
required-check contract.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The trusted pull request workflow currently runs every test, build,
canary, and E2E lane for every pull request in a native stack.

**Subsystem affected**

GitHub Actions pull request verification.

**Current behavior**

A stack with 61 pull requests can start 61 complete CI matrices after a
cascading rebase.

**Proposed behavior**

Run the always-on policy job and stable required-check aggregators for
every layer. Run the complete verification matrix only for ordinary pull
requests, the top stack layer, and the lowest unmerged stack layer.

**Reason and benefit**

The top layer verifies the integrated stack. The lowest unmerged layer
verifies the current merge candidate. Middle layers keep
branch-protection checks without consuming the complete runner matrix.

**Breaking changes**

Middle stack layers no longer run the complete CI matrix. Their `ci /
verify` and `ci / e2e` checks still require the policy job to pass and
require every expensive lane to be intentionally skipped.

## What Changed

- Add a fail-safe stack scope decision to the trusted PR runner gate.
- Run typecheck, general tests, build, serialized tests, canary, and E2E
shards only for ordinary, top, and lowest-unmerged pull requests.
- Preserve the required `ci / verify` and `ci / e2e` names on every
layer.
- Make the required aggregators distinguish valid middle-layer skips
from failures or missing scope decisions.
- Add regression coverage for ordinary, top, bottom, middle, and
malformed stack metadata.

## Verification

- `node --test scripts/__tests__/e2e-shard.test.mjs` — 11 tests passed.
- `actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml`
— passed.
- `git diff --check origin/master...HEAD` — passed.
- The caller remains pinned to the current trusted workflow. A separate
activation change must advance the immutable SHA after this pull request
lands.

## Risks

- Incorrect stack classification could skip important jobs. Missing or
malformed stack metadata defaults to full CI.
- Middle-layer required checks depend on the policy job and verify that
all expensive jobs have the `skipped` result.
- The reusable workflow change does not become active until the
immutable caller SHA advances in a separate change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment suffix and context
window are not exposed. The model used reasoning, repository tools, code
execution, Git, and GitHub API access.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 10:26:38 -05:00
Dotta 40d8cbc41a
ci: activate stacked lockfile regeneration (#12464)
## Thinking Path

> - Paperclip uses trusted GitHub Actions workflows to verify each pull
request.
> - The caller pins the reusable workflow to an immutable merged commit.
> - Native stacked pull requests can inherit a stale lockfile from a
parent layer.
> - Pull request #12461 added safe merge-tree lockfile regeneration to
the trusted workflow.
> - The caller must now select that merged workflow version.
> - This pull request advances the immutable pin to the merge commit
from #12461.
> - The benefit is reliable verification for native stacked pull
requests.

## Linked Issues or Issue Description

Refs #12461

## What Changed

- Pin the trusted pull request workflow to merged commit
`1da6b37fc56dacf5e7ffbd31da35756a1cba41f8`.
- Activate stale lockfile regeneration for native stacked pull requests.

## Verification

- `actionlint .github/workflows/pr.yml`
- `node --test scripts/__tests__/e2e-shard.test.mjs`

## Risks

- Low risk. This changes only the immutable reusable-workflow pin.
- The target commit is merged into `master` and passed all required
checks.
- The target workflow adds one lockfile-only install to the policy job.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with model `gpt-5`, reasoning, tool use, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 20:12:48 -05:00
Dotta 1da6b37fc5
fix(ci): regenerate stale stacked lockfiles (#12461)
## Thinking Path

> - Paperclip uses trusted GitHub Actions workflows to verify every pull
request
> - Native stacked pull requests use another pull request branch as
their base
> - A parent layer can change a package manifest without committing
`pnpm-lock.yaml`
> - A child layer can inherit that manifest change without changing a
manifest itself
> - The current policy skips lockfile regeneration for that child and
downstream frozen installs fail
> - This pull request validates the complete merge tree and shares a
regenerated lockfile only when needed
> - The benefit is reliable stacked pull request verification without
weakening the trusted workflow boundary

## Linked Issues or Issue Description

**What happened?**

A stacked child pull request inherited a package manifest change from
its parent. The child did not change a manifest itself. The policy job
skipped lockfile regeneration. Downstream jobs tried to restore an
artifact that did not exist and then failed during frozen dependency
installation.

**Expected behavior**

The policy job must validate the complete pull request merge tree. It
must upload a regenerated lockfile when the checked-in lockfile is
stale, including on a stacked child layer.

**Steps to reproduce**

1. Create a parent pull request that changes `package.json` without
committing `pnpm-lock.yaml`.
2. Create a child pull request on that branch without another manifest
change.
3. Run the trusted pull request workflow for the child.
4. Observe that frozen dependency installation fails because no
`pr-lockfile` artifact exists.

**Paperclip version or commit**

`f173ee09fa5c2ced7806bba47b54c3df853ab4df`

**Deployment mode**

GitHub Actions trusted pull request workflow.

**Agent adapter(s) involved**

Not adapter-specific. This is a core CI workflow bug.

## What Changed

- Regenerate the lockfile from every checked-out merge tree.
- Compare the generated lockfile with the checked-in copy before upload.
- Download the artifact only when the policy job reports that it
uploaded one.
- Fail closed when a reported artifact is missing.
- Add a workflow contract test for stacked lockfile handling.
- Keep the caller pinned to the last merged trusted SHA; after this
implementation merges, a separate activation PR will advance the
immutable pin to its merge commit.

## Verification

- `actionlint .github/workflows/pr-trusted.yml`
- `node --test scripts/__tests__/e2e-shard.test.mjs`

## Risks

- The policy job runs one lockfile-only install for every pull request.
This can add a small amount of CI time.
- A missing artifact now fails immediately when the policy job reports
an upload. This is intentional because it exposes workflow corruption.
- No runtime or product behavior changes.
- The implementation/activation split is intentional: unmerged
PR-authored workflow code must never execute on trusted runners.

## Model Used

OpenAI Codex with model `gpt-5`, reasoning, tool use, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 15:02:12 -05:00
Dotta f173ee09fa
ci: activate stale base handling (#12463)
## Thinking Path

> - Paperclip uses pull request checks to protect changes.
> - The trusted workflow selects GitHub or AWS runners.
> - Pull request #12462 removed an unreliable stale API equality.
> - The public caller must pin an immutable trusted workflow commit.
> - This pull request changes only that pin.
> - The benefit is correct AWS routing for trusted stacked pull
requests.

## Linked Issues or Issue Description

Refs #12339
Refs #12462

## What Changed

- Pin the thin pull request caller to
f9c32513b2.

## Verification

- actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml
- github-runners/tests/test-workflow.sh .github/workflows/pr-trusted.yml
.github/workflows/pr.yml
- The runner group authorizes this SHA and all in-flight prior SHAs.

## Risks

- Low risk. This changes one immutable workflow reference.
- Older workflow SHAs remain authorized while queued and active jobs
finish.

## Model Used

- OpenAI Codex, GPT-5.6, with reasoning and terminal tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked existing public pull requests
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 14:06:28 -05:00
Dotta f9c32513b2
ci: ignore stale PR API base snapshots (#12462)
## Thinking Path

> - Paperclip uses pull request checks to protect changes.
> - The trusted CI gate selects GitHub or AWS runners.
> - GitHub can return an old pull request base SHA after the live base
advances.
> - The signed event, live Git ref, ancestry, and merge parents provide
the required proof.
> - The stale API field rejects a safe run even when those proofs pass.
> - This pull request removes that unreliable equality.
> - The benefit is correct AWS routing for trusted stacked pull
requests.

## Linked Issues or Issue Description

Refs #12339
Refs #12459

## What Changed

- Stop treating pull request base.sha as a current-state signal.
- Keep the signed event base SHA and live ref descendant check.
- Keep the live base or synthetic base merge-parent proof.
- Keep all numeric identity, repository, head SHA, and triggering actor
checks.

## Verification

- actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml
- github-runners/tests/test-workflow.sh .github/workflows/pr-trusted.yml
.github/workflows/pr.yml
- The corrected gate selected the Fleet label with the exact live event
data from PR #12339 run 33201610330.

## Risks

- The pull request API base SHA can be stale and is no longer compared.
- Replaced ancestry, a changed head, a changed merge parent, and a
changed merge tree still fail closed.

## Model Used

- OpenAI Codex, GPT-5.6, with reasoning and terminal tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked existing public pull requests
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation in the operations repository
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 14:02:36 -05:00
Dotta 30ac5e116e
ci: activate live stacked base validation (#12460)
## Thinking Path

> - Paperclip uses pull request checks to protect changes.
> - The trusted CI workflow selects GitHub or AWS runners.
> - Pull request #12459 fixed validation when a stacked base advances.
> - The public caller must use an immutable trusted workflow SHA.
> - This pull request changes only that SHA.
> - The benefit is automatic AWS routing for the affected trusted stack
runs.

## Linked Issues or Issue Description

Refs #12339
Refs #12459

## What Changed

- Pin the thin pull request caller to
f929355fb9.

## Verification

- actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml
- github-runners/tests/test-workflow.sh .github/workflows/pr-trusted.yml
.github/workflows/pr.yml
- The runner group already authorizes the old and new exact workflow
SHAs.

## Risks

- Low risk. This changes one immutable workflow reference.
- The old workflow SHA remains authorized while current runs finish.

## Model Used

- OpenAI Codex, GPT-5.6, with reasoning and terminal tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked existing public pull requests
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:55:55 -05:00
Dotta f929355fb9
ci: allow validated stacked base advances (#12459)
## Thinking Path

> - Paperclip uses pull request checks to protect changes.
> - The trusted CI workflow selects GitHub or AWS runners.
> - Stacked pull requests can advance their base branch while a gate
waits.
> - The gate already proves that the live base descends from the event
base.
> - An earlier exact base check rejects that safe state before the
ancestry check runs.
> - This pull request removes the conflicting check and verifies live
API consistency.
> - The benefit is automatic AWS routing for trusted stacked pull
requests without weaker identity checks.

## Linked Issues or Issue Description

Refs #12339
Refs #12457

## What Changed

- Allow the live pull request base SHA to advance from the signed event
base snapshot.
- Require the pull request API base SHA to match the live Git ref during
validation.
- Keep the numeric author, sender, and triggering actor checks.
- Keep descendant ancestry and synthetic merge validation.

## Verification

- actionlint .github/workflows/pr-trusted.yml .github/workflows/pr.yml
- github-runners/tests/test-workflow.sh .github/workflows/pr-trusted.yml
.github/workflows/pr.yml
- The routing suite covers a live stacked base advance and a base change
during validation.

## Risks

- A trusted stacked run can use a newer descendant base than its signed
event snapshot.
- Replaced ancestry still fails closed.
- A live base change during gate validation still fails closed.

## Model Used

- OpenAI Codex, GPT-5.6, with reasoning and terminal tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in
this pull request
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation in the operations repository
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open findings
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:53:09 -05:00
Dotta c594811f3c
ci: activate stacked merge validation (#12458)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request checks protect the application and its contributors.
> - The trusted CI gate must validate both direct and stacked GitHub
merge shapes.
> - Pull request #12457 added that validation at an immutable master
SHA.
> - The active caller still pins the prior workflow version.
> - This pull request pins the caller to the newly authorized SHA.
> - The benefit is safe automatic AWS routing for trusted stacked pull
requests.

## Linked Issues or Issue Description

Refs #12457

**What existing behavior does this improve?**

The active caller uses a gate that fails closed on GitHub synthetic
stacked merge parents.

**Subsystem affected**

GitHub Actions pull request routing.

**Current behavior**

Trusted stacked pull requests run on GitHub-hosted runners after the
merge-parent check rejects the synthetic base merge.

**Proposed behavior**

The caller uses the authorized workflow SHA
7b199fcafa.

**Reason and benefit**

The change lets approved stack jobs use the 100-runner AWS Fleet while
all numeric identity and live-state checks remain active.

**Breaking changes**

None for untrusted contributors. Their jobs continue on GitHub-hosted
runners.

**Additional context**

The runner group authorizes both this SHA and the previous SHA during
rotation.

## What Changed

- Pin the thin caller to the authorized stacked-merge validator SHA.

## Verification

- Ran actionlint on both pull request workflows.
- Ran the external routing suite against the trusted workflow and
caller.
- Verified that the runner group authorizes both immutable workflow
versions.

## Risks

A bad pin can stop CI. The target is on master, the previous SHA stays
authorized, and the caller change is a one-line reversible pin update.

> For core feature work, check ROADMAP.md first and discuss it in #dev
before opening the PR. Feature PRs that overlap with planned core work
may need to be redirected — check the roadmap first. See
CONTRIBUTING.md.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, tool use, GitHub API access,
and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked an existing public item and described the issue
in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:37:43 -05:00
Dotta 7b199fcafa
fix(ci): validate stacked PR merge refs (#12457)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request checks protect the application and its contributors.
> - The trusted CI gate verifies signed event data against live GitHub
state.
> - GitHub gives a stacked pull request a synthetic merge commit for its
base stack.
> - The current validator expects the raw base SHA as the first merge
parent.
> - That assumption sends safe stacked pull requests to GitHub-hosted
runners.
> - This pull request validates the synthetic base merge and its
ancestry explicitly.
> - The benefit is safe automatic AWS routing for approved stacked pull
requests.

## Linked Issues or Issue Description

Refs #12455
Refs #12456

**What existing behavior does this improve?**

The trusted runner gate validates merge commits for master-based pull
requests but rejects GitHub synthetic base merges for stacked pull
requests.

**Subsystem affected**

GitHub Actions pull request identity and merge-state validation.

**Current behavior**

A trusted stacked pull request passes all numeric identity checks. The
gate fails closed because the first event merge parent is a GitHub
synthetic base merge instead of the raw base snapshot SHA.

**Proposed behavior**

The gate verifies the current base-ref tip, base-snapshot ancestry,
identical event and live merge parents, the child head parent, the
synthetic base merge parents, and identical event and live merge trees.

**Reason and benefit**

The change preserves fail-closed live-state validation while allowing
approved stacked pull requests to use the isolated AWS Fleet.

**Breaking changes**

None for untrusted contributors. Trusted stacked pull requests can
select AWS after the caller pins this workflow version.

**Additional context**

GitHub builds a stacked test merge in two steps. It first merges the
stack base into its own current base. It then uses that synthetic commit
as the first parent of the child test merge.

## What Changed

- Fetch and validate the current base branch ref.
- Require the event base snapshot to remain an ancestor of that ref.
- Validate direct and synthetic base merge parent shapes.
- Preserve the existing child-head, live-state, merge-tree, repository,
and actor checks.

## Verification

- Ran actionlint on the trusted workflow.
- Ran the external routing suite.
- Added positive coverage for the GitHub stacked merge shape.
- Added fail-closed coverage for replaced base ancestry and a synthetic
merge that omits the current base ref.
- Replayed the checks against the live merge shape for pull request
#12340.

## Risks

The gate has more GitHub API reads. Its five-minute timeout and
fail-closed behavior limit the effect of API errors. The accepted
synthetic commit must be the same in the event and live merge, must
include the current base branch as a direct parent, and must produce the
same merge tree.

> For core feature work, check ROADMAP.md first and discuss it in #dev
before opening the PR. Feature PRs that overlap with planned core work
may need to be redirected — check the roadmap first. See
CONTRIBUTING.md.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, tool use, GitHub API access,
and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked existing public items and described the issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:34:36 -05:00
Dotta e48e0bd3c2
ci: activate trusted stacked PR routing (#12456)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request checks protect the application and its contributors.
> - The trusted CI workflow sends approved contributors to isolated AWS
runners.
> - Pull request #12455 added safe support for non-master base branches.
> - The active caller still filters for master and uses the previous
immutable workflow SHA.
> - This pull request removes the base-branch trigger filter and pins
the caller to the authorized stack-aware SHA.
> - The benefit is that trusted stacked pull requests can use the AWS
fleet automatically.

## Linked Issues or Issue Description

Refs #12455

**What existing behavior does this improve?**

The active pull request caller only triggers for master and selects the
master-only trusted workflow version.

**Subsystem affected**

GitHub Actions pull request routing.

**Current behavior**

Trusted stacked pull requests either do not trigger the default caller
or use a branch-local older caller. Their heavy jobs remain in the
GitHub-hosted queue.

**Proposed behavior**

The caller triggers for every pull request base branch. It uses the
authorized stack-aware workflow at full SHA
d6b33d6c16.

**Reason and benefit**

The change sends heavy jobs from approved stacked pull requests to the
100-runner AWS Fleet. Unlisted contributors still receive the normal
GitHub-hosted checks.

**Breaking changes**

Trusted pull requests with non-master base branches now use AWS.
Untrusted pull requests continue to use GitHub-hosted runners.

**Additional context**

The runner group currently authorizes both the old and new immutable
workflow SHAs. The old SHA stays authorized until outstanding runs
finish.

## What Changed

- Remove the caller base-branch filter so stacked pull requests trigger
CI.
- Pin the thin caller to the authorized stack-aware reusable workflow
SHA.

## Verification

- Ran actionlint on both pull request workflow files.
- Ran the external routing test suite against the new reusable workflow
and thin caller.
- Verified that the runner group authorizes both immutable SHAs and only
the Paperclip repository.

## Risks

A bad pin can stop pull request CI. The target SHA is on master,
contains the reviewed reusable workflow, and is already authorized
beside the prior SHA. This gives the rotation a rollback path. Every
untrusted run still uses a GitHub-hosted gate and route.

> For core feature work, check ROADMAP.md first and discuss it in #dev
before opening the PR. Feature PRs that overlap with planned core work
may need to be redirected — check the roadmap first. See
CONTRIBUTING.md.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, tool use, GitHub API access,
and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked an existing public item and described the issue
in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:25:15 -05:00
Dotta d6b33d6c16
fix(ci): route trusted stacked PRs to AWS (#12455)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Pull request checks protect the application and its contributors.
> - The trusted CI workflow sends approved contributors to isolated AWS
runners.
> - The workflow currently limits AWS routing to pull requests that
target master.
> - Stacked pull requests target another branch and stay in the GitHub
queue.
> - This pull request keeps the identity checks and accepts a live
nonempty base branch in the Paperclip repository.
> - The benefit is that trusted stacked pull requests can use the AWS
fleet automatically.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The trusted pull request runner gate currently sends only master-based
pull requests to AWS.

**Subsystem affected**

GitHub Actions pull request routing.

**Current behavior**

A trusted contributor can pass all numeric identity checks. The gate
still selects GitHub-hosted runners when the pull request targets
another branch in a stack.

**Proposed behavior**

The gate accepts any nonempty base branch in the Paperclip base
repository. It still verifies the live base ref, base SHA, head SHA,
merge SHA, author ID, sender ID, and triggering actor ID.

**Reason and benefit**

Large pull request stacks currently add all heavy jobs to the limited
GitHub-hosted queue. This change lets approved contributors use the
isolated AWS fleet for those jobs.

**Breaking changes**

Trusted pull requests that target a non-master branch now use AWS
instead of GitHub-hosted runners. Untrusted pull requests keep the
current GitHub-hosted route.

**Additional context**

Pull request #12339 is the master-based first item in a stack. Its child
pull requests show this queue pattern.

## What Changed

- Replace the master-only route check with a nonempty base-ref check.
Keep the existing base-repository and live-state checks.

## Verification

- Ran actionlint on .github/workflows/pr-trusted.yml.
- Ran the external routing test suite. It passed the trusted
stacked-base case and all fail-closed identity cases.

## Risks

The AWS trust boundary now includes code from a non-master base branch
when the current pull request author, sender, and triggering actor are
all approved numeric GitHub IDs. An approved account can already submit
arbitrary head code. The runner remains isolated and has no production
access or repository secrets.

> For core feature work, check ROADMAP.md first and discuss it in #dev
before opening the PR. Feature PRs that overlap with planned core work
may need to be redirected — check the roadmap first. See
CONTRIBUTING.md.

## Model Used

OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, tool use, GitHub API access,
and local code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 13:20:30 -05:00
Dotta 66e1c0df8b
ci: pin PR base snapshot validation (#12450)
## Summary

- rotate the thin caller from `5ac66b3fdd5dc22c0c4e5fdb234ac063cf1d9ff8`
to `c119c4bee6ebb9c81791d7a6994f1be06d7cc22b`
- keep both exact SHAs authorized during rotation
- keep AWS routing disabled until verification completes

## Validation

- actionlint passes
- workflow-contract tests pass
- internal trust-routing harness passes
2026-08-28 11:36:35 -05:00