* commit 'a0a78ee60946a5f79f85b2bd0584fc766fae43bb':
ci: bootstrap Node before pnpm setup (#12808)
ci(runner): stamp paid target provenance (#12805)
fix: guard listComments against non-UUID afterCommentId to prevent 500 errors (#8695)
test(plugin-worker): remove the wall-clock race from the duplex buffered-replay tests (#12799)
ci(runner): skip bootstrap registry telemetry (#12797)
fix(ui): honor PAPERCLIP_HIDDEN_SETTINGS in the production switcher menu (#12788)
chore(deps): bump motion from 12.43.0 to 13.1.1 (#12255)
chore(deps): bump dompurify from 3.4.13 to 3.4.14 (#12266)
chore(deps-dev): bump @types/react-dom from 19.2.4 to 19.2.5 (#12253)
ci(runner): inspect Daytona image metadata remotely (#12795)
fix(ui): polish core navigation and task layout (#12793)
chore(deps): bump yjs from 13.6.29 to 13.6.32 (#12256)
chore(deps): bump @aws-sdk/client-s3 from 3.1120.0 to 3.1122.0 (#12261)
chore(deps-dev): bump vitest from 4.1.10 to 4.1.11 (#12262)
chore(deps): bump react-i18next from 17.0.11 to 17.0.12 (#12263)
fix(ui): drop the "Open invite" action from the invites section (#12787)
# Conflicts:
# tests/runner-e2e/workflow-security.test.ts
## Thinking Path
The trusted workflows currently invoke `pnpm/action-setup@v6` before
installing the repository Node version. On hosts whose ambient Node is
older than 22.13, the action downloads standalone `@pnpm/exe`, which has
repeatedly taken several minutes. Supplying Node 24 first lets the same
pinned pnpm action use its normal Node-backed path.
## What Changed
- install Node 24 before every trusted `pnpm/action-setup` invocation
- preserve the existing pnpm-store cache setup, pinned actions,
telemetry suppression, conditions, and secret boundaries
- enforce ordering, modern Node, condition parity, and cache counts in
the workflow security contract
## Verification
- workflow security: 7/7
- Prettier
- actionlint (excluding one pre-existing SC2129 in an untouched Daytona
shell block)
- `git diff --check`
## Risks
Low. Product code, providers, paid-runner selection, credentials, and
pnpm version are unchanged. Jobs that restore pnpm cache run
`setup-node` a second time after pnpm becomes available; the first setup
is deliberately cache-free.
## Model Used
Codex (GPT-5)
## Thinking Path
Trusted workflow-dispatch runs execute an authorized target SHA, but
GitHub context still describes the default-branch workflow revision.
Retained paid results and artifact names were therefore labeling
target-branch executions as master. The workflow must explicitly pass
its authorized target coordinates to target code and trusted reporting.
## What Changed
- emit the canonical authorized target ref alongside the immutable
target SHA
- pass those coordinates to paid cells and the trusted report
- name shared build/provider artifacts with the target SHA rather than
workflow SHA
- add workflow-security coverage for all trusted provenance wiring
## Verification
- focused workflow-security tests: 6/6 passed
- Prettier and git diff checks passed
- run 33823252706 independently proved the pre-fix defect: functionally
green target cells were retained as master SHA 0ad180b85 instead of
feature SHA 33c7646d3
## Risks
The execution checkout and secret boundary were already pinned
correctly; this changes retained attribution and artifact labels only.
Target-side report code on PR #12769 consumes these trusted environment
values and overwrites untrusted cell metadata.
## Model Used
Codex (GPT-5)
## Thinking Path
Every trusted PR and paid-workflow job invokes the pinned pnpm setup
action. Its internal npm install is currently waiting four to seven
minutes on npm audit telemetry before any Paperclip or provider code
runs. Audit, funding, and update notifications are not integrity
controls for this action; its committed lockfile still verifies
installed package bytes.
## What Changed
- disable npm audit, funding, and update-notifier telemetry narrowly on
all seven pinned setup steps in each of the trusted PR and full-stack
workflows
- add a workflow security contract proving every setup invocation
remains covered and the overrides do not leak elsewhere
## Verification
- focused workflow security tests: 6/6 passed
- Prettier and git diff checks passed
- observed unhealthy setup: 4-7+ minutes; historical healthy setup:
about four seconds
## Risks
This skips npm vulnerability-report telemetry for the setup action
bootstrap only. Repository dependency checks, lockfile integrity,
provider-secret authorization, and target-lock verification remain
unchanged.
## Model Used
Codex (GPT-5)
## Thinking Path
The reused Daytona image path already verifies the signed immutable
digest. It then downloads every filesystem layer only to read OCI config
fields. Buildx can retrieve the same config from that immutable digest
without pulling the layers. The assertions can therefore stay intact
while removing the expensive transfer.
## What Changed
- inspect the signed immutable Daytona image config through Buildx after
GHCR logout
- preserve digest, source revision, content ID, platform, user, and
provider-pack assertions
- extend the workflow contract test for the metadata-only path
## Verification
- Daytona image and workflow security tests: 10 passed
- Prettier and git diff checks passed
- observed full pull/prune cost: about 4m55s; metadata inspection: about
one second
## Risks
The current image has one runnable linux/amd64 platform plus its
attestation. A future genuinely multi-platform image would need explicit
linux/amd64 selection.
## Model Used
Codex (GPT-5)
## Thinking Path
> - Paperclip manages AI agents and their provider runtimes.
> - The paid runner workflow installs target dependencies with lifecycle
scripts disabled.
> - The bundled Daytona plugin depends on an audited repo-local plugin
SDK link.
> - The lifecycle-safe install path did not create that link.
> - This pull request restores only the trusted Daytona preparation step
before provider secrets are exposed.
> - The benefit is a working Daytona canary without enabling dependency
lifecycle scripts.
## Linked Issues or Issue Description
**What happened?**
The Daytona paid canary stopped before lease or provider startup. The
trusted paid job disabled root lifecycle scripts, so the repo-local
plugin SDK link was absent. The plugin install returned a missing
runtime dependency error for @paperclipai/plugin-sdk.
**Expected behavior**
The trusted workflow must prepare the bundled Daytona plugin without
running untrusted dependency lifecycle scripts. The paid cell must start
only after its runtime dependencies and entrypoints pass validation.
**Steps to reproduce**
1. Dispatch the runner full-stack paid workflow for
core-compatibility.runner-acpx-claude.daytona.message-marker.
2. Let the trusted job install root dependencies with lifecycle scripts
disabled.
3. Observe the Daytona plugin installation fail before a lease or
provider process starts.
**Paperclip version or commit**
Feature head 781ac7e08c. The failed canary
is Actions run 33803959325.
**Deployment mode**
GitHub Actions paid runner validation.
**Agent adapter(s) involved**
ACPX Claude through the bundled Daytona plugin.
**Additional context**
This is a small trusted-workflow prerequisite for public PR #12769. Old
green run 33118525827 created the SDK link through root postinstall.
This change keeps lifecycle scripts disabled and restores only the
audited prerequisite.
## What Changed
- Install standalone Daytona dependencies with lifecycle scripts
disabled.
- Run the audited repo-local plugin SDK linker before provider secrets
are exposed.
- Build the bundled Daytona plugin and verify its runtime dependency
plus both entrypoints.
- Add a security regression for ordering, scope, and secret isolation.
## Verification
- Five focused workflow-security tests passed.
- Seven focused linker tests passed.
- The exact Daytona preparation command completed locally in nine
seconds.
- Prettier and diff whitespace checks passed.
## Risks
Risk is low and limited to Daytona paid cells. The setup still disables
dependency lifecycle scripts. The trusted step runs before provider
credentials enter the job. Any missing or mismatched path fails closed
before provider startup.
## Model Used
OpenAI GPT-5 Codex with repository tools and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR with the bug template labels
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] Focused local tests pass
- [x] I added tests for the change
- [x] I updated the relevant trusted-workflow security regression
- [x] I documented the risks above
## Thinking Path
> - Paperclip manages AI agents and their provider runtimes.
> - Paid runner validation installs target dependencies with lifecycle
scripts disabled.
> - OpenCode leaves a sentinel executable until its package lifecycle
script runs.
> - Running arbitrary lifecycle code would weaken the paid-secret
boundary.
> - This pull request materializes one exact pinned binary before
secrets are exposed.
> - The benefit is working OpenCode validation without trusting
dependency install scripts.
## Linked Issues or Issue Description
**What happened?**
Every local OpenCode paid cell stopped before provider startup because
`pnpm install --ignore-scripts` correctly retained
`opencode-ai/bin/opencode.exe` as a sentinel.
**Expected behavior**
The trusted workflow must make the exact lockfile-pinned OpenCode
executable available without running package lifecycle scripts.
**Steps to reproduce**
Run a local legacy or native OpenCode paid cell from the trusted
workflow after the target dependency install. The provider health check
reports that the OpenCode postinstall script was not run.
**Paperclip version or commit**
Default branch commit `865b4854fb44d3689f1c0ff17e3e715d52aaea73`.
## What Changed
- Materialize only `opencode-linux-x64-baseline@1.18.17` into the
matching `opencode-ai@1.18.17` package.
- Verify package identity, version, regular-file type, SHA-256 equality,
executable permissions, and runtime `--version`.
- Invoke the helper for local OpenCode and breadth cells and for remote
provider-pack assembly.
- Retain `pnpm install --ignore-scripts`.
- Add helper and trusted-workflow security regressions.
## Verification
- Helper syntax checks passed.
- Helper unit tests passed: 2/2.
- Workflow-security tests passed: 5/5.
- Prettier, actionlint, and diff whitespace checks passed.
## Risks
Risk is low and contained to paid runner setup. The helper supports only
Linux x64, fails closed on package or version drift, and runs before
provider credentials enter the job.
## Model Used
OpenAI GPT-5 Codex with repository tools and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change.
- [x] I have specified the model used.
- [x] I have checked ROADMAP.md and confirmed this does not duplicate
planned core work.
- [x] I have searched GitHub for duplicate or related PRs and found
none.
- [x] I have described the issue in this PR with the bug template
labels.
- [x] I have not referenced internal or instance-local issues.
- [x] My branch name describes the change.
- [x] Focused local tests pass.
- [x] I added tests for the change.
- [x] I updated the runner E2E security documentation.
- [x] I documented the risks above.
## Thinking Path
> - Paid cells repeated the same TypeScript and Rust builds even when
one campaign selected dozens of cells.
> - The trusted workflow can compile once without provider credentials
and distribute run-scoped, digest-verified artifacts.
> - The paid cell can then disable install lifecycle scripts, verify
each artifact before extraction, and expose provider credentials only to
the final test step.
> - Local JS-backed providers also need the setup-node interpreter
permission-qualified before Rust verifies the launch artifact.
## Linked Issues or Issue Description
Run 33786122875 proved target-lock setup and catalog selection, then
failed before provider creation because trusted master did not yet
qualify the setup-node interpreter. The same workflow also rebuilt
TypeScript and Rust inside every matrix cell.
## What Changed
- Build runner TypeScript and native binaries once per campaign in a
credential-free job.
- Build the remote provider pack once only when selected Daytona cells
require it.
- Upload run-scoped bundles with SHA-256 manifests and verify before
extraction in each paid cell.
- Remove repeated TypeScript, provider-pack, and Rust builds from paid
cells.
- Qualify the local provider Node interpreter before verified launch.
- Propagate the resolved target lockfile through all five target-code
jobs.
- Keep local-only selection off Daytona and exclude Xiaomi from the
67-cell catalog.
## Risks
A shared build artifact could fan out a bad payload to many cells. The
producing jobs receive no provider credentials, use the exact authorized
target SHA and resolved lockfile, and publish run-scoped artifacts.
Every consuming job verifies SHA-256 before extraction. Paid dependency
setup keeps lifecycle scripts disabled and provider credentials remain
scoped to the final test step.
## Verification
- Focused runner workflow-security, catalog, and Daytona-image tests:
25/25 passed.
- Prettier passed.
- Actionlint passed with only the two pre-existing SC2129 style notices
ignored.
- Git diff check passed.
## Model Used
OpenAI Codex, GPT-5.
## Checklist
- [x] Build jobs are credential-free.
- [x] Paid installs disable lifecycle scripts.
- [x] Artifacts are run-scoped and digest-verified before extraction.
- [x] Trusted report and history jobs remain isolated from target
artifacts.
- [x] No Daytona or Xiaomi paid run was started for this change.
## Thinking Path
> - The trusted target-branch runner workflow checks out PR code before
paid tests.
> - PR policy intentionally forbids manual lockfile commits.
> - Some runner changes legitimately alter pnpm patch hashes.
> - Frozen installs therefore fail before test selection.
> - Resolve one script-disabled lockfile from the authorized immutable
target SHA and distribute it by exact artifact ID and digest.
> - Keep provider credentials and trusted reporting outside this
resolution job.
## Linked Issues or Issue Description
Target-branch paid runner campaigns currently fail frozen install when a
PR changes pnpm patch content, even though ordinary PR CI regenerates
the lockfile.
## What Changed
- Added one credential-free target-lock job that resolves the authorized
immutable target SHA with lifecycle scripts disabled.
- Uploaded the resolved lockfile with its SHA-256 and restored it by
exact artifact ID before every target-code frozen install.
- Left trusted reporting and history jobs on the workflow SHA.
- Changed the disabled-AWS fallback from unavailable ubuntu-latest-m to
ubuntu-latest.
## Risks
The workflow evaluates pnpm lockfile resolution from authorized target
code. That job receives no provider credentials, disables lifecycle
scripts, rejects unrelated workspace mutations, and exposes only a
digest-verified lockfile artifact. Paid-secret jobs consume only that
lockfile after exact artifact-ID and SHA-256 validation.
## Verification
- Runner workflow-security focused tests pass.
- actionlint passes.
- Prettier and git diff checks pass.
## Model Used
OpenAI Codex, GPT-5.
## Checklist
- [x] Change is narrowly scoped to paid runner orchestration.
- [x] Target lock resolution has no provider credentials and disables
lifecycle scripts.
- [x] Downloaded artifacts are selected by exact artifact ID and
verified by SHA-256.
- [x] Trusted reporting and history jobs remain on the workflow SHA.
## Thinking Path
> - Paperclip uses paid runner tests to qualify agent execution.
> - The runner workflow controls provider secrets and AWS runner access.
> - The trusted workflow must stay on the protected default branch.
> - The code under test often exists on a branch before merge.
> - CODEOWNERS need a safe way to select that branch.
> - This pull request separates workflow authority from the code under
test.
> - The benefit is pre-merge AWS testing without target-controlled
workflow code.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The manual Runner Full-Stack E2E workflow can test only the default
branch.
**Subsystem affected**
GitHub Actions and the paid runner E2E security boundary.
**Current behavior**
A CODEOWNER must merge runner changes before the trusted AWS workflow
can test them.
Selecting another branch as the workflow ref is rejected.
**Proposed behavior**
A CODEOWNER starts the workflow from `master` and supplies a
same-repository branch in `target_branch`.
The authorization job resolves the branch to one commit SHA.
Catalog, image, and paid test jobs check out that SHA after
authorization.
Report sanitization and AWS publication use the trusted workflow SHA.
**Reason and benefit**
This permits paid pre-merge qualification on AWS.
It keeps the workflow definition, report sanitizer, history publisher,
environment deployment, and runner-group permission on `master`.
**Breaking changes**
None.
The new input is optional.
An omitted input still tests the default branch.
## What Changed
- Add the optional `target_branch` workflow input.
- Resolve only a branch in `paperclipai/paperclip` to an immutable SHA.
- Pin catalog, image, paid test, and Daytona provenance to the target
SHA.
- Pin report sanitization and AWS history publication to the trusted
workflow SHA.
- Disable persisted checkout credentials in every job.
- Key cancellation by the selected target branch.
- Add policy regression coverage and operator documentation.
## Verification
- `pnpm test:e2e:runner:unit` passes with 65 tests.
- `actionlint -ignore SC2129
.github/workflows/runner-full-stack-e2e.yml` passes.
- Prettier checks pass for all changed files.
- `git diff --check` passes.
## Risks
A CODEOWNER can authorize selected branch code to receive a cell-scoped
provider credential.
This is the intended trust decision.
The workflow rejects fork refs and target-controlled workflow
definitions.
The trusted workflow SHA owns report sanitization and AWS history
publication.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex, GPT-5.
The exact serving snapshot and context-window size are not exposed.
The model used tool-enabled reasoning and code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip manages AI agents that perform work.
> - The paid runner matrix verifies complete runner behavior with real
providers.
> - Each matrix job currently repeats work on GitHub-hosted runners.
> - Paperclip has an ephemeral AWS runner fleet for trusted workflows.
> - The paid workflow needs a reviewed and fail-closed route to that
fleet.
> - This pull request adds that route and keeps the existing hosted
runner as the disabled-state fallback.
> - The benefit is faster paid campaigns with the same actor,
environment, and secret boundaries.
## Linked Issues or Issue Description
**What happened?**
The Runner Full-Stack E2E workflow always uses `ubuntu-latest-m`. It
limits the matrix to 57 parallel jobs. The repository AWS fleet can run
100 ephemeral jobs, but the paid workflow cannot select it.
**Expected behavior**
An explicit repository flag must select the reviewed AWS fleet label. A
missing or invalid flag must keep the existing hosted runner. The
workflow must authorize the stable actor identity before it routes any
paid job.
**Steps to reproduce**
1. Dispatch the Runner Full-Stack E2E workflow from `master`.
2. Inspect a paid matrix job.
3. Observe that the job requests `ubuntu-latest-m` even when the AWS
fleet should be used.
**Paperclip version or commit**
`da0947d3582ac7779d6bf11851c9938eca6c5c8c`
**Deployment mode**
GitHub Actions paid runner campaign.
## What Changed
- Add a fail-closed `RUNNER_E2E_AWS_ENABLED` switch.
- Select only the reviewed AWS fleet label or the existing hosted label.
- Permit up to 100 parallel jobs in AWS mode.
- Keep the hosted-runner limit at 57.
- Reauthorize paid execution before checkout and provider access.
- Stop paid checkouts from storing GitHub credentials.
- Cancel superseded validation-ref campaigns while preserving `master`
audit runs.
- Add workflow policy checks and operator documentation.
## Verification
- `git diff --check`
- `actionlint -ignore SC2129
.github/workflows/runner-full-stack-e2e.yml`
- The organization runner group permits this workflow only from
`refs/heads/master`.
- The repository AWS switch remains disabled until this pull request is
merged and a one-cell probe succeeds.
## Risks
- A wrong fleet policy can leave jobs queued. The disabled state keeps
the existing hosted runner.
- The AWS fleet uses paid compute. The workflow validates a configured
maximum of 100 jobs.
- The runner group, actor allowlist, and paid environment remain
separate enforcement layers.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, GitHub API coordination, and static
workflow analysis. The exact deployed model identifier and
context-window size are not exposed to this task.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the control plane for agents that perform work.
> - Paperclip Runner connects durable provider sessions to individual
task runs through PRP.
> - Provider continuity and per-run authority are different lifetimes.
> - The existing implementation mixed those lifetimes and lost event
metadata between provider frames, runnerd, persistence, API
sanitization, and the task thread.
> - That caused failed continuation, missing progress and Plans,
duplicate replies, hidden failures, and unsafe recovery.
> - This repair gives every heartbeat fresh authority, preserves
qualified provider-session continuity, and restores one lossless
presentation path without changing direct adapters.
## Linked Issues or Issue Description
**What happened?**
A second native heartbeat could reuse tickets, leases, command receipts,
sequence state, and run identity from the first heartbeat. Provider
phase and item identity could be lost before the UI read them. Redaction
could corrupt protocol discriminators while still missing malformed
credential tails. The task thread could fold progress into the final
response, hide failures, or show more than one final answer. Native
Codex also exposed approval modes that do not yet have a durable
approval bridge.
**Expected behavior**
Each heartbeat uses a new PRP authority epoch. Codex and OpenCode
preserve exact qualified provider sessions; ACPX emits an explicit
continuity event when its qualified process-replacement policy is used.
Every accepted provider event is presented, classified as internal, or
surfaced as unsupported. The task page shows chronological progress,
reasoning summaries, activity, Plans, interactions, terminal failures,
and exactly one final reply. Direct adapters retain their existing path.
**Steps to reproduce**
1. Enable the unified experimental Paperclip Runner setting.
2. Create a local native Codex, OpenCode, ACPX Claude, or ACPX Codex
agent.
3. Run response, Plan, structured-question/resume, restart,
cancellation, and failure scenarios.
4. Reload the task while active, waiting, failed, and settled.
5. On the old implementation, observe stale run authority, missing
classifications, incomplete output, or duplicated/folded replies.
**Paperclip version or commit**
The repair is based directly on `master` at
`87d05e194b643810d16d20612115acd01d735d43`.
**Deployment mode**
Local development with the embedded database.
Related work: Refs #12616, #12646, #12666, #12685, and #12700.
## What Changed
- Rotates PRP control-plane, outbox, ticket, lease, command, receipt,
and sequence authority for each heartbeat while carrying forward only a
validated provider-session identity.
- Reads `control-plane-state.json`, validates both durable schemas and
lifecycle values, resumes coherent current runs, archives qualified
settled authority, and quarantines malformed or mismatched scoped state
without moving ambiguous live legacy state.
- Preserves Codex provider phase and stable item identities so
commentary remains progress and only `final_answer` becomes final.
- Adds raw OpenCode HTTP/SSE boundary coverage and canonical reasoning
lifecycle mapping.
- Makes ACPX normalization lossless for visible reasoning, tool
lifecycle metadata, stable bounded identities, Plan revisions,
structured requests, failures, and qualified process replacement. Only
the compatible terminal assistant message is promoted as final.
- Applies schema-aware redaction before generic JWT-shaped detection and
scans every diagnostic string leaf. Malformed raw/escaped quoted
credential tails are redacted in both server and durable Rust state.
- Restores snapshot-style chronological task presentation, expandable
tool activity, inline Plan cards, visible waiting/resume/cancel/failure
states, and exactly one final answer.
- Makes `never` the only qualified native Codex permission mode and
rejects unsupported persisted native modes with remediation. OpenCode
and ACPX policies remain intact.
- Keeps the unified experimental Runner setting as the only enablement
flag. Onboarding and direct Codex, Claude, and OpenCode stay on their
legacy execution/finalization paths.
- Adds cross-language goldens, authority/recovery/fault coverage, exact
response/count assertions, and native plus legacy acceptance scenarios.
## Verification
- Pull-request GitHub Actions run Rust formatting/tests, TypeScript
checks, server/UI tests, builds, protocol drift checks, browser E2E, and
security scans.
- A separate workflow-only validation ref is pinned directly on this PR
head and runs the 35-cell paid local matrix: three core scenarios plus
structured-question resume and restart/resume for native Codex, native
OpenCode, ACPX Claude, ACPX Codex, and direct Codex/Claude/OpenCode.
Run: https://github.com/paperclipai/paperclip/actions/runs/33682434315
- Acceptance requires exact single visible replies, monotonic sequences,
matching envelope discriminators, one semantic terminal, one run
terminal, no unresolved interaction, no duplicate mutation, no secret
leakage, provider continuity, and zero native rows for direct adapters.
- Per maintainer direction, tests are running in GitHub Actions rather
than on the slower local host. Only formatters and static diff checks
were run locally.
## Risks
- Recovery from old or partial filesystem state is sensitive. The repair
fails closed, preserves active or unverifiable authority, and
quarantines only state whose scoped ownership is safe to move.
- Provider event formats can change. Closed validators and boundary
goldens turn new or malformed events into visible diagnostics instead of
silent drops.
- Shared task presentation could affect direct adapters. Runtime-fact
gating plus the direct-adapter matrix protect the existing path.
- Managed and remote providers are not qualified here. Shared code
continues to compile and fail safely, but live qualification is
deferred.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex based on GPT-5. The exact deployed snapshot and
context-window size are not exposed to this task. It used agentic
reasoning, repository inspection, code editing, Git, parallel subagents,
and GitHub Actions.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] The paid local-provider matrix is green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.
**Problem or motivation**
The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.
**Proposed solution**
Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.
**Alternatives considered**
We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.
**Roadmap alignment**
This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.
Related stack:
- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.
## What Changed
- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.
## Verification
Passed locally:
- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.
I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.
## Risks
The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.
Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.
Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.
The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.
This pull request does not enable the experimental native runner for
existing instances.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Dotta
Dotta