## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Conversations must use the same tasks, controls, and execution
history.
> - Users need an ongoing chat with an agent without managing task
properties.
> - Agents should clarify and plan work, then hand execution to assigned
project tasks.
> - This pull request combines the reviewed Agent Chat stack for one
squash merge.
> - The benefit is persistent conversation with normal task governance
and shared UI.
## Linked Issues or Issue Description
**Subsystem affected**
Task lifecycle, agent runtime tools, shared task UI, and browser/paid
runner tests.
**Problem or motivation**
Users need one persistent conversation with each agent. A separate chat
store or renderer would duplicate task behavior and bypass existing
controls.
**Proposed solution**
Use a task-backed chat per company, user, and agent. Reuse the task
composer and transcript. Clarify and plan in chat, then create assigned
project tasks with the relevant plan. Keep Agent Chat behind its own
disabled-by-default experimental setting.
**Roadmap alignment**
This implements the task-backed direction in [CEO
Chat](https://github.com/paperclipai/paperclip/blob/master/ROADMAP.md#-ceo-chat).
Related proposals: #2504 and #9693. Related request: #7981. The
maintainer requested one squash merge of the complete stack.
Consolidates the reviewed runtime
[#13281](https://github.com/paperclipai/paperclip/pull/13281), backend
[#13282](https://github.com/paperclipai/paperclip/pull/13282), and UI
[#13283](https://github.com/paperclipai/paperclip/pull/13283) layers
with this PR's E2E coverage. All four layers passed CI and received
Greptile 5/5 before consolidation. This PR targets master and includes
the complete feature.
## What Changed
- Add personal canonical chat tasks with ordinary company visibility,
immutable identity, idempotent first sends, and an idle waiting state.
- Process `/new` in queue order. Preserve history, release a chat pause,
and fence old provider context and delayed writes.
- Keep chat lifecycle rules across recovery, finalization, assignment,
task lists, and rollups.
- Support research and plan revision in chat. Hand plans to ordinary
assigned project tasks before execution starts. Reject new chat
subtasks.
- Add repository-aware project creation and discovery tools, including
multiple repository IDs and GitHub URLs, authorization, idempotency, and
durable project-created cards.
- Reuse task UI components for chat, with starred/recent agent
navigation and a separate `enableAgentChat` experimental flag.
- Add deterministic browser tests and 24 paid chat cells across four
Codex/Claude profiles, with validated reports and screenshots.
- Integrate current master recovery, controller lease, queued-message,
and task UI changes. Gate chat interruption and deferred promotion on
ownership/feature policy. Guarantee lease renewal and active controls
are stopped even if teardown fails.
- Preserve master's migration 0273 and generate chat migration 0274 with
idempotent replay for development databases.
## Verification
- Prior exact heads of all four PRs passed Linux CI, including build,
typecheck, general/serialized tests, and browser E2E. Each had Greptile
5/5 and no unresolved findings.
- Integrated local verification passed: full repository typecheck and
production build, Storybook build, token gates, 340 focused UI tests,
all 20 deterministic chat browser tests, two migration replay tests, 88
focused chat/queue/native/controller tests, and provider/session
regressions including real lease expiry. These include the three
lifecycle regressions for the final admission/teardown fixes; server
typecheck also passes. Current head
`1268eda16cc2af892055917e7292f068820be135` has Greptile 5/5 with no
unresolved findings and passing security scans. All final-head CI gates
passed: build, full Runner verification, typecheck/release registry,
canary, all general/serialized test shards, and all browser E2E shards
([CI
run](https://github.com/paperclipai/paperclip/actions/runs/34696739927)).
Local PostgreSQL startup contention required serialized retries; skipped
fixtures do not count as passing coverage.
- The earlier paid campaign passed all 24 chat cells and retained 32
screenshots:
[report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-34648511170-1/index.html?report=agent-chat#suite-agent-chat).
It tested `abacbdfd2f660709ec37312cdb758284c8399d04`; it is prior
evidence, not a paid run of this integrated head.
- Manual check: enable Agent Chat in Experimental settings, open an
agent, clarify and revise a plan, then hand off to an assigned project
task. Stop a reply, send `/new`, and verify fresh context with retained
history. Disable the setting and verify agent shortcuts/new chat turns
are blocked.
## Risks
- Queue/session integration can affect retries and delayed writes. Tests
cover ownership, cancellation, reset boundaries, idle recovery, and
ordinary task behavior.
- Migration 0274 adds conversation fields and constraints. Replay is
idempotent and preserves existing development chat history.
- This combines the previously reviewed stack at the maintainer's
request. Agent Chat remains off by default and is separate from
Conference Room.
## Model Used
OpenAI Codex, GPT-6 Astra (`gpt-6-astra`), with reasoning, code
execution, browser tools, and parallel review. The exact context-window
size is not exposed in this session. Codex and Claude also ran as test
subjects in the linked paid campaign.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip manages AI agents and their work.
> - The new runner gives agents dedicated tools for common tasks.
> - Some API operations and parameters have no dedicated tool.
> - Agents need a controlled way to find and use those operations.
> - This pull request adds API search and calls through the real server
routes.
> - Existing tools remain the preferred path. The new tools are disabled
by default.
> - Paired tests measure correctness, tool choice, cost and time.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner contracts, production tool authority and the server API
catalog.
**Problem or motivation**
The runner cannot use much of the API described by the old Paperclip
skill. A generic HTTP client would also let agents bypass runner control
rules.
**Proposed solution**
Add `search_api` and `call_api`. Resolve calls from the mounted API
catalog. Use server-held, run-bound credentials. Preserve route checks
and runner lifecycle rules. Keep the tools disabled until an operator
enables selected companies.
**Alternatives considered**
A dedicated tool for every endpoint would add a large initial prompt. An
unrestricted HTTP tool would weaken authorization and replay controls.
**Roadmap alignment**
This extends the native runner tooling. The repository owner requested
this design and implementation. The roadmap and related open PRs were
checked. No duplicate API escape-hatch PR was found.
## What Changed
- Register two compact fallback tools in canonical contracts and
provider projections.
- Build deterministic API discovery from OpenAPI, mounted experimental
routes and the old skill reference.
- Execute bounded JSON, text, file and download requests through
authenticated HTTP routes.
- Recheck active runs, company access and work modes. Block runner
lifecycle, scheduling, credential and approval bypasses. Keep routine
annotation collaboration available.
- Retain mutation receipts. Report uncertain outcomes without blindly
repeating writes.
- Add a company rollout gate and a durable eval worker with complete
cost accounting checks.
- Record child-task creation in the activity log with the agent and run.
- Add contract, authorization, file, replay and real runnerd/PRP/HTTP
tests.
- Document rollout gates and paid coverage limits. The companion eval
repository retains immutable attempts and reports.
## Verification
- Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32
checks passed; the unrelated Storybook visual check was skipped.
Greptile 5/5; no unresolved review threads.
- Full Linux build and recursive typecheck passed. Repository tests were
run by project and serialized shard; all 143 serialized server suites
passed.
- Runner TypeScript: 1,599 passed, two skipped. Rust release: 451
passing test reports. Conformance and replay parity passed. The required
API check passed 837 tests, including runnerd → PRP → authority → real
HTTP.
- Bindings cannot enable API tools without the explicit deployment flag.
Unit and real-authority tests prove the default-off boundary.
- The standalone API check builds and stages its own binary. It passed
after existing staged and debug binaries were removed from the test
container.
- UI and CLI tests passed. Initial environment failures (missing jq,
Docker overlay file identity, and parallel linker memory pressure) and
focused passing reruns are retained. The macOS full runner suite has
platform-specific failures; Linux is the qualified full-check platform.
- Eval harness: 27 tests passed; existing CI discovery ran 86 tests with
two unrelated skips. Credential export rejection is tested against the
actual report command.
- Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten
workflows, three repetitions per arm, zero unnecessary API fallback.
- Sonnet passed 11 selected capability/contract cases after fixes.
Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit
and remains unqualified.
- Luna's two cost flags received focused follow-up. The original flags
and a later n=1 latency flag remain visible. Sonnet had no cost or
latency increase above 20%.
- The catalog contains 785 entries; 58 were exercised across all stages.
Most operation probes remain unrun and some need additional fixtures.
Authored probes do not establish successful coverage.
- Total conservative accounted cost: $9.875960. Active paid-campaign
time: 88.16/90 minutes. No missing accounting. Later security and
harness fixes have provider-free verification; no paid validation is
claimed for those revisions.
- Inspect the [qualification
report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md)
and [verification
record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json).
## Risks
- This is a broad authenticated API surface. Keep the default-off gate
until an operator selects initial rollout companies.
- Paid coverage is incomplete. Small regression samples do not prove all
workflows are unchanged.
- A timeout or server failure can follow a committed mutation. The
result reports an unknown outcome and requires state inspection.
- The new definitions add prompt tokens. The report retains cost flags
and cache variation.
- No database migration is required.
- Repository rules require code-owner approval before merge. Technical
CI and automated review are complete.
## Model Used
OpenAI Codex based on GPT-6 assisted with code, tests and review. The
exact serving model ID and context window are not exposed in this
session. It used reasoning, tool calls and code execution.
Eval models: `gpt-5.6-luna` with low reasoning,
`openrouter/anthropic/claude-sonnet-5`,
`openrouter/google/gemini-3.8-flash`, and
`openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime
versions, model identity, usage and source provenance.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Runner executes agents through native and managed provider
drivers.
> - The direct live eval layer had drifted from the current Runner
contracts.
> - The old local workflow did not provide a complete parallel campaign
or durable report history.
> - The Runner also needed current native OpenCode and OpenRouter
qualification.
> - This pull request restores the direct campaign, corrects the runtime
gaps that the campaign found, and adds safe hosted Evalbook history.
> - The benefit is repeatable model comparison against an immutable
Runner and eval source revision.
## Linked Issues or Issue Description
Refs #11297
Refs #11634
**What existing behavior does this improve?**
This improves the direct live `paperclip-runner` eval workflow, provider
execution contract, and static Evalbook reporting path.
**Current behavior**
The direct evals do not have one maintained full campaign on current
`master`. OpenCode has no qualified multi-model OpenRouter roster.
Parallel provider bursts can compact committed events before the
transport observes them. Local reports do not have a separate safe S3
history index.
**Proposed behavior**
Run one immutable roster-plus-case matrix. Use the shared paid AWS
runner fleet. Keep raw artifacts access-controlled. Publish a sanitized
canonical Evalbook report under the separate `runner-protocol-evals` S3
prefix. Keep immutable campaign directories plus root history, latest,
and latest-green pointers.
**Reason and benefit**
Maintainers can compare native Codex, native OpenCode, ACPX, Claude
Managed, and AWS AgentCore behavior over time. They can inspect failures
without mixing this direct protocol layer with browser full-stack E2E.
**Breaking changes**
None. The new workflow and S3 prefix are additive. The existing Runner
full-stack E2E workflow and report remain separate.
## What Changed
- Added a trusted two-shard direct live workflow for up to 393
roster-plus-case cells.
- Reused the numeric actor allowlist, protected paid environment, and
RunsOn fleet controls from Runner full-stack E2E.
- Added immutable Runner and eval revision resolution, exact credential
boundaries, bounded retries, and cost ceilings.
- Added a public report projection that removes sessions, transcripts,
tool payloads, state, traces, raw failures, remote profile identities,
and credential-shaped values.
- Added additive S3 history under `runner-protocol-evals`, with
immutable campaigns and mutable root index pointers.
- Added native OpenCode model injection and current OpenRouter pricing
contracts.
- Fixed direct eval completion, workflow execution, semantic discovery,
warm-attach state reset, executable binding, and event-burst handling.
- Kept Runner browser full-stack E2E behavior and publication separate.
- Documented local and hosted direct eval operation.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `pnpm --filter @paperclipai/paperclip-runner build:typescript` —
passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml
.github/workflows/runner-full-stack-e2e.yml` — passed.
- Local current matrix at the revision in
[paperclip-evals#17](https://github.com/paperclipai/paperclip-evals/pull/17)
— 323 cells across 10 enabled configurations completed.
- Final local current matrix — 269 passed, 11 behavior failures, and 43
expected macOS-only ACPX platform failures.
- Targeted Runner checks — 13/13 eval-session tests, 15/15
publisher/security tests, and package typecheck passed; complete PR CI
is green, including all browser E2E shards.
## Risks
- Paid live campaigns can consume provider budget. Actor authorization,
exact per-cell ceilings, protected environments, and explicit schedule
enablement bound this risk.
- Public reports can leak provider data. The workflow publishes only a
separately projected report and validates every file before upload.
- The new workflow cannot publish until it is present on the default
branch. This pull request does not change the existing
`runner-full-stack-e2e` publication path.
- The campaign is large. It uses two GitHub matrices and caps combined
concurrency at the shared fleet limit.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, code editing, browser
inspection, repository tools, and live provider execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip manages AI agents that perform work.
> - Paperclip Runner connects durable task runs to local provider
processes.
> - The full-stack paid matrix exposed failures after the runner
integrity repair.
> - Verified JavaScript entrypoints lost their relative module graph
when Linux executed them through descriptor paths.
> - Returned provider startup errors also remained pending and became
indeterminate after recovery.
> - Sparse Codex tool lifecycle events lost the `write_document`
identity before task transcript projection.
> - This pull request repairs those three boundaries and makes the
structured-question fixture deterministic.
> - The benefit is repeatable provider startup, exact failure replay,
and correct inline Plan placement.
## Linked Issues or Issue Description
Refs #12721 and #12700.
**What happened?**
The paid runner matrix failed ACPX and OpenCode startup before provider
session creation. The runner journal then replaced the original startup
error with an indeterminate recovery result. Native Codex saved a Plan
but rendered it only as a fallback card. A legacy Claude waiting reply
could also echo the reserved terminal marker before the answer arrived.
**Expected behavior**
Verified JavaScript providers must start from immutable
descriptor-backed artifacts. Returned startup failures must persist as
terminal failed command results. Native tool lifecycle updates must
preserve the `write_document` boundary. Pre-answer fixture output must
not contain the reserved terminal marker.
**Steps to reproduce**
1. Run the local provider cells in the Runner Full-Stack E2E workflow.
2. Observe ACPX and OpenCode fail during `session.open` before provider
execution.
3. Observe recovery report `execution_indeterminate` instead of the
original startup error.
4. Run the native Codex Plan cell and observe the fallback Plan card
after the tool activity row.
5. Run the legacy Claude structured-question resume cell and observe an
early marker echo in waiting prose.
**Paperclip version or commit**
`0f9452101740835ce0b1488a204bf48acd5bafc3`
**Deployment mode**
Local development with the paid GitHub Actions acceptance workflow.
## What Changed
- Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM
entrypoints before hashing and verified descriptor launch.
- Anchor ACPX dynamic provider package resolution at a
controller-derived provider-pack root and keep that root out of the
provider child environment.
- Persist executor-returned startup errors as redacted durable failed
command results while retaining indeterminate recovery for true process
death.
- Coalesce sparse native tool items by stable ID so a late
`write_document` name, input, and result reach the transcript boundary
once.
- Forbid the structured-question fixture from spelling or announcing its
reserved terminal marker before the user answers.
## Verification
- Rust and TypeScript regression tests cover durable failed replay, true
crash ambiguity, bundle closure, package-root derivation, environment
filtering, exact Codex tool lifecycle coalescing, and prompt
determinism.
- Local execution is intentionally limited to formatters and static diff
checks. GitHub Actions will run tests, type checks, builds, and security
checks.
- After ordinary CI is green, scoped paid cells will validate one ACPX
launch, one OpenCode launch, native Codex Plan projection, and legacy
Claude structured resume before a complete matrix rerun.
- Prior failing matrix:
https://github.com/paperclipai/paperclip/actions/runs/33682434315
## Risks
- Bundling changes the bytes covered by provider launch hashes.
Provider-pack generation already hashes the final built files.
- ACPX still loads qualified provider packages dynamically. The
controller supplies a normalized package root, while existing version,
digest, path, and descriptor checks remain active.
- Durable `failed` is terminal. Replays return the same redacted result
and do not execute the provider effect twice.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, parallel subagents, and GitHub Actions
coordination. The exact deployed snapshot and context-window size are
not exposed to this task.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked related public work or described the bug in
this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] No documentation change is required for this runtime repair
- [x] I have considered and documented the risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.
**Problem or motivation**
The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.
**Proposed solution**
Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.
**Alternatives considered**
We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.
**Roadmap alignment**
This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.
Related stack:
- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.
## What Changed
- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.
## Verification
Passed locally:
- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.
I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.
## Risks
The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.
Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.
Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.
The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.
This pull request does not enable the experimental native runner for
existing instances.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner needs repeatable evaluation contracts.
> - Evaluation code must stay separate from provider launch and
production orchestration.
> - Offline fixtures need stable compatibility, scoring, traceability,
and report rules.
> - Published Runner consumers need only the supported evaluation
contract surface.
> - This pull request adds offline evaluation tooling and a
workspace-private matrix kernel.
> - The benefit is deterministic evaluation without credentials or paid
provider calls.
## Linked Issues or Issue Description
Refs #11297
This pull request extracts the offline evaluation unit from the earlier
aggregate Runner work.
## What Changed
- Add a workspace-private, provider-neutral evaluation matrix kernel.
- Add the public `@paperclipai/paperclip-runner/evals` compatibility and
native execution contracts.
- Add fail-closed runnerd artifact and protocol compatibility checks.
- Add deterministic workflow catalogs, scoring, traceability, and report
generation.
- Add sanitized Codex, OpenCode, and ACPX fixtures.
- Add package-boundary and clean-consumer checks.
- Add the eval package manifest to the Docker dependency stage.
- Add the generated protocol fixture digest without changing the
lockfile.
## Verification
GitHub Actions must run:
- Runner TypeScript and Rust type checks.
- Runner unit and protocol tests.
- Evaluation kernel tests.
- Workflow traceability checks.
- Clean-consumer and package-boundary checks.
- Repository test, type-check, build, policy, and security gates.
No local test command was run. The repository owner requested
GitHub-only verification.
## Risks
This is a large greenfield review surface with 51 files. The code does
not launch a live provider or load credentials. Package and protocol
drift fail closed. The workspace lockfile remains under the existing
CI-owned process.
## Model Used
OpenAI Codex with the GPT-5 agent model. The work used high reasoning,
repository inspection, tool use, and parallel code review.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Dotta