## Thinking Path
> - Paperclip manages AI agents and their work.
> - The new runner gives agents dedicated tools for common tasks.
> - Some API operations and parameters have no dedicated tool.
> - Agents need a controlled way to find and use those operations.
> - This pull request adds API search and calls through the real server
routes.
> - Existing tools remain the preferred path. The new tools are disabled
by default.
> - Paired tests measure correctness, tool choice, cost and time.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner contracts, production tool authority and the server API
catalog.
**Problem or motivation**
The runner cannot use much of the API described by the old Paperclip
skill. A generic HTTP client would also let agents bypass runner control
rules.
**Proposed solution**
Add `search_api` and `call_api`. Resolve calls from the mounted API
catalog. Use server-held, run-bound credentials. Preserve route checks
and runner lifecycle rules. Keep the tools disabled until an operator
enables selected companies.
**Alternatives considered**
A dedicated tool for every endpoint would add a large initial prompt. An
unrestricted HTTP tool would weaken authorization and replay controls.
**Roadmap alignment**
This extends the native runner tooling. The repository owner requested
this design and implementation. The roadmap and related open PRs were
checked. No duplicate API escape-hatch PR was found.
## What Changed
- Register two compact fallback tools in canonical contracts and
provider projections.
- Build deterministic API discovery from OpenAPI, mounted experimental
routes and the old skill reference.
- Execute bounded JSON, text, file and download requests through
authenticated HTTP routes.
- Recheck active runs, company access and work modes. Block runner
lifecycle, scheduling, credential and approval bypasses. Keep routine
annotation collaboration available.
- Retain mutation receipts. Report uncertain outcomes without blindly
repeating writes.
- Add a company rollout gate and a durable eval worker with complete
cost accounting checks.
- Record child-task creation in the activity log with the agent and run.
- Add contract, authorization, file, replay and real runnerd/PRP/HTTP
tests.
- Document rollout gates and paid coverage limits. The companion eval
repository retains immutable attempts and reports.
## Verification
- Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32
checks passed; the unrelated Storybook visual check was skipped.
Greptile 5/5; no unresolved review threads.
- Full Linux build and recursive typecheck passed. Repository tests were
run by project and serialized shard; all 143 serialized server suites
passed.
- Runner TypeScript: 1,599 passed, two skipped. Rust release: 451
passing test reports. Conformance and replay parity passed. The required
API check passed 837 tests, including runnerd → PRP → authority → real
HTTP.
- Bindings cannot enable API tools without the explicit deployment flag.
Unit and real-authority tests prove the default-off boundary.
- The standalone API check builds and stages its own binary. It passed
after existing staged and debug binaries were removed from the test
container.
- UI and CLI tests passed. Initial environment failures (missing jq,
Docker overlay file identity, and parallel linker memory pressure) and
focused passing reruns are retained. The macOS full runner suite has
platform-specific failures; Linux is the qualified full-check platform.
- Eval harness: 27 tests passed; existing CI discovery ran 86 tests with
two unrelated skips. Credential export rejection is tested against the
actual report command.
- Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten
workflows, three repetitions per arm, zero unnecessary API fallback.
- Sonnet passed 11 selected capability/contract cases after fixes.
Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit
and remains unqualified.
- Luna's two cost flags received focused follow-up. The original flags
and a later n=1 latency flag remain visible. Sonnet had no cost or
latency increase above 20%.
- The catalog contains 785 entries; 58 were exercised across all stages.
Most operation probes remain unrun and some need additional fixtures.
Authored probes do not establish successful coverage.
- Total conservative accounted cost: $9.875960. Active paid-campaign
time: 88.16/90 minutes. No missing accounting. Later security and
harness fixes have provider-free verification; no paid validation is
claimed for those revisions.
- Inspect the [qualification
report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md)
and [verification
record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json).
## Risks
- This is a broad authenticated API surface. Keep the default-off gate
until an operator selects initial rollout companies.
- Paid coverage is incomplete. Small regression samples do not prove all
workflows are unchanged.
- A timeout or server failure can follow a committed mutation. The
result reports an unknown outcome and requires state inspection.
- The new definitions add prompt tokens. The report retains cost flags
and cache variation.
- No database migration is required.
- Repository rules require code-owner approval before merge. Technical
CI and automated review are complete.
## Model Used
OpenAI Codex based on GPT-6 assisted with code, tests and review. The
exact serving model ID and context window are not exposed in this
session. It used reasoning, tool calls and code execution.
Eval models: `gpt-5.6-luna` with low reasoning,
`openrouter/anthropic/claude-sonnet-5`,
`openrouter/google/gemini-3.8-flash`, and
`openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime
versions, model identity, usage and source provenance.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.
**Problem or motivation**
The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.
**Proposed solution**
Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.
**Alternatives considered**
We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.
**Roadmap alignment**
This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.
Related stack:
- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.
## What Changed
- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.
## Verification
Passed locally:
- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.
I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.
## Risks
The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.
Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.
Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.
The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.
This pull request does not enable the experimental native runner for
existing instances.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner package already provides the production protocol and
execution spine.
> - Contributors still need stable SDK surfaces, deterministic test
tools, and local inspection tools.
> - Those surfaces share generated contracts and must change as one
package boundary.
> - This pull request adds the package-local SDK, labs, examples, and
drift checks.
> - The benefit is a reviewable developer platform that does not change
application execution selection.
## Linked Issues or Issue Description
**Subsystem affected**
`packages/paperclip-runner` — runner SDK, conformance tools, and
developer tooling.
**Problem or motivation**
The production runner spine is present, but package consumers cannot
build deterministic integrations, inspect sessions, or verify
provider-neutral behavior through supported surfaces.
**Proposed solution**
Add browser, React, standalone, live-session, scenario, conformance, and
evaluation surfaces. Add generated contract inventories and
package-local verification scripts. Keep production application routing
unchanged.
**Alternatives considered**
We considered splitting each generated catalog, SDK surface, and demo
into separate pull requests. Those changes share exports, fixtures, and
drift gates. Splitting them would create intermediate package states
that do not build.
**Roadmap alignment**
No overlapping item appears in `ROADMAP.md`. This work extends the
runner package that is already on `master`.
## What Changed
- Add browser, React, standalone, live-session, and issue-thread SDK
surfaces.
- Add deterministic mock control-plane, scenario, conformance, replay,
and evaluation tools.
- Add bounded Codex, OpenCode, and ACPX development transports and
fixtures.
- Keep deferred managed-provider execution fail-closed. Persisted
compatibility data remains readable.
- Add generated capability inventories with their source files and drift
checks.
- Add examples, package documentation, browser checks, and
clean-consumer checks.
- Preserve the reviewed protocol bounds, replay compatibility aliases,
process environment isolation, and semantic redaction limits.
- Update the ACPX package patch that the existing workspace patch
registry already tracks.
- Do not change `pnpm-lock.yaml`, repository workflows, server runtime
selection, or the application UI.
## Verification
GitHub Actions is the verification authority for this pull request. The
repository CI, package TypeScript and Rust checks, package tests,
generated-output drift checks, browser checks, security scans, and
Greptile review must pass on the exact head.
Local test suites were not run because this series uses parallel GitHub
Actions for verification.
## Risks
This is a large greenfield package change. The main risks are public
export drift, generated-output drift, and optional React consumer
compatibility. Package boundary checks, clean-consumer checks, and
browser tests cover those risks. Production adapter selection and server
execution are outside this pull request.
## Stack
1. **This PR:** runner SDK and developer tooling.
2. [Codex production server
integration](https://github.com/paperclipai/paperclip/pull/12616).
3. [Provider-neutral task-thread
UI](https://github.com/paperclipai/paperclip/pull/12617).
## Model Used
OpenAI Codex, GPT-5, high-reasoning mode, with tool use and code
execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the feature request
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Dotta
Dotta