## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner now has protocol, provider, tool, package,
persistence, and hidden server boundaries.
> - The server still cannot select that path for a real agent heartbeat.
> - A new runtime must not change any existing direct adapter.
> - An experimental runtime must fail closed when its rollout flag is
off.
> - This pull request adds one guarded Codex vertical slice through
runnerd.
> - The benefit is a production-built runner path that users cannot
start by default.
## Linked Issues or Issue Description
Refs #11962
Refs #12111
Refs #12169
Refs #12176
**Subsystem affected**
Cross-cutting. The change affects the runner package, server
orchestration, shared settings, and adapter configuration UI.
**Problem or motivation**
The hidden PRP coordinator cannot execute a real heartbeat. The
application also needs an explicit rollout boundary before it can expose
the experimental runner. Existing direct adapters must keep their
current execution and finalization behavior.
**Proposed solution**
Add `paperclip_runner` as a Codex-only adapter behind the default-off
`enableNativeRunner` instance flag. Select the native runtime only for
that adapter. Persist the run binding before runnerd starts. Wait for
the durable PRP result and terminal event. Resume the real Codex
provider thread on later heartbeats. Keep persisted native runs readable
and recoverable after the flag changes.
**Alternatives considered**
The server could route `codex_local` through runnerd. That option would
change an existing adapter and weaken rollback safety. The server could
expose all providers now. That option would add unreviewed provider
behavior. The build could depend on a prebuilt runner binary. That
option would make source builds architecture-dependent and difficult to
verify.
**Roadmap alignment**
This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface. It
is the guarded execution step after the merged hidden runner boundaries.
**Additional context**
This is the next replacement for the closed large runner pull request.
Task-thread presentation remains a separate follow-up so this change can
preserve the current direct-adapter UI.
## What Changed
- Add `paperclip_runner` as an explicit Codex-only adapter.
- Add the default-off `enableNativeRunner` instance flag.
- Reject fresh create, hire, import, switch, and execution requests
while the flag is off.
- Allow edits to persisted runner agents while the flag is off.
- Recover an already persisted native run even after the flag is
disabled.
- Keep every built-in direct adapter on its existing runtime path.
- Persist an immutable native run binding and revisioned completion
contract before runnerd starts.
- Execute server to PRP to runnerd to Codex to server through the hidden
coordinator.
- Validate the durable result against the terminal event and exact
completion criteria before finalization.
- Preserve the Codex provider thread ID and use `thread/resume` on the
next heartbeat.
- Strip unsupported Codex configuration fields from the experimental
adapter.
- Build a target-native release runner binary from source and vendor it
into the server distribution.
- Install Rust only in the Docker build stage. Do not add a workflow or
lockfile change.
- Stop the runner process group on completion, cancellation, and forced
shutdown.
## Verification
- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69
TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay,
formatting, and generated-file checks pass.
- Run the 12 focused adapter, settings, runtime-selection, coordinator,
direct-isolation, and real Codex integration test files. All 186 tests
pass.
- The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd,
and a fake Codex app server. It proves one `thread/start` followed by
one `thread/resume`.
- Run `pnpm -r typecheck`.
- Run `pnpm build`.
- Run `pnpm check:token-gates`.
- Build the Docker `build` target from a clean context. Confirm that the
server distribution contains an executable `paperclip-runnerd` built
with Debian Rust 1.85.
- Start the server through the source-mode tsx entry point with the
package `dist` directory absent. Confirm the vendor shim resolves source
exports and the server boots.
- Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1
file skips. Eight untouched workspace and loopback tests fail because
macOS resolves `/tmp` and `/var` through `/private` and because
PID-derived test ports exceed 65535. Linux CI must pass the full suite.
- Confirm that the diff contains 52 files. Confirm that it contains no
`.github` or `pnpm-lock.yaml` change.
## Risks
- The feature flag is off by default. A fresh native start fails with a
stable error while the flag is off.
- A persisted native run remains recoverable after the flag changes.
This prevents rollout changes from corrupting recorded work.
- Only local Codex execution is accepted. Other providers and remote
work modes fail closed.
- Existing direct adapters do not start runnerd, create native rows, use
native status arbitration, or enter native finalization.
- The runner receives its one-use bootstrap ticket through the child
environment. The server does not put the ticket in command arguments or
logs.
- The server validates the company, task, agent, run, runner, session,
completion contract, result, and terminal binding before it accepts
completion.
- The build compiles a target-native Rust binary. Cross-platform release
packaging remains a later concern. Source builds and Docker builds
compile for their current target.
- Docker needs enough build memory for the existing server TypeScript
compile. The Docker build stage sets a 4 GB V8 heap limit.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and applicable tests pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The package-local runner now has a durable PRP transport, but it
cannot execute a real provider.
> - The first provider must preserve PRP identities while using Codex
native thread and turn identities.
> - Recovery must resume the same Codex thread without starting a
duplicate turn.
> - Provider output must become bounded and provider-neutral before it
crosses PRP.
> - Semantic tools must remain unavailable until the catalog and
authorization layers exist.
> - This pull request adds the Codex provider bridge inside the runner
package only.
> - The benefit is a reviewable provider slice with no server or
user-facing behavior change.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This pull request extends private provider infrastructure
in `packages/paperclip-runner`.
**Problem or motivation**
The durable runner from #12100 has no production provider. It cannot
start Codex app-server, map its events, cancel or steer a turn, deliver
a structured question, or recover a native thread after process restart.
**Proposed solution**
Add a supervised Codex app-server transport and a normalized runner
backend. Persist the Codex thread and active turn identities. Resume and
inspect the exact thread after restart. Convert supported notifications
into bounded PRP events. Keep the dynamic tool inventory empty.
**Alternatives considered**
The combined runner branch implements several providers, semantic tools,
server coordination, and UI integration together. That change is too
large for one review unit. Reusing the direct `codex_local` adapter
would also couple this package layer to the existing server execution
path.
**Roadmap alignment**
This work supports the governed tools and self-healing run direction in
`ROADMAP.md`. It does not add a server endpoint, runtime adapter,
rollout flag, or user-facing behavior.
**Additional context**
Refs #12100 and #11962. Pull request #12100 was squash-merged first.
This branch starts at the resulting `master` commit. Its current delta
is 16 files.
## What Changed
- Added a Codex-only app-server process transport with bounded JSONL
frames and buffered notifications.
- Added strict provider descriptor validation for the Codex driver,
working directory, launch arguments, model, instructions, and
non-interactive approval policy.
- Started new Codex threads with an empty dynamic tool inventory and the
named workspace-only permission profile.
- Added native turn start, steering, interruption, cancellation, thread
reads, and structured question responses.
- Added thread and active-turn binding checks for provider requests and
notifications.
- Added provider-neutral normalization for session, turn, item, plan,
usage, tool execution, notice, and structured input events.
- Bounded and redacted provider text and process output before durable
persistence.
- Added private atomic provider state for the descriptor, thread ID,
account session ID, active turn ID, and unacknowledged normalized
events.
- Added exact-thread recovery through `thread/resume` and `thread/read`.
Recovery does not issue another `turn/start` for an active turn.
- Preserved active native turn identity across unexpected provider exit
and reconciled it before later start, interrupt, or snapshot commands.
- Added stable provider-event identities, per-event durable commit and
acknowledgement, and a bounded fingerprint receipt journal that prevents
duplicate delivery across outbox and provider-ack crash windows.
- Extended the durable command executor with provider event polling and
explicit process shutdown on stop, suspend, revocation, lease expiry,
and runtime expiry.
- Preserved completed shutdown behavior when the command result is
replayed after a disconnect.
- Added a fake Codex app-server and integration tests for response
buffering, structured questions, interruption, provider exit,
unacknowledged-event recovery, durable resume, and duplicate-turn
prevention.
- Added a focused `test:codex` package command for the provider
integration suite.
- Kept server code, UI code, other providers, semantic catalogs, tool
authorization, and production runtime selection out of this pull
request.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner check:all` passes.
- TypeScript contract tests pass: 8 Node tests and 44 Vitest tests.
- Rust tests pass: 43 unit tests, 5 Codex integration tests, 3 public
durable-recovery tests, 2 local-runner tests, and 3 process-supervisor
tests.
- Rust conformance and replay parity checks pass against the shared PRP
fixtures.
- `cargo clippy --workspace --all-targets -- -A
clippy::filter-map-bool-then -D warnings` passes. The narrow allow
covers an unchanged replay implementation from the preceding contract
pull request.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- `pnpm check:token-gates` passes.
- `git diff --check` passes.
- The delta against `master` is 16 files. The package lockfile is
unchanged. The PR workflow generates its temporary lockfile artifact
from the changed package manifest.
- `pnpm test:run` completed locally with 4,690 passing tests, 19 skipped
tests, and 26 failures in 8 unchanged server test files. The failures
reproduce the established local macOS path-alias, listener, port-range,
and workspace-runtime baseline. No changed-file test failed. Linux CI
remains the repository handoff authority.
- Browser and migration tests are not applicable because this pull
request changes no server, UI, database, or migration file.
- The full Linux PR workflow passes. One unchanged heartbeat recovery
test timed out on the first pass and passed on the failed-only rerun;
the aggregate `verify` gate is green.
- Greptile is 5/5 on the final commit. All four review threads are
resolved.
## Risks
Production behavior is unchanged because no server code starts this
provider. The main risks are a provider process escape, cross-thread
event confusion, secret leakage, duplicated turns, duplicated or lost
provider events, lost questions, and unsafe recovery. Process-group
supervision, identity binding, private bounded state, redaction, durable
command replay, retained event acknowledgements, bounded durable
receipts, exact-thread reconciliation, and integration tests cover these
risks. Semantic tools remain undiscoverable in this layer.
I checked `ROADMAP.md`. This change is private provider infrastructure
for planned control-plane work. It does not duplicate a shipped or
public product surface.
## Model Used
OpenAI Codex with GPT-5 was used. The exact serving model ID and context
size were not exposed. The model used high reasoning, repository tools,
GitHub tools, and local code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The package-local runner can supervise a local process, but it
cannot yet survive a broken controller connection.
> - A production transport must authenticate both peers without putting
the bootstrap secret on the wire.
> - Commands and events must remain bounded, ordered, and recoverable
across reconnects and crashes.
> - Retrying an uncertain side effect is unsafe, so indeterminate
outcomes must fail closed instead of running twice.
> - This pull request adds those transport and recovery guarantees
inside the runner package only.
> - The benefit is a durable PRP boundary that can be reviewed before
any provider or server integration exists.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting. This pull request extends private transport
infrastructure in `packages/paperclip-runner`.
**Problem or motivation**
The local runner introduced by #12095 has no authenticated network
handshake, durable outbox, reconnect lease, cumulative acknowledgement,
or crash-safe command journal. A dropped connection could otherwise lose
an event or tempt a controller to repeat a side effect whose outcome is
unknown.
**Proposed solution**
Add an authenticated PRP v1 WebSocket transport, encrypted frames,
lease-based reconnects, a bounded durable event outbox, cumulative
acknowledgements, and an idempotent command journal. Preserve pending
commands before execution and classify the crash window as indeterminate
so an uncertain side effect is never repeated automatically.
**Alternatives considered**
The combined runner branch implements transport together with Codex,
semantic tools, and server coordination. That change is too large for
one review unit. Keeping transport in memory would make reconnect and
crash recovery unverifiable. Re-running a pending command after restart
would weaken the at-most-once side-effect boundary.
**Roadmap alignment**
This work supports the governed tools and self-healing run direction in
`ROADMAP.md`. It does not add a production provider, server endpoint,
adapter, feature flag, or user-facing behavior.
**Additional context**
Refs #12095 and #11962. Pull request #12095 was squash-merged first.
This branch has been rebased onto the resulting `master` commit, and its
current delta is 13 files.
## What Changed
- Added a loopback-only WebSocket connection policy with one-time DNS
resolution and pinned reconnect addresses.
- Added an HMAC mutual-authentication handshake that never sends the
bootstrap ticket over the socket.
- Added AES-256-GCM secure frames with per-direction keys, monotonic
counters, and session-bound authenticated data.
- Added one-use bootstrap-ticket handling and lease-based reconnect
validation with expiry, revocation, and epoch checks.
- Added a private, symlink-resistant state directory with atomic,
synchronized state replacement.
- Added a bounded durable event outbox, priority-zero reserve,
cumulative acknowledgements, and reconnect replay of only the
unacknowledged suffix.
- Added a bounded command journal with contiguous sequence enforcement,
persistent results, and deterministic duplicate responses. Duplicate
replay requires a SHA-256 match over the complete canonical command.
- Persisted commands before their effects. A crash after persistence but
before result storage returns an indeterminate terminal result and does
not execute the command again.
- Migrated pre-fingerprint command journals by compacting through their
persisted controller cursor. Legacy redelivery fails closed instead of
reconstructing an incomplete identity or repeating an uncertain effect.
- Added strict limits and validation for frames, state, results, outbox
entries, command history, and redacted diagnostics.
- Added a transport-only `paperclip-runnerd --connect-url` mode. It
handles lifecycle commands and rejects provider commands because no
provider is present in this pull request.
- Added a full disconnect-before-ack fault test that reconnects with the
lease, replays identical command and event state, and proves the effect
ran once.
- Kept provider transports, semantic tools, server integration, and
production runtime selection out of this pull request.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner check:all` passes.
- TypeScript contract tests pass: 8 Node tests and 44 Vitest tests.
- Rust tests pass: 33 unit tests, 3 public durable-recovery integration
tests, plus the existing 2 local-runner and 3 process-supervisor tests.
- The disconnect-before-ack, lease reconnect, duplicate command,
malformed state, unknown command, bounds, and crash-window tests pass.
- Rust conformance and replay parity checks pass against the shared PRP
fixtures.
- `cargo clippy --workspace --all-targets -- -A
clippy::filter-map-bool-then -D warnings` passes. The narrow allow
covers an unchanged replay implementation from the preceding contract
pull request.
- `pnpm -r typecheck` passes.
- `pnpm build` passes.
- `pnpm check:token-gates` passes.
- `git diff --check` passes.
- The delta against `master` is 13 files. The package lockfile is
unchanged.
- `pnpm test:run` completed locally with 4,686 passing tests, 19 skipped
tests, and 30 failures in 8 unchanged server test files. The failures
reproduce the established local macOS path-alias, listener, port-range,
and workspace-runtime baseline. No changed-file test failed; Linux CI
remains the repository handoff authority.
- Storybook visual regression is not applicable because this pull
request changes no UI or story files.
## Risks
Production behavior is unchanged because no server code starts or
connects to this transport. The main risks are secret disclosure, forged
or replayed frames, state corruption, unbounded disk growth, duplicated
side effects, and incorrect recovery. Mutual authentication, encrypted
counter-bound frames, private atomic state, explicit bounds, cumulative
acknowledgements, a durable command journal, fail-closed indeterminate
recovery, and fault-injection tests cover these risks.
I checked `ROADMAP.md`. This change is private transport infrastructure
for planned control-plane work. It does not duplicate a shipped or
public product surface.
## Model Used
OpenAI Codex with GPT-5 was used. The exact serving model ID and context
size were not exposed. The model used high reasoning, repository tools,
GitHub tools, and local code execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Dotta