Keep runtime-owned command snapshots reusable until cleanup and observe optional prompt-start rejection without hiding it from callers.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Expose the existing finalization timestamp through the scoped sync API and reject periodic-only or out-of-run saves.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Accept bounded Codex deprecation notices after a settled turn and require that capability before reusing a sandbox image runner. Stage replacement artifacts atomically so existing launchers and image symlink targets survive interrupted uploads.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Bind validated OpenCode result notifications to the active provider process and turn, matching semantic tool responses. Reproduce the live shutdown failure and verify exact durable authority after interruption.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Park idle native sandbox sessions before app shutdown, adopt retained legacy workspaces without restaging Git, and carry explicitly bound GitHub access through both OpenCode launch filters.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Select Codex ACP's full-access initial mode only for host-validated external work-folder environments with approve-all authority. Keep local and restrictive permission modes unchanged. The regression failed before the fix; all 50 ACP environment and runtime tests pass.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep custom profile-provided runtimes available while restoring the projected Git launcher before agent startup.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Preserve migration SQL hashes when renumbering and keep sandbox HOME with managed GitHub shell profiles.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Codex filters explicit environment overrides through include_only. Retain the scoped home and standard executable path without allowing provider or host secrets into shell commands.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Codex filters explicit environment overrides through include_only. Retain the scoped home and standard executable path without allowing provider or host secrets into shell commands.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep the execution target identity through workspace realization, restore its saved home for finalization, and distinguish warm setup reuse from replacement dependency recovery. Label direct file updates separately from agent checkpoints.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep the execution target identity through workspace realization, restore its saved home for finalization, and distinguish warm setup reuse from replacement dependency recovery. Label direct file updates separately from agent checkpoints.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Several people can send instructions to the same agent and task.
> - A fixed GitHub token in the provider process can keep the first
person's access after another person's message is accepted.
> - Task ownership cannot select credentials for each accepted
instruction or preserve the identity of an operation already in
progress.
> - This pull request records ordered execution identity contexts and
resolves credentials when managed Git, gh, or GitHub tools start.
> - The benefit is automatic personal GitHub access for shared agents,
with durable continuation rules and no teammate credential fallback.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting: orchestration, connection grants, database, runtime
adapters, native runners, and run details.
**Problem or motivation**
A shared agent must use the person whose instructions it has accepted. A
queued message must retain its author. A retry or approval without new
instructions must retain the originating identity. GitHub must remain
optional for ordinary work.
**Proposed solution**
Persist execution identity separately from task ownership. Give new
processes a run-scoped broker capability and token-free managed
launchers. Capture identity at operation start. Keep an explicit
dedicated-agent grant as an override. Show redacted diagnostics in run
details.
**Alternatives considered**
Per-task ownership, fixed provider tokens, and mutable repository author
configuration do not handle accepted steering or concurrent operations.
A manual account-selection action would add unnecessary setup to each
turn.
**Roadmap alignment**
This completes the existing Multiple Human Users, MCP Tool Gateway &
Apps, Secrets Manager, and Self-healing Runs capabilities. The
implementation follows the maintainer-approved plan.
Related work: Refs #12843, Refs #12907. Existing proposals #4618 and
#8945 cover per-agent or per-worktree author configuration. This change
instead follows the accepted human instruction across runtime types.
Refs #11831 for governed personal connection delegation; this change
preserves connection audience checks and does not use standing
delegation as a personal credential fallback.
## What Changed
- Add durable, ordered identity contexts and active run references.
Preserve message authors through consolidation, steering, retries,
delegation, approvals, routines, and restart.
- Add an authenticated operation-time GitHub credential broker and
local/remote managed git and gh launchers. Keep personal tokens out of
the long-lived provider process.
- Resolve GitHub gateway and server-side Git operations through the same
responsible-person or dedicated-grant selection rules.
- Make absent and unavailable GitHub credentials non-blocking at generic
startup. Clear host and prior-person credentials. Keep anonymous Git
access where supported.
- Add run-detail identity history and the dedicated-account warning.
Keep task ownership and queue-versus-steer decisions unchanged.
- Preserve personal OAuth declarations through connection edits. Retain
exact selected grants in the gateway.
- Fix continuation races found during real acceptance: verify a warm
owner before credential rotation, and wait for bounded durable runner
suspension before the next run starts.
- Make migrations replay-safe. Retain identity through agent/run
deletion, remove it with its company, and clean terminal launcher
directories before releasing execution environments. Document
coordinated release and rollback.
## Verification
- Full workspace typecheck, build, and token gates passed. The complete
local suite passed in its normal test groups: 17,120 passing tests,
including all 143 serialized server suites. After integrating the newly
merged runner API work, full local typecheck and build passed again,
along with 890 focused integration tests. All 31 checks on the
integrated revision passed, including build, browser E2E, release
registry, canary dry run, typecheck, security and all test suites.
Greptile is 5/5 with all review threads resolved.
- Current focused checks passed: 142 native executor tests, 67 runtime
lifecycle tests, 9 durable identity tests, 75 credential/routine tests,
19 low-trust/resumption tests, and the executable migration replay test.
- Authenticated browser acceptance with two Paperclip users and two
GitHub accounts on one shared native agent passed. Real commits and
pushes followed A → B accepted steering → queued A continuation in the
same saved conversation. GitHub commit author and committer identities
matched all three operations. Both runs succeeded and task ownership
stayed unchanged.
- Real GitHub MCP calls switched from A to B after accepted steering. A
delegated subtask retained its originating identity across a server
restart.
- Disabling B's GitHub connection left ordinary work successful. Managed
gh was unauthenticated and the provider had no inherited GH_TOKEN or
GITHUB_TOKEN.
- The browser displayed run-detail diagnostics and the exact
dedicated-account warning. A final controller-restart check followed by
another-person continuation retained the conversation, selected the
correct GitHub login and Git author, and removed each terminal launcher
directory.
- Company-lifetime migration and all five previously failing CI suites
passed locally (167 tests). Same-token gateway A → B → A and six
broker/launcher boundary tests passed.
- Remote callback, launcher, sandbox, and runtime contract tests passed.
Both native and legacy Codex completed actual Daytona executions on the
integrated revision ([campaign
results](https://github.com/paperclipai/paperclip/actions/runs/34155056509)).
The remote package-manager shim staging regression also passed locally.
## Risks
- Deploy the migrations, server broker, launchers, and runner artifacts
together. Existing processes finish with their original contract. New
managed processes need the broker endpoint for GitHub operations.
- Finish or stop new managed executions before rolling application code
back. Keep the additive schema and identity history during rollback.
- Scripts that require a persistent raw GH_TOKEN must use managed git,
gh, or GitHub gateway tools. Run capabilities authorize code executing
within that run to acquire its current identity; this is not
hostile-code isolation within one execution principal. Managed commands
prevent automatic credential carryover; arbitrary code deliberately
copying a credential is outside that boundary.
- Uncertain steering acknowledgement deliberately holds new credential
acquisition until reconciliation. Already-started operations retain
their captured identity.
- GitHub private access and provider outages can still fail the specific
operation that needs them. Dedicated grant failure does not fall back to
personal access.
## Model Used
OpenAI GPT-6 through Codex assisted implementation, review, shell
execution, and browser acceptance. The exact model variant and
context-window size are not exposed in this session. Tool use included
TypeScript and Rust tests, database integration tests, GitHub CLI, and
authenticated browser control.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip manages AI agents and their work.
> - The new runner gives agents dedicated tools for common tasks.
> - Some API operations and parameters have no dedicated tool.
> - Agents need a controlled way to find and use those operations.
> - This pull request adds API search and calls through the real server
routes.
> - Existing tools remain the preferred path. The new tools are disabled
by default.
> - Paired tests measure correctness, tool choice, cost and time.
## Linked Issues or Issue Description
**Subsystem affected**
Paperclip Runner contracts, production tool authority and the server API
catalog.
**Problem or motivation**
The runner cannot use much of the API described by the old Paperclip
skill. A generic HTTP client would also let agents bypass runner control
rules.
**Proposed solution**
Add `search_api` and `call_api`. Resolve calls from the mounted API
catalog. Use server-held, run-bound credentials. Preserve route checks
and runner lifecycle rules. Keep the tools disabled until an operator
enables selected companies.
**Alternatives considered**
A dedicated tool for every endpoint would add a large initial prompt. An
unrestricted HTTP tool would weaken authorization and replay controls.
**Roadmap alignment**
This extends the native runner tooling. The repository owner requested
this design and implementation. The roadmap and related open PRs were
checked. No duplicate API escape-hatch PR was found.
## What Changed
- Register two compact fallback tools in canonical contracts and
provider projections.
- Build deterministic API discovery from OpenAPI, mounted experimental
routes and the old skill reference.
- Execute bounded JSON, text, file and download requests through
authenticated HTTP routes.
- Recheck active runs, company access and work modes. Block runner
lifecycle, scheduling, credential and approval bypasses. Keep routine
annotation collaboration available.
- Retain mutation receipts. Report uncertain outcomes without blindly
repeating writes.
- Add a company rollout gate and a durable eval worker with complete
cost accounting checks.
- Record child-task creation in the activity log with the agent and run.
- Add contract, authorization, file, replay and real runnerd/PRP/HTTP
tests.
- Document rollout gates and paid coverage limits. The companion eval
repository retains immutable attempts and reports.
## Verification
- Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32
checks passed; the unrelated Storybook visual check was skipped.
Greptile 5/5; no unresolved review threads.
- Full Linux build and recursive typecheck passed. Repository tests were
run by project and serialized shard; all 143 serialized server suites
passed.
- Runner TypeScript: 1,599 passed, two skipped. Rust release: 451
passing test reports. Conformance and replay parity passed. The required
API check passed 837 tests, including runnerd → PRP → authority → real
HTTP.
- Bindings cannot enable API tools without the explicit deployment flag.
Unit and real-authority tests prove the default-off boundary.
- The standalone API check builds and stages its own binary. It passed
after existing staged and debug binaries were removed from the test
container.
- UI and CLI tests passed. Initial environment failures (missing jq,
Docker overlay file identity, and parallel linker memory pressure) and
focused passing reruns are retained. The macOS full runner suite has
platform-specific failures; Linux is the qualified full-check platform.
- Eval harness: 27 tests passed; existing CI discovery ran 86 tests with
two unrelated skips. Credential export rejection is tested against the
actual report command.
- Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten
workflows, three repetitions per arm, zero unnecessary API fallback.
- Sonnet passed 11 selected capability/contract cases after fixes.
Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit
and remains unqualified.
- Luna's two cost flags received focused follow-up. The original flags
and a later n=1 latency flag remain visible. Sonnet had no cost or
latency increase above 20%.
- The catalog contains 785 entries; 58 were exercised across all stages.
Most operation probes remain unrun and some need additional fixtures.
Authored probes do not establish successful coverage.
- Total conservative accounted cost: $9.875960. Active paid-campaign
time: 88.16/90 minutes. No missing accounting. Later security and
harness fixes have provider-free verification; no paid validation is
claimed for those revisions.
- Inspect the [qualification
report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md)
and [verification
record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json).
## Risks
- This is a broad authenticated API surface. Keep the default-off gate
until an operator selects initial rollout companies.
- Paid coverage is incomplete. Small regression samples do not prove all
workflows are unchanged.
- A timeout or server failure can follow a committed mutation. The
result reports an unknown outcome and requires state inspection.
- The new definitions add prompt tokens. The report retains cost flags
and cache variation.
- No database migration is required.
- Repository rules require code-owner approval before merge. Technical
CI and automated review are complete.
## Model Used
OpenAI Codex based on GPT-6 assisted with code, tests and review. The
exact serving model ID and context window are not exposed in this
session. It used reasoning, tool calls and code execution.
Eval models: `gpt-5.6-luna` with low reasoning,
`openrouter/anthropic/claude-sonnet-5`,
`openrouter/google/gemini-3.8-flash`, and
`openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime
versions, model identity, usage and source provenance.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Preserve JSON Content-Type when custom idempotency headers are supplied. Expose the latest accepted folder operation timestamp so browser saves remain visible before an agent runs and after all files are deleted.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Resume retained ephemeral leases, pin managed-agent imports against link replacement, carry scoped home settings through native provider boundaries, and keep file transfer arguments below Linux limits. Restrict acceptance exclusions to the engines explicitly deferred by the user.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Use object-storage artifact URLs and Actions build artifacts without creating releases. Fix both native session entry points, preserve the primary workspace environment, and declare upload headers in OpenAPI. Tests cover sandbox home confinement, session dispatch, and generated headers.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Introduce company-scoped task, agent, user and project files, recoverable trash, object-backed repository checkpoints, and a shared 180-second lifecycle for sandbox execution. Wire file browsing and save status into the owner surfaces. This is an implementation checkpoint; staging acceptance and the complete merge gate remain outstanding.
## Thinking Path
- Paperclip Runner needs its runtime preinstalled for fast sandbox
startup.
- Native and local adapters should launch one current CLI installation
per provider.
- An older global copy can shadow that installation, and exact native
compatibility pins must match it.
- Update the qualified releases and binary digests, expose shared CLI
entrypoints from the provider pack, and prefer the image-owned bin
directory.
- Keep dependency installation in the image build; task startup only
discovers, links, and verifies artifacts.
## Linked Issues or Issue Description
**What happened?**
Remote native startup rejected a stale global Codex, while CLI-only
images lacked runnerd entirely.
**Expected behavior**
An image-baked runtime starts without uploading binaries or installing
packages. All adapters share the same current provider CLI.
**Steps to reproduce**
Start a native remote task with the old global Codex and the updated
runtime available only under `/opt/paperclip-runner/bin`.
**Paperclip version or commit**
Discovery behavior at `54a99d884`.
**Deployment mode**
Docker with a remote sandbox.
## What Changed
- Prefer `/opt/paperclip-runner/bin`, then the user's local bin
directory, then PATH. Existing metadata and version validation remains
in force.
- Qualify Codex 0.153.4, OpenCode 1.18.29, and Claude SDK 0.3.263 / CLI
2.1.263. Update binary digests, TypeScript/Rust checks, registry
defaults, and the displayed OpenCode version together.
- Share Codex and Claude's native executable with the ACP bridges
through exact dependency overrides. Preserve the separately qualified
ACP bridge implementations and their security patches.
- Expose shared provider-pack CLI launchers; fail the pack build if
Codex ACP resolves a separate Codex installation. Update the eval
image's other agent CLIs to current stable releases and remove duplicate
global provider installs.
- Document the single-current-CLI policy in source comments and
development guidance. Latest stable releases are resolved at
review/build preparation and pinned; task startup never auto-updates.
## Verification
- Native-session and adapter-registry suites: 158 tests passed.
- Provider suites: 88 tests passed, 7 Linux-only checks skipped on
macOS. One existing macOS temporary-path alias assertion passed when
rerun with canonical `TMPDIR=/private/tmp`.
- Package-contract and OpenCode materialization tests: 11 passed.
- Full typecheck, build, and token gates passed. Rust
native-provider/recovery tests: 19 passed.
- Broad local suite: 5,974 passed, 23 failed, 41 skipped. Failures are
in unchanged macOS workspace/path/port and connection suites; focused
runtime tests pass. All latest-head Linux PR checks passed, including
the full test shards, typecheck, build, runner verification, browser
suites, and canary dry run.
- The standalone fleet image built with one current provider CLI each
and passed native Codex/Claude binary-integrity checks. A disposable
Daytona sandbox reported ready in 798 ms; its baked runner completed an
API-key `gpt-5.6-luna` turn in 2,430 ms and returned the expected marker
with a usage receipt. No runtime artifacts were uploaded or installed.
- The normal shared `codex exec` entrypoint also completed an API-key
`gpt-5.6-luna` turn in 2,321 ms.
- Both image builds verify the complete generated lockfile against a
reviewed SHA-256 before package installation or lifecycle execution.
Root lockfile changes remain CI-owned. Merge and rollout remain on hold
for operator review.
## Risks
- Updating provider CLIs changes their behavior for all adapters;
version probes and live native smoke testing are required before image
promotion.
- The image-owned directory takes precedence. Its entries must launch
the same shared CLI as the global PATH, not a private older/newer copy.
- Application qualification pins and the deployed image must move
together. No startup fallback installation is added.
- No schema or authentication-policy changes.
## Model Used
OpenAI GPT-6 (Codex). The session does not expose a more specific model
ID or context-window size. Used reasoning, repository inspection, code
execution, and browser verification.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub connections give agents access to approved repositories.
> - One GitHub identity can use installations across several
organizations.
> - The permissions page linked to one installation and showed an
unfiltered list.
> - Users could not easily find another organization or inspect a large
selection.
> - This change adds account filtering, search, and access configuration
links.
> - Users can inspect repository access in one compact view.
## Linked Issues or Issue Description
Refs #12993. Related repository-catalog work in #11228 and #11234 was
checked. This change only improves the existing GitHub connection
permissions page.
**What existing behavior does this improve?**
The GitHub connection permissions page and its repository display
metadata.
**Current behavior**
The page links directly to an existing installation. The repository list
has no account filter, search, height limit, or private-repository
marker. Refresh access occupies a separate section.
**Proposed behavior**
Show all authorized repositories by default. Filter by account or
organization and search by name. Open GitHub's account chooser to
configure access across organizations. Show GitHub icons and
private-repository locks. Keep refresh beside configuration and limit
the visible list to about ten rows.
**Reason and benefit**
Users can find repositories across organizations and configure missing
access without creating another GitHub identity. Large repository lists
no longer fill the page.
**Breaking changes**
None. Repository display metadata gains an optional private flag. Older
snapshots remain valid and gain the flag after access refresh. No SQL
migration is required.
## What Changed
- Add an All accounts view, account filter, search, and empty states.
- Link both configuration controls to GitHub's app account chooser.
- Place an accessible refresh icon beside the configuration button.
- Keep the repository heading and list in one section.
- Add GitHub icons and private-repository locks.
- Cap the scrollable list at ten rows using a design token.
- Persist GitHub's private flag only when the provider returns a
boolean.
- Recover missing legacy app configuration from GitHub installation
metadata.
- Update tests and the GitHub connection runbook.
## Verification
- Focused tests passed: 54 permissions-page tests and four GitHub
metadata tests.
- UI and server typechecks passed before submission. Token gates passed.
- Browser checks verified account filtering, search, empty results, and
the configuration destination.
- The live list contained 40 repositories. Its final height was 272
pixels, which fits ten single-line rows with gaps. Scrolling retained
all rows.
- A live access refresh populated 30 private-repository lock icons from
GitHub metadata.
- Full workspace typecheck and build passed. The broad local suite
stopped in the general-server group with 18 failed files. Failures
include macOS temporary-path handling and embedded PostgreSQL startup.
That run also overlapped the legacy fix and retained a stale GitHub
module; the final focused run passed all 58 tests. Clean-runner CI is
tracked separately.
- Latest-head review is 5/5 with the legacy chooser finding resolved.
All CI checks passed on commit
`0ff2b63f348f5c87d8b7df6e43388f60f5d872d9`, including build, typecheck,
all test shards, browser tests, and canary dry run.
## Risks
- Older repository snapshots lack visibility metadata until refreshed.
Unknown visibility does not display a lock.
- The account filter lists authorized installation owners. Users add
other organizations through GitHub's chooser.
- Filtering changes only the displayed list. GitHub remains
authoritative for repository access.
## Model Used
OpenAI GPT-6 (`gpt-6-astra`) via Codex. Reasoning, code execution, and
browser tools were used. The exact context window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub connections give agents an account with selected repository
access.
> - Fresh local instances enroll with production Paperclip Cloud.
> - Enrollment could finish while the GitHub OAuth profile remained
disabled.
> - Setup then switched to a personal access token form without
explanation.
> - This change preserves sign-in intent and shows the connected account
and repositories.
## Linked Issues or Issue Description
Related: #12907, #12943, #12947. Existing open GitHub connection work
was checked. No duplicate was found.
**What happened?**
After Cloud enrollment, a fresh test-drive asked for a GitHub key.
Production did not advertise the managed GitHub profile. Staging did.
The permissions page also omitted the authenticated username and
repository names.
**Expected behavior**
Continue with GitHub OAuth when available. Explain unavailable sign-in
and allow retry otherwise. Show the GitHub username and complete
accessible repository list.
**Steps to reproduce**
Start a fresh test-drive. Choose GitHub and complete instance enrollment
while the Cloud GitHub profile is disabled. Open an existing GitHub
connection's permissions page.
## What Changed
- Preserve managed sign-in intent when the gallery omits its profile.
- Refresh the selected gallery entry on retry without resetting the
audience.
- Fetch all pages of GitHub installations and repositories.
- Store only repository IDs, full names, and installation IDs in grant
metadata.
- Show the GitHub username, repository list, management link, and
refresh action.
- Discard the repository snapshot after newer installation lifecycle
events. Preserve snapshots verified after delayed events.
- Lock and re-read grant metadata when applying installation events or
saving refreshed access. Patch only webhook fields for other events.
Reject snapshots if access changed during the external fetch, using
unique access revisions even when timestamps collide.
- Show repository installation recovery for managed OAuth even when the
app also offers an advanced PAT method.
- Update tests and the GitHub connection runbook. No SQL migration is
required.
## Verification
- Local typecheck, build, and token gates passed. All latest-head CI
gates passed, including the complete test matrix and browser suites.
Greptile is 5/5 with no unresolved findings.
- All 382 focused setup, permissions, metadata, service, and webhook
tests passed across final runs. One socket-hang-up test passed on rerun
with the full service suite. Final service, metadata, and webhook checks
passed all 230 tests.
- The broad local suite was stopped after failures. Seven
workspace-runtime exposure and control-conflict failures reproduce on
base commit `54a99d884`. The broad run also overlapped local iteration;
final focused tests and clean-checkout CI are tracked separately.
- Browser: a fresh production-backed instance completed enrollment,
retried after profile enablement, reached GitHub consent, recovered from
a missing installation, and completed OAuth.
- Browser: the permissions page showed the authenticated username and
the selected private test repository. A real `get_me` call returned the
same account. Reading the selected repository passed; reading an
unselected private repository failed with 404.
- Browser: a second fresh instance completed enrollment and OAuth
without a PAT form or unavailable state. Its username and repository
list survived reload and refresh. A real get_me call on the final code
returned the displayed account.
## Risks
- Repository names are now stored in company-scoped grant metadata and
shown with that credential. They are display data, not authorization
data.
- Large selections require more GitHub API calls. A failed later page
rejects the refresh rather than reporting a partial list.
- Older grants and webhook-invalidated snapshots require Refresh access
to load the list.
- Cloud profile enablement is separate deployment configuration. This PR
does not change OAuth scopes or GitHub App permissions.
## Model Used
OpenAI GPT-6 (`gpt-6-astra`) via Codex. Reasoning, code execution, and
browser tools were used. The exact context window size was not exposed.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Direct Runner evals retain evidence across model configurations.
> - Evalbook already has a grid and a read-only Runner Lab chat viewer.
> - Public projection stripped the view and selected a second plain
result page.
> - This change uses the existing viewer for public and private results.
> - The data access differs, but the presentation does not.
## Linked Issues or Issue Description
Refs #12931, #12945. Related open runtime-contract PR #11634 does not
contain this report-only change.
**What happened?**
The public direct-eval campaign opened plain result pages. The
access-controlled artifact used the chat viewer. Users could not follow
the same recorded interaction from the published grid.
**Expected behavior**
Every newly generated Runner Evalbook opens the existing chat viewer.
The grid and durable run history remain. Public evidence has explicit
redactions.
**Steps to reproduce**
Open campaign gha-34062394019-1 from the direct-eval history. Click a
result, then compare its plain page with the corresponding Actions
artifact.
**Paperclip version or commit**
Reproduced at 83987210d6.
**Deployment mode**
Static GitHub Actions artifacts and S3/CloudFront publication.
Companion site-theme renderer:
https://github.com/paperclipai/paperclip-evals/pull/19. This removes the
Python light theme and links the same built stylesheet.
## What Changed
- Add a closed public chat projection. Require mock isolation evidence
before publishing recorded text. Scrub private references and withhold
tool payloads, reasoning and provider state.
- Validate public HTML against the exact trusted viewer shell and asset
bytes. Validate the public DTO and local links. Keep CSP restrictions on
outbound requests and forms.
- Make the workflow render both data projections with the canonical
viewer. Pass a viewer-only artifact to the trusted publisher. Reject an
old renderer pin before paid execution.
- Fix report-only start position, missing-state inspector, read-only
controls and redaction labels. Tool evidence links select and highlight
the Evidence tab even when reopened. Runner execution and the full-stack
E2E workflow are unchanged.
- Add a no-model report refresh command. Preserve original campaign
identity, measurements and immutable history; label report revisions.
- Document the single presentation and public/private evidence boundary.
- Use one Runner Lab stylesheet and local fonts for the grid, Latest,
test design, inventory, server gate and S3 history index. Keep static
styles scoped away from live chat.
- Emit exact published report/history URLs to the Actions summary and
job outputs only after successful upload. Set the deployment link from
that output.
- Switch public Eval and Evidence panes without rendering both at once.
- Add all-run history with like-for-like pass-rate and cost timelines,
regression/recovery lists, exact commit links, source refs and Actions
links.
- Record all-attempt costs including retries. Keep provider list cost
separate from estimates. Label missing coverage and historical
final-only costs.
- Retain all run records beyond 200, backfill a separate derived
analytics projection, and exclude report refreshes from measurements.
## Verification
- Focused report/publishing/projection/workflow/adapter tests: 29
passed.
- Workflow Evalbook adapter tests: three passed.
- Viewer unit tests: eight passed; Vite viewer build passed.
- Companion renderer tests: 59 total, 57 passed, two inventory tests
skipped because the expected sibling checkout is absent.
- Re-rendered all 375 retained attempts from the completed campaign and
validated the public bundle. Zero new model calls; 356/358 selected
cells still pass.
- Browser walkthrough: grid to failed chat; prompt, named tool calls,
correct blocked status, visible assertions, no loading spinner or
composer. Public payload redactions are explicit.
- Full typecheck and build passed locally. test:run ended with 17 failed
files and 19 failed tests in unrelated server/worktree areas (3772 tests
passed). Latest-head CI is the final merge gate. The browser sources
also typecheck with a temporary TypeScript-7-compatible path
configuration; the checked-in browser config still uses removed baseUrl
options and is unchanged here.
- Real Chromium verification passed for passing, failing and
missing-recording attempts in both the full and public bundles. It
checks tool expansion, navigation, reload, read-only controls, narrow
viewport visibility and the public no-network boundary. Future
publications run it automatically.
- Fresh live proof: three gpt-5.4-mini native Codex cases passed on the
first attempt (get-task-context, create-child-task,
workflow-context-document-progress); estimated total $0.00632625.
Generated the full canonical report and verified all three file://
pages, all seven DevTools tabs, and evidence cross-links in Chromium.
Private screenshots remain local.
- Follow-up 99697c2c5: viewer build, eight unit tests, browser token
gate and browser-source typecheck (existing TS7 configuration
workaround) passed. Added repeated tool-to-evidence navigation to the
publication browser gate.
- Site-theme follow-up: 26 publishing/security tests, 59 Python tests
(57 passed, two existing skips), eight viewer unit tests and the viewer
build passed. Chromium verified shared colors and
grid/test-design/Latest navigation.
- Re-rendered all 375 retained attempts with the shared site theme,
without new provider calls.
- Successful hosted publication: [refreshed
Evalbook](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-34062394019-1-report-site-theme-v3/index.html)
and [themed
history](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/index.html).
Real browser verified grid → test design → chat → All results and
history → run.
- Full maintained live suite completed on AWS: [Actions run
34074939112](https://github.com/paperclipai/paperclip/actions/runs/34074939112),
Paperclip 856813ba3a, evals
34e1846c06a39e641182dadce5de7ea739f657f1. All 358 cells across 11
configurations ran; 355 passed (99.2%), two behavior failures and one
infrastructure failure. Nine configurations are entirely green. All 360
retained attempts were rendered using the new design and published as an
immutable, zero-provider-call report refresh: [full
Evalbook](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-34074939112-1-report-history-v1/index.html).
The trusted master workflow ran the models; this PR's viewer rendered
their results afterward.
- Remaining live failures: native Codex mini did not emit the expected
discovery event for lazy-unauthorized-undisclosed (no mutation
occurred); GLM 5.3 scheduled a wake before approval in
workflow-governed-wait; GLM 5.3 timed out on create-task-document on
both attempts. No scoring rules were relaxed.
- Retry-inclusive recorded estimated model cost is at least $7.067692,
with usage for 357/360 attempts. Provider-reported list cost is a
separate alternative (at least $14.002806), not an additive cost.
Missing usage is unknown, not zero; AWS compute is excluded.
- [Production
history](https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/index.html)
now contains nine model runs and two separately labeled report
refreshes. Matching full-suite comparison reports three newly failing
cells and two recoveries versus the previous run. This is observed
run-to-run variation, not proof of a deterministic code regression. Live
HTTP checks verify the newest report link, shared theme, exact source
SHAs and cost analytics.
- The new full report passed Chromium checks for passing, failing and
missing-recording chat pages, tool-to-Evidence links, grid/design/Latest
navigation, reload, read-only controls and narrow layouts. The Mac was
locked during final hosted verification; the new hosted history was
checked by HTTP and generated-page browser checks, not a fresh
interactive desktop walkthrough.
- History follow-up: 31 publishing/security/metrics tests pass. Viewer
build passes. Chromium checks desktop and narrow history pages with no
document-level horizontal overflow.
### Visual verification
Generated from the scrubbed completed campaign; no private provider
identities or raw tool payloads are included. Full private pages were
also browser-tested, but their private metadata is not published as
screenshots.
Latest shared-theme proof:
## Risks
- Public chat text is newly visible, but only for the isolated mock
boundary. The producer excludes raw payloads and the publisher fails
closed on unknown fields, secrets, shell changes and asset
substitutions.
- Requires the companion canonical renderer revision and an updated
RUNNER_PROTOCOL_EVALS_SHA after merge. Old pins fail before paid
execution.
- Existing published campaigns remain immutable. A report refresh is a
separate history entry, not a new model qualification.
- Successfully published the immutable site-theme refresh with the
configured report-bucket SSO profile. Original run records and
qualification pointers are preserved.
## Model Used
OpenAI Codex, GPT-5-based coding agent with reasoning, shell and browser
tools. Exact deployment model ID and context-window size are not exposed
in this session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green on
cad99dbf04
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
on cad99dbf04
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Runner direct live evals test semantic tools against a mock control
plane.
> - The first complete AWS campaign exercised 358 cells.
> - It exposed setup differences from the working full-stack harness.
> - This pull request corrects those direct-harness differences.
> - It preserves production permission defaults and the full-stack
workflow.
## Linked Issues or Issue Description
**What happened?**
Native Codex cells could not find a global codex executable. ACPX denied
unattended tool requests and lost valid provider usage receipts.
AgentCore hit a 30-second facade timeout while its worker allows 120
seconds for delivery. Several models reported a native run result
without updating the separate mock task state.
**What did you expect?**
The direct harness should use the pinned executable, explicit test
permissions, and a timeout compatible with the provider delivery
contract. Its instructions should explain which operation changes mock
task state.
**Steps to reproduce**
Run the full Runner Direct Live Protocol Evals workflow. Baseline
campaign:
https://github.com/paperclipai/paperclip/actions/runs/34059009921.
**Paperclip version**
Master at b55ce03e86. Related public
change: #12932. No duplicate fix was found.
## What Changed
- Resolve native Codex from the pinned Codex ACP dependency, as the
full-stack launcher does.
- Select explicit unattended ACPX permissions only in the isolated
direct eval harness.
- Add a configurable bounded turn-admission wait. AgentCore direct evals
use up to 125 seconds, capped by their turn budget. Other callers retain
the current default waits.
- Explain mock task-state operations separately from native run-result
reporting. Keep all scoring assertions unchanged.
- Bind the direct turn's scope to the current request, rather than stale
shared fixture notes. A provider's end-of-turn result does not authorize
an unrequested mock task completion or extra completion comment.
- Normalize the qualified Claude/Codex usage semantics without
double-counting reasoning or inventing unknown billable categories.
Forward ACPX's persisted terminal prompt-response usage for the exact
current turn; reject stale or ambiguous receipts.
- Preserve the known numeric ACPX token-counter aliases through durable
redaction. Continue redacting strings and credential-shaped values. A
regression test reproduces the previously redacted usage before
normalization.
- Add regression tests and operating documentation.
## Verification
- Passed Runner TypeScript build.
- Passed 88 focused tests across the eval request contract, provider
setup, and Runner transport.
- Passed 93 focused ACPX adapter, sidecar lifecycle, and
usage-accounting tests, including persisted receipt identity and
missing-field regressions.
- Passed 97 additional runtime-host, in-process ACPX driver, OpenCode
proxy, and MCP bridge tests.
- Reproduced the ACPX counter-redaction failure, then passed 36 Rust
decoder/normalizer and durable-redaction tests after the fix.
Credential-shaped strings and objects remain redacted.
- Native Codex get-task-context passed a real provider probe with no
global CLI dependency.
- Local ACPX probes reached a platform startup rejection on macOS;
qualification therefore used the intended AWS Linux fleet. Complete
campaign 34060573948 reached 341/358 passing with zero infrastructure
failures (up from 250/358 and 100 infrastructure failures). ACPX Claude
reached 34/35, ACPX Codex 7/8, native Codex 34/35, and AgentCore 35/35.
Remaining failures were retained in the canonical report:
https://d1p6rlowie26tp.cloudfront.net/runner-protocol-evals/campaigns/gha-34060573948-1/.
- Final complete campaign 34062394019 exercises the current-request
scope clarification with all 358 cells; results pending.
- Passed git diff --check.
- Repo-wide typecheck, build, and tests are delegated to PR CI. They
were not repeated on this machine.
## Risks
- ACPX approval is scoped to the operator-requested isolated fixture
harness. Production defaults do not change.
- AgentCore admission can wait longer, but remains within the whole-turn
budget.
- The mock fixture has no production task lifecycle service. Explicit
task-state instructions describe that boundary; they do not relax
scoring.
- This change does not modify the browser full-stack workflow or its
test fixtures.
- ACPX receipt corrections apply to all native ACPX sessions. They rely
on the pinned qualified server contracts; unknown or ambiguous receipts
stay unknown, and a final accounting-read error never overwrites the
provider's terminal result.
## Model Used
- OpenAI GPT-6 (`gpt-6-astra`) through Codex, with reasoning, tool use,
and code execution. The session does not expose an exact context-window
limit.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip runs AI agents through qualified provider backends.
> - The direct live eval workflow builds one immutable Runner runtime
for every matrix cell.
> - The workflow reinstalled the packed Runner with npm.
> - That install discarded pnpm patches and selected provider
dependencies outside the qualified lock.
> - The first pnpm deployment model also placed its virtual-store marker
at the wrong level; a real deployment keeps `.pnpm` beside the scoped
Runner package.
> - AgentCore enforced the current context-aware harness but the direct
eval CLI did not supply the production v3 runtime context that harness
requires.
> - This pull request preserves the qualified dependency graph, resolves
the real deployment layout, and makes direct evals exercise the
production runtime-context contract.
> - The benefit is that live eval cells reach their provider turn with
the same artifacts and context contract that Paperclip qualified.
## Linked Issues or Issue Description
Refs: #12931
**What happened?**
The full direct live eval campaign failed every ACPX cell during
`session.open`. The portable runtime had an incorrect dependency root.
Its npm install also discarded the qualified ACP server patches.
AgentCore cells first failed because Runner enforced
`aws-agentcore-harness-v1` while the provisioned stack and eval profile
use `aws-agentcore-harness-context-v2`; after aligning that revision,
the direct eval CLI still omitted the required v3 runtime context.
**Expected behavior**
The direct eval runtime must preserve the frozen pnpm dependency graph
and patched provider bytes. Runner, server validation, OpenAPI, and the
deployed AgentCore stack must use one qualification revision. Direct
eval attempts must supply the same immutable native runtime-context
contract as production.
**Steps to reproduce**
1. Dispatch `Runner Direct Live Protocol Evals` from `master`.
2. Select an ACPX Claude, ACPX Codex, or AgentCore roster.
3. Observe a pre-turn provider bootstrap failure.
**Paperclip version or commit**
`d96452db059338b329b458ba8fe359fef72f1363`
**Deployment mode**
GitHub Actions on the RunsOn Linux x64 fleet.
## What Changed
- Build the reusable direct-eval runtime with `pnpm deploy --prod`.
- Resolve ACPX dependencies from the actual scoped-package layout of a
self-contained pnpm deployment.
- Align AgentCore configuration and qualification checks on
`aws-agentcore-harness-context-v2`.
- Materialize a minimal immutable v3 runtime context for each isolated
direct eval attempt.
- Add workflow, package-authority, runtime-context, Rust, and server
regression coverage.
- Document the qualified packaging, runtime-context, and AgentCore
revision contracts.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/runnerd-codex-transport.test.ts` (70 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/cli/eval-session-contract.test.ts` (14 tests)
- Focused Runner contract tests (36 tests)
- Focused server profile tests (47 tests)
- Focused Rust managed-provider and native-selector tests (19 tests)
- `node --test
packages/paperclip-runner/scripts/runner-protocol-eval-workflow-security.test.mjs`
- `actionlint .github/workflows/runner-protocol-live-evals.yml`
- A local `pnpm deploy --prod` produced both qualified ACP server
digests.
- A Linux reproduction of the first follow-up smoke identified the real
deployment root and the missing AgentCore runtime context.
## Risks
The AgentCore revision change rejects profiles that still use the
obsolete v1 value. This is intentional because the provisioned
context-aware harness and current eval profile use v2. Direct eval
prompts now receive the same fixed runtime-context preamble as
production, so behavior scores may move; that is the intended
qualification surface. The workflow package layout changes, but tests
assert the new entrypoint and dependency root. This change does not
modify the browser full-stack E2E workflow.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex with GPT-5.6. The context-window size is not exposed in
this session. The model used extended reasoning, repository tools, code
execution, Docker-based Linux reproduction, and GitHub Actions
diagnostics.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (for example, `docs/...` or
`fix/...`) and contains no internal Paperclip ticket id or
instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner includes an issue-thread viewer for direct
Evalbook reports.
> - The runner package uses Vite 6.4.3.
> - Dependabot changed the React plugin from version 4.7.0 to version
6.1.1.
> - React plugin 6.1.1 requires Vite 7 or Vite 8 package internals.
> - The live evaluation workflow found this mismatch when it built the
viewer on a clean Linux worker.
> - This pull request restores the compatible plugin and adds the viewer
build to pull request CI.
> - The benefit is that CI detects this class of report-viewer build
failure before a paid evaluation campaign starts.
## Linked Issues or Issue Description
**What happened?**
The direct live evaluation workflow failed before model execution. The
`build:issue-thread` command could not load `@vitejs/plugin-react@6.1.1`
with Vite 6.4.3. The plugin imported the unavailable `vite/internal`
package path.
**Expected behavior**
The Evalbook issue-thread viewer must build from a clean frozen-lockfile
installation before the live evaluation matrix starts.
**Steps to reproduce**
1. Check out commit `165ca56a22adb60e5fda56045442d9c8498116a8`.
2. Run `pnpm install --frozen-lockfile --ignore-scripts`.
3. Run `pnpm --filter @paperclipai/paperclip-runner build:issue-thread`.
4. Observe the `ERR_PACKAGE_PATH_NOT_EXPORTED` error for
`vite/internal`.
**Paperclip version or commit**
`165ca56a22adb60e5fda56045442d9c8498116a8`
**Deployment mode**
Built from source in GitHub Actions on Ubuntu.
## What Changed
- Restore `@vitejs/plugin-react` 4.7.0 in the Vite 6 runner package.
- Follow the repository policy: trusted PR CI regenerates and verifies
the lockfile artifact, and the master refresh workflow commits the
lock-only update after merge.
- Build the Runner Evalbook viewer in pull request CI.
## Verification
- `ci / policy`: regenerated the dependency lock artifact successfully
- `pnpm --filter @paperclipai/paperclip-runner build:issue-thread`
- `actionlint .github/workflows/pr-trusted.yml
.github/workflows/runner-protocol-live-evals.yml`
- `node --test
packages/paperclip-runner/scripts/runner-protocol-eval-workflow-security.test.mjs`
- `git diff --check`
## Risks
Low risk. This change restores the previous React plugin major version
for one package. The selected version declares support for Vite 6. Pull
request CI now builds the affected viewer directly.
> This bug fix does not add or change a roadmap feature.
## Model Used
- OpenAI Codex with GPT-5. Tool use and code execution were enabled. The
Codex app managed the context window.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - The merged direct live eval workflow must read the private
`paperclip-evals` repository at an exact commit.
> - Its first hosted dispatch failed before provider execution because
the GitHub App token was minted from the `paperclip` repository
installation.
> - GitHub returned 404 while resolving the private eval commit, proving
that token did not have the required repository scope.
> - Minting each short-lived token from the exact private eval
repository installation supplies only the cross-repository read boundary
the workflow needs.
> - A workflow regression now verifies every eval-token block keeps that
exact scope.
## Linked Issues or Issue Description
The first default-branch run of Runner Direct Live Protocol Evals failed
in its immutable eval-commit verification step with HTTP 404. No
provider jobs ran and no provider spend occurred.
**What existing behavior does this improve?**
It allows the protected direct live eval workflow to verify and check
out the private `paperclipai/paperclip-evals` repository.
**Current behavior**
All four eval-token blocks set `GH_REPO` to `paperclipai/paperclip`,
selecting a token installation that cannot read the private eval
repository.
**Proposed behavior**
Set `GH_REPO` to the exact `paperclipai/paperclip-evals` repository in
authorization, catalog, matrix, and report jobs.
**Reason and benefit**
The app mints a short-lived token from the correct repository
installation while the main repository continues to use its ordinary
read-only workflow token.
**Breaking changes**
None.
## What Changed
- Scoped all four private-eval installation tokens to
`paperclipai/paperclip-evals`.
- Added a regression requiring that exact scope in every token block.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `node --test .github/scripts/tests/get-bot-token.test.mjs` — 3 passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml` —
passed.
- `git diff --check` — passed.
## Risks
- The workflow reads a private repository. The token is still
short-lived, repository-specific, masked immediately, and used only by
the protected default-branch workflow.
- This changes no provider execution, Runner behavior, report content,
S3 publishing, or browser E2E behavior.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, repository inspection, code
editing, GitHub Actions diagnostics, and test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation where applicable
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Thinking Path
> - Daytona preserves a stopped sandbox filesystem, but deleting or
replacing a sandbox removes its only remote copy.
> - Warm reuse therefore improves latency but cannot be Paperclip's
durability boundary.
> - The host execution workspace must remain authoritative after every
successful turn, while same-run recovery must avoid overwriting
unexported remote work.
> - Result proposal, workspace export/merge, and terminal completion
need a durable, replayable ordering so a crash never starts a duplicate
provider turn.
> - A paid browser acceptance suite must exercise both legacy Codex and
Runner Codex for three real turns on one continuously warm Daytona
sandbox.
## Linked Issues or Issue Description
Refs #12901.
Runner Codex did not previously export successful Daytona workspace
changes back to the authoritative host workspace. That made warm reuse
depend on Daytona's remote filesystem and left deleted/replacement
sandboxes without a reliable reconstruction path. The existing paid
fixture also lacked a focused three-turn continuity case for both Codex
adapters.
## What Changed
- Persist versioned, atomic native workspace-sync descriptors and
durable seeds in `PAPERCLIP_HOME`, without credentials or a database
migration.
- Classify fresh, warm, replacement, and same-run-recovery workspace
preparation explicitly; ambiguous lease/root/digest evidence fails
closed.
- Finalize native workspace export/merge after semantic result proposal
and before run completion, with idempotent replay that never submits a
second provider turn.
- Surface legacy Codex workspace restoration failures instead of masking
them, while preserving an earlier provider error when both fail.
- Keep healthy reusable Daytona leases warm for legacy and native
adapters, stamp finalized workspace generations, and retain existing
cleanup behavior for per-turn or unhealthy leases.
- Preserve Runner Codex's provider process/session across warm turns,
including bounded post-terminal tail draining and exact authority
rotation.
- Add the exact paid `daytona-warm-continuity` matrix:
- `legacy-codex × daytona × warm-three-turn`
- `runner-codex × daytona × warm-three-turn`
- Drive all three turns through the browser, verify ordered file
continuity and stable lease/workspace/runtime identities, capture
per-turn timings, and delete the sandbox immediately after assertions.
- Document `pnpm test:e2e:runner -- --suite daytona-warm-continuity`; no
package script was added.
## Verification
- `pnpm typecheck` — passed, including migration safety (no migration
added)
- Focused server/runner Vitest coverage — 144 passed
- `pnpm test:e2e:runner:unit` — 114 passed
- `pnpm test:e2e:runner:typecheck` — passed
- `pnpm --filter @paperclipai/paperclip-runner test:codex` — 66 passed,
1 helper ignored
- `native-session-executor.test.ts` — 139 passed, including safe
fail-closed cleanup after remote runner identity capture failure
- Paid local browser acceptance, exact post-rebase Linux/amd64 runner
binary:
- Runner Codex — passed in 1.7m; 3 runs; lease outcomes `created,
resumed, resumed`; 10/10 matchers; cleanup passed
- Legacy Codex — passed in 2.7m; 3 runs; lease outcomes `created,
resumed, resumed`; 10/10 matchers; cleanup passed
- [Protected paid GitHub Actions
campaign](https://github.com/paperclipai/paperclip/actions/runs/34026735033)
against `7da42a91b95fa7fb2df126668ef7e37afb3b2b9d` — passed 2/2:
- Runner Codex — 3 runs; lease outcomes `created, resumed, resumed`;
evidence and cleanup passed
- Legacy Codex — 3 runs; lease outcomes `created, resumed, resumed`;
evidence and cleanup passed
- Merge/enforcement, S3 history, and Pages publication jobs passed
- Paid result artifacts were scanned for both provider credentials;
neither secret was present.
- Current PR checks — 31 passed, 1 expected Storybook skip; Greptile
5/5; Superagent security scan passed
- `git diff --check origin/master...HEAD` — passed
- Confirmed no `package.json`, lockfile, migration, or SQL changes.
## Risks
- Workspace synchronization now sits on the terminal-success path, so a
remote export failure deliberately prevents false success. Retryable
state retains its lease/seed; loss of the only unexported remote copy
fails closed.
- Warm provider reuse has strict identity and quiescence checks.
Mismatched or ambiguous evidence blocks reuse rather than risking
concurrent provider work.
- The paid suite incurs Daytona and Codex cost only in the existing
protected scheduled/manual workflow and explicitly destroys its sandbox
after each cell.
## Model Used
OpenAI Codex with GPT-5 agentic reasoning, repository inspection, real
browser E2E execution, Rust/TypeScript test execution, and GitHub
Actions diagnostics.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked an existing issue or described the issue
in-PR
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name contains no internal ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have documented the dedicated suite invocation without adding a
package script
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green on the current revision
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
on the current revision
- [x] I will address all reviewer comments before requesting merge
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Runner executes agents through native and managed provider
drivers.
> - The direct live eval layer had drifted from the current Runner
contracts.
> - The old local workflow did not provide a complete parallel campaign
or durable report history.
> - The Runner also needed current native OpenCode and OpenRouter
qualification.
> - This pull request restores the direct campaign, corrects the runtime
gaps that the campaign found, and adds safe hosted Evalbook history.
> - The benefit is repeatable model comparison against an immutable
Runner and eval source revision.
## Linked Issues or Issue Description
Refs #11297
Refs #11634
**What existing behavior does this improve?**
This improves the direct live `paperclip-runner` eval workflow, provider
execution contract, and static Evalbook reporting path.
**Current behavior**
The direct evals do not have one maintained full campaign on current
`master`. OpenCode has no qualified multi-model OpenRouter roster.
Parallel provider bursts can compact committed events before the
transport observes them. Local reports do not have a separate safe S3
history index.
**Proposed behavior**
Run one immutable roster-plus-case matrix. Use the shared paid AWS
runner fleet. Keep raw artifacts access-controlled. Publish a sanitized
canonical Evalbook report under the separate `runner-protocol-evals` S3
prefix. Keep immutable campaign directories plus root history, latest,
and latest-green pointers.
**Reason and benefit**
Maintainers can compare native Codex, native OpenCode, ACPX, Claude
Managed, and AWS AgentCore behavior over time. They can inspect failures
without mixing this direct protocol layer with browser full-stack E2E.
**Breaking changes**
None. The new workflow and S3 prefix are additive. The existing Runner
full-stack E2E workflow and report remain separate.
## What Changed
- Added a trusted two-shard direct live workflow for up to 393
roster-plus-case cells.
- Reused the numeric actor allowlist, protected paid environment, and
RunsOn fleet controls from Runner full-stack E2E.
- Added immutable Runner and eval revision resolution, exact credential
boundaries, bounded retries, and cost ceilings.
- Added a public report projection that removes sessions, transcripts,
tool payloads, state, traces, raw failures, remote profile identities,
and credential-shaped values.
- Added additive S3 history under `runner-protocol-evals`, with
immutable campaigns and mutable root index pointers.
- Added native OpenCode model injection and current OpenRouter pricing
contracts.
- Fixed direct eval completion, workflow execution, semantic discovery,
warm-attach state reset, executable binding, and event-burst handling.
- Kept Runner browser full-stack E2E behavior and publication separate.
- Documented local and hosted direct eval operation.
## Verification
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-protocol-eval-publish` — 15 passed.
- `pnpm --filter @paperclipai/paperclip-runner build:typescript` —
passed.
- `actionlint .github/workflows/runner-protocol-live-evals.yml
.github/workflows/runner-full-stack-e2e.yml` — passed.
- Local current matrix at the revision in
[paperclip-evals#17](https://github.com/paperclipai/paperclip-evals/pull/17)
— 323 cells across 10 enabled configurations completed.
- Final local current matrix — 269 passed, 11 behavior failures, and 43
expected macOS-only ACPX platform failures.
- Targeted Runner checks — 13/13 eval-session tests, 15/15
publisher/security tests, and package typecheck passed; complete PR CI
is green, including all browser E2E shards.
## Risks
- Paid live campaigns can consume provider budget. Actor authorization,
exact per-cell ceilings, protected environments, and explicit schedule
enablement bound this risk.
- Public reports can leak provider data. The workflow publishes only a
separately projected report and validates every file before upload.
- The new workflow cannot publish until it is present on the default
branch. This pull request does not change the existing
`runner-full-stack-e2e` publication path.
- The campaign is large. It uses two GitHub matrices and caps combined
concurrency at the shared fleet limit.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
OpenAI Codex on GPT-5. The exact deployment ID and context-window size
are not exposed. The model used reasoning, code editing, browser
inspection, repository tools, and live provider execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Dotta
Dotta
