Commit Graph

66 Commits

Author SHA1 Message Date
Nicky Leach d1ba17eeca
fix(adapter-utils): fail fast when the sandbox control channel is lost mid-turn (#13158)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Adapter utilities run agent turns and report their results to the
control plane
> - A lost sandbox control channel can leave an agent turn without a
result
> - The host then waits for the full adapter timeout instead of
reporting the loss
> - This pull request adds a push loss signal and a bounded host wait
> - The benefit is a prompt failure terminal when the agent stops
answering

## Linked Issues or Issue Description

**What happened?**

A sandbox control channel loss during an Agent Client Protocol turn left
the host waiting for the four-hour adapter execution timeout.

**Expected behavior**

The host should detect the terminal channel loss, stop the turn, and
report a safe failure without waiting for the agent.

**Steps to reproduce**

1. Start an Agent Client Protocol turn through a sandbox adapter.
2. Close the duplex control channel while the turn remains active.
3. Observe the host response before the adapter timeout expires.

**Paperclip version or commit**

Test the pull request commit set at
`10b6bbc5525a79fd575298607dd5a25ae448fc8a`.

**Deployment mode**

The change applies to sandbox-backed adapter execution.

## What Changed

- Add `onLoss(listener)` to the duplex bridge handle.
- Register the loss listener at turn start and read losses latched
before turn start.
- Cancel the turn on loss and arm a 30-second host deadline.
- Close the stream locally when the deadline wins and create a host
terminal.
- Derive the public error from the closed `DuplexLossReason` enum.
- Add tests for loss order, cancellation, timeout, and safe error
output.

## Verification

- Run `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`.
- Run `pnpm --filter @paperclipai/adapter-utils exec vitest run
src/acpx-engine/execute.test.ts -t "run-disposition seam"`.
- Confirm that the full pull request workflow passes.

## Risks

The new deadline changes a lost-channel path from a long wait to a
host-built failure after 30 seconds. Orderly completion keeps its
existing behavior. The deadline race against a pending `turn.result` has
no direct test.

## Model Used

OpenAI Codex, GPT-5, with tool use and code execution. The runtime does
not expose a more specific deployment version or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-10 15:37:05 -07:00
Dotta 3bc60dd8bf
fix(adapters): probe Git context in the remote workspace (#13116)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote environments realize the workspace at a provider-owned path.
> - Run startup probes Git and network context before it starts the
agent.
> - The probe used the controller path inside the remote environment.
> - A missing directory stopped the run before any provider work began.
> - This pull request uses the remote execution target's working
directory.

## Linked Issues or Issue Description

Refs #13094, which introduced this probe. Related: #8997 and #10419
address other workspace-directory handoffs; this patch fixes the newer
Git-context probe.

**What happened?**

Remote runs failed with `setup_failed: Could not read execution-target
Git context`, including tasks that did not use Git. The provider shell
could not enter the controller's workspace directory.

**Expected behavior**

Startup must inspect Git and network context in the realized remote
workspace.

**Steps to reproduce**

1. Select a remote sandbox whose workspace is
`/home/daytona/paperclip-workspace`.
2. Start a task whose controller workspace is under
`/paperclip/instances/default/workspaces/`.
3. The startup probe exits when the controller directory does not exist
in the sandbox.

**Paperclip version or commit**

Reproduced on `622376e99` and in the regression test before this patch.

**Deployment mode**

Docker controller with a remote Daytona sandbox. The same helper also
serves SSH targets.

## What Changed

- Use `remote.remoteCwd` for the remote Git-context probe.
- Cover a missing controller directory in both credential modes.
- Verify that SSH reads Git metadata from the remote workspace even when
the caller directory exists.

## Verification

- Before the fix, both new missing-directory tests failed with the
reported error.
- The launcher environment suite passes: 18 tests.
- The adapter-utils suite passes: 1,085 passed, 11 skipped.
- Adapter-utils typecheck passes.
- A disposable sandbox with the affected deployment's image reproduced
the old `cd` failure and passed with the corrected helper. The sandbox
was deleted afterward.
- Full repository typecheck and build pass locally. All CI gates pass at
`61b522c`: regular test shards, serialized server suites, browser
shards, Runner verification, build, policy, and security checks.
- The first CI Build attempt hit a Runner suspension-acknowledgment
timeout outside the changed code. The affected 13-case recovery group
passes locally with its Rust fixtures built, and the unchanged CI job
passed on its single retry.
- The full serial local test run was stopped in favor of the complete CI
matrix; it is not claimed as a local pass.

## Risks

The probe now uses the provider-resolved target directory for sandbox
and SSH execution. Local execution keeps its existing directory. There
are no schema or API changes. Existing local credential and Git metadata
tests pass.

## Model Used

OpenAI GPT-6 in Codex, with tool-assisted code analysis, implementation,
and live and automated testing. The exact model identifier and
context-window size are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-09 16:52:11 -05:00
Dotta 82f662656a
fix(runner): restore legacy Git access and independent networking (#13094)
## Thinking Path

> - Paperclip runs agents for people with different GitHub accounts.
> - Managed operations must use the intended person's eligible
connection.
> - A failed duplicate connection must not hide a healthy grant for the
same account.
> - Legacy hosts also need their existing Git configuration when managed
access is not configured.
> - Runner networking and local Git operations must not depend on GitHub
broker availability.
> - This pull request separates those policies and improves failure
diagnostics.

## Linked Issues or Issue Description

**What happened?** New runs always cleared host Git credentials and
installed managed launchers. Network permission depended on GitHub
environment variables. A launcher failure could stop even local `git
status`. A newer unhealthy duplicate could take precedence over a
healthy connection, and generic health errors were shown as reconnect
requirements.

**Expected behavior:** Use a healthy eligible managed connection for the
intended account. Preserve host authentication only for unconfigured
standard-trust local or SSH execution. Permit local Git during broker
failures and keep network permission independent of GitHub credentials.

**Steps to reproduce:** Configure healthy and unhealthy grants for one
GitHub account, dispatch an agent, and execute Git commands. Separately
run an unconfigured legacy host with existing GitHub CLI authentication.
Stop the broker and run local `git status`.

**Paperclip version or commit:** Master at 668110469. **Deployment
mode:** Self-hosted. **Installation method:** Git checkout. **Agent
adapters involved:** Native Codex runner and shared adapter execution
setup. **Database mode:** Existing instance database; no migration.
**Access context:** Responsible person's managed grant, or explicitly
unconfigured legacy host. **Node.js version:** 26.4.0 locally.
**Operating system:** macOS development and Linux execution hosts.

**Relevant logs or output:** Previously `GitHub credential context
unavailable` hid configuration, transport, and capability errors. New
diagnostics identify these categories without credential values.

**Additional context:** Refs #13005 and #13022. Dependency provisioning
is addressed separately in #13093.

## What Changed

- Prefer healthy eligible grants and retry credential acquisition once
for the same principal and account before starting an operation.
- Preserve host Git configuration only when managed access is
unconfigured on a standard-trust local or SSH target.
- Project authentication mode and validated Git metadata into native
runner boundaries; refresh resumed provider settings when modes change.
- Enable network access through an explicit standard-trust controller
decision, independently of GitHub. Omitted or restricted decisions stay
disabled; replace warm providers when that decision changes.
- Run local Git with cleared credentials when the managed broker fails,
with specific redacted diagnostics.
- Retry access-refresh conflicts once without treating concurrency as
expired authorization.
- Show retry instead of reconnect for transient GitHub health failures.
Add optional authorization and run-diagnostic fields without a database
migration.

## Verification

- All latest-head CI gates are green, including typecheck, general and
serialized suites, browser tests, canary, native runner verification,
and build. Greptile is 5/5 with no remaining findings; the security scan
passed.
- Full recursive typecheck and build passed. UI token gates passed.
- Full general server run: 7,110 passed, one transient socket hangup;
that file passed on retry. All remaining workspace groups passed,
including 5,552 UI and 478 CLI tests. The complete serialized rerun
passed all 144 suites / 2,179 tests after the initial isolated timeout
passed on retry.
- 195 final launcher and native session tests passed, including
host/managed transitions, local/remote warm network-policy changes,
broker rotation, and attempts to override validated controller
filesystem roots.
- GitHub gateway fallback, duplicate connection selection, refresh
conflicts, per-user reauthorization, and native transport/security
suites passed.
- Additional live native fixtures passed SSH public-key authentication
and a Git credential helper in fresh and resumed host-mode sessions. An
unwritable managed configuration directory preserved local Git (exit 0)
while GitHub CLI failed with `configuration_directory_unavailable` (exit
4). The temporary SSH listener and keys were removed.
- Applicable Rust suites passed except two timing failures under load;
each failed case passed in isolation. The final environment contract
test passed.
- Linux native runner acceptance passed in both managed and legacy host
modes: DNS, HTTPS, npm package download, fresh-worktree Git status,
authenticated GitHub user lookup, repository read, and a new run
continuing the same provider conversation. Managed broker outage
preserved local Git and rejected authenticated access without host
fallback.
- Matching Linux server/runner artifacts and the separate provisioning
repair are deployed to the development instance. A fresh UI-dispatched
task and a new run after a server restart both passed all six shell
checks through the live controller and credential broker. Both runs
selected the expected healthy connection/grant and retained the same
provider conversation. Neither connection was repaired or reconnected.

The sandbox roots are assigned from the validated execution-target probe
**after** ordinary bindings are merged. Regression coverage supplies
forged roots and verifies they cannot override the controller paths.
Networking is enabled only for an explicit
`PAPERCLIP_RUNNER_NETWORK_ACCESS=enabled` controller decision; omitted
values remain disabled.

## Risks

Unconfigured standard-trust local and SSH runs regain access to host Git
authentication resources. Managed, sandbox, plugin, and low-trust runs
do not gain this fallback. Revoked managed access never falls back to
another account. Deploy server and runner artifacts together;
already-started operations retain their captured identity. An
unauthenticated command can still fail when it requires GitHub access.

## Model Used

OpenAI GPT-6 through Codex, with code editing, shell execution, tests,
and browser inspection. The exact model variant and context-window size
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-09 10:15:10 -05:00
Nicky Leach 6019e2bd6e
feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip runs agents in local and remote sandboxes through adapter
utilities
> - The HTTP/2 sandbox bridge decoded every body as UTF-8 text and
rejected non-JSON content
> - This stopped agents from uploading or downloading issue attachments
through that bridge
> - This pull request carries raw bytes, permits the two attachment
routes, and enforces a shared body limit
> - The benefit is correct attachment transfer with a process-wide
memory guard

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The HTTP/2 sandbox bridge forwards request bodies between an agent
sandbox and the Paperclip host. It now supports binary bodies and the
issue attachment routes.

**Current behavior**

The bridge decodes each body as UTF-8 text. It returns HTTP 415 for
content types outside the JSON route list. An agent cannot upload or
download an issue attachment through this transport.

**Proposed behavior**

The bridge carries raw bytes through the forward path. It permits the
attachment upload and content routes. The queue transport and file
gateway keep their existing route behavior. A shared 10 MiB body limit
and process-wide byte reservation protect memory use.

**Reason and benefit**

Attachment clients need byte-preserving transfer. The shared limit keeps
the gateway and host aligned. The reservation prevents concurrent
streams from exceeding the accepted process memory ceiling.

**Breaking changes**

The HTTP/2 bridge accepts two attachment routes and permits binary
content. The queue transport and file gateway keep their previous route
lists and HTTP 415 behavior. No schema or external endpoint changes.

## What Changed

- Carry request and response bodies as raw bytes through the HTTP/2
bridge.
- Permit attachment upload and attachment content routes on the HTTP/2
bridge only.
- Raise the resolved per-body limit to 10 MiB and share it between the
gateway and host.
- Reserve body bytes before allocation and release each stream
reservation on every terminal path.
- Document the body limit, process ceiling, and reservation behavior.

## Verification

- Run `pnpm exec vitest run
packages/adapter-utils/src/http2-bridge-server.test.ts
packages/adapter-utils/src/execution-target-sandbox.test.ts
packages/adapter-utils/src/sandbox-callback-bridge.test.ts`; 226 tests
pass.
- Run `pnpm --filter @paperclipai/adapter-utils typecheck`; it passes.
- Run the direct server TypeScript check with `tsc --noEmit` in
`server/`; it passes with zero errors.
- Verify multipart upload and binary download round trips over HTTP/2
without corruption.
- Verify the queue transport and file gateway return HTTP 415 for the
same routes.
- Verify the host rejects bodies over the resolved limit.
- Verify a denied reservation returns HTTP 503 and allocates no copy.
- Verify stream cleanup releases reservations after completion, error,
abort, timeout, and close.

## Risks

The bridge now accepts larger bodies and binary content. The
process-wide reservation limits total live body bytes to 1 GiB. Route
behavior changes only for the HTTP/2 bridge. The security review found
no blocking issue for this commit range.

## Model Used

OpenAI Codex, GPT-5. The runtime used tool calls and code execution. The
runtime did not expose the context window size. No model-generated code
changes were made for this pull request.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-08 14:18:12 -07:00
Dotta 3ad494aacd
fix(adapter-utils): preserve legacy sandbox PATH with managed GitHub (#13051)
## Thinking Path

> - Paperclip lets people manage AI agents and their work.
> - Remote agents need both their installed tools and managed GitHub
credentials.
> - The GitHub launcher replaces a missing remote PATH with a small
system path.
> - Legacy images install agent CLIs outside that path, so those agents
cannot start.
> - This pull request preserves the remote toolchain and puts managed
GitHub commands first.
> - Command checks now use the same environment as execution.

## Linked Issues or Issue Description

Refs #13005. Related: #10239 fixes a separate Cursor environment path
issue.
Searched open issues and PRs for sandbox PATH and GitHub launcher
changes. No duplicate of this launcher fix was found.

**What happened?**

A remote Claude or Codex run passes command discovery, then fails with
`command not found` and exit code 127. Managed GitHub launchers use only
their own directory and `/usr/local/bin:/usr/bin:/bin`. This drops NVM
and other toolchain directories from the sandbox path.

**Expected behavior**

Agent CLIs remain available on legacy and current sandbox images.
Managed `git` and `gh` still resolve first and use the responsible
person's credentials.

**Steps to reproduce**

1. Use a sandbox whose agent CLI is installed in an NVM or other
non-system bin directory.
2. Start an agent run with managed GitHub launchers and no explicit PATH
override.
3. Observe that command discovery succeeds but the agent command exits
with code 127.

**Paperclip version or commit**

Observed on `b97101893f0926f57ed0ce9ef1f8d3e4780c62c2`. The same
launcher behavior remains on the base commit `be6bb768b`.

**Deployment mode**

Hosted server with remote sandbox execution. The shared launcher also
supports SSH targets.

## What Changed

- Read the remote target's effective PATH when no remote override is
set. Do not copy an inherited controller PATH.
- Prepend the managed launcher directory and retain the combined path in
shell startup files.
- Stop startup if path discovery fails. Frame the response so login
banners cannot contaminate PATH.
- Pass the sanitized launch environment to sandbox command checks,
installation, and the second check.
- Add real shell tests for legacy and current CLI layouts, quoted paths,
managed GitHub command execution, explicit overrides, SSH, and failure
cases.
- Document the remote path contract.

## Verification

- Four focused adapter utility suites passed: 153 tests.
- The regression suite passed: 13 tests, including the Linux stdin
handling fix.
- `pnpm --filter @paperclipai/adapter-utils typecheck` passed.
- Full workspace `pnpm -r typecheck` and `pnpm build` passed.
- The regression suite fails on the unchanged base revision (12
failures, 1 pass) and passes with this change (13 passes). The baseline
ran in an isolated scratch copy.
- Full local test coverage was attempted using the official CI shards.
The run was stopped after macOS Postgres shared-memory exhaustion and
CLI timeouts under load. The affected server database suite passed in
isolation (31 tests), as did the five affected DB/CLI suites (89 tests).
- [Full Linux
CI](https://github.com/paperclipai/paperclip/actions/runs/34258333833)
passed on `4e1426f5b`: all general and serialized test shards, all
browser shards, typecheck and release registry checks, native runner
verification, application build, and release canary dry run.
- Greptile scored the latest commit 5/5 with no unresolved findings.
- Shell tests use isolated local fixtures and make no provider or model
requests. No live sandbox qualification is claimed.

## Risks

- Remote startup adds one bounded path query when no explicit override
exists. A failed query stops startup.
- Explicit remote path overrides still control which tools are
available. Invalid overrides now fail the command check earlier.
- Credential selection and GitHub broker policy are unchanged. Tests
verify managed wrappers stay first and can invoke underlying commands.
- No database migration or sandbox image replacement is required.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository editing, and
terminal tools. The exact serving model ID and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-08 12:56:42 -05:00
Dotta 1cc45086d3
feat: use the responsible person's GitHub for shared agent operations (#13005)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Several people can send instructions to the same agent and task.
> - A fixed GitHub token in the provider process can keep the first
person's access after another person's message is accepted.
> - Task ownership cannot select credentials for each accepted
instruction or preserve the identity of an operation already in
progress.
> - This pull request records ordered execution identity contexts and
resolves credentials when managed Git, gh, or GitHub tools start.
> - The benefit is automatic personal GitHub access for shared agents,
with durable continuation rules and no teammate credential fallback.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting: orchestration, connection grants, database, runtime
adapters, native runners, and run details.

**Problem or motivation**

A shared agent must use the person whose instructions it has accepted. A
queued message must retain its author. A retry or approval without new
instructions must retain the originating identity. GitHub must remain
optional for ordinary work.

**Proposed solution**

Persist execution identity separately from task ownership. Give new
processes a run-scoped broker capability and token-free managed
launchers. Capture identity at operation start. Keep an explicit
dedicated-agent grant as an override. Show redacted diagnostics in run
details.

**Alternatives considered**

Per-task ownership, fixed provider tokens, and mutable repository author
configuration do not handle accepted steering or concurrent operations.
A manual account-selection action would add unnecessary setup to each
turn.

**Roadmap alignment**

This completes the existing Multiple Human Users, MCP Tool Gateway &
Apps, Secrets Manager, and Self-healing Runs capabilities. The
implementation follows the maintainer-approved plan.

Related work: Refs #12843, Refs #12907. Existing proposals #4618 and
#8945 cover per-agent or per-worktree author configuration. This change
instead follows the accepted human instruction across runtime types.
Refs #11831 for governed personal connection delegation; this change
preserves connection audience checks and does not use standing
delegation as a personal credential fallback.

## What Changed

- Add durable, ordered identity contexts and active run references.
Preserve message authors through consolidation, steering, retries,
delegation, approvals, routines, and restart.
- Add an authenticated operation-time GitHub credential broker and
local/remote managed git and gh launchers. Keep personal tokens out of
the long-lived provider process.
- Resolve GitHub gateway and server-side Git operations through the same
responsible-person or dedicated-grant selection rules.
- Make absent and unavailable GitHub credentials non-blocking at generic
startup. Clear host and prior-person credentials. Keep anonymous Git
access where supported.
- Add run-detail identity history and the dedicated-account warning.
Keep task ownership and queue-versus-steer decisions unchanged.
- Preserve personal OAuth declarations through connection edits. Retain
exact selected grants in the gateway.
- Fix continuation races found during real acceptance: verify a warm
owner before credential rotation, and wait for bounded durable runner
suspension before the next run starts.
- Make migrations replay-safe. Retain identity through agent/run
deletion, remove it with its company, and clean terminal launcher
directories before releasing execution environments. Document
coordinated release and rollback.

## Verification

- Full workspace typecheck, build, and token gates passed. The complete
local suite passed in its normal test groups: 17,120 passing tests,
including all 143 serialized server suites. After integrating the newly
merged runner API work, full local typecheck and build passed again,
along with 890 focused integration tests. All 31 checks on the
integrated revision passed, including build, browser E2E, release
registry, canary dry run, typecheck, security and all test suites.
Greptile is 5/5 with all review threads resolved.
- Current focused checks passed: 142 native executor tests, 67 runtime
lifecycle tests, 9 durable identity tests, 75 credential/routine tests,
19 low-trust/resumption tests, and the executable migration replay test.
- Authenticated browser acceptance with two Paperclip users and two
GitHub accounts on one shared native agent passed. Real commits and
pushes followed A → B accepted steering → queued A continuation in the
same saved conversation. GitHub commit author and committer identities
matched all three operations. Both runs succeeded and task ownership
stayed unchanged.
- Real GitHub MCP calls switched from A to B after accepted steering. A
delegated subtask retained its originating identity across a server
restart.
- Disabling B's GitHub connection left ordinary work successful. Managed
gh was unauthenticated and the provider had no inherited GH_TOKEN or
GITHUB_TOKEN.
- The browser displayed run-detail diagnostics and the exact
dedicated-account warning. A final controller-restart check followed by
another-person continuation retained the conversation, selected the
correct GitHub login and Git author, and removed each terminal launcher
directory.
- Company-lifetime migration and all five previously failing CI suites
passed locally (167 tests). Same-token gateway A → B → A and six
broker/launcher boundary tests passed.
- Remote callback, launcher, sandbox, and runtime contract tests passed.
Both native and legacy Codex completed actual Daytona executions on the
integrated revision ([campaign
results](https://github.com/paperclipai/paperclip/actions/runs/34155056509)).
The remote package-manager shim staging regression also passed locally.

## Risks

- Deploy the migrations, server broker, launchers, and runner artifacts
together. Existing processes finish with their original contract. New
managed processes need the broker endpoint for GitHub operations.
- Finish or stop new managed executions before rolling application code
back. Keep the additive schema and identity history during rollback.
- Scripts that require a persistent raw GH_TOKEN must use managed git,
gh, or GitHub gateway tools. Run capabilities authorize code executing
within that run to acquire its current identity; this is not
hostile-code isolation within one execution principal. Managed commands
prevent automatic credential carryover; arbitrary code deliberately
copying a credential is outside that boundary.
- Uncertain steering acknowledgement deliberately holds new credential
acquisition until reconciliation. Already-started operations retain
their captured identity.
- GitHub private access and provider outages can still fail the specific
operation that needs them. Dedicated grant failure does not fall back to
personal access.

## Model Used

OpenAI GPT-6 through Codex assisted implementation, review, shell
execution, and browser acceptance. The exact model variant and
context-window size are not exposed in this session. Tool use included
TypeScript and Rust tests, database integration tests, GitHub CLI, and
authenticated browser control.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-07 14:32:20 -05:00
Dotta 1dceee9a4e
fix(runner): persist warm Daytona workspaces (#12901)
## Thinking Path

> - Paperclip manages AI agent work and the execution state for each
task.
> - Remote agents run in sandbox environments such as Daytona.
> - Daytona keeps files while a sandbox is stopped, but deletion removes
those files.
> - Runner Codex did not copy successful remote workspace changes back
to the host workspace.
> - A warm sandbox could therefore hide data loss until Daytona replaced
or deleted the sandbox.
> - This pull request makes the host workspace durable after every
successful turn and keeps verified reusable sandboxes warm.
> - The benefit is reliable multi-turn work across warm reuse, restart,
stop, and sandbox replacement.

## Linked Issues or Issue Description

**What happened?**

A successful native Codex turn in Daytona could leave workspace changes
only in the remote sandbox. A later warm turn appeared to work because
it reused that filesystem. A replacement sandbox could start from stale
host data and lose the successful changes.

**Expected behavior**

Paperclip must merge each successful remote turn into the authoritative
host workspace before it completes the run. A verified warm lease may
reuse its remote files. A replacement lease must reconstruct the exact
durable workspace seed.

**Steps to reproduce**

1. Run Codex in a reusable Daytona environment.
2. Write a file during one successful turn.
3. Replace the Daytona sandbox before the next turn.
4. Observe that the next turn can start without the prior file on the
unpatched code.

Related remote workspace foundation: #10070.

## What Changed

- Added explicit `host_current`, `durable_seed`, and `adopt_remote`
workspace preparation modes.
- Added atomic, versioned native workspace descriptors and seed archives
under `PAPERCLIP_HOME`.
- Added real native sandbox export and three-way host merge before
terminal result completion.
- Added workspace-only recovery after a proposed result. Recovery does
not submit another provider turn or consume the provider retry budget.
- Added fail-closed handling when a sandbox with unexported changes is
gone.
- Kept healthy reusable Daytona sandboxes started for legacy Codex and
Runner Codex.
- Kept the Runner Codex process and provider session across verified
warm turns.
- Added the paid `daytona-warm-continuity` browser suite. It contains
exactly the legacy Codex and Runner Codex cells. Each cell performs
three measured turns.
- Documented `pnpm test:e2e:runner -- --suite daytona-warm-continuity`.
No package script was added.
- Added no database migration. The metadata format is backward
compatible and idempotent.

## Verification

- `pnpm typecheck`
- `pnpm test:e2e:runner:unit` — 114 passed
- Native workspace, finalizer, session, and environment tests — 232
passed
- Daytona provider tests — 150 passed
- Workspace staging and merge tests — 98 passed
- Runner transport tests — 63 passed
- Legacy Codex restore tests — 5 passed
- Rust format and compile checks pass through root typecheck
- The paid Daytona suite was not run locally because the required
Daytona, OpenAI, and immutable image credentials are not present.

## Risks

- The main risk is an incorrect workspace identity or merge after a
crash. Durable descriptors bind the run, workspace, lease, provider
lease, local root, remote root, and baseline digest. Ambiguous evidence
fails closed.
- The host merge may conflict with concurrent host edits. The existing
three-way merge and exclusion rules handle this case and surface
failures.
- A deleted sandbox cannot recover unexported bytes. Paperclip now
blocks with `workspace_sync_out_unrecoverable` instead of reporting
success or rerunning the provider.
- There is no database migration. Descriptor writes and recovery are
atomic and idempotent.

## Model Used

OpenAI Codex with GPT-5. The run used agentic reasoning, repository
inspection, code execution, test execution, Git, and GitHub CLI tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-05 13:00:57 -05:00
Dotta 0a422fda52
feat(runner): add remote execution substrate (#12638)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner gives native runs a durable and governed execution
path.
> - The current native path runs on the control-plane host.
> - Remote environments need an authenticated execution-target contract.
> - The contract must not change direct adapters or enable new runtimes
by default.
> - This pull request adds the remote execution substrate and Daytona
ingress.
> - The benefit is a bounded base for later remote runner transport
work.

## Linked Issues or Issue Description

Refs #12616.
Refs #12352.

**Subsystem affected**

Cross-cutting. This change touches runner transport, server
orchestration, plugin contracts, and shared settings.

**Problem or motivation**

Native execution cannot resolve an authenticated runner ingress through
a remote environment. The server also lacks one provider-neutral
contract for remote execution targets.

**Proposed solution**

Add a default-off runner preview ingress capability. Add
transport-neutral runner connectivity. Add remote execution target and
lifecycle handling. Add a Daytona ingress implementation with redacted
credentials.

**Alternatives considered**

A provider-specific server path would duplicate orchestration and
authorization. A public endpoint without an environment contract would
weaken the trust boundary.

**Roadmap alignment**

This work supports the Cloud and Sandbox agents milestone. It also
supports self-healing runs and governed tool access.

## What Changed

- Added execution-target traits for local, SSH, and sandbox
environments.
- Added plugin RPC contracts for runner ingress endpoints.
- Added authenticated Daytona preview ingress.
- Added transport-neutral PRP outbound connections.
- Added remote runner artifact verification and fail-closed provider
selection.
- Added bounded native session resume, cancellation, and lifecycle
recovery.
- Preserved Codex-only selection for fresh experimental runner starts.
- Preserved all direct adapter execution and finalization paths.
- Removed stale Pi provider-pack requirements that security review
rejected.
- Kept the rollout controls off by default.
- Did not change pnpm-lock.yaml, Cargo, database migrations, or GitHub
workflows.

## Verification

- GitHub Actions will run the repository test, typecheck, build,
security, and policy gates.
- Focused tests cover ingress validation, redaction, execution targets,
remote lifecycle, cancellation, resume, and legacy adapter selection.
- Local tests were not run. The requested verification policy uses
GitHub Actions for this series.
- `git diff --check origin/master...HEAD` passes.
- The diff contains 52 files.

## Risks

- Remote execution crosses a trust boundary.
- The implementation validates target capabilities, artifact digests,
provider-pack pins, and connection metadata.
- The feature remains default-off.
- Fresh native selection remains Codex-only.
- Existing direct adapters remain on the legacy path.
- This PR does not yet make remote Codex runnable. The next PR adds the
Rust WSS and TLS transport.

## Model Used

OpenAI Codex with GPT-5.6. The work used high-reasoning agent mode,
repository tools, GitHub tools, and parallel code-audit agents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: / Closes /
Refs OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 01:29:06 -05:00
Dotta 0834a0c1f7
feat(runner): bind ACPX profile boundary (#12387)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip runner needs a safe boundary before it can launch
ACP-compatible agents.
> - A caller-controlled command, model, environment, or frame could
bypass that boundary.
> - The ACPX transport contract in #12386 defines the allowed messages
but does not bind a runtime profile.
> - This pull request defines closed, versioned profiles and validates
the launch inputs around that contract.
> - The benefit is a small and reviewable trust boundary before any ACPX
process can become available.

## Linked Issues or Issue Description

**Agent or provider**

ACPX sidecar support for the qualified Pi, Claude, and Codex ACP
servers.

**Why this adapter is useful**

The runner needs one bounded process boundary for ACP-compatible
providers. A closed profile prevents an untrusted run from selecting an
arbitrary executable, package version, or model.

**How the agent is invoked**

A later pull request will launch an internal sidecar from an exact
profile. This pull request only validates profiles, environment values,
and protocol frames. It does not add an executable dependency or enable
an adapter.

**Additional context**

This pull request is stacked on #12386. It keeps the existing direct
adapters and the Codex runner path unchanged.

## What Changed

- Add a closed profile table for the qualified Pi, Claude, and Codex ACP
servers.
- Require the exact qualified model and return an isolated profile value
to callers.
- Add an agent-specific environment allowlist with entry and aggregate
size limits.
- Add strict parsing for bounded sidecar requests and structured plan
values.
- Reject unknown fields, unsupported protocol versions, invalid
identifiers, null bytes, cyclic values, and oversized input.

## Verification

- Runner TypeScript typecheck — passed.
- Runner TypeScript tests — 40 files and 362 Vitest tests passed; 11
Node contract tests passed.
- `pnpm -r typecheck` — passed for all applicable workspaces.
- `pnpm build` — passed, including runner binary, server, UI, and
workspace packages.
- Prettier and `git diff --check` — passed.
- The diff contains 6 files and does not change `pnpm-lock.yaml`, a
workflow, a package dependency, or a public export.

## Risks

The main risk is accepting more launch state than the sidecar needs. The
implementation uses an agent-specific allowlist, rejects null bytes, and
enforces per-entry and aggregate bounds. This pull request does not
launch a process or expose a new adapter, so production and
direct-adapter behavior remain unchanged.

## Model Used

OpenAI Codex with GPT-5 and repository tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked an existing public item or described the
issue in this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal task
identifier
- [x] I have run the affected tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have documented the compatibility and security boundary
- [ ] All applicable GitHub Actions are green
- [ ] Greptile is 5/5 with every actionable comment resolved
- [x] I will address all review findings before requesting merge
2026-08-30 12:37:13 -05:00
Nicky Leach 64b7dce0ad
refactor(adapter-utils): replace the process-wide byte ledger with route-local byte bounds (#12465)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The adapter layer carries sandbox requests to host processes.
> - The HTTP/2 bridge used one process-wide byte ledger for all routes.
> - One busy route could exhaust that shared budget and move another
route to file transport.
> - This pull request gives each host retention site a fixed byte bound
and limits concurrent HTTP/2 streams.
> - The benefit is local protection: one route cannot consume the byte
budget of another route.

## Linked Issues or Issue Description

**What happened?**

The HTTP/2 bridge used one aggregate byte ledger for retained bytes
across all routes. A busy route could exhaust the shared budget and
force an unrelated route to use file transport.

**Expected behavior**

Each route should protect its own retained bytes. A reset on one HTTP/2
stream should cancel only that stream's host forward.

**Steps to reproduce**

1. Start the HTTP/2 bridge with multiple sandbox routes.
2. Send enough retained data through one route to reach the aggregate
byte limit.
3. Send a request through a sibling route.
4. Observe that the sibling route can fall back to file transport
because the first route used the shared ledger.

**Paperclip version or commit**

`47639e227e78e3c5e0dd1a3c0e2d792fe86895a3`

**Deployment mode**

Built from source with the adapter-utils and server test suites.

## What Changed

- Bound each host retention site with a fixed local byte limit.
- Limited concurrent live HTTP/2 streams with one built-in stream limit.
- Bound each host forward and response-body read to its own HTTP/2
stream lifetime.
- Removed the process-wide byte ledger, its environment override, its
metrics, and its file-transport fallbacks.
- Added tests for the stream limit, host body budget, and sibling-stream
cancellation.

## Verification

- Run `pnpm vitest run --project adapter-utils`.
- Confirm that 996 adapter-utils tests pass.
- Confirm that `test_live_forward_work_never_passes_the_stream_limit`
passes.
- Confirm that `test_the_host_body_budget_matches_the_stream_limit`
passes.
- Confirm that the sibling-stream cancellation test passes.
- Run `pnpm tsc --noEmit`.
- Confirm that all pull request checks pass.

## Risks

The bridge no longer uses a process-wide byte ledger. A local bound or
stream limit that is too low can reject or delay valid work. The tests
cover the new limits and stream cancellation behavior.

## Model Used

OpenAI GPT-5 Codex. Runtime model ID: GPT-5. The model used code
execution and repository tools. The runtime does not expose the context
window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-28 14:36:18 -07:00
Nicky Leach b2752b21d5
fix(adapter-utils): allow process sessions without birthtime (#12451)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote agent adapters use a process-session wrapper inside a
sandbox.
> - Some supported sandbox filesystems do not report inode creation
time.
> - The wrapper rejects a zero creation time before it launches the
agent.
> - This pull request accepts that filesystem shape and keeps the
existing change-time probe.
> - The benefit is that valid sandbox runs can start without a
birth-time field.

## Linked Issues or Issue Description

**What happened?**

The remote process-session wrapper exited before it launched the agent
when the sandbox filesystem reported `birthtimeMs` as zero.

**Expected behavior**

The wrapper must start on a filesystem that does not report inode
creation time.

**Steps to reproduce**

1. Start the remote process-session wrapper.
2. Make `lstat()` report a zero `birthtimeMs` for its session directory.
3. Observe that the pre-fix wrapper terminates before the child process
starts.

**Paperclip version or commit**

Reproduced from `66e1c0df8b23cb8354b36dd446d9548dc4389191`.

**Deployment mode**

Self-hosted server with a remote sandbox runtime.

## What Changed

- Allow a zero reported creation time for process-session directories.
- Keep the probe that rejects a creation time copied from change time.
- Add a regression test that launches and stops a session with zero
birth time.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-stdin-race.test.ts`
- `pnpm --filter @paperclipai/adapter-utils typecheck`

## Risks

A filesystem without creation time can reduce the precision of
sandbox-local path-swap detection. This change does not change host-file
or Paperclip API authority. A follow-up will review that larger security
posture alignment.

> I checked `ROADMAP.md`. This is a focused compatibility bug fix for
the existing sandbox-agent roadmap area.

## Model Used

OpenAI Codex — GPT-5.6. The exact deployment suffix and context-window
size are not exposed. The model used reasoning, shell tools, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-28 10:33:22 -07:00
Nicky Leach e628cf35da
fix(adapter-utils): harden the wrapper birth-time probe with exclusive create and identity-aware cleanup (#12248)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Adapter utilities run process sessions in local and remote
environments.
> - The remote process-session wrapper uses a probe file to verify
directory creation time.
> - A peer could pre-create the probe path or replace it before cleanup.
> - This pull request uses exclusive create and file-descriptor identity
checks to protect the probe.
> - The benefit is safer cleanup and fail-closed behavior at the sandbox
boundary.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The remote process-session wrapper creates and removes a birth-time
probe file. The old path-based flow did not prove that the wrapper
created the path or that the path still named the same file.

**Current behavior**

A sandbox peer can race with the probe path. The peer can pre-create a
symbolic link or replace the probe before cleanup. The wrapper can then
inspect or remove an object that it did not create.

**Proposed behavior**

The wrapper creates the probe with exclusive create. It reads `(dev,
ino, ctimeMs)` from the open file descriptor. It removes the path only
when a final identity read matches the created file.

**Reason and benefit**

This change prevents symlink-following during creation and avoids
removal of a peer's replacement object. The wrapper still fails closed
when it cannot prove a real creation time.

**Breaking changes**

None. The wrapper keeps its existing fail-closed capture behavior.

## What Changed

- Create the birth-time probe with `fs.open(path, "wx")`.
- Read probe identity with `fstat` from the open descriptor.
- Remove the probe only after a matching final identity read.
- Add focused race tests for ordinary cleanup and file, directory, and
symbolic-link replacement.

## Verification

- Run `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`.
- Run the focused suite
`packages/adapter-utils/src/execution-target-stdin-race.test.ts`.
- Confirm that the focused suite passes all 33 tests.

## Risks

The change affects shared wrapper source for local and remote process
sessions. An identity read or cleanup failure leaves the probe in place
and stops capture. The focused tests cover the new race paths.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. The model reviewed and
prepared this pull request from the supplied implementation and test
results.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-26 17:51:36 -07:00
Nicky Leach 4277ecbb2e
fix(adapter-utils): terminate the remote process-session wrapper deterministically on bridge stop (#12244)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Adapter utilities run remote process-session wrappers for sandbox
work.
> - A wrapper can outlive its host run when the host removes its session
directory during shutdown.
> - A failed directory read can look like an empty queue, so the wrapper
can poll forever.
> - This pull request adds an explicit shutdown acknowledgement and
fail-closed identity checks.
> - The benefit is deterministic wrapper cleanup without killing an
unrelated session.

## Linked Issues or Issue Description

Refs: #11916

**What happened?**

Remote process-session wrappers could remain after a host run ended. The
host could remove the session directory before the wrapper read the
shutdown marker. The wrapper then treated directory errors as an empty
queue and continued to poll.

**Expected behavior**

The host must receive an explicit shutdown acknowledgement before it
treats the wrapper as stopped. The wrapper must stop when its session
identity becomes invalid or untrusted.

**Steps to reproduce**

1. Start a remote process-session wrapper.
2. Stop the bridge while the wrapper polls its session directory.
3. Remove the session directory during the poll.
4. Observe that the wrapper must terminate with its child.

**Paperclip version or commit**

`7cfbd1ecbe4a40261ba51fed07f624524352ada2`

**Deployment mode**

Built from source with the adapter-utils test suite.

## What Changed

- Add a shutdown control file and wait for a bounded `shutdownAck`
before session cleanup.
- Require `shutdownAck` as proof of host-side shutdown.
- Capture and verify session and stdin directory identity before each
poll.
- Terminate and latch the wrapper on missing, changed, linked,
non-directory, or untrusted paths.
- Reject unusable creation times and treat all identity-check `lstat`
errors as terminal.
- Add focused regression coverage for shutdown races and identity
failures.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-stdin-race.test.ts` passes.
- The full execution-target set passes: 175 tests across three files.
- The `packages/adapter-utils` typecheck passes with `tsc --noEmit`.
- CI will run on this pull request.
- Greptile will review the pull request.

## Risks

- A platform with unreliable directory creation times can stop a wrapper
earlier than before. This fail-closed result prevents an orphan.
- A transient identity-check error now stops the wrapper. This favors
cleanup over continued polling when the session identity cannot be
trusted.
- Session cleanup remains unconditional after the bounded
acknowledgement wait.

## Model Used

OpenAI Codex — GPT-5. Context window size is not exposed in this run.
The model used tool calls and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-26 15:12:49 -07:00
Nicky Leach 802f2af154
refactor(adapter-utils): delete the dead duplex body-chunk protocol code (#12186)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The adapter utilities package provides transport code for sandbox
agents
> - The retired `duplex_v1` broker no longer produces or consumes
body-chunk frames
> - Dead protocol code remains in the host codec, gateway copy, bridge
options, and tests
> - This pull request removes that dead code and keeps the READY
handshake unchanged
> - The benefit is a smaller transport surface with fewer unused paths
to maintain

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This change improves the adapter utilities code that supports sandbox
duplex readiness and frame handling.

**Subsystem affected**

`packages/adapter-utils/` — sandbox transport codecs, execution targets,
and callback bridge tests.

**Current behavior**

The repository keeps body-chunk frame types, validators, a body spool,
decoder limits, and tests after the `duplex_v1` broker removal. No live
producer or consumer uses this code.

**Proposed behavior**

Remove the unused body-chunk protocol code and retain the READY
handshake, its strict checks, and its size limits.

**Reason and benefit**

The removal reduces dead code and keeps the host and embedded gateway
paths easier to inspect. It adds no new behavior.

**Breaking changes**

The removed frame types now decode as `unknown_type`. The live readiness
gate already ignores those frames. The READY handshake stays
byte-for-byte compatible.

**Additional context**

This cleanup follows [PR
#12171](https://github.com/paperclipai/paperclip/pull/12171), which
removed the duplex broker.

## What Changed

- Remove `duplex-body-spool.ts` and its test.
- Remove unused body-chunk frame types, validators, decoder code,
vectors, and limits.
- Remove the unused `reassembledBody` option and decoder limit
environment entry.
- Remove the embedded gateway decoder copy and the unused frame type
map.
- Keep the READY handshake and its existing boundary tests unchanged in
behavior.

## Verification

- `pnpm -F @paperclip/adapter-utils typecheck` passes.
- The duplex frame codec test passes with 30 tests.
- The sandbox execution-target test passes with 136 tests.
- The sandbox callback bridge test passes with 46 tests.
- CI must confirm all required checks after it starts.

## Risks

Low risk. The change removes code only. The READY handshake, HTTP/2 body
path, and byte-ledger path remain unchanged.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 14:33:38 -07:00
Nicky Leach 6880213de5
fix(adapter-utils): honor .gitignore for referenced-project staging (#12184)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Sandbox adapters stage project files before an agent starts.
> - Referenced projects ignored Git-ignored paths and copied large local
directories.
> - This behavior increased staging time and disk use, and it differed
from anchor workspaces.
> - This pull request resolves Git-ignored paths once and shares that
result across all referenced-project consumers.
> - The benefit is smaller, faster, and consistent project staging.

## Linked Issues or Issue Description

No public GitHub issue exists for this bug.

**What happened?**
Referenced-project staging copied Git-ignored paths, except for a fixed
list of heavy directory names. A large repository therefore used much
more time and disk space than the same repository in an anchor
workspace.

**Expected behavior**
Referenced-project staging should exclude the same Git-ignored paths
that the workspace staging path excludes.

**Steps to reproduce**
1. Create a referenced project with a large Git-ignored directory.
2. Start a sandbox or SSH run that stages the referenced project.
3. Observe that the ignored directory enters the staged content.

**Paperclip version or commit**
Commit `9964b034bbff24e700c8eccf5a8b1fc3daa44bf2`.

**Deployment mode**
Built from source.

## What Changed

- Resolve each referenced project's Git-ignored paths once before
staging.
- Carry the resolved paths as a required field on
`SandboxAdditionalSource`.
- Reuse the resolved paths in sandbox staging, SSH staging, and
content-signature code.
- Harden the read-only Git helper with a bounded process, a reduced
environment, and disabled system and global configuration.
- Fail closed on Git errors, timeouts, and invalid path relations.
- Escape tar glob metacharacters in ignore-derived exclude entries.
- Add and update unit tests for the resolver and its three consumers.

## Verification

- `pnpm vitest run --config packages/adapter-utils/vitest.config.ts`
passes 266 tests locally.
- `pnpm exec tsc --noEmit -p packages/adapter-utils/tsconfig.json`
passes locally.
- CI must pass on this pull request.
- Greptile must report 5/5 with no unresolved comments before merge.

## Risks

- A Git error or timeout now prevents staging for the affected
referenced project.
- The resolver uses a bounded read-only Git process and fails closed by
design.
- The change stays inside `packages/adapter-utils` and does not change
the database schema.

## Model Used

Claude Sonnet 5 (Anthropic) assisted the implementation with code
execution and tool use. The exact context window and reasoning mode are
not recorded.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 14:22:47 -07:00
Nicky Leach 2862e18484
refactor(adapter-utils): remove the retired duplex_v1 sandbox bridge transport (#12171)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The adapter utilities provide sandbox transport paths for agent
execution
> - The retired `duplex_v1` path remains in host, gateway, and test code
after `http2_v1` replaced it
> - Retired transport code adds maintenance cost and leaves an unsafe
fallback for unknown gateway modes
> - This pull request removes the retired path, moves shared `http2_v1`
contracts to a leaf module, and closes mode dispatch to a fixed
allowlist
> - The benefit is a smaller transport surface and explicit failure for
unsupported modes

## Linked Issues or Issue Description

Refs #12120

The `http2_v1` transport replaced `duplex_v1`, but the retired broker,
gateway, constants, and tests remain in the adapter utilities. An
unknown bridge mode can also fall through to the queue gateway when a
queue directory exists. This change removes the retired code and rejects
unsupported modes before gateway selection.

## What Changed

- Delete the host `duplex_v1` broker and its transport-only tests.
- Delete the in-sandbox duplex gateway and retired mode constants.
- Move shared `http2_v1` symbols into `bridge-transport-contract.ts`.
- Update the remaining importers and repair their focused tests.
- Validate bridge modes against `http2_v1` and `queue_v1` before queue
lookup.
- Keep `queue_v1`, `duplex-frame-codec.ts`, and duplex telemetry
dimensions unchanged.

## Verification

- [x] `npx tsc --noEmit -p packages/adapter-utils` passes.
- [x] `npx vitest run packages/adapter-utils/src` passes: 48 files and
968 tests pass, with 4 pre-existing platform skips.
- [x] Full CI is green on this pull request.
- [x] Greptile review is complete and every finding is resolved.

## Risks

The change removes an internal transport that no host path selects. The
main risk is an overlooked import or test dependency. Targeted typecheck
and tests cover the adapter utility package. Full CI must confirm
workspace-wide compatibility.

## Model Used

Anthropic Claude Sonnet 5 assisted with the implementation, as recorded
in the commit. The commit does not record a context-window size or
reasoning mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 08:47:43 -07:00
Nicky Leach 445547c989
feat(duplex): run the Daytona sandbox callback bridge over Node HTTP/2 (#12120)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox providers carry agent work through controlled execution
channels
> - The Daytona callback bridge uses a bespoke line-framed protocol over
its duplex channel
> - The bespoke protocol adds framing work and does not use the Node
transport that already supports multiplexed streams
> - This pull request carries raw bytes across the channel, adds a Node
HTTP/2 bridge, and selects it for Daytona
> - The benefit is one authenticated, multiplexed callback session with
queue_v1 as the bounded fallback

## Linked Issues or Issue Description

**Subsystem affected**

The packages/plugins Daytona provider and the shared duplex execution
path.

**Problem or motivation**

The Daytona callback bridge uses a bespoke line-framed protocol over the
provider duplex channel. This adds protocol work and limits stream
handling.

**Proposed solution**

Carry raw bytes through the cross-layer channel. Add an authenticated
Node HTTP/2 host server and sandbox client gateway. Select http2_v1 for
Daytona and retain queue_v1 as the fallback.

**Alternatives considered**

Keep the current duplex_v1 protocol. This keeps the bespoke framing path
and does not provide one HTTP/2 session for callback streams.

**Roadmap alignment**

ROADMAP.md lists Daytona under cloud and sandbox agents. This change
improves the shipped Daytona provider path.

**Additional context**

The branch adds no dependency. Node 24 provides the http2 module. The
host token check and canonical path parser remain the single dispatch
path.

## What Changed

- Carry raw Uint8Array chunks through the adapter, plugin, worker,
runtime, and Daytona layers.
- Encode bytes as base64 only across the JSON-RPC hop, because JSON has
no binary type.
- Add the bounded host HTTP/2 server and the in-sandbox HTTP/2 client
gateway.
- Authenticate every stream with the per-run bridge token before route
work.
- Parse the path once and reuse the canonical result for route and
forwarding work.
- Select http2_v1 for Daytona and fall back once to queue_v1 when the
client preface is absent.
- Add transport, session, stream, and fallback telemetry.
- Mark HTTP/2 as the preferred transport and queue_v1 as the
soft-deprecated fallback.

## Verification

- `npx vitest run packages/adapter-utils/src` — 990 passed and 4
skipped.
- `npx vitest run
server/src/__tests__/plugin-worker-manager-duplex.test.ts` — 32 passed.
- `npx vitest run --config
packages/plugins/sandbox-providers/daytona/vitest.config.ts` — 220
passed and 6 skipped.
- `npx tsc --noEmit` in `packages/adapter-utils`, `packages/shared`,
`packages/plugins/sdk`, and `server` — clean.
- No `package.json` or `pnpm-lock.yaml` file changed.
- The live Daytona test skips when `DAYTONA_API_KEY` is absent.
- The root `npx tsc --noEmit` command has a pre-existing missing
`packages/adapters/droid-local` reference on this branch and on
`master`.

## Risks

- The transport change affects several duplex layers and could expose
byte-boundary errors.
- A missing HTTP/2 client preface falls back once to queue_v1 and
records `preface_missing`.
- The host token check and canonical path parser must remain on the
shared dispatch path.
- The live Daytona test needs `DAYTONA_API_KEY` and does not run in this
agent sandbox.

## Model Used

OpenAI GPT-5, tool-enabled coding agent with repository inspection,
GitHub CLI, and shell execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 07:35:39 -07:00
Nicky Leach b6854e61c7
refactor(adapter-utils): rename EffectiveSandboxCapabilities to EffectiveExecutionCapabilities (#12119)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The adapter utilities package defines shared types for agent
execution targets
> - The type name EffectiveSandboxCapabilities describes only one
transport
> - All execution target drivers return the same resolved capability
snapshot
> - This pull request gives the snapshot a general name and keeps the
old type as a deprecated alias
> - The benefit is clearer public vocabulary with source compatibility
for current consumers

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The exported capability snapshot type uses the name
`EffectiveSandboxCapabilities`, although local, SSH, sandbox, and plugin
drivers return it.

**Subsystem affected**

The change affects `packages/adapter-utils` and its server consumers.

**Current behavior**

The public type name points to the sandbox transport. The private parser
also uses the sandbox-only name.

**Proposed behavior**

Use `EffectiveExecutionCapabilities` for the public type and
`parseEffectiveExecutionCapabilities` for the private parser. Keep a
deprecated alias for the old public type.

**Reason and benefit**

The new name matches the established execution-target vocabulary. The
alias keeps existing type imports working during the migration.

**Breaking changes**

None. The runtime field, capability flags, parsed shape, and package
versions do not change.

**Additional context**

GitHub search found no duplicate or related open issue or pull request.

## What Changed

- Rename the exported interface to `EffectiveExecutionCapabilities`.
- Keep `EffectiveSandboxCapabilities` as a deprecated type alias.
- Rename the private parser and update its call site and references.
- Add a type-level test for the deprecated alias.

## Verification

- `npx tsc --noEmit -p packages/adapter-utils`
- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts`
- `npx vitest run
server/src/__tests__/environment-execution-target-capabilities.test.ts
server/src/__tests__/environment-execution-target-duplex.test.ts`
- The local checks passed with 133 adapter-utils tests and 31 server
tests.
- Reviewers can confirm that the runtime field and capability flags stay
unchanged.

## Risks

Low risk. The alias protects existing type imports. The change does not
alter runtime behavior or serialized data.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. The runtime does not
expose the context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 07:13:55 -07:00
Nicky Leach d1573244b5
refactor: disambiguate the Telemetry and Observability data paths (#12128)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip records first-party events, OpenTelemetry data, and local
run-log events
> - The code and documents used one term for these three data paths
> - This naming made the required review level unclear
> - This pull request names each data path in the module names,
documents, and code comments
> - The benefit is a clear review rule without a runtime change

## Linked Issues or Issue Description

**Issue type**

Unclear or confusing.

**Where is the issue?**

`packages/shared/src/telemetry/README.md`, `doc/observability.md`,
`doc/run-log-events.md`, and the duplex instrumentation modules.

**What's wrong?**

The repository used Telemetry for first-party events, OpenTelemetry
data, and local run-log events. This usage made the data path and review
level unclear.

**Suggested fix**

Use Telemetry only for Paperclip first-party events. Use Observability
for OpenTelemetry data. Use the run log for rows in
`heartbeat_run_events`.

Related public pull requests: #8476 and #9672.

## What Changed

- Rename the duplex instrumentation modules and identifiers from
`Telemetry` to `Observability`.
- Move the Observability and run-log contracts out of the Telemetry
README.
- Add `doc/observability.md` and `doc/run-log-events.md` as the
canonical documents.
- Add a file-path review rule to `AGENTS.md`.
- Correct the remaining code comments that name the wrong data path.
- Keep all event names, payloads, database records, spans, configuration
keys, environment variables, and runtime paths unchanged.

## Verification

- `npx vitest run packages/shared/src/telemetry/readme-contract.test.ts`
passes.
- `npx vitest run packages/adapter-utils/src/published-exports.test.ts`
passes.
- `npx vitest run
packages/adapter-utils/src/acpx-engine/startup-timing.test.ts` passes
with 42 tests.
- `pnpm --filter @paperclipai/adapter-utils typecheck` passes.
- `pnpm --filter server typecheck` passes.
- The old module name does not remain in TypeScript or JSON files,
except for the intentional publication guard.
- CI and Greptile checks remain pending after PR creation.

## Risks

- The old duplex module subpath no longer has a compatibility shim. The
board accepted this intentional hard break.
- The new duplex module subpath stays blocked from package publication.
- The change has no runtime effect. The main risk is an incorrect
document or module reference.

## Model Used

OpenAI GPT-5 Codex, exact model ID `gpt-5`, with tool use and code
review support.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR with the documentation issue
fields
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-24 16:42:33 -07:00
Nicky Leach 16b59c9315
feat(adapter-utils): stream duplex bridge bodies as sequenced chunks with receive-side spill (#12006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters use a duplex bridge to send requests and responses
across an isolated boundary
> - The bridge held each request body and response body in memory on
both ends
> - Large bodies can exhaust memory and reduce the safe size of adapter
traffic
> - This pull request sends receive-side bodies as sequenced chunks and
spills large bodies to disk
> - The benefit is bounded memory use with strict size, order, and
cleanup checks

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The adapter-utils duplex bridge transports request and response bodies
across the sandbox boundary.

**Current behavior**

The bridge stores each complete body in memory on both ends of the
duplex channel.

**Proposed behavior**

The bridge sends body chunks with sequence checks. The receive side
keeps bodies up to 1 MiB in memory and spills larger bodies to a
temporary file.

**Reason and benefit**

This change reduces memory pressure and keeps malformed or oversized
input on a terminal error path.

**Breaking changes**

The duplex frame version changes to version 2. The request and response
envelopes now carry bodyByteCount, and body_chunk frames carry the body
data.

## What Changed

- Add version 2 body_chunk frames with 256 KiB raw slices encoded as
canonical base64 text.
- Add receive-side memory and spill reassembly with per-channel disk and
file limits.
- Reject malformed, reordered, oversized, truncated, and non-canonical
body chunks.
- Stream reassembled request bodies to the host forward handler with a
web stream and half-duplex request.
- Remove spill files on success, failure, channel death, and startup
cleanup.

## Verification

- Run the adapter-utils type-check.
- Run the adapter-utils duplex test suite.
- Run all pull request checks.
- Run the Greptile review and confirm a 5/5 score with no open findings.

## Risks

The wire format changes from version 1 to version 2. Older bridge peers
cannot use this protocol. The receive path adds temporary file
operations and cleanup paths. The implementation fails closed when a
body violates size or sequence rules.

## Model Used

OpenAI GPT-5 Codex, tool-enabled coding agent. The exact context-window
size and reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
- [x] My branch name describes the change (e.g. docs/..., fix/...) and
contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-23 08:52:31 -07:00
Nicky Leach 05b35d4669
feat(duplex): bound aggregate duplex route resource consumption with a process-owned byte ledger (#12003)
## Thinking Path

> - Paperclip runs AI agents through adapters and sandboxed execution
targets.
> - Duplex routes retain bytes across route data, broker messages,
decoder buffers, and readiness replay.
> - Per-route limits bound each route but do not bound the total
retained bytes across many routes.
> - A process-owned ledger must charge each retained buffer before
allocation and release the charge during cleanup.
> - This pull request adds the aggregate ledger, connects it to host and
sandbox duplex paths, and adds route coverage.
> - The benefit is a fail-closed process-wide byte limit that keeps
concurrent duplex work within a safe resource budget.

## Linked Issues or Issue Description

**Subsystem affected**

This change affects packages/adapter-utils and server duplex
orchestration.

**Problem or motivation**

Many routes can each stay below their per-route limits while their
combined retained bytes exceed a safe process budget.

**Proposed solution**

Add a process-owned aggregate byte ledger. Charge route data, broker
bytes, decoder buffers, and readiness replay bytes before allocation.
Release each charge during cleanup. Use a separate sandbox_process
decoder cap for the in-sandbox path.

**Alternatives considered**

Keep only per-route limits. This does not bound the combined process
use. Set a fixed limit at one call site. This misses retained bytes in
other duplex paths.

**Roadmap alignment**

This is a tightly scoped reliability and resource-safety improvement. It
does not duplicate a roadmap feature.

**Additional context**

The aggregate ceiling uses a safe 256 MiB default. An invalid override
falls back to that default and reports the rejected value.

## What Changed

- Add a process-owned aggregate byte ledger for duplex route resource
use.
- Charge and release route data, broker forward and response bytes,
decoder buffers, and readiness replay bytes.
- Bound host-to-worker pending writes and standard input transport
bytes.
- Add a separate decoder cap for the sandbox_process path.
- Make invalid aggregate-ceiling overrides fall back to the safe default
without host startup failure.
- Add adapter-utils and server tests for charging, release, rejection,
cleanup, and many-route aggregate limits.

## Verification

- pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit
- pnpm --filter @paperclipai/server exec tsc --noEmit
- Run the focused adapter-utils duplex ledger and execution-target
tests.
- Run the server aggregate-ledger route test.
- Confirm all required pull request checks pass on this branch.

## Risks

The ledger touches several duplex buffer paths. A missed release could
reduce later capacity until process restart. The tests cover charge,
release, rejection, cleanup, and route aggregation. The change uses a
safe default when configuration input is invalid.

## Model Used

OpenAI GPT-5 Codex. The runtime model ID and context window are not
exposed to this task. The model used tool calls, shell commands, and
code review workflow support.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes/Closes/Refs OR
(b) described the issue in-PR following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub issue references)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-22 22:14:35 -07:00
Nicky Leach cc42a67e7e
fix(adapter-utils): extend the duplex fail-closed run disposition to the CLI lane (#11966)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip runs agents through adapter execution lanes
> - Duplex adapters can lose their control channel before a process
completes
> - The ACP lane already fails closed, but the CLI lane can report false
success
> - This pull request applies the same completion rule to the CLI lane
and shares the loss code
> - The benefit is consistent failure reporting when a duplex channel
closes during a run

## Linked Issues or Issue Description

**What happened?**

A CLI-lane duplex run can lose its control channel before clean process
completion. The run can then report `succeeded` with exit code 0 and no
error code.

**Expected behavior**

The execution target must fail closed when the channel dies before clean
completion. It must return exit code 1, the typed `duplex_channel_lost`
error code, and a short stderr note.

**Steps to reproduce**

1. Start a duplex adapter run through the CLI execution lane.
2. Close the duplex control channel before the process completes
cleanly.
3. Inspect the run result and error code.

**Paperclip version or commit**

Commit `5e01523d4eb6df4a20a0bddd05374c9c42225203`.

**Deployment mode**

Built from source.

**Installation method**

Built from source with pnpm.

**Agent adapter(s) involved**

Claude Code, Codex, Cursor, Gemini, Kimi, OpenCode, and Pi local
adapters.

**Database mode**

Not database-related.

## What Changed

- Add an optional `errorCode` field to `RunProcessResult`.
- Add a one-read completion seam to the execution target process
options.
- Fail closed when a duplex channel dies before clean process
completion.
- Add `settleRunDisposition()` to atomically read and mark orderly
completion.
- Share the typed duplex loss error code across the ACP and CLI lanes.
- Mark non-success terminal results as orderly completion before
teardown.
- Wire the seam through the seven duplex adapters.
- Add regression tests for channel loss, clean completion, and non-clean
terminal results.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts` — 118
passed.
- `npx vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts
-t "sandbox duplex run-disposition seam"` — 4 passed.
- The author confirmed a clean type-check for
`@paperclipai/adapter-utils` and the seven duplex adapter packages.
- Pre-existing environment failures remain outside this change. They
include `EACCES mkdir '/srv/paperclip'` and remote file-size setup
failures.

## Risks

The change alters terminal status for CLI duplex runs that lose control
before clean completion. The typed error code and stderr note keep the
failure visible. The broker marks failed, cancelled, and timed-out
results as orderly completion to prevent false loss events during
teardown.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution, with the standard
GPT-5 context window. The model assisted with the implementation and
test work.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-22 11:12:51 -07:00
Nicky Leach 10d2781a29
feat(sandbox): add the duplex bridge broker, gated transport selection, and fixed observability (#11769)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Sandbox adapters provide controlled execution for untrusted provider
environments.
> - The sandbox channel needs one persistent duplex transport with
strict host control.
> - The transport must remain off unless the instance setting and
provider capability both allow it.
> - The host must detect loss, bound resource use, and expose only safe
telemetry.
> - This pull request adds the broker, gated selection, kill-switch
wiring, fixed observability, and real-process proof.
> - The benefit is safer sandbox execution with bounded failure behavior
and inspectable transport results.

## Linked Issues or Issue Description

No public issue exists for this change. The related pull requests are
#11738 and #11750.

**Problem or motivation**

The sandbox duplex channel needs a host-controlled broker, strict
transport gates, bounded provider input, and safe loss telemetry.
Without these controls, a provider can cause replay, resource growth,
unsafe endpoint selection, or data exposure through telemetry.

**Proposed solution**

Add a host broker with nested time limits, request limits, one-shot
loss, and per-id deduplication. Select duplex transport only when the
instance setting and provider capability both equal true. Assign the
endpoint and nonce on the host. Reject invalid readiness data and use
the file bridge on failure. Add fixed redacted telemetry and a
real-process end-to-end test harness.

**Alternatives considered**

Keep the file bridge as the only transport. This avoids new channel
behavior but does not provide persistent duplex operation for supported
sandbox providers.

**Roadmap alignment**

This change supports the Cloud / Sandbox agents section in ROADMAP.md.

## What Changed

- Add the duplex bridge broker with bounded forward, response, and
gateway wait budgets.
- Bound concurrent requests, lifetime requests, and request-id bytes
before retention or forwarding.
- Select duplex transport only when both required gates are true.
- Assign the loopback port and nonce on the host and enforce a
liveness-only READY frame.
- Fall back to the file bridge after invalid readiness, contamination,
bind failure, or timeout.
- Carry the kill switch through the server, acpx engine, and six local
adapters.
- Add fixed, redacted duplex telemetry with a provider allowlist.
- Add a real-process end-to-end harness for readiness, round trips,
loss, and teardown.
- Add regression coverage for limits, loss, UTF-8 splits, concurrency,
and telemetry dimensions.

## Verification

- Adapter-utils, server, and Daytona typechecks pass locally.
- Adapter-utils tests pass, including the codec, broker,
execution-target sandbox, and real-process harness.
- Server kill-switch tests pass.
- Live Daytona tests pass with the required provider key and skip
without that key.
- The root pnpm-lock.yaml file has no diff.
- The branch contains ten commits after origin/master.

## Risks

- Duplex transport remains disabled unless both gates equal true.
- A provider remains an untrusted boundary and needs least-privilege
credentials and quotas.
- The server telemetry recorder stays deferred; the default recorder
does nothing.
- A provider that pre-binds the host port causes a fail-closed fallback
to the file bridge.
- The change adds no database migration and changes no root lockfile.

## Model Used

OpenAI GPT-5, exact model family GPT-5, large context window, reasoning,
and tool use. The model assisted with Git handoff validation and PR
preparation. The implementation commits came from the engineering
worktree.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
- [x] My branch name describes the change (e.g. docs/... or fix/...) and
contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
- [x] I searched the GitHub PR list for similar PRs and confirmed this
is not a duplicate

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-22 09:01:31 -07:00
Devin Foley adfbe2d4b9
feat(environments): refer to the managed default environment by name, not the sandbox driver key (#11838)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Managed deployments provision a platform-managed default environment
for agent runs; the UI shows this environment in selectors, the agent
form, run details, and the environments page
> - Those surfaces append the raw driver key to the environment name, so
users see labels like "Paperclip Computer (sandbox)", "Paperclip
Computer · sandbox", and fallback copy such as "Managed sandbox" and
"The sandbox has no ready authentication"
> - "sandbox" is infrastructure vocabulary, not the product name of the
environment; showing it next to the managed environment's name is
confusing and off-brand
> - This pull request renders platform-managed environments by name
alone and rewords the sandbox-phrased copy, while user-created
environments keep the driver suffix so mixed lists stay distinguishable
> - The benefit is that the default environment reads as one clear
product name everywhere, and self-hosted users lose nothing: their own
environments still show the driver

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Display of the platform-managed default environment across the UI.

**Subsystem affected**

UI (environment selectors, agent config form, environments page, agents
page, run details) and the claude-local/codex-local adapter auth checks.

**Current behavior**

The agent form labels the inherited default environment as "Name
(sandbox)". Environment selectors and the environments list render "Name
· sandbox". The agents page describes the environment as "<provider>
sandbox provider". The agent form's fallback label is "Managed sandbox".
Adapter auth checks say "The sandbox has no ready authentication for
this adapter."

**Proposed behavior**

Platform-managed environment rows (`metadata.managedByPaperclip`) render
their name alone. The fallback label is "Paperclip Computer". The agents
page describes managed environments as "Managed by Paperclip". Run
details omit the driver suffix for sandbox-driver environments (the
adjacent Provider entry already identifies the mechanism). Adapter auth
checks say "This environment has no ready authentication for this
adapter."

**Reason and benefit**

The managed environment carries a product name. Appending the raw driver
key ("sandbox") to it is noise and contradicts the product naming.
User-created environments keep the driver suffix, so mixed lists stay
distinguishable.

**Breaking changes**

None. Message text of the auth check is not read programmatically; the
UI keys off `ADAPTER_AUTH_MISSING_CHECK_CODE`. Rows without the managed
marker render exactly as before.

## What Changed

- New `environmentDisplayLabel` helper in
`ui/src/lib/managed-sandbox-environment.ts`: managed rows → name alone;
other rows → "Name · driver".
- `AgentConfigForm`: inherited-default label uses the helper; fallback
copy "Managed sandbox" → "Paperclip Computer"; environment options use
the helper.
- `ProjectProperties`, `CompanyEnvironments`: environment selector
options use the helper; the environments-list row hides the driver
suffix on managed rows; the managed detail page's fallback description
no longer says "sandbox".
- `Agents` page: managed environments are described as "Managed by
Paperclip" instead of "<provider> sandbox provider".
- `CommentThread` run details: the driver suffix is omitted for
sandbox-driver environments.
- claude-local and codex-local adapters: auth-missing check message/hint
reworded from "sandbox" to "environment" (ACP and environment-test
paths); claude-local probe/effort/login hints reworded the same way.
- Run status lines: "Syncing workspace to sandbox", "Exporting git
changes from sandbox", "Starting adapter in sandbox", and friends now
say "environment"; "Finalizing sandbox workspace" → "Finalizing
workspace". Templated transfer-progress lines map the `sandbox`
transport key to "environment" for display (`runtime-progress.ts`).
- Agent form sign-in panel: "Sign in to the sandbox" → "Sign in to the
environment"; "Authenticated. The sandbox has credentials now." → "…The
environment has credentials now."
- Feature catalog + instance settings card: "Managed Sandbox Only" →
"Managed Environment Only" (setting key unchanged; the card keeps its
alphabetical slot).
- Server agents routes: execution-target failure and test-identity copy
no longer say "sandbox"; workspace-mode label "Cloud sandbox" → "Cloud
environment".
- Tests: new `environmentDisplayLabel` unit cases; new `AgentConfigForm`
render case asserting the managed default renders without "(sandbox)" or
"· sandbox"; status-line assertions updated across adapter-utils, server
heartbeat/live-run, and UI chat suites.

## Verification

- `pnpm --filter @paperclipai/ui typecheck` — clean.
- `pnpm --filter @paperclipai/adapter-claude-local typecheck` and
`--filter @paperclipai/adapter-codex-local typecheck` — clean.
- `vitest run` for `managed-sandbox-environment.test.ts`,
`AgentConfigForm.render.test.tsx`, `CompanyEnvironments.test.tsx`,
`Agents.test.tsx`, `CommentThread.test.tsx`, `NewAgent.test.tsx` — all
green (118 tests across the two runs).

## Risks

Low risk. Cosmetic label changes only; no data or API changes. Rows
without `metadata.managedByPaperclip` render exactly as before, so
self-hosted deployments with their own environments see no change. The
only self-hosted-visible wording changes are the adapter auth-check
message and the driver suffix omission on sandbox-driver rows in run
details.

## Model Used

- Claude (Anthropic) — claude-fable-5 (Claude Fable 5), Claude Code CLI,
extended thinking, tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
docs reference these labels)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-21 12:51:22 -07:00
Nicky Leach 8161244284
feat(sandbox): add opt-in duplex command-stream foundation (capability, protocol, bounded host route, frame codec) (#11738)
## Thinking Path

> - Paperclip provides a control plane for companies that run AI agents.
> - Sandboxed agents need a safe execution path for persistent command
streams.
> - The existing callback transport does not provide a bounded, generic
duplex route.
> - The host must control capability access, route identity, protocol
limits, and close behavior.
> - This pull request adds an opt-in duplex command-stream foundation
across the sandbox layers.
> - The feature stays inert because no current provider declares the
capability.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting (multiple of the above)

**Problem or motivation**

Sandbox command execution needs a persistent host-to-sandbox stream. The
current callback bridge uses a file transport and does not provide this
generic route.

**Proposed solution**

Add a fail-closed provider capability, generic worker protocol messages,
a host-owned bounded route, cross-layer service mediation, and a
versioned newline-delimited frame codec.

**Alternatives considered**

Keep the file transport and add feature-specific commands. This does not
provide one reusable duplex contract or host-owned route bounds.

**Roadmap alignment**

This work supports the completed Cloud / Sandbox agents roadmap area and
the safe autonomy goal in the product definition.

**Additional context**

The change passed a two-stage security review. The final code review
verdict was approve after fixes for active-stream bounds and
service-layer capability mediation.

## What Changed

- Add the opt-in `duplexCommandStream` provider capability with
fail-closed narrowing.
- Add duplex open, write, stop, and close requests and data and exit
notifications to the plugin worker protocol.
- Add a host-owned route with bounds for chunk size, cumulative bytes,
lifetime, protocol errors, pending requests, and pre-bind buffering.
- Add close acknowledgement handling with worker retirement when the
close remains unconfirmed.
- Wire `openDuplexChannel` through the execution target, runtime
service, and plugin worker.
- Add a versioned frame codec with shared wire-compatibility vectors and
split UTF-8 handling.

## Verification

- `server/src/__tests__/plugin-worker-manager-duplex.test.ts` passes 18
tests.
- `server/src/__tests__/environment-execution-target-duplex.test.ts`
passes 11 tests.
- `packages/adapter-utils/src/duplex-frame-codec.test.ts` passes 38
tests.
- `server/src/__tests__/sandbox-capability-contract.test.ts` passes 15
tests.
- Setup-token pseudo-terminal regression tests pass 47 tests.
- Server TypeScript check passes.
- Continuous integration will run the full required test, typecheck,
build, and policy checks.

## Risks

- Providers that opt into the capability must implement the complete
worker protocol.
- Route limit defaults can close a stream when a workload exceeds the
configured bounds.
- The capability remains disabled for current providers, so current
production behavior does not change.

## Model Used

OpenAI GPT-5 (`gpt-5`), with tool use and code execution. The model
reviewed and prepared this pull request from the supplied implementation
and verification record.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-19 13:39:13 -07:00
Nicky Leach 233be4b36c
feat: parallelize sandbox file-sync behind a provider opt-in capability (#11736)
## Thinking Path

> - Paperclip runs AI agents through local and remote execution
adapters.
> - Sandbox providers move workspace and asset files before and after
agent runs.
> - Serial file transfers delay startup and teardown when several
operations do not depend on each other.
> - Providers need an opt-in contract so existing providers keep their
serial behavior.
> - This pull request adds a bounded scheduler and routes inbound and
outbound sync operations through it.
> - The benefit is shorter sandbox setup and teardown with stable
errors, clear telemetry, and a safe opt-in path.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting (multiple of the above): packages/shared,
packages/adapter-utils, packages/plugins, and server.

**Problem or motivation**

Sandbox sync processes the workspace, assets, and referenced projects in
series. This adds avoidable wait time to agent startup and teardown.

**Proposed solution**

Add a fail-closed provider capability named concurrentSyncOperations.
Use a bounded scheduler with a limit of four operations. Preserve
operation order for error reporting. Keep non-opted-in providers on the
serial path.

**Alternatives considered**

Increase the serial transfer speed or add provider-specific schedulers.
Those options do not provide one shared contract or stable behavior
across providers.

**Roadmap alignment**

ROADMAP.md lists cloud and sandbox agents as a product area. This change
improves sandbox execution without changing the control-plane contract.

**Additional context**

The Daytona provider opts in. Board trials on this commit showed overlap
for inbound sync and outbound restore, with no referenced-project
staging failures.

## What Changed

- Add the concurrentSyncOperations sandbox capability and fail-closed
parsing.
- Add a bounded settle-all scheduler with stable input-order errors.
- Parallelize inbound workspace, asset, and referenced-project sync
operations when the provider opts in.
- Parallelize outbound workspace and asset restore operations when the
provider opts in.
- Surface referenced-project failure text in run logs and server
telemetry.
- Add Daytona sync spans and the capability declaration.
- Preserve in-flight upload scratch tarballs during workspace wipe.
- Add unit and regression tests for the scheduler, coordinators,
provider behavior, telemetry, and wipe race.

## Verification

- Run the adapter-utils and server type checks.
- Run the targeted adapter-utils, server, and Daytona test suites.
- Run the full automated sweep.
- Review six cold Daytona trials, with three serial and three parallel
runs.
- Confirm that parallel trials show inbound overlap and outbound restore
overlap.
- Confirm that providers without the capability keep serial behavior.

## Risks

- Providers must opt in only when their file operations can run safely
at the same time.
- A provider that declares the capability incorrectly can expose
transfer races.
- The scheduler keeps a limit of four to bound resource use.
- Providers without the capability keep the prior serial behavior.

## Model Used

OpenAI GPT-5 in the Codex runtime. The model used tool calls, code
inspection, and GitHub workflow support. The model did not author the
implementation commits.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-19 12:34:11 -07:00
scotttong 1c366a9059
fix(server): reject invalid agent credentials instead of downgrading to the local user actor (#11589)
<!-- Write all pull request text in Simplified Technical English
(ASD-STE100): short sentences, one instruction per sentence, simple
approved vocabulary, and the active voice. -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server authenticates each agent request in `actorMiddleware`
before it attributes chat comments
> - When an agent bearer token failed verification, the middleware
called `next()` with no error and the request continued without an agent
actor
> - The request then fell back to the local user actor, so the server
stored agent replies as user comments
> - The task chat UI renders user comments in blue bubbles, so agent
messages appeared as blue user bubbles
> - This pull request rejects invalid agent credentials with 401 instead
of a silent downgrade
> - The benefit is that agent messages keep agent attribution, and
broken credentials fail loudly with a clear retry message

## Linked Issues or Issue Description

**What happened?**

A user cancelled an onboarding question card. The agent posted a
follow-up reply. The reply appeared in a blue bubble, which the UI
reserves for human messages. The agent run held an expired local agent
JWT. The auth middleware could not verify the token, called `next()`
without an actor, and the request fell back to the local user identity.
The server stored the agent comment as a user comment.

**Expected behavior**

Agent messages always render as agent bubbles. A request with invalid
agent credentials must fail with 401 so the adapter can refresh
credentials and retry. It must not post content under a human identity.

**Steps to reproduce**

1. Start a local Paperclip instance.
2. Give an agent run an expired or malformed agent JWT.
3. Let the agent post an issue comment through the API bridge.
4. Before this change: the comment is stored with the local user
identity and renders as a blue bubble. After this change: the request
fails with 401 and a message that tells the caller to obtain fresh
credentials.

## What Changed

- `server/src/middleware/auth.ts`: a bearer token that fails
verification now produces a 401 `unauthorized` error instead of a silent
fall-through to the anonymous/local-user actor.
- The 401 message states the cause: expired token, unverifiable token,
empty bearer token, missing agent record, agent record in another
company, terminated agent, or agent pending approval.
- The API-key path now also rejects an agent record whose company does
not match the key.
- `packages/adapter-utils/src/execution-target.ts`: the bridge proxy now
writes a `comment id: <id>` marker to the run log for each posted issue
comment, so misattributed comments can be traced to a run.
- `ui/src/components/task-chat/task-chat-adapter.test.ts`: a regression
test asserts that a recovered `local-board` comment with a derived agent
author renders as an agent bubble, not a user bubble.
- `server/src/__tests__/agent-auth-middleware.test.ts` and
`packages/adapter-utils/src/execution-target-sandbox.test.ts`: new tests
cover each rejection path and the log marker.

## Verification

- Run `pnpm vitest run src/__tests__/agent-auth-middleware.test.ts` in
`server/` — 14 tests pass.
- Run `pnpm vitest run execution-target-sandbox` at the repo root — 44
tests pass.
- Run `pnpm vitest run
src/components/task-chat/task-chat-adapter.test.ts` in `ui/` — 4 tests
pass.
- Manual check: post an issue comment with an expired agent JWT; the API
returns 401 with a retry message and no comment is stored.

## Risks

- Behavioral shift: requests that previously continued as anonymous or
local-user actors after a failed agent-token verification now receive
401. Any caller that relied on the silent downgrade must refresh its
credentials. This is the intended fix, and the adapters already handle
401 with a credential refresh.
- No schema or migration changes. Low risk otherwise.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic), model ID `claude-fable-5`, via Claude Code with
extended thinking and tool use (agent harness with shell, file, and git
tools).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 21:36:11 -07:00
Nicky Leach 3061ce6901
feat(sandbox): stream session output by capability, drop three operator flags (#11557)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandboxed agents use provider capabilities to select safe execution
paths
> - Session output still depends on three operator flags that duplicate
capability data
> - Duplicate flags can drift from the verified sandbox capability
snapshot
> - This pull request makes the capability snapshot the only streaming
decision and removes the obsolete flags
> - The benefit is default streaming with a poll fallback when a
capability or stream fails

## Linked Issues or Issue Description

**What existing behavior does this improve?**

ACP sandbox session-output streaming and sandbox execution
configuration.

**Subsystem affected**

Cross-cutting (multiple of the above): server/, packages/shared/,
packages/adapter-utils/, and packages/plugins/.

**Current behavior**

Session-output streaming requires operator flags in the server and
Daytona plugin configuration. Saved configurations can retain a removed
key.

**Proposed behavior**

The verified capability snapshot selects streaming. The Daytona plugin
uses persistent sessions by default, keeps bypass commands one-shot, and
falls back from the log stream to polling. Removed configuration keys
become inert.

**Reason and benefit**

One capability source prevents configuration drift. The fallback keeps
output available when capability resolution or log streaming fails.

**Breaking changes**

The three operator flags no longer control session-output streaming.
Existing saved keys load but have no effect.

## What Changed

- Remove `useSessions` and `useLogStream` from the Daytona plugin
configuration and manifest.
- Remove `streamAgentSessionOutput` from server configuration, shared
types, and execution-target plumbing.
- Select streaming from `persistentProcessSessions` and
`independentControlCommands`.
- Keep poll fallback on capability resolution failure and stream
failure.
- Strip removed keys from strict fake-sandbox and catchall plugin
configuration.
- Update the sandbox capability documentation and focused tests.

## Verification

- `tsc --noEmit` passed in `packages/shared`, `packages/adapter-utils`,
`server`, and the Daytona plugin.
- Daytona `plugin.test.ts` passed 139 tests.
- Server capability, configuration, route, and runtime suites passed 160
tests.
- `packages/adapter-utils` `execution-target-sandbox.test.ts` passed 44
tests.
- The capability matrix covers stream, poll, and resolution-failure
paths.
- Removed-key tests cover strict fake-sandbox and catchall plugin
schemas.

## Risks

- A capability snapshot that lacks either required session capability
uses polling.
- A log stream failure uses polling and can increase request count.
- Existing removed configuration keys no longer change behavior.
- The isolated-worktree Daytona Vitest run has a pre-existing missing
`packages/adapters/droid-local` reference. CI and standard checkouts use
the committed configuration.

## Model Used

OpenAI Codex, GPT-5, tool use and code review assistance. The exact
runtime context window is managed by the Codex platform.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-17 13:25:30 -07:00
Nicky Leach e71ce9a9d3
feat: sandbox provider capability contract with fail-closed effective resolution (#11463)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip runs work through adapters and sandbox providers
> - Providers need a clear contract so the server can use only verified
capabilities
> - A declared capability must not grant a method that the live worker
did not verify
> - This pull request adds manifest declarations and fail-closed
effective capability resolution
> - The benefit is safe provider reuse across execution targets and run
lifecycles

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting (multiple of the above)

**Problem or motivation**

Sandbox providers expose different runtime methods. The server needs one
safe capability contract that accounts for provider declarations, worker
verification, and narrowing configuration.

**Proposed solution**

Add strict manifest validation for five sandbox capabilities. Resolve
effective capabilities as the subset of verified, declared, and narrowed
values. Store the result as a frozen execution-target snapshot.

**Alternatives considered**

Trusting the manifest alone could grant methods that the worker does not
support. Trusting only a fixed built-in list would reject valid
third-party providers. The intersection rule keeps the verified runtime
ceiling and supports both provider types.

**Roadmap alignment**

This change supports the ACP run lifecycle track and the sandbox
provider contract work in the current roadmap.

**Additional context**

The legacy `supportsReusableLeases` field remains supported. The nested
capability validator rejects unknown keys. Missing or unavailable
verification resolves all capabilities to `false`.

## What Changed

- Add strict `sandboxCapabilities` manifest validation with legacy
reusable-lease compatibility.
- Carry declarations through the ready-driver projection.
- Add fail-closed effective resolution from verified, declared, and
narrowed capabilities.
- Add narrowing for provider configuration, Kubernetes Job leases, and
Daytona sessions.
- Add a frozen read-only capability snapshot to execution targets.
- Add focused tests and keep existing characterization baselines
covered.
- Add and update sandbox provider capability documentation.

## Verification

- `npx vitest run packages/shared/src/validators/plugin.test.ts`
- `npx vitest run
server/src/__tests__/plugin-environment-driver-sandbox-capabilities.test.ts`
- `npx vitest run
server/src/__tests__/sandbox-capability-contract.test.ts`
- `npx vitest run
server/src/__tests__/environment-execution-target-capabilities.test.ts`
- `npx vitest run
packages/adapter-utils/src/acpx-engine/startup-characterization.test.ts
packages/adapter-utils/src/acpx-engine/turn-characterization.test.ts
packages/adapter-utils/src/acpx-engine/settlement-characterization.test.ts
packages/adapter-utils/src/acpx-engine/composed-run-characterization.test.ts`
- Package typechecks for shared, server, and adapter-utils pass.
- Stage-2 security review suites pass with 28 tests.

## Risks

The resolver fails closed when verification is absent or unavailable.
Providers that rely on undeclared capabilities may see narrower behavior
until they expose verified worker methods. The change does not alter the
existing native-sync guard.

## Model Used

OpenAI Codex, GPT-5, exact runtime model ID `gpt-5`, tool use and code
execution. The implementation author used this model to assist with the
change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-17 10:55:40 -07:00
Nicky Leach ed8075b535
fix(adapter-utils): order stdin file writes in the sandbox process-session bridge (#11406)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters can run through a sandbox process-session bridge
> - The bridge writes streamed standard input to files before the remote
process reads them
> - Concurrent file writes can make a later chunk visible before an
earlier chunk
> - The remote process can then parse a tail fragment and wait forever
for the missing head
> - This pull request serializes host writes and makes an unexpected
file gap a loud error
> - The benefit is ordered input with a bounded failure path for sandbox
ACP sessions

## Linked Issues or Issue Description

No public GitHub issue exists for this change. The description below
follows `.github/ISSUE_TEMPLATE/bug_report.yml`.

**What happened?**

A sandbox ACP process-session bridge could stall after its handshake
when a run sent a large prompt. The host sent one un-awaited file write
for each standard input chunk. A small later chunk could finish before a
large earlier chunk. The remote poller then sent the tail bytes first.
The agent parser raised an error on the tail fragment, and the head
bytes stayed buffered without a newline.

**Expected behavior**

The bridge must expose standard input files in sequence. The remote
poller must report a clear error when an earlier file remains missing
beyond the retry budget.

**Steps to reproduce**

1. Start an ACP session through a sandbox process-session bridge.
2. Send a prompt that produces multiple standard input file chunks.
3. Delay finalization of an earlier chunk while a later chunk completes.
4. Observe that the remote parser can receive the later chunk first and
the session can stop without a clear error.

**Paperclip version or commit**

The change targets the current `master` branch at the submitted commit.

**Deployment mode**

The bug affects sandbox execution.

**Agent adapter(s) involved**

The failure affects the ACP process-session bridge.

**Database mode**

Not database-related.

**Additional context**

The fix keeps the existing per-file atomic write behavior. It adds
ordering at the host write boundary and a bounded ordering check in the
shared wrapper poll tail.

## What Changed

- Add a per-session promise chain for host standard input file writes.
- Keep a failed write from blocking later chain entries.
- Track the next expected sequence number in the shared wrapper poll
tail.
- Hold later files while an earlier file is missing within the existing
retry budget.
- Emit a loud error and advance after the retry budget expires.
- Add regression tests for host ordering, gap holding, and the loud
error path.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-stdin-race.test.ts` — 9
tests passed.
- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts` — 43 tests
passed.
- `pnpm --filter @paperclipai/adapter-utils typecheck` — clean.
- With the source fix reverted, the 3 new tests fail and the 6 original
tests pass.
- CI must pass on the pull request before merge.

## Risks

Low risk.

- The host now serializes writes for each session, which can reduce
write parallelism.
- A failed write still emits one error and destroys the socket, as
before.
- The wrapper can emit a loud error after the existing retry budget when
a file gap persists.
- The change does not alter the atomic per-file write behavior.

## Model Used

OpenAI GPT-5, model ID `gpt-5`, with tool use and code execution. The
context window and internal reasoning details are not disclosed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-14 18:45:31 -07:00
Nicky Leach 6f26f2a450
fix(adapter-utils): add per-iteration timeout and watchdog to the sandbox callback bridge poll loop (#11341)
## Thinking Path

> - Paperclip runs AI agents through adapters and sandboxed execution
paths
> - The sandbox callback bridge carries file requests between the host
and a sandbox
> - The poll loop waited forever when a sandbox call stopped responding
> - A permanent wait stranded queued requests and hid the run failure
> - This pull request adds bounded timeouts, abort handling, recovery
backstops, and trace reporting
> - The benefit is prompt request failure, safe mutation outcomes,
run-level error reporting, and trace visibility

## Linked Issues or Issue Description

**What happened?**

The sandbox callback bridge could wait forever when a client call
stopped responding without a rejection.

**Expected behavior**

The bridge should fail queued requests and report a run-level error when
the sandbox channel stops responding.

**Steps to reproduce**

1. Start a sandbox callback bridge.
2. Queue a request.
3. Make the sandbox call stop responding.
4. Observe that the request does not receive a failure response.

**Paperclip version or commit**

Commit `edc4f71b460c600f97cf44cb486d5cac72ca2db9`.

**Deployment mode**

Built from source.

**Installation method**

Built from source with pnpm.

**Agent adapter(s) involved**

Custom or external sandbox callback bridge.

**Database mode**

Not database-related.

## What Changed

- Add a per-iteration timeout for `listJsonFiles` and
`processRequestFile`.
- Add a watchdog that fails pending requests when the loop makes no
progress.
- Abort a hung handler and use a non-retryable 504 backstop when its
outcome can be indeterminate.
- Retry recovery writes and keep queued requests when a recovery write
fails.
- Forward the indeterminate-outcome header through the execution target.
- Record worker failures through the
`sandbox.callbackBridge.workerFailed` trace span.
- Add tests for timeout, watchdog, recovery, mutation safety, header
forwarding, and fast-request behavior.

## Verification

- Run `pnpm exec vitest run
packages/adapter-utils/src/sandbox-callback-bridge.test.ts
packages/adapter-utils/src/execution-target-sandbox.test.ts`.
- Confirm that the PR test, typecheck, build, end-to-end, serialized
test, and security checks pass.
- Confirm that the current PR head is
`edc4f71b460c600f97cf44cb486d5cac72ca2db9`.
- Confirm that the PR changes four files: the callback bridge, its
tests, the execution target, and its tests.

## Risks

The default timeout can fail a slow but valid sandbox call. The defaults
remain configurable, and the iteration timeout stays below the sandbox
response deadline. A mutation that may have committed returns a
non-retryable 504 outcome so the caller does not apply it twice.

## Model Used

OpenAI Codex, GPT-5 current runtime, with extended reasoning and tool
use. The exact context window is not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-14 18:26:52 -07:00
Jannes Stubbemann 8a5c0615f9
fix(adapter-utils): forward sandbox callback bridge traffic to the local listen origin (#10017)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents can execute in remote sandboxes, where a callback bridge
relays in-sandbox Paperclip API calls back to the host server process
> - The bridge worker resolves its forward target from
PAPERCLIP_RUNTIME_API_URL / PAPERCLIP_API_URL, which now prefer a
configured public base URL and therefore mean "the origin browsers and
external agents use"
> - The bridge worker runs inside the same process that serves the API,
so forwarding through the public origin routes an in-process loopback
hop through the network edge
> - On a deployment whose public origin sits behind a session-gated edge
proxy, every forwarded agent API call is rejected at the edge, so agents
in sandboxes cannot read their identity, comment, or hire
> - This pull request resolves the bridge forward target from the
explicit hostApiUrl override or the local listen host and port only,
never the public URL exports
> - The benefit is that sandbox agent API calls keep working regardless
of how the public base URL is configured or gated

## Linked Issues or Issue Description

No existing issue. Describing in-PR following the bug report template:

**What happened?**
On a cloud deployment with a session-gated public edge, setting a public
base URL (PAPERCLIP_PUBLIC_URL) caused every in-sandbox agent API call
through the sandbox callback bridge to fail with `403 text/plain "Access
denied"` from the edge proxy. With PAPERCLIP_BRIDGE_DEBUG enabled, the
bridge logs show the forward target is the public origin, and every
proxied request (for example `GET /api/agents/me`) returns the edge
proxy's 403 instead of reaching the API.

**Expected behavior**
The bridge worker runs in the same server process that serves the API,
so forwarded calls should target the local listen origin and succeed
regardless of how the public origin is configured or gated.

**Steps to reproduce**
1. Run the server with a public base URL configured, fronted by a proxy
that requires a browser session on API routes.
2. Start a sandbox-executed agent run (any adapter using the sandbox
callback bridge).
3. Observe every in-sandbox call to the Paperclip API fail with the
proxy's 403; with PAPERCLIP_BRIDGE_DEBUG the forward URL is the public
origin.

**Paperclip version or commit**
Current `master`.

**Deployment mode**
Self-hosted server behind a reverse proxy.

**Agent adapter(s) involved**
All sandbox-executed adapters (the bridge is adapter-agnostic).

## What Changed

- `packages/adapter-utils/src/execution-target.ts`:
`startAdapterExecutionTargetPaperclipBridge` now resolves its forward
target as `input.hostApiUrl?.trim() || resolveDefaultPaperclipApiUrl()`.
It no longer consults `PAPERCLIP_RUNTIME_API_URL` / `PAPERCLIP_API_URL`,
which now describe the public origin for browsers and external agents,
exactly the wrong target for an in-process loopback hop.
`resolveDefaultPaperclipApiUrl()` builds
`http://<PAPERCLIP_LISTEN_HOST>:<PAPERCLIP_LISTEN_PORT>` (exported by
server boot before any run executes) and maps wildcard listen hosts to
the loopback address of the same family (`0.0.0.0` to `127.0.0.1`, `::`
to `[::1]`), so the forward target always matches the address family the
server is bound to. `input.hostApiUrl` remains the explicit override
seam. A comment documents the reasoning.
- `packages/adapter-utils/src/execution-target-sandbox.test.ts`: two new
tests. One sets both public URL env vars to an unreachable public https
origin and asserts the bridge forwards to the local listen origin (fails
before this fix with a 502 because the worker targets the public
origin). One asserts an explicit `hostApiUrl` input still overrides
everything.
- The acpx-engine bridge start
(`packages/adapter-utils/src/acpx-engine/execute.ts`) passes no
`hostApiUrl` and goes through the same resolution site, so it is covered
by the same fix. The sandbox-facing env builder in `server-utils.ts` is
intentionally untouched; the bridge env overrides `PAPERCLIP_API_URL`
inside the sandbox separately.

## Verification

- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts` (28 tests
pass; the new local-origin test fails without the fix)
- `pnpm --filter @paperclipai/adapter-utils typecheck` (clean)
- Full adapter-utils suite run; the only failures are pre-existing
environment-dependent tests (bubblewrap and shallow-clone tests on
macOS) identical on a clean `master` checkout

## Risks

- Low risk. Deployments where the bridge previously worked did so
precisely because the forward target already resolved to the local
origin (no public URL configured, so the chain fell through to the same
`resolveDefaultPaperclipApiUrl()` result). The only behavioral shift is
for deployments with a public URL configured, where forwarding through
the edge was either wasteful (an unnecessary network round trip) or
broken (session-gated edge). The explicit `hostApiUrl` override seam is
preserved for callers that need a nonlocal target.

## Model Used

- Claude Fable 5 (claude-fable-5), extended thinking, via Claude Code

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 16:09:26 -07:00
Nicky Leach 04bf7a6ab5
feat(observability): instrument stage.sync host steps and home the agent process span (#11301)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - It runs each agent in a remote sandbox and emits OpenTelemetry spans
for the sandbox bring-up and the run.
> - A real trace showed two gaps. `stage.sync` had about 3 seconds of
unattributed host work before its `pack` span. The persistent agent
process showed a `sandbox.exec` span that outlived its parent by about
50 seconds.
> - The gaps hide real cost and make the trace read as a sequencing bug,
so an operator cannot see where startup time goes.
> - This pull request wraps the two pre-`pack` host steps in their own
spans. It also homes the long-lived process in a run-scoped
`sandbox.agentProcess` span.
> - The benefit is that startup time is fully attributed and the process
reads as a resource that overlaps the turn, not a child that outlives
its parent.

## Linked Issues or Issue Description

No public issue exists. This is an enhancement to existing telemetry. It
is described inline below, following
`.github/ISSUE_TEMPLATE/enhancement.yml`. Prior related work: the merged
PR #10999 added the run-time wrapper spans and the telemetry
data-contract section this PR extends.

**What existing behavior does this improve?**

The sandbox bring-up and run OpenTelemetry trace. It closes two
attribution gaps in that trace.

**Subsystem affected**

Observability for sandbox execution. The code lives in
`packages/adapter-utils`. The span contract lives in
`packages/shared/src/telemetry`.

**Current behavior**

`stage.sync` opens a `pack` span, but the git enumeration and the
baseline content-hash walk that run before `pack` have no span, so about
3 seconds read as a gap. On the streamed process-session path the agent
process launches fire-and-forget inside the ~2.3 second
`bridge.process-session` bring-up step, so its `sandbox.exec` span
parents to that step and then runs about 50 seconds. The child dangles
past its parent and overlaps `agent.turn`.

**Proposed behavior**

Wrap the two pre-`pack` host operations in `snapshot.git` and
`snapshot.baseline` spans under `stage.sync`. Wrap the streamed launch
in a run-scoped `sandbox.agentProcess` span that parents to the live run
root (`task.run` at launch).

**Reason and benefit**

Startup time is fully attributed. The long-lived process reads as a
resource that overlaps the sibling `agent.turn`, not a mis-parented
child.

**Breaking changes**

None. The spans are opt-in and export only when an OTLP endpoint is
configured. The span seam is a no-op when no runner is injected. No
first-party telemetry event changes.

## What Changed

- `sandbox-managed-runtime.ts`: add `snapshot.git` and
`snapshot.baseline` spans around the git enumeration and the baseline
content-hash walk, nested under `stage.sync`, through a shared
`runStepSpan` helper that `pack` now also uses.
- `execution-target.ts`: wrap the fire-and-forget streamed launch in a
run-rooted `sandbox.agentProcess` span, so it parents to the live run
root and holds the inner `sandbox.exec`. The `.then`/`.catch` chain
became try/catch inside the span callback, with identical
frame-ingestion behavior.
- `packages/shared/src/telemetry/README.md`: update the span table and
the parenting prose. Add `snapshot.git`, `snapshot.baseline`, `pack`,
and `sandbox.agentProcess`, and document the intended
`sandbox.agentProcess` / `agent.turn` overlap.
- Tests: update the executor span-tree test (`childNames` and parent
assertions), update the `sandbox-managed-runtime` span-set and nesting
tests, and add two `execution-target-sandbox` tests (the launch opens
`sandbox.agentProcess`; it parents to the run root, not the bring-up
step).

## Verification

- Run `npx vitest run` on the three affected test files. Result: 174
tests pass. This includes the updated executor span-tree test and the
new `sandbox.agentProcess` open and parenting tests.
- Run `tsc --noEmit` in `packages/adapter-utils`. Result: no errors in
the changed source or test files.
- The full 37-test streamed process-session suite passes unchanged. This
confirms the try/catch restructure preserves frame delivery and
exit/error behavior.
- Pre-existing and unrelated to this PR (present on `master`): `tsc`
errors in `execute.ts` / `execute.test.ts` /
`remote-spawn-smoke.test.ts` (`onAgentStderr` / `spawnCwd`), and a
`check:forbidden-tokens` failure from internal `PAP-###` ids in
`ui/src/components/IssueRecoveryActionCard.test.tsx`. This PR does not
touch those files, and its own diff is token-clean.

## Risks

Low. The change adds instrumentation on the opt-in span path and does
not change control flow on the default path. The one production
restructure is the streamed launch, which stays fire-and-forget, so
bring-up does not block on it. Only the streamed path gains
`sandbox.agentProcess`; the legacy poll path launches the process
detached and has no host-side long-lived span to home.

## Model Used

Anthropic Claude Opus 4.8 (`claude-opus-4-8`), about 200K-token context,
agentic tool use through Claude Code. The trace was reviewed through the
Honeycomb MCP. The code was written and tested with the model.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-12 16:06:05 -07:00
Nicky Leach 8f478242f1
fix(adapter-utils): close sandbox stdin file race with atomic write and fault-tolerant poller (#11235)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters use execution targets to exchange input and output
with sandbox processes
> - The sandbox input path can expose partial files, and the poller can
delete or stop on invalid input
> - These timing windows can lose agent input without a clear error
> - This pull request makes host writes atomic and makes the poller
retry invalid files before it drops them
> - The benefit is reliable sandbox input delivery with visible failure
after bounded retries

## Linked Issues or Issue Description

Closes #10874

## What Changed

- Decode host input into a temporary file, then rename it onto the final
JSON path.
- Apply the same atomic write pattern to the filesystem client.
- Parse each input file before deletion.
- Retry parse failures and drop a file after the bounded retry limit
with an error event.
- Add regression tests for empty, partial, and permanently malformed
input files.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/execution-target-stdin-race.test.ts` passes
5/5 tests.
- Related sandbox callback, execution target, and sandbox execution
suites pass 71/71 tests.
- TypeScript checks pass for the changed files.
- The regression suite fails on the old code and passes on this change.

## Risks

- Low risk. The change affects sandbox input file handling and adds
bounded retry behavior.
- A permanently malformed file now creates an error event after the
retry limit.

> This bug fix does not add a core feature, so a roadmap change is not
needed.

## Model Used

Codex, OpenAI GPT-5, current agent runtime, large context window, tool
use and code review support.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-11 16:05:41 -07:00
Nicky Leach d5208d30c1
feat(adapter-utils): add host-side pack span to managed-runtime tarball build (#11072)
## Thinking Path

> - Paperclip runs AI agents through adapter execution services.
> - The adapter runtime records spans for each stage of agent startup.
> - The managed runtime packs workspace tarballs before it uploads them.
> - These pack operations had no host span, so `stage.sync` omitted pack
time.
> - This pull request adds one `pack` span around both tarball builds.
> - The span nests under the active `stage.sync` step and improves trace
detail.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The managed runtime workspace sync builds a git-history tarball and a
workspace-overlay tarball before upload.

**Subsystem affected**

The change affects `packages/adapter-utils`, which provides adapter
execution and managed runtime support.

**Current behavior**

The host builds both tarballs without an OpenTelemetry span. The
`stage.sync` trace therefore omits the host pack duration.

**Proposed behavior**

The host wraps both tarball builds in one `pack` span. The executor
parents this span under the active startup step.

**Reason and benefit**

The trace shows the time that the host spends packing workspace data.
Operators can use the existing runtime span tree to find sync delays.

**Breaking changes**

None. The default span runner remains a no-op runner, and the existing
control flow remains unchanged.

## What Changed

- Add an optional `runtimeSpan` runner to the managed runtime
preparation path.
- Create one host `pack` span around the two workspace tarball builds.
- Parent the `pack` span under the active startup step.
- Add unit coverage for span emission and span nesting.

## Verification

- Run `tsc --noEmit` for `@paperclipai/adapter-utils`.
- Run the `@paperclipai/adapter-utils` Vitest suite.
- Confirm the suite reports 448 passed tests and 4 skipped tests.
- Confirm the trace test records `pack` under `stage.sync`.

## Risks

The change adds optional tracing only. The default no-op runner
preserves behavior when tracing is not configured.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. The model reviewed the
handoff and opened this pull request. The implementation author supplied
the commit.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-07 17:47:30 -07:00
Nicky Leach 6b7e0814a0
feat(acp): stream Daytona sandbox agent output and remove the host output poll (#11049)
## Thinking Path

> - Paperclip is the open source app that manages AI agents for work
> - Sandbox providers let agents run in remote and isolated environments
> - Daytona session commands need a path that sends agent output to the
host without host polling
> - Host polling adds delay and repeats provider output work
> - This pull request adds typed execute.log notifications and a log
sink for incremental output
> - This pull request adds an optional ACP session stream with
final-result replay protection
> - The benefit is lower output delay while the default flags keep
current behavior unchanged

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting. The change spans the plugin SDK, Daytona provider,
adapter utilities, and server execution services.

**Problem or motivation**

The Daytona ACP bridge polls a host output file while an agent command
runs. This adds delay and can repeat work. The host also needs a safe
route for provider output chunks.

**Proposed solution**

Add a typed `execute.log` notification with host-issued invocation
correlation. Add an ordered log sink to the environment execute path.
Add an optional ACP session-log path that parses newline-delimited JSON
frames and removes the host output poll for that path.

**Alternatives considered**

Keep the output-file poll as the only path. This keeps the current
behavior but does not provide timely output. The new path stays behind
flags, so the existing path remains the default fallback.

**Roadmap alignment**

This change supports the shipped Cloud / Sandbox agents milestone in
`ROADMAP.md`, including Daytona support.

## What Changed

- Add the typed `execute.log` worker-to-host notification and
company-scoped host route.
- Add ordered `stdout` and `stderr` chunk delivery before the final
execute result.
- Add the Daytona session log sink and the optional ACP streamed session
path.
- Add monotonic frame handling so live and final output reach the host
once.
- Keep `useLogStream` and `streamAgentSessionOutput` off by default.
- Add unit and integration coverage for the notification, execution
target, runtime, and Daytona paths.

## Verification

- Run adapter-utils tests: 445 tests pass locally.
- Run server environment tests: 73 tests pass locally.
- Run Daytona plugin tests: 131 tests pass locally.
- Run TypeScript checks for shared, adapter-utils, and server.
- Review the pull request checks after GitHub completes them.
- All required GitHub checks pass on the current head.

## Risks

The new paths change output delivery only when a feature flag enables
them. The final execute result remains available for parsing and
fallback. The main risk is a provider stream or frame-order error; the
final-result parser limits that risk.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. The runtime did not
supply a context-window value.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes

No operator documentation change applies because both new flags remain
disabled by default.
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-07 14:29:58 -07:00
Nicky Leach 9ace548fd2
feat(observability): rename sandbox provider spans and add run-time wrapper spans (#10999)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip uses adapter and sandbox code to start agents and run
sandbox work
> - The current sandbox spans use mixed names and do not group related
run-time work
> - Mixed names make traces harder to read and compare across providers
> - This pull request renames provider spans, adds run-time wrapper
spans, and keeps the host allowlist closed
> - The benefit is clearer traces with the same sandbox behavior and
trust boundary

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves OpenTelemetry span names and grouping for sandbox startup,
execution, callback relay, and agent session work.

**Subsystem affected**

Cross-cutting (multiple of the above): adapter utilities, sandbox
providers, shared telemetry documentation, and server instrumentation.

**Current behavior**

Sandbox provider spans use mixed names. Related run-time operations
expose inner `sandbox.exec` spans without a named wrapper span. The host
mapper uses a closed allowlist for provider span names.

**Proposed behavior**

Use descriptive provider-scoped span names. Add wrapper spans for agent
session input, agent session output polling, and callback relay. Keep
the host mapper allowlist closed and map unknown names to `other`.

**Reason and benefit**

Clear names make traces easier to read and reduce ambiguity during
sandbox operation analysis. Wrapper spans show the full operation while
preserving the inner execution spans.

**Breaking changes**

None. This change updates telemetry span names and grouping only. It
does not change sandbox behavior, endpoint behavior, or the host trust
boundary.

**Additional context**

Related prior work:
[#10758](https://github.com/paperclipai/paperclip/pull/10758).

## What Changed

- Rename Daytona provider sync and session spans with descriptive
provider-scoped names.
- Add three run-time wrapper spans for agent session input, output
polling, and callback relay.
- Add a shared span runner that preserves no-op behavior without a real
tracer.
- Keep the host mapper allowlist closed and map unknown names to
`other`.
- Update telemetry documentation and span-name tests.

## Verification

- Focused adapter-utils span tests pass for startup timing, callback
relay, and sandbox execution.
- Focused Daytona plugin span tests pass for renamed leaf spans and
session open or close spans.
- Focused server tests pass for host mapping and instrumentation.
- The stacked diff contains one commit on top of
`feat/daytona-persistent-session-model`.

## Risks

- Span names change for existing telemetry consumers.
- The wrapper spans add trace structure but do not change sandbox
execution.
- The host mapper keeps the existing closed allowlist and `other`
bucket.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI GPT-5 (Codex agent); exact deployment revision and context window
are not exposed in this run; tool use and code execution enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-06 22:31:15 -07:00
Nicky Leach c647b8cc2e
feat(acpx-engine): give sandbox.exec spans real parents (#10852)
## Thinking Path

> - Paperclip uses spans and traces to show how work moves through
agents and tools
> - sandbox.exec spans need a real parent so the trace tree matches the
work tree
> - Wrong parent links make execution history hard to read and hard to
debug
> - This pull request adds a single task.run root span and re-parents
live work to the nearest active span
> - The change keeps detached work under the closest live span instead
of the HTTP root
> - The benefit is a clear trace tree for sandbox.exec work and better
execution diagnosis

## Linked Issues or Issue Description

**What happened?**

sandbox.exec spans attached to the wrong parent or to no live parent in
some paths.

**Expected behavior**

Each sandbox.exec span should attach to the nearest live span.

**Steps to reproduce**

1. Run work that creates sandbox.exec spans during startup and callback
bridge paths.
2. Inspect the trace tree.
3. Observe an orphaned span or a span with the wrong parent.

**Paperclip version or commit**

`672e9de9c8b004aebc1f08e24b612ab067735ad1`

**Deployment mode**

Local dev.

**Additional context**

The branch adds the task.run root span, parents sandbox.startup to it,
and re-parents detached bridge work to the nearest live span.

## What Changed

- Added a task.run root span for the run tree.
- Re-parented sandbox.startup, agent.turn, and detached bridge work to
the nearest live span.
- Added end-to-end trace-tree assertions for the full parent chain.
- Added negative coverage so sandbox.exec does not parent to the HTTP
root.

## Verification

- Focused Vitest suite passed:
`packages/adapter-utils/src/acpx-engine/execute.test.ts`,
`packages/adapter-utils/src/acpx-engine/startup-timing.test.ts`,
`packages/adapter-utils/src/execution-target-sandbox.test.ts`,
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, and
`server/src/__tests__/environment-execution-target.test.ts`.
- Result: 5 files passed, 204 tests passed.
- The submitted branch also reported `adapter-utils` checks, `server`
seam checks, and `tsc` exit 0 in the handoff state.

## Risks

- This change can alter trace tree shape in tools that read parent
spans.
- A missed bridge path could still point to the wrong live span.
- Low risk for runtime behavior, because the change only changes span
parent attribution.

## Model Used

OpenAI GPT-5, tool-use capable.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-04 15:32:41 -07:00
Nicky Leach 2ab797dcbe
fix(sandbox): reset step store for long-lived bridge work (#10813)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Bridge workers keep startup setup separate from long-lived runtime
work
> - The callback bridge must keep queue-directory setup inside the
startup step
> - The long-lived poll loop must run with no active startup step
> - This pull request keeps that boundary in the right place
> - The benefit is correct span parents and correct runtime exec
metadata

## Linked Issues or Issue Description

**Bug**

**What happened?**
Long-lived bridge continuations kept a stale startup step store during
the queue-directory setup path.

**Expected behavior**
Runtime exec spans should start with no active startup step.

**Steps to reproduce**
1. Start a bridge lane.
2. Let the startup step end.
3. Run later runtime exec work on the same lane.

**Paperclip version or commit**
223068e2ff

**Deployment mode**
Self-hosted server

## What Changed

- Added `runWithoutActiveStep` in
`packages/adapter-utils/src/acpx-engine/startup-timing.ts`.
- Wrapped the long-lived poll timer, socket handlers, and
callback-bridge worker loop in both bridge lanes.
- Added unit tests for store leak and reset behavior.
- Added continuation tests for both bridge lanes and the `criticalPath`
flag.

## Verification

- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`
- `pnpm exec vitest run
packages/adapter-utils/src/acpx-engine/startup-timing.test.ts`
- `pnpm exec vitest run
server/src/__tests__/environment-execution-target.test.ts`

## Risks

- Low risk.
- The change alters async context handling in bridge continuations.
- If a caller depends on inherited step state, this change removes it.
- The tests cover the intended bridge lanes.

## Model Used

OpenAI Codex, GPT-5, tool use enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-03 23:55:56 -07:00
Nicky Leach 53bcf3897f
feat: sync @-mentioned projects into remote sandboxes (confined sandbox transport, flag ON) (#10564)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The run layer must move project context into the sandbox that
executes the agent
> - Local sandboxes already stage referenced projects for @-mentions
> - Remote confined sandboxes dropped the whole referenced set, so the
agent lost needed files and paths
> - This pull request keeps the confined sandbox transport aligned with
the local behavior for referenced projects
> - It does this behind a remote-only flag that defaults on, while SSH
keeps the old drop-only path
> - The benefit is that remote runs can read the same referenced project
context that local runs already provide

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting. This change touches server orchestration, sandbox
transport, and observability.

**Problem or motivation**

A run can @-mention another project. Local targets stage each referenced
project and give the agent a path. Remote confined sandboxes dropped the
full referenced set, so the agent could not read those project files or
paths.

**Proposed solution**

Enable referenced-project sync for the confined sandbox transport behind
`PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC_REMOTE`, which defaults on. Keep
the SSH transport out of scope and keep it dropping referenced projects.
Repoint each referenced workspace hint at its staged
`project-<projectId>` sandbox directory. Publish
`PAPERCLIP_WORKSPACES_JSON` on the confined sandbox lane. Count each
per-project remote staging failure as a `staging` failure in the
requested-vs-synced metrics.

**Alternatives considered**

Keep the remote path drop-only. That keeps the gap open. Move the change
into SSH too. That expands scope beyond the target transport and adds
risk.

**Roadmap alignment**

No matching item in `ROADMAP.md` showed up in this review.

**Additional context**

The change lands in three commits. The first commit opens the gate for
the confined sandbox transport. The second commit repoints the workspace
hints and publishes the workspace map. The third commit records
per-project staging failure data.

## What Changed

- Opened remote referenced-project sync for the confined sandbox
transport behind `PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC_REMOTE`.
- Repointed referenced workspace hints to the staged
`project-<projectId>` sandbox directories and published
`PAPERCLIP_WORKSPACES_JSON`.
- Counted per-project remote staging failures as first-class `staging`
failures in the requested-vs-synced observability.

## Verification

- The pushed ref
`refs/heads/feat/sync-referenced-projects-remote-sandbox` resolves to
the authorized submit SHA.
- `git log --oneline
origin/master..origin/feat/sync-referenced-projects-remote-sandbox`
shows exactly the three expected commits.
- The handoff reports server typecheck clean, adapter-utils typecheck
clean, and the listed unit tests passing.
- The handoff also reports no open review comments and no Greptile score
yet.

## Risks

- The change touches authorization and sandbox path handling, so
regressions could block remote runs or expose the wrong project context.
- The new flag defaults on, so any bug in the remote path affects normal
remote use.
- SSH stays out of scope, so the two transport paths must remain
distinct.

## Model Used

OpenAI GPT-5 via Codex. Tool use enabled. Context window not reported in
this run.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-31 13:31:17 -07:00
Nicky Leach 674d71548a
perf(sandbox): fold dead sandbox start round trips (bridge dirs + handle-cache seed) (#10485)
## Thinking Path

> - Paperclip is the control plane for AI agents.
> - Sandbox startup uses bridge directories and a Daytona workspace
handle.
> - The cold path made repeat directory creation calls and one avoidable
handle fetch.
> - Those calls add delay but do not change state.
> - This pull request folds the bridge directory setup into one exec,
removes redundant process-session setup, and seeds the Daytona handle
cache at acquire.
> - The benefit is fewer deterministic host-to-sandbox round trips and
faster cold starts.

## Linked Issues or Issue Description

- Problem: Cold sandbox start does extra directory creation work and
re-fetches a handle it already has.
- Expected result: The startup path should create each directory once
and reuse the fresh handle.
- Related PRs I found on GitHub: #9280, #9293.

## What Changed

- Added `makeDirs` to the bridge queue client and used one `mkdir -p`
exec for the callback bridge directories.
- Removed the two upfront `mkdir` execs for the process-session bridge
stdin and events directories.
- Seeded the Daytona sandbox handle cache at acquire so realize can
reuse the fresh handle.
- Reset the process-scoped cache in the compatibility test so the second
sync run sees the expected exec count.

## Verification

- `pnpm --filter @paperclipai/adapter-utils exec vitest run` - 351
passed, 4 skipped.
- `pnpm --filter @paperclipai/sandbox-provider-daytona exec vitest run`
- 91 passed.
- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit` - clean.
- I checked `ROADMAP.md` for sandbox round-trip work. I found no
duplicate planned core work for this change.

## Risks

- Low risk. The change removes redundant calls and adds cache seeding.
- A wrong cache scope would hide the handle. The seed now checks the
lease scope and fails loudly.
- The daytona package `tsc --noEmit` still depends on SDK types that are
not installed in this isolated workspace. CI covers that path.

## Model Used

- OpenAI GPT-5, tool-using, with code execution in the current
workspace.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked public issues or described the issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-29 22:23:45 -07:00
Nicky Leach 6a267e0328
feat(sandbox): stage referenced projects into the run sandbox (#10469)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Runs can reference more than one project
> - Each referenced project must land in its own sandbox tree so one run
does not mix files across projects
> - The anchor workspace must keep its own git history and overlay rules
> - Fail closed on sync and confinement errors, and keep the other
projects alive
> - This pull request stages referenced projects into isolated project
directories under the run sandbox root
> - The benefit is safer multi-project runs with clear failure isolation

## Linked Issues or Issue Description

No public issue exists.

Related PR: #10448.

## What Changed

- Thread additional referenced-project sources through the run prepare
path and the runtime layers.
- Stage each referenced project into its own `project-<projectId>`
directory under the runtime root.
- Keep the anchor workspace history and overlay semantics unchanged.
- Fail closed on confinement or sync errors for one project, and keep
the other projects running.
- Keep the path inert by default behind the multi-project workspace-sync
kill-switch.
- No documentation update was needed for this code-only runtime change.

## Verification

- `pnpm --filter @paperclipai/adapter-utils exec vitest run
src/sandbox-file-sync.test.ts src/command-managed-runtime.test.ts
src/sandbox-managed-runtime.test.ts src/remote-managed-runtime.test.ts`
- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`
- The branch points at `871378e4064a95adc8e4647e442ec1359768de61` on
`origin/feat/stage-referenced-projects-into-sandbox`.
- GitHub CI is green on PR #10469.

## Risks

- A referenced project can skip if confinement or sync fails.
- A new runtime tree layout can affect tools that assume one project
root.
- The kill-switch keeps the path inert until operators enable it.

## Model Used

OpenAI Codex, GPT-5, tool use enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes or
confirmed no docs update was needed
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-29 16:09:57 -07:00
Nicky Leach 9ff7e1caf3
perf(adapter-utils): content-hash-skip process-session remote script write + lock inbound round-trip count (#10377)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The sandbox runtime depends on efficiently staging files into remote
environments
> - The process-session bridge was rewriting a static remote script on
every start, even when the remote copy already matched
> - That caused unnecessary round trips and made the startup path
noisier than it needed to be
> - This pull request adds a hash-skip path for the process-session
script write and keeps the existing bridge-entrypoint hash gate on the
same helper
> - It also locks in the reduced inbound round-trip count so the
collapse from the prior PR cannot silently regress
> - The benefit is fewer remote execs on warm starts and a stronger
guard against performance regressions

## Linked Issues or Issue Description

- Refs: https://github.com/paperclipai/paperclip/pull/10354

### Feature request-style description

**Subsystem affected**
- Cross-cutting (packages/adapter-utils and sandbox staging helpers)

**Problem or motivation**
- The process-session bootstrap path was rewriting a static remote
script on every start even when the remote file already matched the host
content.
- That added avoidable remote execs and latency to warm starts.
- The inbound staging collapse from the previous PR also needed a
regression lock so it could not silently drift back to extra round
trips.

**Proposed solution**
- Route the process-session remote script upload through the existing
hash-skip helper used by the sandbox callback bridge entrypoint.
- Keep the bridge-entrypoint behavior unchanged by delegating it to the
same helper.
- Add a regression test that asserts the inbound staging path still
collapses to the expected round-trip count.

**Alternatives considered**
- Keep the existing unconditional write path and accept the extra execs
on warm starts. Rejected because it preserves avoidable overhead.
- Add a second specialized helper just for process-session scripts.
Rejected because the bridge-entrypoint logic already solved the same
problem and should stay aligned.

**Roadmap alignment**
- This fits the roadmap items around cloud/sandbox agents and enforced
outcomes by reducing bootstrap waste and preventing performance
regressions.
- It does not introduce new product surface area, telemetry, or
user-facing workflow changes.

**Additional context**
- The implementation preserves the existing upload/verify/rename
behavior when the remote content differs and only skips the write when
the hash already matches.
- PR #10354 collapsed the inbound staging path; this PR preserves that
win.

## What Changed

- Added a shared hash-skip helper for remote text-file synchronization
so unchanged content skips the write path after a single remote hash
check.
- Switched the process-session remote script upload to use that helper,
preserving the existing write/verify/rename behavior when the remote
content differs.
- Refactored the sandbox callback bridge entrypoint sync to use the same
helper without changing its observable behavior.
- Added a mocked-native-runner regression test that locks the inbound
staging path to one `client.syncIn` round-trip per step and zero direct
write/run execs.

## Verification

- `tsc --noEmit`
- `pnpm --filter @paperclipai/adapter-utils exec vitest run`
- Full adapter-utils vitest sweep: 338 passed / 4 skipped

## Risks

- Low risk: the helper changes when a write occurs, not the script
contents or the remote execution surface.
- A hash-check failure now fails loudly instead of silently rewriting,
which is safer but could surface provider-side issues earlier than
before.
- The regression test is intentionally specific to the current collapsed
staging path, so future architectural changes will need test updates.

## Model Used

OpenAI Codex (GPT-5, tool-using coding agent)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-28 08:33:12 -07:00
Nicky Leach a3b293e26d
perf(adapter-utils): parallelize the two Daytona sandbox bridge setups (#10334)
## Thinking Path

> - Paperclip runs AI-agent work through adapter-backed execution paths.
> - Sandbox-backed local adapters pay startup overhead in two host-side
bridge setups.
> - Those setups were happening serially even though most of the work is
independent.
> - The remaining dependency is only the merged env that must reach the
process-session launch.
> - This pull request overlaps the independent setup work while
preserving that launch-time dependency.
> - The benefit is lower end-to-end startup latency without changing
execution semantics.

## Linked Issues or Issue Description

No public GitHub issue exists for this change.

Problem / motivation:
- Sandbox-backed local adapter startup waited for two largely
independent bridge setups in series, so users paid roughly the sum of
both setup times.
- The only hard dependency is the merged Paperclip env that must reach
the process-session launch.

Proposed solution:
- Start the paperclip callback bridge and the process-session bridge
concurrently.
- Keep the env merge as the single sequencing point before
process-session launch.
- Stop whichever bridge started if either startup path fails.
- Keep the concurrent bridge telemetry duration-only so shared runner
counters are not double-counted.

Alternatives considered:
- Keep the bridges serial, which preserves the current telemetry shape
but leaves the startup latency unchanged.
- Move env merging earlier, which would complicate launch ordering and
risk changing runtime semantics.

Roadmap alignment:
- This is the approved Daytona start-speedup work, focused on bridge
startup concurrency rather than broader runtime behavior.

## What Changed

- Started the paperclip callback bridge and the process-session bridge
concurrently in `execute.ts`.
- Added a memoized env finalizer so the process-session launch still
waits for the merged paperclip env at the correct moment.
- Updated `execution-target.ts` so the process-session bridge can accept
a deferred env resolver and consume it only at launch time.
- Added tests that cover the overlapped launch path and the
failure-cleanup path when one bridge start fails.

## Verification

- `pnpm --filter @paperclipai/adapter-utils exec vitest run
src/acpx-engine/execute.test.ts`
- `pnpm --filter @paperclipai/adapter-utils test`
- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`
- `git log --oneline
origin/master..origin/perf/parallelize-daytona-bridge-setups`
- `git diff --stat
origin/master...origin/perf/parallelize-daytona-bridge-setups`

## Risks

- A regression in the launch-time env merge could change what the
process-session bridge sees at startup.
- The concurrent start/cleanup logic could leak a bridge if the stop
paths were incorrect, which is why the failure cleanup test matters.
- This is low-to-moderate risk because the change is isolated to
adapter-utils startup plumbing and is covered by targeted tests.

## Model Used

OpenAI Codex, GPT-5, tool-using code execution mode.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-27 15:26:11 -07:00
Dotta 4e00818574
fix(runtime): support in-place workspace realization (#10230)
## Thinking Path

> - Paperclip is the control plane people use to coordinate AI agents
and their execution environments.
> - Environment realization decides where an agent runs and which
filesystem and toolchain are authoritative.
> - Copy-based realization is unsafe for container-anchored tasks
because absolute paths such as `/app` can point outside the synchronized
tree and task-specific binaries may be absent.
> - That mismatch can let an agent successfully verify work in a phantom
writable path while sync-back silently discards the result.
> - Existing task environments already provide the authoritative
filesystem and toolchain, so they should be executed in place rather
than copied.
> - Copy mode still needs explicit confinement rules so aliases target
the synchronized workspace and unsynchronized writable paths fail
visibly.
> - This pull request adds typed realization metadata, propagates the
authoritative root through orchestration, and teaches Codex to honor it.
> - The benefit is that container-anchored tasks operate on
verifier-visible state with the intended tools, while copy mode remains
safe and backward compatible.

## Linked Issues or Issue Description

No public GitHub issue exists for this defect.

GitHub duplicate searches for in-place execution, workspace realization,
and authoritative workspace roots found no related pull request to link.

### What happened?

Environment-backed agent runs were always realized through a copied
workspace. Tasks anchored to absolute container paths could therefore
write outside the synchronized tree, and task-provided toolchains were
unavailable in the copy. A run could report success even though
sync-back discarded its output.

### Expected behavior

Existing task environments should run against their real authoritative
root and toolchain. Copy-mode runs should map declared absolute aliases
into the synchronized tree and reject writable paths that cannot be
restored.

### Steps to reproduce

1. Run a Codex task environment whose required files live under `/app`
or `/workspace` and whose required binary exists only in the task
container.
2. Observe that copy realization changes the effective
filesystem/toolchain or permits writes outside the synchronized root.
3. Complete and verify the task inside the agent sandbox.
4. Observe that the verifier cannot see out-of-tree artifacts or that
task-specific commands were unavailable.

### Reproduction context

- Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924`
- Deployment: built from source in a task-container execution
environment
- Adapter: Codex local
- Database: not database-related
- Access context: agent execution

## What Changed

- Added typed `copy | in_place` workspace-realization metadata,
authoritative roots, confined aliases, and outbound restore paths to
shared execution-target contracts.
- Selected in-place realization for existing task environments and
skipped archive prepare/restore when the authoritative environment is
used directly.
- Propagated the authoritative root into adapter context so Codex uses
it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP
execution.
- Bound copy-mode aliases such as `/app` to the synchronized workspace
and rejected writable out-of-tree paths without explicit restore
mappings.
- Added focused regression coverage while preserving existing copy-mode
archive restore behavior.

## Verification

- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/execute.remote.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed,
4 skipped.
- `pnpm -r typecheck` — passed.
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u
AWS_SESSION_TOKEN pnpm test:run` — passed across all general and
serialized Vitest shards.
- `pnpm build` — passed.
- Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC
with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`,
`openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate`
each received terminal grade 1.0 against real task-environment paths and
toolchains.

## Risks

- In-place mode deliberately exposes the authoritative task root to the
adapter; incorrect environment metadata could point execution at the
wrong root. Typed metadata and focused orchestration tests cover
selection and propagation.
- Copy-mode writable-path validation is stricter and may reject
previously accepted unsafe configurations. The rejection is intentional
and produces a visible error instead of silently losing output.
- The acceptance run is focused on four Codex task-environment
workloads, not a broad cross-adapter benchmark. Existing copy-mode
archive tests and the full repository suite remain green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI coding agent; exact model ID and context-window size
were not exposed to this runtime. Capabilities used: extended reasoning,
repository editing, shell execution, test/build execution, Git, GitHub
CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:29:26 -05:00
Nicky Leach cf5ba4bbea
feat(adapter-utils): generic per-asset lifecycle-contribution seam (#9778)
## Thinking Path

> - Paperclip's sandbox managed runtime is responsible for provisioning
the agent's execution environment — it extracts a home directory asset
into the sandbox before the adapter runs.
> - The sandbox runtime core was directly branching on the adapter key
(`codex`) to decide which merge scripts to stage and which merge-extract
command to run, coupling generic infrastructure to a specific adapter's
credential-merge protocol.
> - This makes it harder to add, remove, or modify per-adapter asset
provisioning without touching the runtime core; it also prevents other
adapters from contributing staged files or a custom extract command at
all.
> - The fix is to move the adapter-specific knowledge into the adapter
itself: the asset descriptor gains optional `provision` (stageFiles +
extractCommand) and `restore` contribution fields that any adapter can
populate, and the runtime core consumes them generically.
> - This pull request introduces those contribution fields, wires the
Codex adapter's inbound credential-merge as a `provision` contribution,
and removes the adapter-specific branching from the runtime core.
> - The benefit is a clean seam: the runtime core is now
adapter-agnostic for asset provisioning, the inbound behavior is
unchanged (same merge matrix, same scripts), and other adapters can
attach custom staged files or extract commands without modifying shared
infrastructure.

## Linked Issues or Issue Description

No pre-existing public GitHub issue. Describing the problem inline per
the feature template:

**Problem or motivation**

The sandbox managed-runtime asset provisioning in
`sandbox-managed-runtime.ts` branched directly on the adapter key
(`codex`) to decide which merge scripts to stage and which shell command
to use during asset extraction. This tight coupling prevents other
adapters from customizing their provisioning without modifying the
runtime core, and it means the runtime core must import and know about
adapter-specific merge scripts.

**Proposed solution**

Add an optional `provision` contribution (array of `stageFiles` entries
+ an `extractCommand` string) and an optional `restore` contribution to
the asset descriptor returned by adapters. The runtime core now consumes
these generically — if a `provision` contribution is present, it stages
those files and uses the supplied command; otherwise it falls back to
the default `tar -xf` extraction. The Codex adapter populates the
`provision` contribution where it previously depended on core branching.

**Alternatives considered**

Keeping the adapter-specific logic in the core as a documented
exception; rejected because it makes the seam inextensible.

**Roadmap alignment**

Decoupling — removes a latent coupling between the runtime core and a
specific adapter.

## What Changed

- Added `provision` contribution field (`stageFiles: Array<{src, dest}>`
+ `extractCommand: string`) to the `SandboxManagedRuntimeAsset`
descriptor type in `adapter-utils`.
- Added `restore` contribution field (hook for post-restore logic,
populated in a later phase) to the descriptor.
- Removed adapter-key branching (`if adapterKey === 'codex'`) from the
runtime core in `sandbox-managed-runtime.ts`; the core now reads
`provision.stageFiles` and `provision.extractCommand` generically.
- Extracted Codex-specific merge-script paths and the merge-extract
command into `codex-auth-merge-scripts.ts` in `adapter-utils`; the Codex
adapter's `execute.ts` now attaches them as a `provision` contribution
when it builds its managed-home asset descriptor.
- Updated `execution-target.ts` to pass the extended asset type through
to the adapter call site so the new fields are load-bearing end-to-end.
- Added seam-proving unit tests in `sandbox-managed-runtime.test.ts`:
contribution-less asset uses the default path; a non-adapter asset
round-trips the generic provision+restore seam; a structural assertion
verifies the runtime core carries no Codex-specific string literals.
- Added one test in `workspace-restore-merge.test.ts` confirming the
inbound merge matrix is unaffected.

## Verification

```bash
# Unit tests (20 pass):
npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/workspace-restore-merge.test.ts

# Type-check both affected packages:
cd packages/adapter-utils && npx tsc --noEmit
cd packages/adapters/codex-local && npx tsc --noEmit

# Structural: runtime core carries no adapter string literals
grep -n 'codex\|auth\.json' packages/adapter-utils/src/sandbox-managed-runtime.ts
# Expected: zero matches
```

## Risks

**Low risk.** This is a behavior-preserving refactor: the inbound
provisioning output (which files get staged, which command runs) is
identical to before, now driven by the adapter-supplied contribution
instead of core branching. The existing inbound merge matrix tests are
the regression guard. No change to which bytes cross the sandbox
boundary. The SSH transport is untouched.

## Model Used

- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200 K tokens
- **Capabilities used:** tool use (file read/edit, bash execution,
Paperclip API), extended reasoning over multi-file TypeScript refactor
- **Mode:** agentic (Paperclip ACPX platform)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Harold Kim <harold@paperclip.ing>
2026-07-17 14:18:01 -05:00
Dotta 98d9360658
feat(adapters): confine local coding processes (#9504)
## Thinking Path

> - Paperclip is the open source control plane people use to manage AI
agents for work.
> - Local coding adapters currently spawn their CLI processes directly
on the Paperclip host.
> - CLI-native approval and sandbox flags do not provide a reliable host
filesystem or network boundary.
> - An agent can therefore inspect unrelated host files or fetch
external material when an operator needs stronger isolation.
> - The confinement must stay opt-in so existing local adapter behavior
does not change unexpectedly.
> - This pull request adds a shared Linux Bubblewrap spawn layer for
workspace filesystem and deny/allowlist network scopes.
> - The benefit is enforceable defense in depth around Codex and Claude
local runs while preserving explicit provider connectivity.

## Linked Issues or Issue Description

### What happened?

`codex_local` and `claude_local` processes could read arbitrary host
paths and make unrestricted outbound network requests because Paperclip
did not impose a spawn-level boundary.

### Expected behavior

Operators can opt into a workspace-only filesystem view and either deny
network egress or allow exact provider/API hosts, independently of CLI
approval flags.

### Steps to reproduce

1. Run current `master` on Linux and configure a Codex or Claude local
adapter.
2. Ask the agent to read a canary file outside its active workspace.
3. Ask the agent to `curl` a public host.
4. Observe that both operations succeed without a Paperclip-level
confinement option.

### Environment

- Paperclip commit: `c36f1a4af` / current `master` base.
- Deployment mode: Linux local dev or self-hosted server.
- Installation: built from source.
- Adapters: Codex and Claude Code.
- Database: not related.

## What Changed

- Added a shared Bubblewrap process wrapper with opt-in
`filesystemScope: "workspace"`, managed/extra path mounts, private
`/tmp`, and Linux-only validation.
- Added `networkScope: "deny" | "allowlist"`; both use a private network
namespace, while allowlist mode exposes an exact-host HTTP(S) proxy over
a Unix-socket bridge.
- Wired Codex and Claude local CLI execution through the wrapper and
forced scoped auto runs onto the CLI lane because ACP processes are not
covered.
- Added unit and gated Bubblewrap canaries for outside-file denial,
workspace writes, direct network denial, allowlisted forwarding, and
rejected destinations.
- Documented both scopes, provider allowlist examples, Bubblewrap
requirements, and default-off behavior.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/claude-local/src/server/acp.test.ts` — 34 passed, 4
gated Bubblewrap tests skipped by default.
- `pnpm --filter @paperclipai/adapter-utils typecheck`
- `pnpm --filter @paperclipai/adapter-codex-local typecheck`
- `pnpm --filter @paperclipai/adapter-claude-local typecheck`
- `pnpm --filter @paperclipai/adapter-utils build`
- `pnpm --filter @paperclipai/adapter-codex-local build`
- `pnpm --filter @paperclipai/adapter-claude-local build`
- Attempted the gated tests with a vendored Bubblewrap binary; this
container blocks unprivileged namespace setup (`setting up uid map:
Permission denied` / loopback `RTM_NEWADDR: Operation not permitted`),
so kernel-level execution remains for CI or a namespace-enabled Linux
host.

## Risks

- Bubblewrap must be installed and unprivileged user/mount/network
namespaces must be enabled on the host; scoped runs fail clearly if the
prerequisite is missing.
- Allowlist mode depends on the coding CLI honoring standard
`HTTP_PROXY` / `HTTPS_PROXY` variables; custom providers must list every
required exact hostname and port.
- Exact-host allowlists intentionally reject wildcards, which is safer
but may require operators to enumerate multi-host provider setups.
- No behavior changes unless an operator enables `filesystemScope` or
`networkScope`.

> This aligns with the ROADMAP direction toward safer remote and
sandboxed agent environments and does not duplicate an open PR or issue
found in the repository search.

## Model Used

- OpenAI GPT-5.5 (`gpt-5.5`) via Codex CLI, with reasoning, repository
tool use, shell execution, code editing, and test execution. The serving
context-window size is not exposed to the agent.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 11:01:19 -05:00
Devin Foley 9cde4e128c
feat: run ACP sessions in sandbox execution targets (#9390)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters (Claude, Codex, Gemini) default to the ACP engine
lane, which needs a live bidirectional stdio session with the agent
process
> - Sandbox execution targets only exposed one-shot command execution,
so every ACP-capable adapter refused remote targets and fell back to the
CLI lane with a "supports only the local Paperclip host" warning
> - Running agents in sandboxes is a core deployment mode, and losing
ACP there means losing streaming updates, structured events, and
default-lane parity with local runs
> - This pull request adds a provider-agnostic process-session bridge
that relays the ACP stdio session into the sandbox over the existing
sandbox runner contract, and updates the adapters to use it
> - The benefit is that the default ACP lane now behaves the same on the
local host and in any sandbox provider, with CLI fallback reserved for
targets that genuinely cannot host a bidirectional session

## Linked Issues or Issue Description

No existing public issue covers this; inline description following the
feature request template:

**Problem or motivation**

Configuring an ACP-capable adapter (e.g. Claude) with a sandbox
environment made every run fall back to the CLI lane with the warning
"Claude ACP currently supports only the local Paperclip host, but this
run targets a remote environment." The ACP engine only knew how to spawn
a local subprocess, while sandbox providers only expose one-shot command
execution — so there was no way to hold the bidirectional stdio session
ACP requires.

**Proposed solution**

Add a process-session bridge in `adapter-utils`: a local ACPX-spawnable
proxy script connects to a token-authenticated loopback TCP server,
which relays JSON-framed stdin/stdout/stderr events to and from a small
relay script executed inside the sandbox via the provider's ordinary
runner. Claude/Codex/Gemini adapters now treat sandbox targets with a
runner as ACP-capable, resolve agent commands against the remote target,
and fall back to CLI only when the sandbox exposes no bidirectional
path. The sandbox callback bridge injects a run-scoped API endpoint and
bridge token so the agent inside the sandbox can reach Paperclip
(including work-product handoffs) without ever receiving the host run
JWT.

**Alternatives considered**

A provider-specific lane was prototyped first: Daytona minting SSH
access metadata at lease time, converted into an SSH execution target.
It was dropped because it only worked for providers able to advertise
SSH, added per-provider surface area, and left every other sandbox
provider on the CLI fallback. The merged design rides the one-shot
runner contract all providers already implement; a regression test pins
that sandbox targets stay on the bridge lane even when lease metadata
advertises SSH access.

**Roadmap alignment**

Directly advances the "Cloud / Sandbox agents" roadmap item — agents
running in remote and sandboxed environments keep the same control-plane
behavior as local ones. No overlap with other planned core work.

## What Changed

- `packages/adapter-utils/src/execution-target.ts`: new
`startAdapterExecutionTargetProcessSessionBridge()` plus helpers —
writes a token-authenticated local proxy script (spawnable by ACPX) and
a remote relay script synced into the sandbox, with a loopback TCP
server streaming JSON-framed stdio between them; events emitted before
the ACP client attaches are buffered so none are lost.
- `packages/adapter-utils/src/acpx-engine/execute.ts`: the ACP engine
can execute against remote sandbox targets through the bridge instead of
requiring a local subprocess, including remote cwd/env shaping.
- `packages/adapter-utils/src/sandbox-callback-bridge.ts`:
sandbox-scoped API bridging extended to allow work-product handoffs; the
sandbox payload env carries a bridge token, never the host run JWT.
- `packages/adapters/claude-local`, `codex-local`, `gemini-local`
(`src/server/acp.ts`): default-lane selection no longer rejects all
remote targets; command resolution is remote-aware
(`ensureAdapterExecutionTargetCommandResolvable`,
`resolveAdapterExecutionTargetCwd`); the fallback reason is now scoped
to sandboxes that expose only one-shot execution.
- `server/src/__tests__/environment-execution-target.test.ts`: pins that
sandbox targets resolve to the bridge lane, including when lease
metadata advertises SSH access.
- Non-sandbox remote targets (e.g. SSH) keep the CLI lane: the ACP
engine's remote transport is sandbox-only, so default-lane selection
falls back for those targets across all three adapters, and tests
covering CLI-specific remote behavior pin `engine: "cli"` explicitly.
- The bridge authenticates loopback connections before they can own the
session or receive buffered output (token required, idle unauthenticated
peers dropped), and remote event writes are serialized so the exit event
always lands after stdout/stderr have drained.
- Daytona plugin: formatting-only residue from the earlier iteration; no
functional change.

## Verification

- `vitest run` over the touched suites —
`packages/adapter-utils/src/acpx-engine/execute.test.ts`,
`packages/adapter-utils/src/execution-target-sandbox.test.ts`,
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, the three
adapter `acp.test.ts` files, and
`server/src/__tests__/environment-execution-target.test.ts` — 102 tests
pass.
- End to end: with a Claude agent configured on a Daytona sandbox
environment, the primary-model test now selects the default ACP lane (no
fallback warning), and the full round trip (wake → sandbox execution →
API bridge → comment post) was exercised twice from inside a live
sandbox.

## Risks

- Behavioral shift: adapters that previously always fell back to CLI on
sandbox targets now default to ACP there; `engine=cli` still pins the
CLI lane explicitly.
- The bridge relays stdio as JSON lines over loopback TCP guarded by a
per-session random token; the remote relay runs inside the sandbox under
the provider's runner. Providers with slow one-shot execution will see
higher session startup latency — the CLI fallback remains for genuinely
incapable targets.
- No schema or migration changes.

## Model Used

- Claude Fable 5 (`claude-fable-5`, Anthropic) — extended thinking
enabled, agentic tool use via the Claude Agent SDK harness;
implementation iterated with local Vitest verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
shipped docs describe the old local-only ACP limitation)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Cody <noreply@paperclip.ing>
Co-authored-by: Cody <cody@paperclip.local>
2026-07-10 17:13:53 -07:00
Devin Foley cc17f29e7e
fix(timeouts): raise sandbox wall-clock backstop to 4h and make acpx_local timeouts self-describing (#9232)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs execute through adapters (e.g. `acpx_local`), which can
run locally, over SSH, or inside sandbox execution targets, each with a
wall-clock execution timeout
> - Sandbox-backed runs defaulted to a 30-minute wall-clock backstop
(`DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800`), which kills
healthy long agent runs that are still making progress — long before the
recovery watchdog's 4h critical threshold would even consider them stuck
> - On top of that, `acpx_local` resolved its timeout directly from
`adapterConfig.timeoutSec` instead of the shared execution-target
resolver, and its timeout failures surfaced as a bare `Timed out after
Ns` — giving operators no clue which timer fired or which knob raises it
> - This pull request raises the sandbox backstop to 4h (aligned with
the recovery watchdog), routes `acpx_local` through the shared timeout
resolver, logs the effective timeout and its source at run start, and
makes every timeout error message self-describing
> - The benefit is that long-running sandbox agent runs no longer die at
30 minutes, and when a wall-clock timeout does fire, the run log states
exactly which timer fired and how to configure it

## Linked Issues or Issue Description

Refs #4535 (related: wall-clock execution timeouts killing agent runs
that are still making progress — that issue covers a different hardcoded
600s timer, but the operator pain is the same).

No exact public issue exists for this one, so describing it in-PR:

**Bug:** A long sandbox-backed `acpx_local` agent run was killed with a
bare `Timed out after 1800s` even though the agent was actively working.

- **What happened:** The run hit the 30-minute
`DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` backstop. `acpx_local`
never consulted the shared execution-target timeout resolution (it read
`adapterConfig.timeoutSec` directly, default 0), so on sandbox targets
the sandbox-provider default applied with no adapter-level say. The
resulting error named neither the timer that fired nor the knob that
controls it.
- **Expected:** Healthy long runs should not be killed by a 30-minute
wall-clock backstop when the recovery watchdog only treats runs as
critically stuck after 4h of output silence; and any timeout error
should say which timeout fired and how to raise it.
- **Impact:** Long, legitimate agent runs in sandboxes fail mid-work;
operators waste time reverse-engineering which of several timers
produced "Timed out after Ns".

## What Changed

- `packages/adapter-utils/src/execution-target.ts`
- `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` raised from `1_800` to
`14_400` (4h), with a comment explaining it intentionally matches the
recovery watchdog's `ACTIVE_RUN_OUTPUT_CRITICAL_THRESHOLD_MS` (4h) so
the adapter backstop never fires before the watchdog path.
Output-inactivity monitors remain the primary hang detectors.
- New `resolveAdapterExecutionTargetTimeout(target,
configuredTimeoutSec)` returns `{ timeoutSec, source }` where `source`
is `configured` / `sandbox_default` / `unlimited`. The existing
`resolveAdapterExecutionTargetTimeoutSec` is preserved as a thin
wrapper, so current callers are unaffected.
- New `formatAdapterExecutionTimeoutErrorMessage(resolution)` and
`formatAdapterExecutionTimeoutStartLogLine(resolution)` produce
self-describing messages that name the timer that fired and the
`adapterConfig.timeoutSec` knob that controls it.
- `packages/adapters/acpx-local/src/server/execute.ts`
- `buildRuntime` now resolves the wall-clock timeout through the shared
resolver: sandbox targets default to the 4h backstop, local/SSH keep the
historical "0 = no adapter timeout", and a configured
`adapterConfig.timeoutSec` always wins.
- The executor logs the effective timeout and its source at run start
(`[paperclip] Adapter execution timeout: …`), so a later timeout is
diagnosable from the run log alone.
- All three bare timeout messages (timer cancel reason, turn result
`errorMessage`, catch-path `messageOverride`) now use the
self-describing format.
- `packages/adapters/acpx-local/src/index.ts` — the adapter
configuration doc for `timeoutSec` states the sandbox default and that
the output-inactivity monitor remains the primary hang detector.
- Tests: `packages/adapter-utils/src/execution-target-sandbox.test.ts`
and `packages/adapters/acpx-local/src/server/execute.test.ts` (see
Verification).

## Verification

- `pnpm --filter @paperclipai/adapter-utils typecheck` — passes
- `pnpm --filter @paperclipai/adapter-acpx-local typecheck` — passes
- `npx vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts
packages/adapters/acpx-local/src/server/execute.test.ts` — 2 files, 40
tests, all pass
- New/updated test coverage:
- sandbox default resolves to 4h (and the constant is asserted to be `4
* 60 * 60`)
- `resolveAdapterExecutionTargetTimeout` reports `configured` /
`sandbox_default` / `unlimited` sources with the correct precedence
(configured > sandbox default; local/SSH stay unlimited)
- exact wording of the self-describing error message and the
start-of-run log line
- `acpx_local` runtime picks up the sandbox default into `timeoutMs`,
keeps the unlimited local default, honors configured-over-default
precedence, emits the start-of-run log line, and surfaces the
self-describing `errorMessage`/cancel reason when the wall-clock timer
kills a turn

## Risks

- **Behavioral shift:** sandbox-backed adapter runs that previously hit
the 30-minute backstop now run up to 4h before the adapter kills them.
Genuinely hung runs are still caught much earlier by the adapters'
output-inactivity monitors and by the recovery watchdog; the wall-clock
timer is a last-resort kill switch. Operators who relied on the
30-minute default can restore it explicitly via
`adapterConfig.timeoutSec`.
- **Error-message consumers:** any tooling that pattern-matched the
exact `Timed out after Ns` string from `acpx_local` will see the new
self-describing message instead.
- No API or schema changes; `resolveAdapterExecutionTargetTimeoutSec`
keeps its exact signature and behavior (modulo the raised sandbox
default).

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude (Anthropic) via Claude Code CLI — model ID `claude-fable-5`,
extended thinking enabled, agentic tool use (file edits, shell, test
execution)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-08 13:50:00 -07:00
Devin Foley c48feee190
Improve live agent feedback during sandboxed runs (#8915)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A core part of that experience is watching active agent runs without
dropping into raw logs first
> - Local and sandbox-backed adapters already record useful run output,
progress, and tool activity
> - But active issue threads could sit visually stale while the agent
was syncing workspaces, tailing sandbox output, or emitting incremental
tool-call updates
> - Operators need timely, human-readable progress while preserving the
raw transcript underneath
> - This pull request streams sandbox run-log progress into runtime
status, keeps visible issue threads refreshed, and folds repeated ACPX
tool updates into stable transcript cards
> - The benefit is that long-running agent work becomes easier to
supervise without changing the task/comment control-plane model

## Linked Issues or Issue Description

No public GitHub issue exists for this exact change.

Problem/motivation:

- During long-running sandboxed agent work, the issue UI can appear idle
even though the agent is actively syncing, running tools, or producing
incremental output.
- Operators need realtime feedback at the issue-thread layer, not only
after opening raw logs or waiting for the final heartbeat result.
- Related public context: #1808 previously added live-run status dots to
Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate
of this runtime/UI feedback change.

## What Changed

- Added sandbox run-log streaming support and defaulted sandbox-capable
local adapters into the richer live-feedback path.
- Surfaced environment/sandbox sync progress through heartbeat runtime
status with bounded, redacted snippets.
- Added live issue-thread cache patching so visible active runs update
as progress events arrive.
- Folded repeated ACPX `tool_call` updates into one transcript card
instead of stacking duplicate cards.
- Updated adapter docs and added focused regression coverage for sandbox
log streaming, runtime status, ACPX parsing, live updates, transcript
rendering, and issue chat messages.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts`
- `pnpm exec vitest run
server/src/services/heartbeat-run-runtime-status.test.ts
server/src/__tests__/heartbeat-runtime-state.test.ts
ui/src/context/LiveUpdatesProvider.test.ts`
- `pnpm exec vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
server/src/services/heartbeat-run-runtime-status.test.ts
server/src/__tests__/agent-live-run-routes.test.ts
server/src/__tests__/heartbeat-runtime-state.test.ts
packages/adapters/acpx-local/src/ui/parse-stdout.test.ts
ui/src/context/LiveUpdatesProvider.test.ts
ui/src/components/transcript/RunTranscriptView.test.tsx
ui/src/lib/issue-chat-messages.test.ts
ui/src/components/IssueChatThread.test.tsx`
- GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`:
`verify`, build, typecheck/release-registry, e2e, general shards,
serialized server shards, and canary dry run passed.
- Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`:
Confidence Score 5/5, no unresolved review threads.

## Risks

- Live issue-thread cache patching could miss an edge case for a route
shape not covered by tests.
- Surfacing active-run snippets needs continued care around redaction;
this PR keeps snippets bounded and adds redaction-focused coverage.
- More frequent active-run UI refreshes could expose performance issues
on very large issue threads, though updates are scoped to visible
run/query caches.

## Model Used

OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with
shell, git, and repository-editing capabilities. Context window size is
not exposed in this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-02 22:21:56 -07:00