Commit Graph

8 Commits

Author SHA1 Message Date
Nicky Leach 6880213de5
fix(adapter-utils): honor .gitignore for referenced-project staging (#12184)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Sandbox adapters stage project files before an agent starts.
> - Referenced projects ignored Git-ignored paths and copied large local
directories.
> - This behavior increased staging time and disk use, and it differed
from anchor workspaces.
> - This pull request resolves Git-ignored paths once and shares that
result across all referenced-project consumers.
> - The benefit is smaller, faster, and consistent project staging.

## Linked Issues or Issue Description

No public GitHub issue exists for this bug.

**What happened?**
Referenced-project staging copied Git-ignored paths, except for a fixed
list of heavy directory names. A large repository therefore used much
more time and disk space than the same repository in an anchor
workspace.

**Expected behavior**
Referenced-project staging should exclude the same Git-ignored paths
that the workspace staging path excludes.

**Steps to reproduce**
1. Create a referenced project with a large Git-ignored directory.
2. Start a sandbox or SSH run that stages the referenced project.
3. Observe that the ignored directory enters the staged content.

**Paperclip version or commit**
Commit `9964b034bbff24e700c8eccf5a8b1fc3daa44bf2`.

**Deployment mode**
Built from source.

## What Changed

- Resolve each referenced project's Git-ignored paths once before
staging.
- Carry the resolved paths as a required field on
`SandboxAdditionalSource`.
- Reuse the resolved paths in sandbox staging, SSH staging, and
content-signature code.
- Harden the read-only Git helper with a bounded process, a reduced
environment, and disabled system and global configuration.
- Fail closed on Git errors, timeouts, and invalid path relations.
- Escape tar glob metacharacters in ignore-derived exclude entries.
- Add and update unit tests for the resolver and its three consumers.

## Verification

- `pnpm vitest run --config packages/adapter-utils/vitest.config.ts`
passes 266 tests locally.
- `pnpm exec tsc --noEmit -p packages/adapter-utils/tsconfig.json`
passes locally.
- CI must pass on this pull request.
- Greptile must report 5/5 with no unresolved comments before merge.

## Risks

- A Git error or timeout now prevents staging for the affected
referenced project.
- The resolver uses a bounded read-only Git process and fails closed by
design.
- The change stays inside `packages/adapter-utils` and does not change
the database schema.

## Model Used

Claude Sonnet 5 (Anthropic) assisted the implementation with code
execution and tool use. The exact context window and reasoning mode are
not recorded.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 14:22:47 -07:00
Nicky Leach 6a267e0328
feat(sandbox): stage referenced projects into the run sandbox (#10469)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Runs can reference more than one project
> - Each referenced project must land in its own sandbox tree so one run
does not mix files across projects
> - The anchor workspace must keep its own git history and overlay rules
> - Fail closed on sync and confinement errors, and keep the other
projects alive
> - This pull request stages referenced projects into isolated project
directories under the run sandbox root
> - The benefit is safer multi-project runs with clear failure isolation

## Linked Issues or Issue Description

No public issue exists.

Related PR: #10448.

## What Changed

- Thread additional referenced-project sources through the run prepare
path and the runtime layers.
- Stage each referenced project into its own `project-<projectId>`
directory under the runtime root.
- Keep the anchor workspace history and overlay semantics unchanged.
- Fail closed on confinement or sync errors for one project, and keep
the other projects running.
- Keep the path inert by default behind the multi-project workspace-sync
kill-switch.
- No documentation update was needed for this code-only runtime change.

## Verification

- `pnpm --filter @paperclipai/adapter-utils exec vitest run
src/sandbox-file-sync.test.ts src/command-managed-runtime.test.ts
src/sandbox-managed-runtime.test.ts src/remote-managed-runtime.test.ts`
- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`
- The branch points at `871378e4064a95adc8e4647e442ec1359768de61` on
`origin/feat/stage-referenced-projects-into-sandbox`.
- GitHub CI is green on PR #10469.

## Risks

- A referenced project can skip if confinement or sync fails.
- A new runtime tree layout can affect tools that assume one project
root.
- The kill-switch keeps the path inert until operators enable it.

## Model Used

OpenAI Codex, GPT-5, tool use enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes or
confirmed no docs update was needed
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-29 16:09:57 -07:00
Dotta 4e00818574
fix(runtime): support in-place workspace realization (#10230)
## Thinking Path

> - Paperclip is the control plane people use to coordinate AI agents
and their execution environments.
> - Environment realization decides where an agent runs and which
filesystem and toolchain are authoritative.
> - Copy-based realization is unsafe for container-anchored tasks
because absolute paths such as `/app` can point outside the synchronized
tree and task-specific binaries may be absent.
> - That mismatch can let an agent successfully verify work in a phantom
writable path while sync-back silently discards the result.
> - Existing task environments already provide the authoritative
filesystem and toolchain, so they should be executed in place rather
than copied.
> - Copy mode still needs explicit confinement rules so aliases target
the synchronized workspace and unsynchronized writable paths fail
visibly.
> - This pull request adds typed realization metadata, propagates the
authoritative root through orchestration, and teaches Codex to honor it.
> - The benefit is that container-anchored tasks operate on
verifier-visible state with the intended tools, while copy mode remains
safe and backward compatible.

## Linked Issues or Issue Description

No public GitHub issue exists for this defect.

GitHub duplicate searches for in-place execution, workspace realization,
and authoritative workspace roots found no related pull request to link.

### What happened?

Environment-backed agent runs were always realized through a copied
workspace. Tasks anchored to absolute container paths could therefore
write outside the synchronized tree, and task-provided toolchains were
unavailable in the copy. A run could report success even though
sync-back discarded its output.

### Expected behavior

Existing task environments should run against their real authoritative
root and toolchain. Copy-mode runs should map declared absolute aliases
into the synchronized tree and reject writable paths that cannot be
restored.

### Steps to reproduce

1. Run a Codex task environment whose required files live under `/app`
or `/workspace` and whose required binary exists only in the task
container.
2. Observe that copy realization changes the effective
filesystem/toolchain or permits writes outside the synchronized root.
3. Complete and verify the task inside the agent sandbox.
4. Observe that the verifier cannot see out-of-tree artifacts or that
task-specific commands were unavailable.

### Reproduction context

- Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924`
- Deployment: built from source in a task-container execution
environment
- Adapter: Codex local
- Database: not database-related
- Access context: agent execution

## What Changed

- Added typed `copy | in_place` workspace-realization metadata,
authoritative roots, confined aliases, and outbound restore paths to
shared execution-target contracts.
- Selected in-place realization for existing task environments and
skipped archive prepare/restore when the authoritative environment is
used directly.
- Propagated the authoritative root into adapter context so Codex uses
it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP
execution.
- Bound copy-mode aliases such as `/app` to the synchronized workspace
and rejected writable out-of-tree paths without explicit restore
mappings.
- Added focused regression coverage while preserving existing copy-mode
archive restore behavior.

## Verification

- `pnpm --filter @paperclipai/shared typecheck` — passed.
- `pnpm exec vitest run
packages/adapter-utils/src/local-process-sandbox.test.ts
packages/adapters/codex-local/src/server/acp.test.ts
packages/adapters/codex-local/src/server/execute.remote.test.ts
server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed,
4 skipped.
- `pnpm -r typecheck` — passed.
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u
AWS_SESSION_TOKEN pnpm test:run` — passed across all general and
serialized Vitest shards.
- `pnpm build` — passed.
- Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC
with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`,
`openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate`
each received terminal grade 1.0 against real task-environment paths and
toolchains.

## Risks

- In-place mode deliberately exposes the authoritative task root to the
adapter; incorrect environment metadata could point execution at the
wrong root. Typed metadata and focused orchestration tests cover
selection and propagation.
- Copy-mode writable-path validation is stricter and may reject
previously accepted unsafe configurations. The rejection is intentional
and produces a visible error instead of silently losing output.
- The acceptance run is focused on four Codex task-environment
workloads, not a broad cross-adapter benchmark. Existing copy-mode
archive tests and the full repository suite remain green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex CLI coding agent; exact model ID and context-window size
were not exposed to this runtime. Capabilities used: extended reasoning,
repository editing, shell execution, test/build execution, Git, GitHub
CLI, and Paperclip API tool use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-25 08:29:26 -05:00
Devin Foley 2c98c8e1e5
Fix sandbox git publishing and large workspace uploads (#8422)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox and SSH runtimes need to preserve agent work across isolated
execution environments
> - Git-backed workspaces were being copied mostly as filesystem
archives, which breaks when `.git` points outside the mounted workspace
and makes sandbox agents unable to publish their own branches
> - Large ignored dependency trees could also be swept into the sandbox
overlay, causing multi-GB transfers and max-string failures in some
sandbox clients
> - This pull request makes sandbox runtime setup use a git-backed HEAD
sync plus a small dirty/untracked overlay, and bounds sandbox file
transfers so large archives do not need one huge string
> - The benefit is that sandbox agents can commit and push from a usable
git checkout without uploading dependency trees such as `node_modules`

## Linked Issues or Issue Description

Refs #8395

No public duplicate issue or PR was found after searches for `sandbox
git workspace`, `git push sandbox`, and `node_modules sandbox upload`.

Bug report:

- What happened: sandbox-backed agent workspaces could receive a `.git`
file that pointed at host-only git state, leaving the sandbox unable to
run normal git workflows. The sandbox overlay upload could also include
ignored dependency directories, creating very large transfers.
- Expected behavior: sandbox and remote runtimes should prepare a usable
git-backed workspace, copy only the necessary workspace overlay, and
restore git history plus file changes without depending on a host-only
`.git` path.
- Steps to reproduce:
1. Run an agent in a sandbox-backed workspace whose local git checkout
is a worktree.
2. Ask the agent to complete a GitHub workflow that requires commit/push
access.
3. Observe that git operations can fail inside the sandbox, and ignored
dependency trees can be uploaded as part of the workspace overlay.
- Paperclip version or commit: reproduced against `master` before this
PR, base `7aa212296eb1`.
- Deployment mode: local dev / sandbox-backed runtime.
- Installation method: built from source.
- Agent adapters involved: local adapters using shared adapter-utils
runtime preparation.
- Database mode: not database-related.
- Access context: agent runtime.
- Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS
arm64.
- Privacy checklist: all pasted output was reviewed for secrets, private
hostnames, local usernames, and internal instance links.

## What Changed

- Added a GitHub workflow push preflight so agent runs can detect
missing push credentials when a workflow explicitly needs GitHub
publishing.
- Added shared git workspace sync helpers for shallow HEAD import/export
and dirty/untracked overlay tracking.
- Updated sandbox managed runtime setup to use git history plus a
selected overlay instead of uploading the full local workspace for
git-backed workspaces.
- Bounded sandbox archive upload/download paths so large payloads stream
or chunk instead of materializing one oversized string.
- Excluded `.git` and ignored dependency trees from sandbox upload,
download, and restore baselines while preserving local ignored
directories during sync-back.
- Added focused tests for git workspace sync, sandbox overlay selection,
transfer chunking, and heartbeat push-preflight behavior.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/git-workspace-sync.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
packages/adapter-utils/src/command-managed-runtime.test.ts
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/heartbeat-workspace-session.test.ts`
- `pnpm --filter @paperclipai/adapter-utils typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm build`
- Public-hygiene scan of the PR diff and commit messages for internal
issue ids, local paths, private hostnames, and obvious token patterns.

## Risks

- Medium risk: this changes sandbox runtime synchronization semantics
for git-backed workspaces, especially around dirty tracked files,
untracked files, deleted paths, and ignored files.
- The main mitigation is focused test coverage for upload contents,
restore exclusions, and git round-trip behavior.
- The SSH runtime keeps the current bundle-based implementation from
`master`; this PR only aligns shared excludes and sandbox behavior with
that model.

## Model Used

OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub
workflow, with code execution and repository inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 22:03:55 -07:00
Devin Foley 07e98d2b2c
feat(adapter-utils): add observable sandbox sync progress (#8395)
## Thinking Path

> - Paperclip is the control plane for running AI-agent companies, so
long-running remote work needs to stay observable to human operators.
> - Cloud / sandbox agents are an active roadmap area, and their
workspace sync path is part of the runtime substrate every remote coding
run depends on.
> - In the sandbox and SSH execution-target flows, Paperclip logged that
sync had started, then often went silent for the full transfer window.
> - That made large remote syncs feel stalled and also hid a real
performance problem in the command-managed sandbox upload path.
> - The first part of this pull request threads a throttled
progress-reporting surface through the adapter execution-target stack so
sync and restore work can emit meaningful updates.
> - The second part fixes the command-managed sandbox transport itself:
it removes the old serial 32KB append bottleneck, but also falls back
away from the single-stream path when a provider-backed sandbox runner
cannot surface mid-flight stdin progress.
> - The result is that sandbox and SSH transfers are both faster and
more observable, including the live Daytona-style sandbox case that
previously only emitted `0%` and `100%`.

## Linked Issues or Issue Description

No public GitHub issue exists for this bug, so it is described inline
below following the bug report template.

### What happened

- Remote sandbox and SSH workspace syncs could spend a long time
transferring data while only logging a start line (`Syncing workspace
and runtime assets to sandbox environment`) and, at best, a terminal
line.
- In the command-managed sandbox path, the original upload
implementation also paid a large performance cost by appending base64
data in many small sequential remote writes (thousands of serial 32KB
round-trips on a large workspace).
- After the initial transport rewrite, live provider-backed sandbox runs
still only emitted `0%` and `100%` because the single-stream stdin RPC
buffered progress until completion.

### Expected behavior

- Long-running sandbox and SSH syncs should periodically report how much
of the transfer is complete (a percentage and/or MB transferred) so an
operator can tell the run is healthy and making progress rather than
stuck.
- The main sandbox upload path should not be artificially slow.
- A transfer that fails partway should leave an explicit failure marker
in the log rather than a dangling intermediate percentage.

### Steps to reproduce

1. Run an agent against a sandbox (command-managed) or SSH
(remote-managed) execution target with a non-trivial workspace.
2. Watch the run log during the workspace/runtime asset sync phase.
3. Observe that the log shows the sync start line and then stays silent
for the full transfer (live provider-backed sandbox runs only show `0%`
then `100%`).

### Paperclip version or commit

- Branch `PAPA-825-provide-status-updates-when-syncing-sandboxes` off
`master`.

### Deployment mode

- Self-hosted / local instance using sandbox (command-managed) and SSH
(remote-managed) execution targets, including provider-backed sandbox
runners.

## What Changed

- Added shared throttled runtime progress reporting and threaded
`onProgress` through the adapter execution-target surface and adapter
`execute.ts` entrypoints.
- Added sync and restore progress reporting for the command-managed
sandbox path and the SSH/remote-managed path, including git
import/export progress where totals are known.
- Reworked command-managed sandbox transfer behavior so uploads use the
faster single-stream path when appropriate, but fall back to chunked
progress-emitting writes when the runner cannot expose mid-stream stdin
progress.
- Marked provider-backed environment sandbox runners as not supporting
single-stream stdin progress so live sandbox runs emit meaningful
intermediate updates instead of only `0%` and `100%`.
- Emit an explicit terminal failure marker (`failed at NN% (x/y MB)`)
when an SSH/tar transfer rejects, so a failed sync no longer leaves a
dangling intermediate percentage in the log.
- Run the SSH sync/restore size estimate (local directory walk / remote
`du` probe) concurrently with the transfer instead of awaiting it before
opening the pipe, so progress instrumentation no longer adds startup
latency proportional to workspace file count.
- Added and extended focused regression coverage for runtime progress
throttling and the new failure marker, command-managed sandbox
transfers, sandbox orchestration, SSH transfer progress, and environment
execution-target wiring.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/runtime-progress.test.ts
packages/adapter-utils/src/ssh-fixture.test.ts
packages/adapter-utils/src/command-managed-runtime.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts`
- `pnpm exec vitest run
packages/adapter-utils/src/command-managed-runtime.test.ts
server/src/__tests__/environment-execution-target.test.ts`
- `npx tsc --noEmit` for `packages/adapter-utils`

## Risks

- The provider-backed sandbox fallback now prefers chunked
command-managed writes when progress hooks are active, so
small-to-medium uploads may trade some raw throughput for observable
intermediate progress on runtimes that cannot surface true mid-stream
stdin progress.
- Progress percentages on tar-based transfers still depend on estimates
in some cases, so operators may briefly see MB-only lines before the
estimate resolves, then near-final clamping before the terminal `100%`
line.
- This PR changes shared execution-target behavior used by multiple
adapters, so regressions would most likely appear in remote runtime
setup/teardown flows rather than in a single adapter.

## Model Used

- Initial implementation: OpenAI GPT-5.4 via Codex local agent
(`codex_local`), high reasoning mode.
- Observability follow-ups (failure marker, concurrent size estimate,
added tests): Claude Opus 4.8 via Claude Code (`claude_local`).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 13:03:42 -07:00
Devin Foley 12cb7b40fd
Harden remote workspace sync and restore flows (#5444)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - When an agent runs against a remote target, Paperclip syncs the
workspace out to the remote at run start and restores changes back to
the local workspace at run end
> - The previous restore flow naïvely overwrote local files with
whatever the remote returned, so files that the remote run never touched
but had timestamp/mode drift could be needlessly rewritten — and a
single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH
workspace exports race on the same git ref
> - This pull request adds a `workspace-restore-merge` module that diffs
a pre-run snapshot against the post-run remote state and only writes
back files the remote actually changed; SSH workspace exports now use a
per-import unique ref so concurrent runs can't trample each other
> - Every adapter's execute path threads the snapshot through
`prepareAdapterExecutionTargetRuntime` so the merge has the baseline it
needs
> - The benefit is workspace restores no longer churn untouched files,
and concurrent SSH runs no longer collide on the import ref

## What Changed

- `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new
module — directory snapshot (kind/mode/sha256/symlink target) plus
snapshot-aware merge that writes only the files the remote changed
- `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a
per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`);
restore goes through the new merge helper; `ssh-fixture.test.ts` covers
the unique-ref + merge paths
- `packages/adapter-utils/src/sandbox-managed-runtime.ts` +
`remote-managed-runtime.ts`: thread the snapshot/merge through the
sandbox and SSH paths
- `packages/adapter-utils/src/server-utils.{ts,test.ts}` +
`execution-target.ts`: helpers for capturing the pre-run snapshot;
`prepareAdapterExecutionTargetRuntime` gains required `runId` and
optional `workspaceRemoteDir`, and returns the realized
`workspaceRemoteDir`
- Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini,
opencode, pi) takes the snapshot at run start and passes it through to
the runtime restore
- Remote execute test mocks updated to match the new
`prepareWorkspaceForSshExecution` return shape and the per-run
`${managedRemoteWorkspace}` cwd subdirectory

## Verification

- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-acpx-local --project
@paperclipai/adapter-claude-local --project
@paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project
@paperclipai/adapter-gemini-local --project
@paperclipai/adapter-opencode-local --project
@paperclipai/adapter-pi-local` — 196/196 passing
- `pnpm typecheck` clean across the workspace

## Risks

Medium. The restore path now writes a strict subset of what it
previously did — files the remote did not touch are no longer rewritten.
If any flow was relying on a touch-without-content-change being copied
back (timestamp or permission propagation only), that behavior is now
skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at
run start; the cost is bounded by the existing exclude list. The `runId`
parameter on `prepareAdapterExecutionTargetRuntime` is now required —
every in-tree caller is updated; out-of-tree adapter authors need to
pass it.

## Model Used

Claude Opus 4.7 (1M context)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new module +
every adapter execute path covered
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-05-07 14:44:45 -07:00
Devin Foley 076067865f
Migrate SSH environment callback to bridge (#5116)
> **Stacked PR (part 3 of 7).** Depends on:
  - PR #5114
  - PR #5115
> Diff against `master` includes commits from earlier PRs in the stack —
the new commit in this PR is the topmost one.

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Agents executing on a remote SSH-backed environment need a way to
call back into
>   the Paperclip control plane (run events, log streaming, signals)
> - When the SSH host can't reach the Paperclip host (NAT, firewalls, or
simply not
> on the same network), the run silently fails or hangs — a recurring
class of
>   failure during SSH testing
> - In sandboxed environments we already solved this with a callback
bridge that
> tunnels back through the existing connection; SSH was the odd one out
> - This PR migrates SSH execution to use the same callback bridge, so
every
> adapter's remote run uses one consistent reverse-channel. Per-adapter
SSH glue
> is deleted in favour of a shared `CommandManagedRuntimeRunner` built
from the
>   SSH spec
> - The benefit is fewer SSH-specific failure modes, a smaller code
surface, and
>   one place to evolve the callback contract going forward

## What Changed

- Added `createSshCommandManagedRuntimeRunner` in
`packages/adapter-utils/src/ssh.ts` that adapts an SSH spec into a
generic
  command-managed-runtime runner (with cwd, env, and timeout handling)
- Removed `paperclipApiUrl` from `SshRemoteExecutionSpec`; the bridge
URL now flows
  through the shared runner
- Reworked `execution-target.ts` to use the SSH runner alongside sandbox
runners
  via a unified `CommandManagedRuntimeRunner` interface
- Simplified `remote-managed-runtime.ts` and
`sandbox-managed-runtime.ts` to consume
  the shared runner abstraction
- Deleted per-adapter SSH callback wiring from claude-local,
codex-local,
  cursor-local, gemini-local, opencode-local, pi-local execute.ts files
- Removed `environment-runtime-driver-contract.test.ts` (the contract is
now
  enforced by `environment-execution-target.test.ts`)
- Added/updated `execute.remote.test.ts` cases for each adapter to cover
the SSH
  runner path

## Verification

- `pnpm --filter @paperclipai/adapter-utils test`
- `pnpm test -- execute.remote` (covers all six local adapters' SSH
paths)
- Manual QA: ran a claude-local agent against an SSH-backed environment,
confirmed
the agent successfully called back to `/api/agent-callback/*` endpoints
during
  the run

## Risks

- Refactor touches all six local adapters. If any adapter had subtle
SSH-specific
behaviour that wasn't captured in tests, it could regress. Mitigation:
each
  adapter's `execute.remote.test.ts` was extended.
- `paperclipApiUrl` removal from `SshRemoteExecutionSpec` is a breaking
type change
for any internal consumer. Verified no external plugins consume this
type.
- The new `CommandManagedRuntimeRunner` shape is a public surface in
`@paperclipai/adapter-utils`; downstream plugins implementing custom
runners may
  need updates, but no such plugins exist in this repo.

## Model Used

- OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI
- Provider: OpenAI
- Used to author the code changes in this PR

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots — N/A
- [ ] I have updated relevant documentation to reflect my changes — N/A
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-05-03 12:43:52 -07:00
Devin Foley e4995bbb1c
Add SSH environment support (#4358)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - The environments subsystem already models execution environments,
but before this branch there was no end-to-end SSH-backed runtime path
for agents to actually run work against a remote box
> - That meant agents could be configured around environment concepts
without a reliable way to execute adapter sessions remotely, sync
workspace state, and preserve run context across supported adapters
> - We also need environment selection to participate in normal
Paperclip control-plane behavior: agent defaults, project/issue
selection, route validation, and environment probing
> - Because this capability is still experimental, the UI surface should
be easy to hide and easy to remove later without undoing the underlying
implementation
> - This pull request adds SSH environment execution support across the
runtime, adapters, routes, schema, and tests, then puts the visible
environment-management UI behind an experimental flag
> - The benefit is that we can validate real SSH-backed agent execution
now while keeping the user-facing controls safely gated until the
feature is ready to come out of experimentation

## What Changed

- Added SSH-backed execution target support in the shared adapter
runtime, including remote workspace preparation, skill/runtime asset
sync, remote session handling, and workspace restore behavior after
runs.
- Added SSH execution coverage for supported local adapters, plus remote
execution tests across Claude, Codex, Cursor, Gemini, OpenCode, and Pi.
- Added environment selection and environment-management backend support
needed for SSH execution, including route/service work, validation,
probing, and agent default environment persistence.
- Added CLI support for SSH environment lab verification and updated
related docs/tests.
- Added the `enableEnvironments` experimental flag and gated the
environment UI behind it on company settings, agent configuration, and
project configuration surfaces.

## Verification

- `pnpm exec vitest run
packages/adapters/claude-local/src/server/execute.remote.test.ts
packages/adapters/cursor-local/src/server/execute.remote.test.ts
packages/adapters/gemini-local/src/server/execute.remote.test.ts
packages/adapters/opencode-local/src/server/execute.remote.test.ts
packages/adapters/pi-local/src/server/execute.remote.test.ts`
- `pnpm exec vitest run server/src/__tests__/environment-routes.test.ts`
- `pnpm exec vitest run
server/src/__tests__/instance-settings-routes.test.ts`
- `pnpm exec vitest run ui/src/lib/new-agent-hire-payload.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- `pnpm -r typecheck`
- `pnpm build`
- Manual verification on a branch-local dev server:
  - enabled the experimental flag
  - created an SSH environment
  - created a Linux Claude agent using that environment
- confirmed a run executed on the Linux box and synced workspace changes
back

## Risks

- Medium: this touches runtime execution flow across multiple adapters,
so regressions would likely show up in remote session setup, workspace
sync, or environment selection precedence.
- The UI flag reduces exposure, but the underlying runtime and route
changes are still substantial and rely on migration correctness.
- The change set is broad across adapters, control-plane services,
migrations, and UI gating, so review should pay close attention to
environment-selection precedence and remote workspace lifecycle
behavior.

## Model Used

- OpenAI Codex via Paperclip's local Codex adapter, GPT-5-class coding
model with tool use and code execution in the local repo workspace. The
local adapter does not surface a more specific public model version
string in this branch workflow.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-04-23 19:15:22 -07:00