## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The adapter/runtime layer has to move files into sandboxes safely
and efficiently
> - The current sync-in path needs a provider-delegable seam so
providers can use their native upload transport when available
> - The upload contract also needs ordered post-upload commands so
extracted content can be finalized fail-fast after transfer
> - The fallback path still has to preserve current behavior when the
provider does not expose native sync verbs
> - This pull request adds the contract and runtime seam for
provider-delegable sync-in, plus the single-stream collapse flag
> - The benefit is fewer round trips, a cleaner provider-owned upload
path, and a compatible fallback for existing runners
## Linked Issues or Issue Description
No public GitHub issue exists for this change. The underlying feature
request is described below in the repository's feature-request format.
### Problem or motivation
Paperclip needs a sync-in path that lets each provider choose the best
available upload transport instead of forcing the harness to orchestrate
uploads the same way every time. The runtime also needs a way to
describe ordered post-upload commands so providers can finalize
extracted content fail-fast after transfer.
### Proposed solution
Extend the sync contract with ordered post-upload commands, forward that
contract through the plugin and environment runtime layers, and make
client syncIn always available. When a provider advertises native sync
verbs, the client should delegate to that transport; otherwise it should
fall back to the existing tarball/write/extract behavior and then run
the post-upload commands in order.
### Alternatives considered
Keeping upload orchestration entirely host-side would avoid a contract
change, but it would block provider-specific transport optimizations and
keep the harness responsible for a path the provider can do more
efficiently. A separate post-upload API would add another surface
without improving the existing sync flow.
### Roadmap alignment
This work aligns with the broader runtime and adapter roadmap because it
improves provider integration without changing the external product
model. It is an additive contract change that preserves backward
compatibility for providers that do not expose native sync verbs.
### Additional context
The fallback path still needs to preserve existing observable behavior,
including command ordering, cwd confinement, and fail-fast execution.
The single-stream progress flag is part of the same transport
improvement so smaller writes can collapse to a single round trip when
the runner supports it.
## What Changed
- Added ordered `postUploadCommands` support to the sync operation
contract and SDK mirror.
- Plumbed the sync-in contract through the plugin and environment
runtime layers.
- Implemented a runtime client `syncIn` path that delegates to native
provider transport when available, otherwise uses the generic
tarball/write/extract fallback.
- Preserved fail-fast execution of ordered post-upload commands in the
fallback path.
- Flipped the sandbox runner's single-stream stdin progress flag to
collapse small `writeFile` operations to a single round trip.
- Added and updated tests for contract forwarding, fallback behavior,
cwd rejection, fail-fast behavior, and single-stream collapse.
## Verification
- `pnpm --filter @paperclipai/plugin-sdk exec vitest run
protocol.postupload.test.ts`
- `pnpm --filter @paperclipai/plugin-sdk exec vitest run
environment-sync-negotiation.test.ts`
- `pnpm --filter @paperclipai/adapter-utils exec vitest run
command-managed-runtime.test.ts`
- `pnpm --filter @paperclipai/server exec vitest run
environment-execution-target.test.ts`
- Local typecheck and targeted suite runs reported in the handoff passed
before PR creation.
## Risks
- The new fallback path could diverge from the previous inline upload
behavior if the tarball/extract contract changes.
- Provider-native sync handling may expose provider-specific edge cases
if a runner advertises sync verbs but does not fully honor the contract.
- The single-stream flag changes transport behavior for small uploads,
so regressions would likely show up as round-trip or upload failures.
## Model Used
OpenAI Codex (GPT-5, tool-using coding agent).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is an open-source AI-agent management platform; agents run
tasks inside sandboxed environments (Daytona, Kubernetes, E2B, etc.)
> - The control-plane ↔ sandbox file-transfer path flows through the
`environmentExecute` seam in `protocol.ts` — the only verb available to
plugins — which forces a base64-over-exec chunked loop for every file
move: workspace files, assets, Codex home sync
> - This transport is correct and safe, but it bypasses provider-native
bulk/streaming APIs (Daytona `uploadFiles`, K8s `FastUploadInterceptor`
/ volume mounts), leaving significant throughput on the table for large
workspaces
> - The right fix is an opt-in seam extension: providers with faster
native transfer declare two optional verbs; providers that do not opt in
stay on the existing fallback with zero code or behavior change required
> - This PR adds the first layer of that extension — two optional verbs
(`environmentSyncIn` / `environmentSyncOut`) in the plugin SDK, the
runtime plumbing to prefer the native path for the two clean
destroy-then-replace cases, and a doc for the contract
> - The core correctness invariant is byte-identical fallback: if no
provider opts in, execution is exactly what ships today;
`assertSyncOperationsConfined` enforces host-side path confinement for
providers that do opt in
> - No provider advertises the verbs yet → zero production behavior
change; future PRs wire up Daytona and K8s providers against this
contract
## Linked Issues or Issue Description
No public GitHub issue exists for this feature. Description follows the
`feature_request` issue template:
**Subsystem affected:**
packages/plugins — plugin system; packages/adapter-utils — adapter
runtime; server/ — EnvironmentRuntimeService
**Problem or motivation:**
Sandbox file transfers currently always use a base64-over-exec chunked
loop regardless of what the underlying provider supports. For workspaces
larger than a few MB this becomes the dominant wall-clock cost of every
sandbox run, and it bypasses bulk/stream APIs that providers like
Daytona already expose natively.
**Proposed solution:**
Add two optional, opt-in plugin hooks — `onEnvironmentSyncIn` /
`onEnvironmentSyncOut` — to the plugin SDK. When a provider defines both
hooks and both are advertised via the existing `supportedMethods`
negotiation, the runtime prefers the native path for the two clean
destroy-then-replace transfer cases; all other cases fall back to the
existing byte-identical base64 transport.
**Alternatives considered:**
An unconditional verb would require every provider to implement or stub
the verb. The opt-in / `METHOD_NOT_IMPLEMENTED` pattern (already used by
`environmentExecute`) preserves backward compatibility with zero
provider changes required.
**Roadmap alignment:**
Consistent with the ✅ "Cloud / Sandbox agents" and ✅ "Plugin system"
milestones; extends the plugin seam rather than adding
control-plane-level logic.
**Additional context:**
Searched open pull requests and issues for duplicate sandbox file-sync /
native-transfer work; none found.
## What Changed
- **`packages/plugins/sdk`**
- `protocol.ts`: two new optional `HostToWorkerMethods` —
`environmentSyncIn` / `environmentSyncOut` — plus generic
`SyncOperation`, `SyncFileMapping`, and `SyncOutcome` types
- `define-plugin.ts`: optional `onEnvironmentSyncIn` /
`onEnvironmentSyncOut` fields on `PluginDefinition`; worker advertises
each verb only when its hook is defined (else `METHOD_NOT_IMPLEMENTED`,
mirroring `environmentExecute`)
- `worker-rpc-host.ts`: route new verbs to plugin hooks
- `index.ts`: re-export new public types
- **`packages/adapter-utils`**
- `command-managed-runtime.ts`: expose optional `syncIn` / `syncOut` on
`CommandManagedRuntimeRunner` (available only when both verbs are
advertised); add `assertSyncOperationsConfined` host-side
path-confinement guard
- `sandbox-managed-runtime.ts`: `SandboxManagedRuntimeClient` gains
optional `syncIn` / `syncOut`; orchestrator prefers native path for
default-provision asset inbound and workspace-download-into-fresh-dir
outbound; all other paths keep the existing base64 fallback
- `sandbox-file-sync.test.ts` (new): 234-line characterization suite —
native-opt-in branch, fallback branch, `assertSyncOperationsConfined`
escape-path rejection, `followSymlinks` → tar `-h`
- `command-managed-runtime.test.ts`: negotiation + native-sync +
confinement tests
- **`server/src/services/environment-runtime.ts`**:
`EnvironmentRuntimeService` delegates to `syncIn` / `syncOut`, gated on
advertised support
- **`server/src/services/environment-execution-target.ts`**: minor
typing fix alongside the new verbs
- **`doc/plugins/SANDBOX_FILE_SYNC_HOOKS.md`** (new): documents the full
contract — opt-in / no-op guarantee, operation ordering,
provider-may-tar, atomicity, `followSymlinks`, secret modes (0600, no
window), path confinement, `operationId` opacity, resource bounds,
shell-quoting
## Verification
```bash
# SDK suite
pnpm --filter packages/plugins/sdk test
# Adapter-utils suite (includes new sandbox-file-sync characterization tests)
pnpm --filter packages/adapter-utils test
# Expected: 255 pass / 4 skip
# Type-check across affected packages
pnpm --filter packages/plugins/sdk typecheck
pnpm --filter packages/adapter-utils typecheck
# Server changed-file spot check:
cd server && npx tsc --noEmit --skipLibCheck 2>&1 | grep -E "environment-(runtime|execution-target)" | head -20
```
Key behavioral invariant to spot-check: with no provider opting in (the
current state), run any sandbox task and confirm file-transfer behavior
is byte-for-byte identical to what the pre-PR code produces. The
characterization tests assert this at the unit level.
## Risks
- **Zero production risk today**: no provider advertises
`environmentSyncIn` / `environmentSyncOut`, so the new code paths are
unreachable in production; all real traffic stays on the existing base64
fallback
- **Path confinement**: `assertSyncOperationsConfined` rejects any
`targetPath` that escapes the declared root — this is the primary
security boundary for future providers. The test suite covers
escape-path rejection
- **Atomicity**: the contract delegates atomicity to providers; the doc
explicitly calls out that directory-level ops are not guaranteed atomic
- **Secret transport**: credential assets (e.g., Codex `auth.json`,
directory mappings) continue to use the existing tar path — they do not
go through the new verbs in any current provider
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
Provider: Anthropic
Model: `claude-sonnet-4-6` (Claude Sonnet 4.6)
Context window: 200 K tokens
Capabilities: extended tool use, multi-file code generation, agentic
reasoning via the Paperclip agent framework
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Nicky Leach