paperclip/packages/adapter-utils/src
Dotta 9031516a7e
fix: recover legacy Daytona startup failures from task and inbox (#13272)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Legacy conversation adapters can run in Daytona sandboxes.
> - A server restart during provisioning can occur before the invocation
event exists.
> - Recovery then lacks the old adapter identity and leaves a hold that
ordinary user retries cannot clear.
> - A remote launch can also fail when its host relay looks for Node in
the sandbox PATH.
> - This pull request records the adapter at claim time and restores
explicit user continuation after verified cleanup.
> - Users can recover from the task or inbox while the failed run and
uncertain action history remain intact.

## Linked Issues or Issue Description

Refs #13237, #13239, #13254. Those changes cover recorded conversation
runs, native user continuation, and explicit remote Stop. This change
covers legacy failure before `adapter.invoke` and exact task/inbox
Retry.

Refs #9771 for overlapping generated-command quoting. This change also
supplies the absolute host Node executable. Refs #13163 and #13264 for
the separate native restart and retained-workspace work.

**What happened?**
A legacy Daytona run interrupted during provisioning became
`process_lost` without an invocation event. Recovery preserved an
execution hold, and Retry or a new task reply could not resume it.
Cleanup could also run before the Daytona plugin was ready. On a macOS
host, a subsequent ACP relay launch failed with `env: node: No such file
or directory` because the remote launch environment did not contain the
host Node path.

**Expected behavior**
An interrupted conversation can continue after its previous execution
stops. Explicit Retry and new user replies should start a fresh turn
with the task history. Cleanup failures must remain visible and
recoverable. The host relay must use the host Node executable.

**Steps to reproduce**
1. Use a legacy Claude adapter with a Daytona environment.
2. Interrupt the server after it acquires the sandbox lease and before
it records `adapter.invoke`.
3. Restart and inspect the task hold.
4. Retry from the task or inbox, or send a new task reply.
5. Confirm the old sandbox has stopped and one new response arrives.

**Paperclip version or commit**
Reproduced from master at `3bafac12f796fbea02e609e1074a9639f872e9c4`.
The branch is rebased on `51b0e01ea`, including #13261 and #13270.

**Deployment mode**
Built from source on macOS with a real Daytona sandbox and the legacy
Claude ACP adapter.

## What Changed

- Count new browser specs with the scheduler's median duration in the
shard-balance check. This fixes a false policy failure after new specs
arrive from both branches. The balance threshold is unchanged.

- Persist server-owned adapter identity in the queued-to-running claim
before provisioning starts.
- Wait for provider plugin startup before restart cleanup. Keep failed
cleanup leases as active ownership blockers.
- Admit exact board retries and new user comments after verified
termination. Retain the old run, task history, approvals, and unknown
action outcomes.
- Adopt repeated Retry requests. Permit one scoped cleanup attempt per
explicit user Retry after the automatic limit, with an activity record.
A later user Retry can recover after a transient provider failure;
automatic attempts remain capped.
- Resume replies deferred during cleanup, including historical legacy
startup failures.
- Launch the host ACP relay through the absolute host Node executable.
- Add a task-level Retry button and return actionable blockers when
retry admission is refused.
- Add database regressions and three browser recovery journeys. Exclude
installed third-party dependency skills from the shipped-skill audit.

## Verification

- Current head: `d23c84181`, rebased on `51b0e01ea`. Conflict resolution
retains the saved-message recovery, local stop receipts, and wait
reasons from #13270 alongside exact legacy Retry support.
- Real Daytona: interrupted the server after lease acquisition and
before adapter invocation. Restart cleanup confirmed provider
termination. Task Retry cleared a seeded historical hold and a real
Claude agent returned `Recovery verified.` in the task. Removed the
disposable sandbox and environment after testing.
- All three browser recovery journeys passed again after the final
rebase. Task Retry, Inbox Retry, and a new reply each produced one fresh
successor, completed the task, preserved the failed run, and retained
the answer after reload.
- All 29 e2e/server shard-partition tests passed. The balance check now
uses the scheduler's median fallback for unmeasured specs, with the same
balance threshold.
- Server typecheck passed after rebuilding the generated runner
dependencies. The combined recovery/route run passed 136 of 137 tests.
Its remaining route test timed out during the first cold module import
at its explicit 10-second limit; an isolated rerun reproduced that
timeout and passed the other 51 route cases. The complete CI suite
passed on this head. The same route file passed all 52 cases in CI,
including the first cold import in 7.5 seconds.
- Before the final rebase, recursive typecheck, full build, UI token
gates, 132 targeted server tests, and the complete [CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/34650004085)
passed. The subsequent CI failure was the shard-balance accounting
mismatch fixed here.
- Greptile reviewed `d23c84181` at 5/5 with no outstanding actionable
findings. The complete [current CI
workflow](https://github.com/paperclipai/paperclip/actions/runs/34653327949)
passed on attempt 2. All test, typecheck, build, and canary jobs passed
on the first attempt. Docker setup timed out fetching BuildKit from
Docker Hub; retrying that job and its dependent aggregate succeeded.

## Risks

- Recovery admission changes executable authority. Company, task, agent,
user, approvals, process ownership, and provider termination checks
remain required.
- Explicit continuation starts a fresh conversation with history. It
does not certify unknown external action outcomes or rerun
non-conversation adapters automatically.
- Changing task status alone does not clear an execution hold. The task
now offers an explicit Retry action.
- Historical adapter claims and invocation events take precedence over
current agent settings. Known process or webhook runs retain their hold.
Pre-upgrade rows with no adapter evidence may receive only a new
explicit user turn after termination proof; they do not become eligible
for automatic replay.
- No schema migration or sandbox-image change is required. This branch
has not been deployed to production.

## Model Used

OpenAI GPT-6 through Codex, with repository inspection, code execution,
browser automation, and test execution. The exact deployment model ID
and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-11 18:08:14 -05:00
..
acpx-engine fix: recover legacy Daytona startup failures from task and inbox (#13272) 2026-09-11 18:08:14 -05:00
test-support feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559) 2026-07-14 15:07:30 -05:00
billing.test.ts feat(costs): add billing, quota, and budget control plane 2026-03-16 15:11:01 -05:00
billing.ts feat(costs): add billing, quota, and budget control plane 2026-03-16 15:11:01 -05:00
bridge-transport-contract.ts refactor(adapter-utils): remove the retired duplex_v1 sandbox bridge transport (#12171) 2026-08-25 08:47:43 -07:00
chat-file-delivery.test.ts feat: add experimental native chat connectors (#13038) 2026-09-10 10:06:45 -05:00
chat-file-delivery.ts feat: add experimental native chat connectors (#13038) 2026-09-10 10:06:45 -05:00
command-managed-runtime.test.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
command-managed-runtime.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
command-redaction.test.ts fix(runner): restore local session and task integrity (#12721) 2026-09-02 16:11:26 -05:00
command-redaction.ts fix(runner): restore local session and task integrity (#12721) 2026-09-02 16:11:26 -05:00
duplex-frame-codec.test.ts refactor(adapter-utils): delete the dead duplex body-chunk protocol code (#12186) 2026-08-25 14:33:38 -07:00
duplex-frame-codec.ts refactor(adapter-utils): replace the process-wide byte ledger with route-local byte bounds (#12465) 2026-08-28 14:36:18 -07:00
duplex-frame-vectors.json refactor(adapter-utils): replace the process-wide byte ledger with route-local byte bounds (#12465) 2026-08-28 14:36:18 -07:00
duplex-observability.test.ts feat(duplex): run the Daytona sandbox callback bridge over Node HTTP/2 (#12120) 2026-08-25 07:35:39 -07:00
duplex-observability.ts refactor(adapter-utils): replace the process-wide byte ledger with route-local byte bounds (#12465) 2026-08-28 14:36:18 -07:00
env-bindings.test.ts fix(adapters): keep user-scoped env bindings on the agent Test action (#10926) 2026-08-05 17:18:41 -07:00
env-bindings.ts fix(adapters): keep user-scoped env bindings on the agent Test action (#10926) 2026-08-05 17:18:41 -07:00
exclude-patterns.ts Fix sandbox git publishing and large workspace uploads (#8422) 2026-06-20 22:03:55 -07:00
execution-target-sandbox.test.ts feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923) 2026-09-08 14:18:12 -07:00
execution-target-stdin-race.test.ts fix(adapter-utils): allow process sessions without birthtime (#12451) 2026-08-28 10:33:22 -07:00
execution-target.test.ts feat: use the responsible person's GitHub for shared agent operations (#13005) 2026-09-07 14:32:20 -05:00
execution-target.ts fix(adapter-utils): fail fast when the sandbox control channel is lost mid-turn (#13158) 2026-09-10 15:37:05 -07:00
git-workspace-sync.test.ts fix(adapter-utils): harden the referenced-project ignore scan (#12214) 2026-08-26 08:30:42 -07:00
git-workspace-sync.ts fix(adapter-utils): harden the referenced-project ignore scan (#12214) 2026-08-26 08:30:42 -07:00
github-launcher-environment.test.ts fix(adapters): probe Git context in the remote workspace (#13116) 2026-09-09 16:52:11 -05:00
github-launcher.test.ts fix(runner): restore legacy Git access and independent networking (#13094) 2026-09-09 10:15:10 -05:00
github-launcher.ts fix(runner): restore legacy Git access and independent networking (#13094) 2026-09-09 10:15:10 -05:00
http2-bridge-server.test.ts feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923) 2026-09-08 14:18:12 -07:00
http2-bridge-server.ts feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923) 2026-09-08 14:18:12 -07:00
index.ts fix: repair runner configuration, macOS execution, and artifact galleries (#13062) 2026-09-08 19:10:09 -05:00
local-process-sandbox.test.ts feat: use the responsible person's GitHub for shared agent operations (#13005) 2026-09-07 14:32:20 -05:00
local-process-sandbox.ts fix(adapter-utils): keep sandbox proxy sockets within Linux path limit (#10221) 2026-07-26 10:52:16 -05:00
log-redaction.ts fix(ui): external adapter selection, config field placement, and transcript parser freshness 2026-04-03 21:11:22 +01:00
login-capability.test.ts feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020) 2026-08-23 09:44:59 -07:00
login-capability.ts feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020) 2026-08-23 09:44:59 -07:00
login-pty-transport.test.ts feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020) 2026-08-23 09:44:59 -07:00
login-pty-transport.ts feat(login): use the login pseudo-terminal for Codex device login and de-Claude the shared channel (#12020) 2026-08-23 09:44:59 -07:00
login-runner-lifecycle.test.ts feat(auth): normalize agent login in the sandbox onto one session table and a capability contract (#11730) 2026-08-19 11:51:31 -07:00
login-runner-lifecycle.ts feat(auth): normalize agent login in the sandbox onto one session table and a capability contract (#11730) 2026-08-19 11:51:31 -07:00
mcp-isolation.integration.test.ts feat(skills): import skills from projects (#9620) 2026-07-15 18:01:44 -05:00
paperclip-runner-permissions.test.ts fix(runner): restore local session and task integrity (#12721) 2026-09-02 16:11:26 -05:00
paperclip-runner-permissions.ts fix: repair runner configuration, macOS execution, and artifact galleries (#13062) 2026-09-08 19:10:09 -05:00
published-exports.test.ts refactor: disambiguate the Telemetry and Observability data paths (#12128) 2026-08-24 16:42:33 -07:00
remote-execution-env.ts Sanitize remote execution envs at the boundary (#5325) 2026-05-05 19:30:14 -07:00
remote-managed-runtime.test.ts fix(adapter-utils): harden the referenced-project ignore scan (#12214) 2026-08-26 08:30:42 -07:00
remote-managed-runtime.ts fix(adapter-utils): honor .gitignore for referenced-project staging (#12184) 2026-08-25 14:22:47 -07:00
runner-connectivity.test.ts Unify Paperclip Runner experimental controls (#12666) 2026-09-01 09:21:23 -05:00
runner-connectivity.ts Unify Paperclip Runner experimental controls (#12666) 2026-09-01 09:21:23 -05:00
runtime-progress.test.ts feat(environments): refer to the managed default environment by name, not the sandbox driver key (#11838) 2026-08-21 12:51:22 -07:00
runtime-progress.ts feat(environments): refer to the managed default environment by name, not the sandbox driver key (#11838) 2026-08-21 12:51:22 -07:00
sandbox-callback-bridge.test.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
sandbox-callback-bridge.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
sandbox-file-sync.test.ts fix(adapter-utils): honor .gitignore for referenced-project staging (#12184) 2026-08-25 14:22:47 -07:00
sandbox-install-command.test.ts fix(build): enforce Node 24 across Paperclip (#11792) 2026-08-21 10:17:52 -07:00
sandbox-install-command.ts fix(build): enforce Node 24 across Paperclip (#11792) 2026-08-21 10:17:52 -07:00
sandbox-managed-runtime.test.ts fix(runner): persist warm Daytona workspaces (#12901) 2026-09-05 13:00:57 -05:00
sandbox-managed-runtime.ts fix(runner): persist warm Daytona workspaces (#12901) 2026-09-05 13:00:57 -05:00
sandbox-run-log-stream.ts Improve live agent feedback during sandboxed runs (#8915) 2026-07-02 22:21:56 -07:00
sandbox-shell.ts Add secrets provider vaults and remote import (#5429) 2026-05-09 18:22:17 -05:00
server-utils-env.test.ts fix: preserve control plane access in Codex sandbox (#10152) 2026-07-24 15:25:59 -05:00
server-utils.test.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
server-utils.ts feat(connections): add AgentMail inboxes and email tasks (#13256) 2026-09-11 16:56:38 -05:00
session-compaction.ts feat: add kimi-local adapter for Kimi Code CLI (CLI + ACP engines) (#9967) 2026-08-20 12:06:33 -07:00
skill-library-manifest.test.ts feat: agents see the company skill library at runtime (#12147) 2026-08-25 14:01:52 -07:00
skill-library-manifest.ts feat: agents see the company skill library at runtime (#12147) 2026-08-25 14:01:52 -07:00
skills-staging-follow-symlinks.test.ts fix(adapter-utils): stage selected skills into the sandbox for a remote Claude ACP run (#13196) 2026-09-11 07:50:06 -07:00
ssh-fixture.test.ts fix(adapter-utils): make SSH env-lab fixture teardown deterministic (#12238) 2026-08-26 14:03:47 -07:00
ssh.ts fix(adapter-utils): make SSH env-lab fixture teardown deterministic (#12238) 2026-08-26 14:03:47 -07:00
sync-operation-schedule.test.ts feat: parallelize sandbox file-sync behind a provider opt-in capability (#11736) 2026-08-19 12:34:11 -07:00
sync-operation-schedule.ts feat: parallelize sandbox file-sync behind a provider opt-in capability (#11736) 2026-08-19 12:34:11 -07:00
types.ts feat: add experimental native chat connectors (#13038) 2026-09-10 10:06:45 -05:00
workspace-restore-merge.test.ts fix(runner): persist warm Daytona workspaces (#12901) 2026-09-05 13:00:57 -05:00
workspace-restore-merge.ts fix(runner): persist warm Daytona workspaces (#12901) 2026-09-05 13:00:57 -05:00
workspace-restore-teardown.test.ts refactor(adapter-utils): extract the shared workspace-restore teardown factory (#12196) 2026-08-25 21:38:27 -07:00
workspace-restore-teardown.ts refactor(adapter-utils): extract the shared workspace-restore teardown factory (#12196) 2026-08-25 21:38:27 -07:00