paperclip/doc
Dotta cb0009b097
fix: preserve recovery retries across restarts (#11817)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The control plane must keep each active issue on a clear execution
or recovery path.
> - A missing issue disposition can require more than one bounded repair
attempt.
> - A server restart could lose that repair path or move source
ownership to the recovery owner.
> - A parked or expired retry could also make the user interface show a
false healthy state.
> - Concurrent recovery loops must not schedule the same repair attempt
twice.
> - This pull request keeps retry state durable, makes scheduling
atomic, and keeps source ownership stable.
> - The benefit is that recovery continues after a restart and operators
see the correct state.

## Linked Issues or Issue Description

**What happened?**

A run that ended without a valid issue disposition could lose its repair
path after a server restart. Manager recovery could also change the
source owner. In addition, a parked or expired retry could make the
issue look healthy when no active work existed. Concurrent
reconciliation could also schedule the same repair attempt twice.

**Expected behavior**

Paperclip must keep bounded source and manager repair attempts across
restarts. Recovery ownership must stay separate from source issue
ownership. The server and user interface must report only a live retry
as active work. Each repair attempt must be scheduled at most once per
company.

**Steps to reproduce**

1. Start an agent run on an issue.
2. End the run without a valid issue disposition.
3. Let the first repair attempt schedule a retry.
4. Restart the server, let the retry time pass without a live run, or
start two reconciliation loops together.
5. Observe that the repair path can stop, the issue can show a false
healthy state, or duplicate retries can be created.

**Paperclip version or commit**

The problem existed on `master` before candidate head
`d8e620fe86bade7df18decac332007f5821ae04f`.

**Deployment mode**

The problem affects self-hosted servers and local builds that use
automatic recovery.

## What Changed

- Persist bounded source-owner and manager repair lineages with stable
fingerprints and retry limits.
- Resume incomplete disposition repairs after a server restart.
- Keep recovery ownership separate from source issue ownership and
enforce source mutation authority.
- Project live retry evidence into issue and blocker summaries.
- Show recovery owner, return owner, attempt count, and retry state in
the board user interface.
- Treat expired or parked retries as attention states unless a queued or
running attempt exists.
- Atomically deduplicate disposition-repair wake requests with a
company-scoped partial unique index.
- Reuse the winning run when concurrent reconciliation loses the
uniqueness race, without duplicate scheduling activity.
- Honor disabled on-demand wake policy before recovery scheduling and
again before delayed retry promotion.
- Keep the new index migration safe for lagging seeded databases that
already contain the index.
- Add server and user interface tests for recovery, restart, ownership,
retry, concurrency, and blocker states.
- Update the implementation and execution semantics documents.

## Verification

- Focused server recovery and ownership suites: 282 tests passed on the
repaired base candidate.
- Focused user interface recovery suites: 128 tests passed on the
repaired base candidate.
- Atomic-deduplication schema and recovery suites: 111 tests passed on
the first Greptile repair.
- Recovery and scheduled-retry wake-policy suites: 126 tests passed at
`d8e620fe86bade7df18decac332007f5821ae04f`.
- The exact lagging-source migration-order test passed after the index
migration became idempotent: 1 test passed and 62 unrelated tests were
skipped.
- `@paperclipai/db` and `@paperclipai/server` typechecks passed at the
current head.
- Migration generation and migration safety checks passed for migration
`0226_tan_colossus.sql`.
- `pnpm check:token-gates` passed on the repaired base candidate.
- `pnpm -r typecheck` passed on the repaired base candidate.
- `pnpm build` passed on the repaired base candidate.
- `pnpm test:run` passed 4,540 tests on the repaired base candidate.
Four fixed-port cases met listeners that already existed on the host.
- The two unchanged fixed-port files passed in an isolated network
namespace: 129 tests passed and 27 tests were skipped.
- Independent Security and QA reviews approved
`63c0423aab54c66f2293a20b0fb3f3b013ee3ba8`; exact-head re-review is
required after automated checks settle on
`d8e620fe86bade7df18decac332007f5821ae04f`.

## Risks

- Recovery orchestration affects issue liveness and ownership. The new
paths use bounded attempts, stable fingerprints, row locks, authority
checks, and database uniqueness.
- A conservative attention state can show more warnings when a scheduled
retry has no queued or running attempt. It does not hide stopped work.
- Migration `0226_tan_colossus.sql` creates a partial unique index on a
known-large table. Migrations run transactionally, so `CONCURRENTLY` is
unavailable. The matching disposition-repair key namespace is introduced
by this release, so deployed databases have no matching rows before the
index is added.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex from the GPT-5 model family used agentic reasoning, tool
use, and code execution. The runtime did not expose the exact model ID
or context window.
- Anthropic Claude Opus 5 used a 1M context window, tool use, and code
execution for part of the user interface repair, as recorded in the
commit history.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:09:42 -05:00
..
assets docs: refresh README roadmap and four pillars (#9922) 2026-07-20 18:00:12 -05:00
connections docs(connections): MCP-direct/DCR playbook section + Notion dry-run appendix (#11030) 2026-08-07 10:52:04 -05:00
design fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400) 2026-08-14 22:11:16 -07:00
experimental feat(server): configure shared workspace concurrency (#10759) 2026-08-03 14:25:05 -05:00
logs Improve CLI API parity coverage (#6626) 2026-06-02 17:13:29 -07:00
plans feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
plugins feat(sandbox): stream session output by capability, drop three operator flags (#11557) 2026-08-17 13:25:30 -07:00
pr/5429 Add secrets provider vaults and remote import (#5429) 2026-05-09 18:22:17 -05:00
screenshots feat(mcp) [split 7/8]: activate Apps and gateway UI (#9562) 2026-07-14 15:40:08 -05:00
spec feat(adapters): confine local coding processes (#9504) 2026-07-14 11:01:19 -05:00
AGENT-ARTIFACTS.md Add workspace file viewer and artifact links (#7681) 2026-06-09 17:17:43 -05:00
AGENTCOMPANIES_SPEC_INVENTORY.md Add routine support to recurring task portability 2026-03-23 16:57:38 -05:00
CHANNELS.md feat(cli): add 'paperclipai channels' to show release lanes and the current one (#11210) 2026-08-11 08:47:32 -07:00
CLI.md fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400) 2026-08-14 22:11:16 -07:00
CLIPHUB.md refactor: rename packages to @paperclipai and CLI binary to paperclipai 2026-03-03 08:45:26 -06:00
DATABASE.md fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400) 2026-08-14 22:11:16 -07:00
DEPLOYMENT-MODES.md fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400) 2026-08-14 22:11:16 -07:00
DEVELOPING.md fix(workspaces): seed managed worktrees when the base checkout has no config (#11752) 2026-08-20 08:42:16 -07:00
DOCKER.md feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
GOAL.md [codex] Refresh docs and agent skills (#4693) 2026-04-28 16:12:03 -05:00
HERMES_GATEWAY_ONBOARDING.md fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400) 2026-08-14 22:11:16 -07:00
HERMES_GATEWAY_SMOKE.md [codex] Add built-in Hermes adapters (#8543) 2026-06-26 16:04:58 -05:00
INSTALLING.md feat(cli): add managed install, update, and service lifecycle (#10045) 2026-07-31 18:52:23 -07:00
LOW-TRUST-PRESETS.md docs(execution-semantics): define routable blocking and watchdog restoration (#10094) 2026-07-27 19:10:28 -05:00
MCP-ACCESS-GOVERNANCE.md feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
MCP-DEMO-SCRIPT.md feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
MCP-RUNTIME-OPERATIONS.md feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
OPENCLAW_ONBOARDING.md Introduce bind presets for deployment setup 2026-04-11 07:09:07 -05:00
PRODUCT.md feat(skills): open-by-default company skill policy and core UX (#9564) 2026-07-15 11:42:40 -05:00
PUBLISHING.md feat(release): bootstrap new npm packages with a placeholder publish (#11757) 2026-08-19 19:38:17 -07:00
README-draft.md docs: add README, draft README, and adapter logo assets 2026-03-02 10:31:59 -06:00
RELEASE-AUTOMATION-SETUP.md feat(release): add human-gated beta channel with stable soak enforcement (#11008) 2026-08-10 16:52:59 -07:00
RELEASE-CHECKLIST.md feat(release): draft stable notes at beta publish, read them from master at promotion (#11567) 2026-08-17 20:47:23 -07:00
RELEASE-NOTES-mcp-access-governance.md feat(mcp) [split 8/8]: add e2e coverage and operator docs (#9563) 2026-07-14 15:48:57 -05:00
RELEASING.md feat(release): draft stable notes at beta publish, read them from master at promotion (#11567) 2026-08-17 20:47:23 -07:00
SECRETS-AWS-PROVIDER.md feat(secrets): add user-specific runtime secrets (#8825) 2026-07-05 05:58:20 -05:00
SPEC-implementation.md fix: preserve recovery retries across restarts (#11817) 2026-08-20 17:09:42 -05:00
SPEC.md [codex] Add agent permissions and controls plan (#6386) 2026-05-22 08:12:52 -05:00
TASK-WATCHDOG.md fix(interactions): authorize resolvers consistently (#11376) 2026-08-16 13:46:50 -05:00
TASKS-mcp.md Add product spec and MCP task interface docs 2026-02-16 19:07:30 -06:00
TASKS.md Add task management data model spec 2026-02-16 14:25:00 -06:00
TELEMETRY_WORKFLOW.md Add telemetry proposal extractor (#9544) 2026-07-13 23:47:27 -07:00
UNTRUSTED-PR-REVIEW.md chore(docker): improve base image and organize docker files 2026-04-01 11:36:27 +00:00
acp-run-lifecycle.md refactor(acpx-engine): coordinator-owned ACP run lifecycle with a typed resource ledger (#11576) 2026-08-17 22:02:16 -07:00
execution-semantics.md fix: preserve recovery retries across restarts (#11817) 2026-08-20 17:09:42 -05:00
memory-landscape.md chore: improve worktree tooling and security docs 2026-04-10 22:26:30 -05:00
observability.md fix(server): stamp built commit into service.version (#11748) 2026-08-19 21:20:40 -07:00