paperclip/server
Dotta 85404b46c5
fix(server): throttle serial recovery repeats (#9651)
## Thinking Path

> - Paperclip is the open source control plane people use to coordinate
AI-agent companies.
> - Its recovery services create productivity reviews and liveness
escalations when work stops making progress.
> - Existing uniqueness guards prevent concurrent duplicates, but
terminal recovery tasks can still be recreated serially without enough
time for conditions to change.
> - That creates noisy review churn for persistently stalled issues and
immediate liveness re-escalation after a recovery task closes.
> - This pull request adds bounded, configurable cooldown and no-action
suppression behavior to those two recovery paths.
> - The benefit is quieter recovery automation that still resumes
automatically after source activity or cooldown expiry.

## Linked Issues or Issue Description

### Pre-submission checklist

- [x] I have searched existing open and closed issues and this is not a
duplicate.
- [x] I can reproduce this behavior on `master`.
- [x] I have confirmed the behavior originates in Paperclip core
recovery orchestration, not an adapter, provider, or local
configuration.

### What happened?

Recovery reconciliation can serially recreate equivalent system-origin
tasks after previous tasks become terminal. Productivity reviews allowed
multiple creations for the same source issue within a rolling day, and a
closed liveness escalation could be recreated immediately for the same
incident or recovery leaf.

### Expected behavior

Productivity review creation should be limited to once per rolling 24
hours, repeated completed reviews that produced no source action should
eventually suppress further creation until activity resumes, and
recently terminal liveness escalations should receive a short cooldown
before recreation.

### Steps to reproduce

1. Create a stalled assigned issue that meets productivity-review
eligibility.
2. Complete repeated productivity-review tasks without adding
source-issue activity, then reconcile again within 24 hours.
3. Create and close a liveness escalation for a blocked issue graph,
then immediately reconcile the same graph.
4. Observe that equivalent system tasks can be recreated serially
without a meaningful state change.

### Paperclip version or commit

`5588ddf68175eea448f9d19677b97d7393c38c3d` (`master` when reproduced)

### Deployment mode

Local dev (`pnpm dev`)

### Installation method

Built from source (`pnpm dev` / `pnpm build`)

### Agent adapter(s) involved

- [x] Not adapter-specific (core bug)

### Database mode

Embedded PGlite (default — `DATABASE_URL` unset)

### Access context

Unclear / not applicable

### Node.js version

Current repository-supported Node.js runtime.

### Operating system

Linux development environment.

### Relevant logs or output

No error is emitted; the bug is repeated task creation visible in
persisted issue history.

### Relevant config (if applicable)

No special configuration is required.

### Additional context

The concurrent/open-task uniqueness guards work as designed; this change
targets serial repeats after matching tasks become terminal.

### Privacy checklist

- [x] I have reviewed all pasted output for PII and redacted where
necessary.

## What Changed

- Tightened the productivity-review creation cap to one review per
source issue in a rolling 24-hour window.
- Added configurable suppression after three consecutive completed
reviews with no source-issue activity, with automatic reset when source
activity occurs.
- Added a configurable one-hour default cooldown for matching terminal
liveness escalations.
- Exposed the liveness reconciliation clock/cooldown inputs for
deterministic orchestration tests.
- Added focused tests for daily enforcement, no-action suppression and
reset, and cooldown expiry.

## Verification

- `pnpm exec vitest run
server/src/__tests__/productivity-review-service.test.ts
server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts` — 36
tests passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- Confirm the focused tests demonstrate creation after source activity
and after the liveness cooldown expires.

## Risks

- Low-to-moderate behavioral risk: recovery tasks intentionally appear
less often, so overly aggressive thresholds could delay intervention for
a persistently stalled issue.
- Thresholds are configurable through reconciliation inputs, and source
activity resets productivity-review suppression.
- No database migration, public API change, telemetry contract change,
or UI behavior change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex using GPT-5.3-Codex, with reasoning, repository/terminal
tool use, code execution, and test execution. The runtime did not expose
a reliable context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-16 09:22:50 -05:00
..
scripts
…
src fix(server): throttle serial recovery repeats (#9651) 2026-07-16 09:22:50 -05:00
CHANGELOG.md
…
package.json build(deps): bump dompurify from 3.4.8 to 3.4.12 (#9478) 2026-07-14 10:03:46 -07:00
tsconfig.json
…
vitest.config.ts refactor(deps-dev): bump vitest from 3.2.4 to 4.1.8 (#7581) 2026-06-05 21:11:32 -07:00