## Thinking Path
> - Paperclip is the open source control plane people use to manage AI
agents and their work
> - The server coordinates agent heartbeats and preserves eligible live
runs during a hot restart
> - Shutdown previously waited for all heartbeat scheduler work before
capturing the hot-restart snapshot
> - A deployment heartbeat can itself be in that scheduler set while
waiting for the restart, creating a circular wait
> - The missing snapshot prevents startup from classifying and adopting
the still-running agent process
> - This pull request captures the snapshot first and skips
scheduler/drain waits only for an eligible hot restart
> - The benefit is a single SIGTERM can restart the server without
losing eligible live agent runs
## Linked Issues or Issue Description
- **Preflight:** Searched open and closed PRs for the hot-restart
shutdown deadlock; no duplicate found. Reproduced on `master` and
confirmed this is core Paperclip behavior.
- **What happened:** During a hot restart initiated by a running
deployment heartbeat, the SIGTERM handler waited for
`heartbeatSchedulerInFlight` before calling
`prepareHotRestartShutdown()`. The heartbeat was itself in that set and
waited for restart completion, so shutdown never wrote the adoption
snapshot.
- **Expected behavior:** An eligible hot restart captures its snapshot
before waiting for scheduler work, preserves live child processes, and
exits after one SIGTERM.
- **Steps to reproduce:**
1. Start a heartbeat that remains active while requesting a hot restart.
2. Send SIGTERM to the server process.
3. Observe shutdown waiting on the active scheduler task and startup
finding an intent without a shutdown snapshot.
- **Paperclip commit:** `992389480a243b97bda214227e0767eb8c3672af`
- **Deployment/install:** Self-hosted server built from source.
- **Adapter:** Not adapter-specific; reproduced with a Codex heartbeat.
- **Database/access:** Embedded PGlite; agent bearer context.
- **Environment:** Node `v22.22.2` on `Linux 6.17.0-1015-aws aarch64
GNU/Linux`.
- **Privacy:** No secrets, private logs, user paths, or internal issue
references are included.
## What Changed
- Add a focused shutdown coordinator that prepares hot-restart state
before waiting for heartbeat scheduler idleness.
- Skip scheduler-idle and graceful-drain waits only when the hot-restart
service returns `skipDrain: true`.
- Preserve normal graceful shutdown behavior when no eligible intent
exists or preparation fails.
- Add regression coverage for pending scheduler work, normal shutdown,
and preparation failure.
## Verification
- `pnpm exec vitest run server/src/shutdown.test.ts` — 3 passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t
'hot-restart'` — 3 passed, 88 skipped.
- `git diff --check origin/master...HEAD` — passed.
## Risks
- Low-to-moderate risk: shutdown ordering changes, but only the
explicitly eligible hot-restart path bypasses scheduler-idle and
run-drain waits.
- Normal shutdown and hot-restart preparation failures retain the
existing graceful behavior.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. This is a focused bug fix
and does not duplicate planned roadmap work.
## Model Used
- OpenAI Codex coding agent; exact runtime model ID and context-window
size are not exposed to the agent. Tool use, shell execution, repository
editing, and test execution were enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (no
documentation change required for this internal shutdown-order fix)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
263316609e