test(heartbeat): await execution drain before cleanup (#10023)

## Thinking Path

> - Paperclip is the open source control plane people use to manage AI
agents and their work
> - Heartbeat scheduling tests protect the orchestration rules that
serialize an agent's runs
> - The dependency scheduling suite waits for run rows to become
terminal before deleting shared database fixtures
> - A terminal row is persisted before asynchronous execution
finalization and successful-run handoff work fully drain
> - The test then clears process tracking and deletes heartbeat events
while finalization can still append another event
> - This pull request waits for each tracked run's execution promise to
drain before resetting mocks or deleting fixtures
> - The benefit is deterministic cleanup that preserves the production
lifecycle ordering and prevents release CI flakes

## Linked Issues or Issue Description

### What happened?

Release run `29936031931` failed in
`heartbeat-dependency-scheduling.test.ts` while deleting
`heartbeat_runs`. Asynchronous heartbeat finalization inserted a new
`heartbeat_run_events` row after the test had already deleted existing
events, causing the run-row delete to violate the event foreign key.

### Expected behavior

The serialized heartbeat test suite should finish all asynchronous run
execution work before destructive fixture cleanup.

### Steps to reproduce

1. Check out commit `2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d`.
2. Run `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/heartbeat-dependency-scheduling.test.ts
--pool=forks --isolate` repeatedly with PostgreSQL test support enabled.
3. Observe that teardown can delete heartbeat events while execution
finalization is still able to append another event, causing a
foreign-key failure when heartbeat runs are deleted.

### Paperclip version or commit

`2aef4641b48e88f5ce7e75ce69fbe3bf6bbfc60d`

### Deployment mode

Other — GitHub Actions release verification.

### Installation method

Built from source with pnpm.

### Agent adapter(s) involved

Not adapter-specific (core heartbeat test lifecycle).

### Database mode

External PostgreSQL test database.

### Relevant logs or output

`delete from "heartbeat_runs"` failed because the run remained
referenced by `heartbeat_run_events_run_id_heartbeat_runs_id_fk`.

## What Changed

- Collect heartbeat run IDs after queued/running rows settle and await
`heartbeat.waitForRunExecutionDrain()` for each run.
- Reset the adapter mock and clear process tracking only after
asynchronous heartbeat finalization has completed.

## Verification

- Ran `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/heartbeat-dependency-scheduling.test.ts
--pool=forks --isolate` 10 consecutive times; all 10 runs passed with
6/6 tests.

## Risks

- Low risk: test-only cleanup ordering change using an existing
heartbeat service drain API. Production behavior is unchanged.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with exact model IDs `gpt-5.5` for this heartbeat and
`gpt-5.6-sol` for the recovered initial implementation run; tool-enabled
code inspection, GitHub diagnostics, and shell test execution. Runtime
context-window sizes were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta 2026-07-22 11:53:29 -05:00 committed by GitHub
parent 2aef4641b4
commit d54ff52fc3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 16 additions and 12 deletions

View File

@ -100,17 +100,6 @@ describeEmbeddedPostgres("heartbeat dependency-aware queued run selection", () =
}, 20_000);
afterEach(async () => {
mockAdapterExecute.mockReset();
mockAdapterExecute.mockImplementation(async () => ({
exitCode: 0,
signal: null,
timedOut: false,
errorMessage: null,
summary: "Dependency-aware heartbeat test run.",
provider: "test",
model: "test-model",
}));
runningProcesses.clear();
let idlePolls = 0;
for (let attempt = 0; attempt < 100; attempt += 1) {
const runs = await db
@ -125,7 +114,22 @@ describeEmbeddedPostgres("heartbeat dependency-aware queued run selection", () =
}
await new Promise((resolve) => setTimeout(resolve, 50));
}
await new Promise((resolve) => setTimeout(resolve, 50));
const runIds = await db
.select({ id: heartbeatRuns.id })
.from(heartbeatRuns)
.then((runs) => runs.map((run) => run.id));
await Promise.all(runIds.map((runId) => heartbeat.waitForRunExecutionDrain(runId)));
mockAdapterExecute.mockReset();
mockAdapterExecute.mockImplementation(async () => ({
exitCode: 0,
signal: null,
timedOut: false,
errorMessage: null,
summary: "Dependency-aware heartbeat test run.",
provider: "test",
model: "test-model",
}));
runningProcesses.clear();
await db.delete(environmentLeases);
await db.delete(activityLog);
await db.delete(companySkills);