Commit Graph

5 Commits

Author SHA1 Message Date
Nicky Leach b57aa9950c
fix(test): stop flaky server-suite afterAll hook timeouts (#10024)
## Thinking Path

> - Paperclip is an open-source AI agent management platform; its test
suite spans a `server` package that mounts real embedded Postgres
databases in `beforeAll`/`afterAll` hooks
> - The `server` package CI shard runs all ~93 suites serially
(`maxWorkers=1`) on a loaded CI host; each suite boots and tears down
its own embedded Postgres in hook callbacks
> - vitest's default `hookTimeout` is 10 seconds; under load, graceful
embedded-Postgres shutdown occasionally crosses that threshold
> - This produces intermittent `Error: Hook timed out in 10000ms`
failures in `afterAll` hooks — not test assertion failures — and the
suites pass on re-run, making them textbook flaky tests
> - Inspecting `embedded-postgres@18.1.0-beta.16` shows that `stop()`
takes no argument (no fast-shutdown mode), SIGINTs postgres (already
PostgreSQL "fast shutdown"), and resolves only on the child's `exit`
event with no internal time bound
> - Two targeted fixes: (1) raise `hookTimeout` and `teardownTimeout` to
30 s in `server/vitest.config.ts` — one config change that eliminates
the flake for all ~93 suites at once; (2) wrap `stop()` in a 5 s bounded
`Promise.race` in the test helper so a slow shutdown can never hang the
hook regardless of OS scheduling variance
> - This PR changes only test-infra and test-config; no production-code
behavior changes

## Linked Issues or Issue Description

No public GitHub issue exists for this flake. Inline bug description
(bug report template):

**What happened?**

The `General tests (server (N/3))` CI shards intermittently fail with
`Error: Hook timed out in 10000ms` in `afterAll` hooks and pass on
re-run. Every test assertion passes; only the teardown hook exceeds
vitest's default timeout.

**Expected behavior**

CI passes reliably. Teardown timeouts should not be a source of flake.

**Steps to reproduce**

Run the server test suite repeatedly on a loaded host or in CI with
`maxWorkers=1` — the shard occasionally crosses 10 s in `afterAll`
during embedded-Postgres shutdown.

**Paperclip version**

`master`, any build that includes `server/vitest.config.ts` without an
explicit `hookTimeout`.

**Deployment mode**

Self-hosted (CI).

## What Changed

- **`server/vitest.config.ts`** — added `hookTimeout: 30000` and
`teardownTimeout: 30000`. Removes flake across all ~93 server suites at
once. 30 s gives generous headroom over observed worst-case teardown
while still catching a genuinely hung hook.
- **`packages/db/src/test-embedded-postgres.ts`** — added
`stopEmbeddedPostgresBounded()`, a 5 s `Promise.race` wrapper around
`stop()`. Applied at all three call sites inside `cleanup()`. Data dir
is still removed unconditionally; errors are still swallowed; the
null-instance guard is preserved. Existing behavior unchanged except the
shutdown can no longer block indefinitely.

## Verification

- `tsc --noEmit` clean on `packages/db` (built against worktree-local
`shared`)
- `packages/db` `client.test.ts` passes 14/14 — boots embedded Postgres
and exercises the bounded teardown via `cleanup()` in `afterEach`
- Standalone bounded-race semantics verified: hang resolves at the 5 s
bound; late or immediate `stop()` rejection swallowed; no unhandled
rejection; null-instance path safe
- CI: all 3 server shards + split-verify lane (Async-Verification Gate)
expected green after this PR

```bash
# Reproduce the teardown test locally:
cd packages/db && npx vitest run src/client.test.ts

# Type-check packages/db:
npx tsc --noEmit -p packages/db/tsconfig.json
```

## Risks

Low risk. No product-code changes — test-infra and test-config only. The
vitest timeout increase is additive (raises the ceiling; never lowers
it). The bounded race wrapper preserves prior teardown behavior exactly:
data dir always removed, errors always swallowed, stop is still
attempted. A worst-case outcome is that a genuinely hung `stop()` now
surfaces as a test timeout at 30 s instead of 10 s — still caught, just
later.

## Model Used

- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200 k tokens
- **Capabilities:** tool use, code execution, extended reasoning
- **Mode:** Paperclip agent heartbeat (autonomous execution with human
board oversight)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 11:51:30 -07:00
Devin Foley 20aea356cc
refactor(deps-dev): bump vitest from 3.2.4 to 4.1.8 (#7581)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Test infrastructure across server, ui, packages/* runs on Vitest
> - Dependabot opened a narrow bump (3.2.4 → 3.2.6), but the wider
workspace is on 3.2.4 and the major-version bridge to v4 needs a
coordinated change set across configs and tests
> - Staying on 3.x indefinitely leaves us behind on Vitest 4 (perf,
pool, and config improvements) and forces repeated patch-only dependabot
churn
> - This pull request upgrades Vitest to 4.1.8 across the workspace,
updates `server/vitest.config.ts` and `scripts/run-vitest-stable.mjs`
for the new API, and adjusts two UI tests for the new assertion
semantics
> - The benefit is a single, coherent Vitest 4 upgrade that supersedes
#7570 and gets us on the supported major line

## What Changed

- Bump `vitest` from `3.2.4` to `4.1.8` across root, `server`, `ui`, and
all `packages/*` (including plugin examples and sandbox providers)
- Update `server/vitest.config.ts` for Vitest 4 config surface
- Update `scripts/run-vitest-stable.mjs` to match the new runner
behavior
- Adjust `ui/src/components/CommentThread.test.tsx` and
`ui/src/components/MarkdownEditor.test.tsx` for Vitest 4 matcher/timing
semantics
- Refresh `pnpm-lock.yaml`

## Verification

- `pnpm install` resolves cleanly with the new lockfile
- `pnpm -w -r test` (server, ui, packages) runs under Vitest 4.1.8

## Risks

- Major-version Vitest bump: behavioral changes in pools, fake timers,
and matcher strictness can surface flake. Test config and the two UI
tests were updated to match v4 semantics; broader test runs should be
watched on CI before merge.
- Supersedes dependabot PR #7570 (3.2.4 → 3.2.6); that PR should be
closed.

## Model Used

- Claude (Anthropic) — `claude-opus-4-7`, extended thinking, tool use
enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Closes #7570
2026-06-05 21:11:32 -07:00
Dotta deba60ebb2
Stabilize serialized server route tests (#4448)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - The server route suite is a core confidence layer for auth, issue
context, and workspace runtime behavior
> - Some route tests were doing extra module/server isolation work that
made local runs slower and more fragile
> - The stable Vitest runner also needs to pass server-relative exclude
paths to avoid accidentally re-including serialized suites
> - This pull request tightens route test isolation and runner
serialization behavior
> - The benefit is more reliable targeted and stable-route test
execution without product behavior changes

## What Changed

- Updated `run-vitest-stable.mjs` to exclude serialized server tests
using server-relative paths.
- Forced the server Vitest config to use a single worker in addition to
isolated forks.
- Simplified agent permission route tests to create per-request test
servers without shared server lifecycle state.
- Stabilized issue goal context route mocks by using static mocked
services and a sequential suite.
- Re-registered workspace runtime route mocks before cache-busted route
imports.

## Verification

- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/agent-permissions-routes.test.ts
server/src/__tests__/issues-goal-context-routes.test.ts
server/src/__tests__/workspace-runtime-routes-authz.test.ts --pool=forks
--poolOptions.forks.isolate=true`
- `node --check scripts/run-vitest-stable.mjs`

## Risks

- Low risk. This is test infrastructure only.
- The stable runner path fix changes which tests are excluded from the
non-serialized server batch, matching the server project root that
Vitest applies internally.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5 coding agent, tool-enabled with
shell/GitHub/Paperclip API access. Context window was not reported by
the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-24 19:27:00 -05:00
Dotta 9a8d219949
[codex] Stabilize tests and local maintenance assets (#4423)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - A fast-moving control plane needs stable local tests and repeatable
local maintenance tools so contributors can safely split and review work
> - Several route suites needed stronger isolation, Codex manual model
selection needed a faster-mode option, and local browser cleanup missed
Playwright's headless shell binary
> - Storybook static output also needed to be preserved as a generated
review artifact from the working branch
> - This pull request groups the test/local-dev maintenance pieces so
they can be reviewed separately from product runtime changes
> - The benefit is more predictable contributor verification and cleaner
local maintenance without mixing these changes into feature PRs

## What Changed

- Added stable Vitest runner support and serialized route/authz test
isolation.
- Fixed workspace runtime authz route mocks and stabilized
Claude/company-import related assertions.
- Allowed Codex fast mode for manually selected models.
- Broadened the agent browser cleanup script to detect
`chrome-headless-shell` as well as Chrome for Testing.
- Preserved generated Storybook static output from the source branch.

## Verification

- `pnpm exec vitest run
src/__tests__/workspace-runtime-routes-authz.test.ts
src/__tests__/claude-local-execute.test.ts --config vitest.config.ts`
from `server/` passed: 2 files, 19 tests.
- `pnpm exec vitest run src/server/codex-args.test.ts --config
vitest.config.ts` from `packages/adapters/codex-local/` passed: 1 file,
3 tests.
- `bash -n scripts/kill-agent-browsers.sh &&
scripts/kill-agent-browsers.sh --dry` passed; dry-run detected
`chrome-headless-shell` processes without killing them.
- `test -f ui/storybook-static/index.html && test -f
ui/storybook-static/assets/forms-editors.stories-Dry7qwx2.js` passed.
- `git diff --check public-gh/master..pap-2228-test-local-maintenance --
. ':(exclude)ui/storybook-static'` passed.
- `pnpm exec vitest run
cli/src/__tests__/company-import-export-e2e.test.ts --config
cli/vitest.config.ts` did not complete in the isolated split worktree
because `paperclipai run` exited during build prep with `TS2688: Cannot
find type definition file for 'react'`; this appears to be caused by the
worktree dependency symlink setup, not the code under test.
- Confirmed this PR does not include `pnpm-lock.yaml`.

## Risks

- Medium risk: the stable Vitest runner changes how route/authz tests
are scheduled.
- Generated `ui/storybook-static` files are large and contain minified
third-party output; `git diff --check` reports whitespace inside those
generated assets, so reviewers may choose to drop or regenerate that
artifact before merge.
- No database migrations.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agent based on GPT-5, with shell, git, Paperclip
API, and GitHub CLI tool use in the local Paperclip workspace.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Note: screenshot checklist item is not applicable to source UI behavior;
the included Storybook static output is generated artifact preservation
from the source branch.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-24 15:11:42 -05:00
Forgotten c9d7cbfe44 Add API server with routes, services, and middleware
Express server with CRUD routes for agents, goals, issues, projects,
and activity log. Includes validation middleware, structured error
handling, request logging, and health check endpoint with tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 13:31:58 -06:00