Commit Graph

10 Commits

Author SHA1 Message Date
Devin Foley dbf052577d
Follow the current onboarding arc in the release smoke (#12423)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A release is gated by the release smoke: it installs the published
`paperclipai` artifact into a Docker container and drives the sign-in →
onboarding → first-agent path with Playwright
> - That suite runs only from the release pipeline, never on a pull
request, so it sees the UI only after the UI has already changed
> - The onboarding wizard was rebuilt into the agent arc. The "Name your
organization" step, the "Start Onboarding" launcher, and the agent role
picker are all gone
> - The spec still waited for those, so it failed on its first assertion
and blocked every nightly and beta release
> - The failure was also hard to read. The workflow uploaded no
container logs, because it learned the container's name only after the
harness succeeded, and the harness ran the container with `--rm` and
deleted it before anything read it
> - This pull request rewrites the spec to follow the current arc, and
repairs the log capture at both ends
> - The benefit is that nightly and beta releases are unblocked, and the
next failure arrives with the logs attached

## Linked Issues or Issue Description

No existing issue. Describing it inline, following
`.github/ISSUE_TEMPLATE/bug_report.yml`.

Refs #12274 (removed the company-naming step from the wizard).
Refs #12135 (the previous alignment of this spec, before #12274).
Refs #12316 (open; also edits `scripts/docker-onboard-smoke.sh`, in the
bootstrap helpers rather than the container lifecycle, so the two
changes do
not overlap. Whichever lands second should rebase and re-run).

**What happened?**

The release smoke fails.
`tests/release-smoke/docker-auth-onboarding.spec.ts`
never gets past its first wait:

```
✘ tests/release-smoke/docker-auth-onboarding.spec.ts:43:3 › Docker authenticated onboarding smoke › logs in, completes onboarding, and hires the lead agent
Error: expect(locator).toBeVisible() failed — element(s) not found (timeout 20000ms)
> 33 | await expect(wizardHeading.or(startButton)).toBeVisible({ timeout: 20_000 });
```

The spec waits for an `h3` reading "Name your organization" or a
"Start Onboarding" button. Neither exists. #12274 removed the
company-naming
step; the string now survives only in a code comment and in
`ui/src/components/OnboardingWizard.step.test.tsx`, which asserts it is
*absent*. The steps after the first wait are stale too: the CTA on step
1 is
"Continue" and not "Next", the organization input's placeholder changed,
and
the agent step's `#onboarding-agent-role` picker is gone, so every
onboarding
hire is filed under the neutral `general` role.

The suite runs only from the release pipeline, so nothing on a pull
request
saw the drift. Both `smoke_nightly` and `smoke_beta` call the same
reusable
workflow, so every nightly and every beta was blocked.

The failure also arrived without diagnostics. The job's "Capture Docker
logs"
step is `if: always()`, but it is guarded on `SMOKE_CONTAINER_NAME`,
which the
"Launch Docker smoke harness" step writes to `$GITHUB_ENV` only *after*
the
harness returns. On any failure before that the guard is false, the step
does
nothing, and the upload reports "No files were found". Below that,
`scripts/docker-onboard-smoke.sh` starts the container with
`docker run -d --rm`, so the `docker stop` in its EXIT trap deletes the
container and its logs together — and a container that crashes on its
own is
removed the instant its process exits.

**Expected behavior**

The spec walks the onboarding arc the app actually presents, and proves
the
company is created, the lead agent is hired, and the first task is
seeded and
dispatched. When the smoke fails, the run's artifact carries the
container's
logs.

**Steps to reproduce**

1. Run the Release Smoke workflow against a published artifact that
carries
   #12274, or run it locally:
`PAPERCLIPAI_VERSION=2026.828.0-canary.3 SMOKE_DETACH=true
./scripts/docker-onboard-smoke.sh`
2. Run `pnpm run test:release-smoke` against that container.
3. The single spec fails at `openOnboarding()` after 20 seconds.
4. In CI, open the run's `release-smoke` artifact. It has no
   `docker-onboard-smoke.log`.

**Paperclip version or commit**

`2026.828.0-canary.3` (commit 8316ceb0b).

**Deployment mode**

Docker.

**Installation method**

npm / pnpm global install (the container runs `npx
paperclipai@<version>`).

**Node.js version**

v24.20.0 inside the container.

**Relevant logs or output**

```
Running 1 test using 1 worker
  ✓  1 [chromium] › tests/release-smoke/docker-auth-onboarding.spec.ts:76:3 › Docker authenticated onboarding smoke › logs in, completes onboarding, and hires the lead agent (7.0s)
  1 passed (8.7s)
```

That is the result after this change. Before it, the same command failed
at
the first wait, as quoted above.

## What Changed

- `tests/release-smoke/docker-auth-onboarding.spec.ts` now follows the
current
arc. It signs in, opens `/onboarding`, names the organization and
presses
"Continue" (which creates the company and routes straight to the agent
step,
because onboarding no longer asks for a mission), names the lead and
presses
  "Next", presses "Connect" on the default adapter to hire, then presses
  "Get started" to launch.
- The spec addresses controls by role and accessible name, or by id
where one
exists (`#onboarding-agent-name`). Step 1's field has no id and no
associated
  label, so it is found as the wizard's only text box rather than by its
  placeholder copy.
- The spec asserts the hired agent's role is `general`, which is what
the arc
files every onboarding hire under. Every other API assertion is
unchanged.
- The spec navigates to `/onboarding` explicitly and drops any saved
onboarding
draft first, so it can run twice against one instance. The suite retries
once
  in CI. It still asserts that a company-less board routes sign-in into
  onboarding, guarded on the board actually being empty.
- `scripts/docker-onboard-smoke.sh` accepts `SMOKE_CONTAINER_NAME`,
drops
  `--rm`, removes the container itself, and dumps `docker logs` to
  `SMOKE_LOG_FILE` before the teardown.
- `.github/workflows/release-smoke.yml` pins the container name in the
job's
`env`, so every `always()` step has it before anything runs. The capture
step
  refreshes the log from a live container when there is one, keeps the
harness's dump when there is not, and writes a one-line explanation when
  there is neither. The upload's paths are literals, and
`if-no-files-found: error` makes a broken diagnostics path fail rather
than
  warn.
- `scripts/docker-onboard-smoke.test.mjs` pins that wiring. It is added
to
  `test:release-registry`, which runs on every pull request.
- `doc/DOCKER.md` documents `SMOKE_CONTAINER_NAME` and `SMOKE_LOG_FILE`.

## Verification

The spec was run against a real container built from the published
`2026.828.0-canary.3` artifact, exactly as the workflow runs it.

```sh
SMOKE_CONTAINER_NAME=release-smoke-onboard \
HOST_PORT=3232 DATA_DIR=<tmp>/smoke-data \
PAPERCLIPAI_VERSION=2026.828.0-canary.3 \
SMOKE_READY_TIMEOUT_SECONDS=420 SMOKE_DETACH=true \
SMOKE_METADATA_FILE=<tmp>/release-smoke.env \
SMOKE_LOG_FILE=<tmp>/docker-onboard-smoke.log \
  ./scripts/docker-onboard-smoke.sh

PAPERCLIP_RELEASE_SMOKE_BASE_URL=http://localhost:3232 \
PAPERCLIP_RELEASE_SMOKE_EMAIL=smoke-admin@paperclip.local \
PAPERCLIP_RELEASE_SMOKE_PASSWORD=paperclip-smoke-password \
PAPERCLIP_PLAYWRIGHT_CHANNEL=chrome \
  pnpm run test:release-smoke
```

```
Running 1 test using 1 worker
  ✓  1 [chromium] › tests/release-smoke/docker-auth-onboarding.spec.ts:76:3 › Docker authenticated onboarding smoke › logs in, completes onboarding, and hires the lead agent (7.0s)
  1 passed (8.7s)
```

The same command was run a second time against the same, now non-empty,
instance. That covers the retry path, and it also passes.

The log capture was verified by making the container die during startup:

```sh
PAPERCLIPAI_VERSION=0.0.0-no-such-version \
SMOKE_CONTAINER_NAME=release-smoke-onboard SMOKE_LOG_FILE=<tmp>/fail.log \
  ./scripts/docker-onboard-smoke.sh
```

`<tmp>/fail.log` was written and carried the cause:

```
npm error code ETARGET
npm error notarget No matching version found for paperclipai@0.0.0-no-such-version.
```

The container was removed afterwards. On `master` this file is never
written,
because `--rm` deletes the container the moment its process exits.

The workflow's capture step was run by hand against three states: a live
container (258 lines), a removed container with the harness's dump
already on
disk (258 lines kept), and neither (a one-line explanation).

Unit coverage:

```sh
pnpm run test:release-registry   # 93 tests, 93 pass
```

Nothing under `ui/` changed, so `pnpm --filter @paperclipai/ui
typecheck` was
not required. `tests/release-smoke` is outside the TypeScript project
references; Playwright compiles it at run time, which the runs above did
three
times.

## Risks

Low risk. Nothing ships to users. The change touches one Playwright
spec, one
smoke script, and one workflow.

Points worth a reviewer's attention:

- **This suite gates every nightly and beta, and it runs only
post-merge.**
`smoke_nightly` and `smoke_beta` both call `release-smoke.yml`, and no
pull
  request runs it. Drift between the wizard and this spec is therefore
invisible until a release is already blocked, which is how this bug
reached
a release train. I think the arc deserves an earlier check. The cheapest
version is the one added here: `scripts/docker-onboard-smoke.test.mjs`
runs
on every pull request and pins the harness wiring. The full container
smoke
is too slow for the pull request path, but a UI-level test of the arc's
step
  sequence would catch exactly this class of drift, and
`ui/src/components/OnboardingWizard.step.test.tsx` is already the right
  home for it. I did not add it here, to keep this change to the repair.
- **Dropping `--rm`.** The container is now removed by the script's
cleanup
instead of by Docker. The script already ran `docker rm -f` before
starting,
  and the workflow's final step removes it too, so a leaked container is
cleaned up on the next run either way. A developer who kills the script
with
`SIGKILL` will leave a stopped container behind, where previously they
would
  not.
- **`if-no-files-found: error` on the upload.** The capture step now
always
writes the log file, so the upload always has at least one path to
match. If
  that ever stops being true, the job fails instead of warning. That is
  deliberate.
- **The spec drops the saved onboarding draft before it walks.** A stale
draft
makes step 1 skip company creation and hire into the previous run's
company.
That state only exists when the spec runs twice against one instance. A
fresh
  release-smoke container never has it.

## Model Used

Claude (Anthropic), Claude Opus, 1M context, extended thinking, agentic
tool
use via Claude Code. The container, the Playwright runs, and the failure
injection were driven as real commands on a local Docker host.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-28 07:21:08 -07:00
Devin Foley 30f6999cbe
fix(release-smoke): configurable readiness timeout and diagnostics for slow containers (#11187)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release subsystem's nightly lane (#11006) gates every nightly
publish on the release smoke suite, which boots the published artifact
in a Docker container
> - The suite's first CI execution failed at the health readiness check:
the harness hard-codes a 90 second budget, but a CI container
cold-installs paperclipai from npm and initializes embedded postgres
with no warm caches
> - When the timeout expired with the container still running, the
harness printed no container logs, so the failure gave no diagnostics
> - This pull request makes the readiness budget configurable, raises it
for CI, and dumps container logs on timeout
> - The benefit is that the nightly gate measures the artifact, not the
runner's cold caches, and a red smoke run is diagnosable from its logs

## Linked Issues or Issue Description

**Subsystem affected**

Release smoke testing: `scripts/docker-onboard-smoke.sh`,
`.github/workflows/release-smoke.yml`.

**Problem or motivation**

Run 31426044332 (first forced nightly after #11006) failed in
`smoke_nightly` with `server did not become ready at
http://localhost:3232/api/health` after exactly 90 seconds. The
harness's readiness window is hard-coded to 90 attempts at 1 second.
Locally that works because the npm cache is warm; in CI the container
downloads the full package set and embedded postgres first. The timeout
path also printed no container logs when the container was still
running, so there was no way to see how far boot had progressed.

**Proposed solution**

Make the readiness budget an environment variable
(`SMOKE_READY_TIMEOUT_SECONDS`, default unchanged at 90 for local use),
set it to 420 in the CI workflow, and dump the last 150 container log
lines when the readiness check times out on a still-running container.

## What Changed

- `scripts/docker-onboard-smoke.sh`: `SMOKE_READY_TIMEOUT_SECONDS` env
var (default 90) replaces the hard-coded readiness budget; timeout with
a still-running container now prints the tail of `docker logs`
- `.github/workflows/release-smoke.yml`: sets
`SMOKE_READY_TIMEOUT_SECONDS=420` for CI runs

## Verification

- `bash -n` on the harness and YAML parse of the workflow
- The real proof is the next `channel: nightly` dispatch of
`release.yml`, which re-runs this suite in CI with the new budget

## Risks

- Low. The local default is unchanged; CI runs simply wait longer before
declaring failure, and a genuinely broken artifact still fails (with
logs now)

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with
extended thinking and full tool use. Diagnosis from CI run logs; patch
model-authored under human direction.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending — will confirm before
merge)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending — will confirm before merge)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-10 13:19:33 -07:00
Cody (Radius Red) 420cd4fd8d chore(docker): improve base image and organize docker files
- Add wget, ripgrep, python3, and GitHub CLI (gh) to base image
- Add OPENCODE_ALLOW_ALL_MODELS=true to production ENV
- Move compose files, onboard-smoke Dockerfile to docker/
- Move entrypoint script to scripts/docker-entrypoint.sh
- Add Podman Quadlet unit files (pod, app, db containers)
- Add docker/README.md with build, compose, and quadlet docs
- Add scripts/docker-build-test.sh for local build validation
- Update all doc references for new file locations
- Keep main Dockerfile at project root (no .dockerignore changes needed)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-01 11:36:27 +00:00
dotta 19f4a78f4a feat: add release smoke workflow 2026-03-18 07:59:32 -05:00
Dotta 01c5a6f198 Unblock canary onboard smoke bootstrap 2026-03-09 16:06:16 -05:00
Dotta 64f5c3f837 Fix authenticated smoke bootstrap flow 2026-03-09 15:30:08 -05:00
Dotta c62266aa6a tweaks to docker smoke 2026-03-09 14:41:00 -05:00
Dotta b66c6d017a Adjust docker onboard smoke defaults and console guidance 2026-03-04 10:48:36 -06:00
Dotta bbf7490f32 Fix onboard smoke Docker flow for clean npx runs 2026-03-04 10:42:07 -06:00
Dotta ea637110ac Add Ubuntu onboard smoke flow and lazy-load auth startup 2026-03-04 10:15:11 -06:00