Commit Graph

1 Commits

Author SHA1 Message Date
Devin Foley 0a01444514
test(release-smoke): cover the background-service leg of onboarding (#12151)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release pipeline gates each nightly and beta on a smoke suite
that onboards the published npm artifact and drives the golden path
> - That smoke runs onboarding inside a Docker container, and containers
have no service manager, so the background-service leg of onboarding has
zero automated coverage
> - v2026.824.0 shipped a service install that crash-looped on a missing
shim, and every smoke check stayed green (#12148 fixed the defect
itself)
> - This pull request adds a `smoke_service` job that runs the same
published artifact directly on the runner VM's systemd and requires the
installed service to end up serving
> - The benefit is that a release with a broken service install can no
longer pass the release smoke suite

## Linked Issues or Issue Description

Refs #12148 — the fix for the defect this coverage gap let through. The
gap: the release smoke runs `onboard` with `--yes` inside Docker, which
both skips the service prompt and lacks systemd, so no CI job ever
executed `manager.install()` against a real service manager.

## What Changed

- New `scripts/service-onboard-smoke.sh`: onboards the published
artifact with `--yes --install-service` on a systemd host, then fails
unless the managed shim exists and is executable, `paperclipai.service`
is active, and `/api/health` answers. A health response while the unit
is not active also fails, because that is the signature of something
other than the service serving. The script refuses to run over an
existing managed install unless `SMOKE_FORCE=true`, and cleans up after
itself by default so it is safe to run locally.
- New `smoke_service` job in `.github/workflows/release-smoke.yml`:
starts a user systemd session on the hosted runner (`loginctl
enable-linger` + exported `XDG_RUNTIME_DIR`/`DBUS_SESSION_BUS_ADDRESS`),
runs the script against `inputs.paperclip_version`, and uploads
`systemctl status` + journal output as diagnostics.
- No `release.yml` changes needed: `smoke_nightly` and `smoke_beta` call
this reusable workflow, and a `workflow_call` result aggregates all
jobs, so the new job gates nightly promotion automatically.

## Verification

- `bash -n scripts/service-onboard-smoke.sh` passes and the workflow
YAML parses.
- End-to-end: dispatched this branch's Release Smoke workflow against
the published canary that contains #12148; the `smoke_service` job
onboards, installs the service, and verifies the service serves health.
(Run link in PR comments.)
- Negative case: the same assertions fail against v2026.824.0 —
reproduced in a systemd container during the #12148 investigation: shim
missing, unit in a 203/EXEC restart loop.

## Risks

- Low risk to the product: no application code changes.
- Pipeline risk: a flaky user-session setup on the hosted runner would
block nightly promotion. Mitigated by validating the job end-to-end from
this branch before merge, a 30-minute job timeout, and diagnostics
uploaded on every run.
- The service leg only covers systemd. launchd (macOS) still has no CI
coverage; a macOS runner job is a possible follow-up.

## Model Used

- Claude Fable 5 (Anthropic, model ID `claude-fable-5`), extended
thinking, agentic tool use via Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 01:05:27 -07:00