From 71e9d6bb0b7af397ddf7e3665455ea36972b9b22 Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Tue, 11 Aug 2026 10:05:13 -0700 Subject: [PATCH] feat(release): candidate-branch beta builds and the release checklist (#11209) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit > Follow-up to #11208 (merged): rebased onto master and ready for review. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release channels promote artifacts along canary → nightly → beta → stable, with the happy path being promotion of an existing build > - When one or two targeted fixes are needed before a beta or stable, the only options today are waiting for the next nightly or absorbing a whole day of unrelated master changes > - The channel model was designed with an escape hatch for exactly this: short-lived candidate branches carrying only cherry-picked fixes > - This pull request implements candidate-branch beta builds with full verification, documents the stable fix path through the soak-justification gate, and adds the release captain's checklist > - The benefit is that a surgical fix can ship forward without either delay or blast radius, with its provenance recorded ## Linked Issues or Issue Description Refs #11008 — completes the fix-path half of the channel model introduced there. **Subsystem affected** Release automation: `scripts/release.sh`, `.github/workflows/release.yml`, `doc/RELEASING.md`, new `doc/RELEASE-CHECKLIST.md`, tests. **Problem or motivation** Beta promotion only accepts commits that already shipped as a nightly, and stable promotion expects a soaked beta. There is no supported way to ship one or two cherry-picked fixes between lanes: an urgent fix must wait for the nightly cycle or pull in every unrelated master change from the day. The original channel design called for candidate branches to cover this, and they were deferred from the initial implementation. **Proposed solution** Candidate-branch beta builds: cut `candidate/beta-` from a nightly's source commit, cherry-pick the fixes, and dispatch `channel: beta` with the new `candidate_branch` input. Selection enforces the naming convention, rejects heads that already shipped as a beta or predate the candidate tooling, and records the cherry-picked commits in the job summary. Because candidate heads never went through a canary or nightly, publication is gated on a full `release-verify` run (promoted nightlies keep skipping re-verification). The stable fix path (`candidate/release-` as `source_ref`) works through the existing soak gate: the justification requirement is the deliberate, recorded trade-off for shipping unsoaked bits, and is now documented as such. ## What Changed - `scripts/release.sh`: `--from-candidate` flag (beta only) waives the shipped-a-nightly requirement while keeping the duplicate-beta guard - `.github/workflows/release.yml`: `candidate_branch` dispatch input; candidate mode in `select_beta` (naming validation, duplicate and tooling-era rejection, cherry-pick recording); new `verify_beta_candidate` job gating candidate publishes on full verification - `doc/RELEASING.md`: beta fix-path and stable fix-path sections - `doc/RELEASE-CHECKLIST.md` (new): the release captain's checklist for all four lanes as built - Tests: dry-run fixture coverage for `--from-candidate` (waives the nightly guard, keeps the duplicate guard, rejected outside beta) and wiring tests for candidate validation plus the verification gate ## Verification - `node --test` on the four affected suites: 42 pass in total (17 + 25 across the two runs), including the 5 new tests - `bash -n` on `release.sh`; YAML parse of the workflow - After merge: exercise the path end to end the first time a real cherry-picked beta is needed — dispatch with a `candidate/beta-*` branch and confirm the summary records the picks and verification runs ## Risks - Candidate builds bypass the smoke-tested-nightly provenance by design; the compensating controls are full verification before publish, the post-publish beta smoke, the human `npm-beta` gate, and recorded cherry-picks - The stable fix path rides the existing justification mechanism rather than adding a second bypass — one recorded escape hatch, not two ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use. All changes model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge --- .github/workflows/release.yml | 79 ++++++++++++++++++- doc/RELEASE-CHECKLIST.md | 72 +++++++++++++++++ doc/RELEASING.md | 31 ++++++++ .../__tests__/release-dry-run-notes.test.mjs | 20 +++++ .../release-verify-workflow.test.mjs | 13 +++ scripts/release.sh | 21 ++++- 6 files changed, 233 insertions(+), 3 deletions(-) create mode 100644 doc/RELEASE-CHECKLIST.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3e07b36372..931694db1b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -31,6 +31,10 @@ on: description: For nightly, the explicit canary version to promote (empty selects the newest canary on master). For beta, the explicit nightly version to promote (empty selects the newest nightly on master). required: false type: string + candidate_branch: + description: (beta) candidate/beta-* branch to build a cherry-picked beta from. Leave empty to promote a nightly. Mutually exclusive with source_version. + required: false + type: string skip_soak_justification: description: (stable) Written justification for publishing a stable whose source has not soaked as a beta for 3 days. Leave empty for normal releases. required: false @@ -362,6 +366,7 @@ jobs: outputs: sha: ${{ steps.select.outputs.sha }} nightly_version: ${{ steps.select.outputs.nightly_version }} + mode: ${{ steps.select.outputs.mode }} steps: - name: Checkout repository uses: actions/checkout@v7 @@ -373,11 +378,65 @@ jobs: id: select env: EXPLICIT_NIGHTLY_VERSION: ${{ inputs.source_version }} + CANDIDATE_BRANCH: ${{ inputs.candidate_branch }} run: | set -euo pipefail git fetch origin --tags --prune --quiet + # Candidate mode: build a cherry-picked beta from a short-lived + # candidate branch instead of promoting a nightly. + if [ -n "${CANDIDATE_BRANCH:-}" ]; then + if [ -n "${EXPLICIT_NIGHTLY_VERSION:-}" ]; then + echo "Error: candidate_branch and source_version are mutually exclusive." >&2 + exit 1 + fi + case "$CANDIDATE_BRANCH" in + candidate/beta-*) ;; + *) + echo "Error: candidate branches must be named candidate/beta- (got: $CANDIDATE_BRANCH)." >&2 + exit 1 + ;; + esac + git fetch origin "$CANDIDATE_BRANCH" --quiet + sha="$(git rev-parse --verify "origin/${CANDIDATE_BRANCH}^{commit}" 2>/dev/null || true)" + if [ -z "$sha" ]; then + echo "Error: candidate branch $CANDIDATE_BRANCH does not exist on origin." >&2 + exit 1 + fi + + existing_beta="$(git tag --points-at "$sha" | grep '^beta/v' | head -1 || true)" + if [ -n "$existing_beta" ]; then + echo "Error: candidate head $sha already shipped as $existing_beta." >&2 + exit 1 + fi + + if ! git show "${sha}:scripts/release.sh" | grep -qF -- '--from-candidate'; then + echo "Error: candidate head $sha predates candidate-build release tooling; rebase the candidate onto a newer base." >&2 + exit 1 + fi + + merge_base="$(git merge-base origin/master "$sha")" + echo "mode=candidate" >> "$GITHUB_OUTPUT" + echo "sha=$sha" >> "$GITHUB_OUTPUT" + echo "nightly_version=" >> "$GITHUB_OUTPUT" + { + echo "## Beta candidate branch" + echo "" + echo "- Branch: \`$CANDIDATE_BRANCH\`" + echo "- Head: \`$sha\`" + echo "- Base (merge-base with master): \`$merge_base\`" + echo "- Cherry-picked commits:" + echo "" + echo '\`\`\`' + git log --oneline "${merge_base}..${sha}" + echo '\`\`\`' + } >> "$GITHUB_STEP_SUMMARY" + exit 0 + fi + + echo "mode=promote" >> "$GITHUB_OUTPUT" + if [ -n "${EXPLICIT_NIGHTLY_VERSION:-}" ]; then tag="nightly/v${EXPLICIT_NIGHTLY_VERSION}" sha="$(git rev-list -n 1 "$tag" 2>/dev/null || true)" @@ -425,8 +484,23 @@ jobs: echo "- Source nightly: \`$nightly_version\`" } >> "$GITHUB_STEP_SUMMARY" - publish_beta: + # Candidate-branch heads are new commits that never went through a canary + # or nightly, so they must pass full verification before publishing. + # Promoted nightlies were already verified by their canary run and skip it. + verify_beta_candidate: needs: select_beta + if: needs.select_beta.outputs.mode == 'candidate' + uses: ./.github/workflows/release-verify.yml + with: + ref: ${{ needs.select_beta.outputs.sha }} + + publish_beta: + needs: [select_beta, verify_beta_candidate] + if: >- + !cancelled() && + needs.select_beta.result == 'success' && + (needs.verify_beta_candidate.result == 'success' || + (needs.verify_beta_candidate.result == 'skipped' && needs.select_beta.outputs.mode == 'promote')) runs-on: ubuntu-latest timeout-minutes: 45 environment: npm-beta @@ -480,6 +554,9 @@ jobs: GITHUB_ACTIONS: "true" run: | args=(beta --skip-verify) + if [ "${{ needs.select_beta.outputs.mode }}" = "candidate" ]; then + args+=(--from-candidate) + fi if [ "${{ inputs.dry_run }}" = "true" ]; then args+=(--dry-run) fi diff --git a/doc/RELEASE-CHECKLIST.md b/doc/RELEASE-CHECKLIST.md new file mode 100644 index 0000000000..3ab74932a8 --- /dev/null +++ b/doc/RELEASE-CHECKLIST.md @@ -0,0 +1,72 @@ +# Release Checklist + +The release captain's checklist for every lane. The mechanics live in +[`RELEASING.md`](RELEASING.md); the user-facing channel guide is +[`CHANNELS.md`](CHANNELS.md). + +## Canary (automatic, every `master` push) + +- [ ] the push's `Release` run is green (verify + publish) +- [ ] `npm view paperclipai@canary version` matches the expected canary +- [ ] Docker `:canary` updated (the same push's `Docker` run) +- [ ] a canary publish failure is a release-infra regression — fix it before + trusting later promotions + +## Nightly (automatic, 09:00 UTC) + +- [ ] the scheduled run selected the newest green canary, or skipped with a + job-summary reason (no new candidate / already shipped / red smoke) +- [ ] the release smoke suite passed against the exact candidate canary + before anything published +- [ ] `npm view paperclipai@nightly version` shows the new `-nightly.N` +- [ ] `nightly/v*` tag pushed; `:nightly` and `:nightly-cloud` images built +- [ ] on a tag-push rejection (workflows-permission error), follow the + recovery commands in the job summary + +To force a nightly: dispatch `release.yml` with `channel: nightly` +(optional exact canary in `source_version`; `dry_run` to preview). + +## Beta (manual promotion) + +Happy path: + +- [ ] pick the nightly to promote (empty `source_version` selects the newest) +- [ ] dispatch `release.yml` with `channel: beta` +- [ ] approve the `npm-beta` environment gate +- [ ] `npm view paperclipai@beta version` shows the new `-beta.N` +- [ ] `beta/v*` tag pushed; `:beta` and `:beta-cloud` images built +- [ ] post-publish smoke (`smoke_beta`) is green + +Fix path (cherry-picked candidate): + +- [ ] cut `candidate/beta-` from the chosen nightly's source commit +- [ ] cherry-pick only the required fixes; push the branch +- [ ] dispatch `channel: beta` with `candidate_branch` +- [ ] confirm the job summary records the cherry-picked commits and that + full verification ran on the candidate head +- [ ] after shipping: reconcile the fixes to `master`, delete the branch + +## Stable (manual promotion) + +- [ ] pick the beta to promote; its source commit is `source_ref` +- [ ] the beta has soaked ≥ 3 days with no open beta-blocker issues +- [ ] author `releases/vYYYY.MDD.P.md` on that source ref +- [ ] dispatch `release.yml` with `channel: stable` (a dry run first shows + the resolved version and soak state without publishing) +- [ ] approve the `npm-stable` environment gate +- [ ] `npm view paperclipai version` (dist-tag `latest`) shows the stable +- [ ] `vYYYY.MDD.P` tag pushed; GitHub Release created; `:latest` and the + versioned Docker tags built +- [ ] if the soak gate was bypassed, `skip_soak_justification` carries a + real written reason (it lands in the job summary) + +Fix path: `candidate/release-` from the beta's source commit; the +soak gate will demand a justification because the exact bits were not +soaked — write one that stands on its own. + +## After any incomplete run + +The failure playbooks in [`RELEASING.md`](RELEASING.md) cover: red canary, +skipped or failed nightly, a beta that looks bad during soak, partial +stable releases, broken `latest`, and rejected tag pushes. Every publish +job's summary names what completed and what remains. diff --git a/doc/RELEASING.md b/doc/RELEASING.md index f6f3e6d41c..1e1891663d 100644 --- a/doc/RELEASING.md +++ b/doc/RELEASING.md @@ -166,6 +166,28 @@ Users install betas with: npx paperclipai@beta onboard ``` +#### Beta fix path: candidate branches + +When one or two targeted fixes are needed before beta and waiting for the +next nightly (or absorbing a whole day of `master`) is wrong, build the beta +from a short-lived candidate branch: + +1. cut `candidate/beta-` from the chosen nightly's source commit + (for example `candidate/beta-2026.811.0`) +2. cherry-pick only the required fix commits onto it and push the branch +3. dispatch `release.yml` with `channel: beta` and `candidate_branch: + candidate/beta-` +4. selection validates the branch name, rejects heads that already shipped + as a beta, records the cherry-picked commits in the job summary, and the + head runs **full verification** before publishing (it never went through + a canary or nightly) +5. after the beta ships, land the fixes on `master` normally and delete the + candidate branch + +Use this sparingly: the happy path is promoting a nightly. A candidate build +has its own `-beta.N` identity and is never pretended to be the nightly it +was cut from. + ### Stable Use [`.github/workflows/release.yml`](../.github/workflows/release.yml) from the Actions tab with the manual `workflow_dispatch` inputs. @@ -193,6 +215,14 @@ The stable preflight enforces the beta soak: the source commit must carry a the run fails unless `skip_soak_justification` is provided; the justification is echoed into the job summary. Dry runs report soak state without blocking. +For a cherry-picked stable (the release fix path), cut +`candidate/release-` from the chosen beta's source commit, +cherry-pick the required fixes, push the branch, and use it as +`source_ref`. The candidate head carries no `beta/v*` tag, so the soak gate +requires `skip_soak_justification` — that is deliberate: the exact bits were +not soaked, and the justification is the recorded trade-off. Reconcile the +fixes back to `master` and delete the branch after shipping. + Before running stable: 1. pick the beta you are promoting (its source commit is the `source_ref`) @@ -429,5 +459,6 @@ Then fix forward with a new stable release. - [`scripts/release-package-map.mjs`](../scripts/release-package-map.mjs) - [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) - [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) +- [`doc/RELEASE-CHECKLIST.md`](RELEASE-CHECKLIST.md) - [`doc/PUBLISHING.md`](PUBLISHING.md) - [`doc/RELEASE-AUTOMATION-SETUP.md`](RELEASE-AUTOMATION-SETUP.md) diff --git a/scripts/__tests__/release-dry-run-notes.test.mjs b/scripts/__tests__/release-dry-run-notes.test.mjs index 64f70fcc0f..6f1d12d003 100644 --- a/scripts/__tests__/release-dry-run-notes.test.mjs +++ b/scripts/__tests__/release-dry-run-notes.test.mjs @@ -218,6 +218,26 @@ test("beta refuses commits that already shipped as a beta", () => { assert.doesNotMatch(result.calls, /^pnpm /m); }); +test("beta --from-candidate waives the nightly requirement but keeps the duplicate guard", () => { + const result = runRelease(["beta", "--from-candidate", "--skip-verify", "--dry-run"], { + FAKE_MISSING_CHANNEL_TAG: "nightly", + }); + + assert.equal(result.status, 42); + assert.doesNotMatch(result.output, /require_channel_tag_at_head nightly/); + assert.match(result.output, /\[fixture\] require_channel_tag_absent_at_head beta/); + assert.match(result.output, /Beta version: 2026\.710\.0-beta\.0/); + assert.match(result.calls, /^pnpm build$/m); +}); + +test("--from-candidate is rejected outside the beta channel", () => { + const result = runRelease(["nightly", "--from-candidate", "--skip-verify", "--dry-run"]); + + assert.equal(result.status, 1); + assert.match(result.output, /--from-candidate only applies to the beta channel/); + assert.doesNotMatch(result.calls, /^pnpm /m); +}); + test("beta refuses commits that never shipped a nightly", () => { const result = runRelease(["beta", "--skip-verify", "--dry-run"], { FAKE_MISSING_CHANNEL_TAG: "nightly", diff --git a/scripts/__tests__/release-verify-workflow.test.mjs b/scripts/__tests__/release-verify-workflow.test.mjs index df30f9e953..354bd81c4c 100644 --- a/scripts/__tests__/release-verify-workflow.test.mjs +++ b/scripts/__tests__/release-verify-workflow.test.mjs @@ -47,6 +47,19 @@ test("promotion selection guards against sources that predate their channel tool assert.match(releaseWorkflow, /git show "\$\{sha\}:scripts\/release\.sh" \| grep -qF 'canary\|nightly\|beta\|stable\)'/); }); +test("candidate-branch betas are validated and fully verified before publish", () => { + const releaseWorkflow = readWorkflow("release.yml"); + + // Candidate heads are new commits: selection must pin the naming + // convention and publication must be gated on full verification. + assert.match(releaseWorkflow, /candidate\/beta-\*\)/); + assert.match( + releaseWorkflow, + /verify_beta_candidate:\n\s+needs: select_beta\n\s+if: needs\.select_beta\.outputs\.mode == 'candidate'\n\s+uses: \.\/\.github\/workflows\/release-verify\.yml/, + ); + assert.match(releaseWorkflow, /needs\.verify_beta_candidate\.result == 'success'/); +}); + test("every lane's tag push degrades to recovery instructions when rejected", () => { const releaseWorkflow = readWorkflow("release.yml"); diff --git a/scripts/release.sh b/scripts/release.sh index db3a4ea90a..5d51aab69a 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -11,6 +11,7 @@ release_date="" dry_run=false skip_verify=false print_version_only=false +from_candidate=false tag_name="" cleanup_on_exit=false @@ -25,6 +26,7 @@ Examples: ./scripts/release.sh canary --date 2026-03-17 --dry-run ./scripts/release.sh nightly --dry-run ./scripts/release.sh beta --dry-run + ./scripts/release.sh beta --from-candidate --dry-run ./scripts/release.sh stable ./scripts/release.sh stable --date 2026-03-17 --dry-run ./scripts/release.sh stable --date 2026-03-18 --print-version @@ -41,6 +43,9 @@ Notes: - Beta releases republish a commit that already shipped a nightly (HEAD must carry a nightly/v* tag) as YYYY.MDD.P-beta.N under the npm dist-tag "beta", with the git tag beta/vYYYY.MDD.P-beta.N. + - --from-candidate (beta only) waives the nightly-tag requirement for + cherry-picked candidate-branch builds; callers are responsible for + validating the candidate branch before using it. - Stable releases publish YYYY.MDD.P under the npm dist-tag "latest" and create the git tag vYYYY.MDD.P. - Non-dry-run stable release notes must already exist at releases/vYYYY.MDD.P.md. @@ -108,6 +113,7 @@ while [ $# -gt 0 ]; do --dry-run) dry_run=true ;; --skip-verify) skip_verify=true ;; --print-version) print_version_only=true ;; + --from-candidate) from_candidate=true ;; -h|--help) usage exit 0 @@ -124,6 +130,10 @@ done exit 1 } +if [ "$from_candidate" = true ] && [ "$channel" != "beta" ]; then + release_fail "--from-candidate only applies to the beta channel." +fi + PUBLISH_REMOTE="$(resolve_release_remote)" fetch_release_remote "$PUBLISH_REMOTE" @@ -167,8 +177,15 @@ elif [ "$channel" = "nightly" ]; then DIST_TAG="nightly" tag_name="$(prerelease_tag_name nightly "$TARGET_PUBLISH_VERSION")" elif [ "$channel" = "beta" ]; then - # Beta promotes an already-shipped nightly commit. - require_channel_tag_at_head nightly + if [ "$from_candidate" = true ]; then + # Candidate builds carry targeted cherry-picks that never shipped as a + # nightly, so the nightly-tag requirement does not apply; the workflow + # validates the candidate branch identity before invoking this path. + : + else + # Beta promotes an already-shipped nightly commit. + require_channel_tag_at_head nightly + fi require_channel_tag_absent_at_head beta TARGET_PUBLISH_VERSION="$(next_prerelease_version beta "$TARGET_STABLE_VERSION" "${PUBLIC_PACKAGE_NAMES[@]}")" DIST_TAG="beta"