feat(release): thorough notes skeletons — nest each PR's summary at creation (#12124)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release workflow drafts the upcoming stable's notes skeleton the
moment a beta publishes
> - That skeleton was a bare list of commit subjects, so the notes only
reached the shipped stable's depth after a later authoring pass during
the soak
> - Stable release notes are consistently verbose and thorough; the
initial draft should start that way too
> - This pull request nests each referenced PR's own summary under its
subject line at creation time, and states the density bar in the
authoring skill
> - The benefit is a thorough raw document from day one of the soak,
with no LLM tokens in Actions

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The `draft_stable_notes` skeleton generated at beta publish
(`scripts/draft-stable-notes.sh`).

**Current behavior**

The skeleton groups bare commit subjects by conventional-commit type.
All substance arrives later, when a maintainer or agent rewrites it —
reviewed maintainer feedback: stable notes are a lot more verbose, and
the initial beta notes should be consistent with that.

**Proposed behavior**

Each subject that references a PR carries that PR's own summary nested
beneath it — the PR template's "What Changed" bullets, else the first
prose lines — fetched best-effort via `gh` and skipped silently when
unavailable. The release-changelog skill now states the density bar
explicitly: the beta-keyed draft ships verbatim as the stable's notes
and is written at the previous stable's depth from the first pass.

**Reason and benefit**

The notes author starts from a thorough raw document instead of a commit
list, and beta-time notes match the verbosity the stable will ship with.

## What Changed

- `scripts/draft-stable-notes.sh`: `enrich_pr` nests PR summaries under
subjects; best-effort (`gh` failure or
`DRAFT_NOTES_SKIP_PR_ENRICHMENT=1` degrades to today's output);
pipefail-safe when a "What Changed" section has no bullets.
- `.github/workflows/release.yml`: the `draft_stable_notes` step gets
`GH_TOKEN` so `gh` can read PR bodies.
- `.agents/skills/release-changelog/SKILL.md`: "write at full stable
depth from the first pass" guideline.
- `scripts/draft-stable-notes.test.mjs`: three new tests — enrichment
rendering via a fake `gh`, silent degradation without one, and the
sparse-body case that previously killed the script under `set -o
pipefail`.

## Verification

- `node --test scripts/draft-stable-notes.test.mjs` — 11 pass.
- Live run against the real repository for the current beta
(`2026.818.0-beta.1`, 172 commits): exit 0, 439 nested summary lines;
spot-checked entries carry the correct PRs' What Changed bullets.
- `bash -n` on the script; `release.yml` re-parsed as YAML.

## Risks

- Low: the publish path is untouched; enrichment is read-only `gh` calls
in the post-publish draft job and degrades to the current skeleton on
any failure. Roughly one API call per commit in the range (~170 today) —
well inside the token's rate budget, adds a couple of minutes to a job
with a 10-minute timeout.

## Model Used

Claude Fable 5 (Claude Code)

## Pre-submission checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
This commit is contained in:
Devin Foley 2026-08-24 20:51:33 -07:00 committed by GitHub
parent ae9711da48
commit 890ab9acfe
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 110 additions and 2 deletions

View File

@ -168,6 +168,11 @@ Guidelines:
- write from the user perspective
- keep highlights short and concrete
- spell out upgrade actions for breaking changes
- **write at full stable depth from the first pass**: the beta-keyed
draft ships verbatim as the stable's notes, so the previous stable's
file is the density bar the moment the draft is first written — never
leave it at generated-skeleton density for the soak. The skeleton's
nested PR summaries are raw material to rewrite, not a format to keep.
- **describe deltas, not repeats**: read the previous stable's notes
(`releases/v<last-stable>.md`) before writing. When they already
introduced a feature, this release's entry covers only what changed —

View File

@ -647,6 +647,9 @@ jobs:
timeout-minutes: 10
permissions:
contents: write
# gh pr view needs PR read for the skeleton's nested summaries;
# without it the enrichment silently degrades to bare subjects.
pull-requests: read
steps:
- name: Checkout repository
uses: actions/checkout@v7
@ -661,6 +664,9 @@ jobs:
- name: Draft stable notes from the published beta
env:
# gh needs a token so the generator can nest each PR's summary
# under its subject line (best-effort thoroughness).
GH_TOKEN: ${{ github.token }}
BETA_VERSION: ${{ needs.publish_beta.outputs.beta_version }}
SOURCE_SHA: ${{ needs.select_beta.outputs.sha }}
run: |

View File

@ -123,6 +123,26 @@ fi
subjects="$(git -C "$repo_dir" log --no-merges --format='%s' "$range")"
# Best-effort thoroughness: nest each referenced PR's own summary under its
# subject line, so the skeleton is a genuinely thorough raw document at
# creation time instead of a bare commit list. Prefers the PR template's
# "What Changed" bullets, falls back to the first prose lines. Degrades
# silently when gh or the network is unavailable (tests, offline runs);
# set DRAFT_NOTES_SKIP_PR_ENRICHMENT=1 to disable explicitly.
enrich_pr() {
local pr_num="$1" pr_body excerpt
[ "${DRAFT_NOTES_SKIP_PR_ENRICHMENT:-0}" = "1" ] && return 0
pr_body="$(cd "$repo_dir" && gh pr view "$pr_num" --json body --jq .body 2>/dev/null || true)"
[ -n "$pr_body" ] || return 0
if printf '%s\n' "$pr_body" | grep -q '^## What Changed'; then
excerpt="$(printf '%s\n' "$pr_body" | sed -n '/^## What Changed/,/^## /p' | grep -E '^- ' | head -3 || true)"
else
excerpt="$(printf '%s\n' "$pr_body" | grep -vE '^[[:space:]]*$|^#|^>|^<!--' | head -2 || true)"
fi
[ -n "$excerpt" ] || return 0
printf '%s\n' "$excerpt" | sed 's/^/ > /'
}
section() {
local title="$1" pattern="$2" invert="${3:-false}" body
if [ "$invert" = true ]; then
@ -132,7 +152,14 @@ section() {
fi
[ -n "$body" ] || return 0
printf '## %s\n\n' "$title"
printf '%s\n' "$body" | sed 's/^/- /'
while IFS= read -r subject_line; do
[ -n "$subject_line" ] || continue
printf -- '- %s\n' "$subject_line"
pr_ref="$(printf '%s' "$subject_line" | grep -oE '\(#[0-9]+\)$' | tr -dc '0-9' || true)"
if [ -n "$pr_ref" ]; then
enrich_pr "$pr_ref"
fi
done <<< "$body"
printf '\n'
}

View File

@ -1,6 +1,6 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { mkdtempSync, readFileSync } from "node:fs";
import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
@ -137,6 +137,76 @@ test("writes to releases/beta/v<version>.md inside the repo by default", () => {
assert.match(body, /- feat: default path \(#1\)/);
});
test("nests PR summaries under subjects when gh can serve them", () => {
const dir = makeFixtureRepo();
commit(dir, "feat: enriched work (#42)");
git(dir, "tag", "beta/v2026.100.0-beta.0");
const binDir = join(dir, "fake-bin");
execFileSync("mkdir", ["-p", binDir]);
writeFileSync(
join(binDir, "gh"),
`#!/usr/bin/env bash
echo "## Thinking Path"
echo ""
echo "## What Changed"
echo ""
echo "- Adds the enriched thing"
echo "- Covers it with tests"
echo ""
echo "## Risks"
`,
{ mode: 0o755 }
);
const out = join(dir, "draft.md");
execFileSync(
"bash",
[script, "2026.100.0-beta.0", "--repo-dir", dir, "--out", out],
{ encoding: "utf8", env: { ...process.env, PATH: `${binDir}:${process.env.PATH}` } }
);
const body = readFileSync(out, "utf8");
assert.match(body, /- feat: enriched work \(#42\)\n > - Adds the enriched thing\n > - Covers it with tests/);
});
test("survives a PR body whose What Changed section has no bullets", () => {
const dir = makeFixtureRepo();
commit(dir, "feat: sparse body (#9)");
git(dir, "tag", "beta/v2026.100.0-beta.0");
const binDir = join(dir, "fake-bin");
execFileSync("mkdir", ["-p", binDir]);
writeFileSync(
join(binDir, "gh"),
`#!/usr/bin/env bash
echo "## What Changed"
echo ""
echo "## Risks"
`,
{ mode: 0o755 }
);
const out = join(dir, "draft.md");
execFileSync(
"bash",
[script, "2026.100.0-beta.0", "--repo-dir", dir, "--out", out],
{ encoding: "utf8", env: { ...process.env, PATH: `${binDir}:${process.env.PATH}` } }
);
const body = readFileSync(out, "utf8");
assert.match(body, /- feat: sparse body \(#9\)\n/);
});
test("skeleton stays clean when enrichment is unavailable", () => {
const dir = makeFixtureRepo();
commit(dir, "feat: plain work (#7)");
git(dir, "tag", "beta/v2026.100.0-beta.0");
const { body } = runDraft(dir, "2026.100.0-beta.0");
assert.match(body, /- feat: plain work \(#7\)\n/);
assert.doesNotMatch(body, / > /);
});
test("rejects a malformed beta version", () => {
const dir = makeFixtureRepo();
assert.throws(() =>