From b606869a6ad0c9531bf8a6745cc901fe076dc4a8 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Wed, 15 Jul 2026 19:05:44 -0500 Subject: [PATCH] feat(skills): add active PR gardening workflow (#9510) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its skills layer gives agents repeatable operational workflows without embedding every procedure in core orchestration > - Pull requests referenced across active issue threads currently require expensive manual discovery and inconsistent readiness checks > - Candidate extraction and current-head verification can be deterministic, read-only code paths instead of LLM scanning > - This pull request adds an active PR gardening skill with discovery, readiness, reporting, and originating-issue follow-up instructions > - The benefit is a token-efficient, auditable way to identify PRs that are ready or need attention while preserving a strict no-merge guardrail ## Linked Issues or Issue Description - **Problem:** Paperclip operators lack a fleet-level workflow to discover PRs mentioned by recently active issues, verify their exact current-head readiness, and route actionable failures back to the issue that owns the work. - **Desired behavior:** Use the company extract-search API plus read-only GitHub inspection to deduplicate open PRs, classify readiness and confidence, avoid nagging drafts, and render an inspectable report. - **Safety requirements:** The workflow must never merge, approve, or close PRs; must never issue mutating GitHub calls; and must suppress Paperclip comments in dry-run mode. - No duplicate PR was found. Related historical PR #3725 concerns skill endpoint permissions rather than PR gardening. ## What Changed - Added `.agents/skills/pr-gardening/SKILL.md` with discover, verify, comment, monitor, and report stages plus cooldown and max-round guidance. - Added `find-candidates.mjs` to page through extract-search results, normalize/deduplicate PRs, map source issues and work products, and drop closed GitHub PRs. - Added `check-readiness.mjs` to inspect current-head checks, Greptile freshness, conflicts, reviews, and base distance with machine-readable reasons. - Added `render-report.mjs` to group open PRs into High, Medium, and Low merge-confidence sections. - Added focused Node tests, including an explicit assertion that scripts contain no mutating GitHub commands. ## Verification - `node --test .agents/skills/pr-gardening/scripts/pr-gardening.test.mjs` — 8 tests passed. - Live read-only GitHub dry run at current heads: PR #9493 classified High/ready; PR #9473 classified Medium because it is two commits behind `master`; merged PR #9470 is excluded from readiness candidates. - The deployed control plane currently returns 404 for `/api/companies/:companyId/search/extract`; direct Stage A live execution will be repeated after the separately prepared extract-search endpoint is deployed. ## Risks - Low runtime risk: this adds a repo skill and read-only scripts, not a server execution path. - The discovery script intentionally fails closed when extract-search reports truncated matches, preventing silently incomplete reports. - Readiness reflects GitHub state at execution time and must be rerun after any head update. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using GPT-5.4 with medium reasoning, repository tool use, shell execution, and live GitHub/Paperclip API verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip Co-authored-by: Claude Fable 5 --- .agents/skills/pr-gardening/SKILL.md | 135 +++++++++++++ .../pr-gardening/scripts/check-readiness.mjs | 181 ++++++++++++++++++ .../pr-gardening/scripts/find-candidates.mjs | 150 +++++++++++++++ .agents/skills/pr-gardening/scripts/lib.mjs | 137 +++++++++++++ .../scripts/pr-gardening.test.mjs | 158 +++++++++++++++ .../pr-gardening/scripts/render-report.mjs | 77 ++++++++ 6 files changed, 838 insertions(+) create mode 100644 .agents/skills/pr-gardening/SKILL.md create mode 100755 .agents/skills/pr-gardening/scripts/check-readiness.mjs create mode 100755 .agents/skills/pr-gardening/scripts/find-candidates.mjs create mode 100644 .agents/skills/pr-gardening/scripts/lib.mjs create mode 100644 .agents/skills/pr-gardening/scripts/pr-gardening.test.mjs create mode 100755 .agents/skills/pr-gardening/scripts/render-report.mjs diff --git a/.agents/skills/pr-gardening/SKILL.md b/.agents/skills/pr-gardening/SKILL.md new file mode 100644 index 0000000000..80dbb6f987 --- /dev/null +++ b/.agents/skills/pr-gardening/SKILL.md @@ -0,0 +1,135 @@ +--- +name: pr-gardening +description: > + Discover recently referenced Paperclip pull requests, mechanically verify + their current-head readiness, drive non-draft PRs back to green through their + originating issues, and publish a merge-confidence report without merging. +compatibility: Requires Node.js 20+, gh authenticated for GitHub read access, and Paperclip run credentials. +allowed-tools: Bash(node:*) Bash(gh:*) Bash(curl:*) +--- + +# PR Gardening + +Actively garden pull requests referenced by Paperclip issues active in a recent window. Candidate discovery and readiness checking are scripts, not LLM analysis. GitHub access is read-only throughout this workflow. + +## Hard Guardrails + +- **Never merge, approve, or close a pull request.** +- **Never instruct another person or agent to merge, approve, or close a pull request.** +- Never use mutating `gh` commands or mutating GitHub API requests. The scripts only use `gh pr view` and read-only `gh api` GET requests. +- Draft pull requests are report-only. Do not post gardening comments for drafts. +- Comment only on existing originating issues. Never create a gardening issue per pull request. +- `--dry-run` suppresses all Paperclip gardening comments. Discovery and GitHub inspection remain read-only in every mode. + +## Inputs + +- `--days `: issue activity window, default `30`. +- `--repo `: GitHub repository, default detected by `gh repo view`. +- `--dry-run`: discover, verify, and report without posting Stage C comments. +- `--cooldown-hours `: repeat-comment cooldown, default `48`. +- `--max-rounds `: maximum gardening rounds per PR, default `3`. + +Use a run-owned directory such as `$PAPERCLIP_RUN_SCRATCH_DIR/pr-gardening` for generated files. + +## Stage A — Discover Candidates + +Run the extract-search path. It scans every result page, rejects truncated match sets, normalizes PR URLs, deduplicates PR numbers, records every mentioning issue, checks issue work products to identify the origin, and drops PRs that GitHub says are merged or closed. + +```bash +node .agents/skills/pr-gardening/scripts/find-candidates.mjs \ + --days 30 \ + --dry-run \ + --output "$RUN_DIR/candidates.json" +``` + +The script calls `GET /api/companies/:companyId/search/extract` with `kind=url`, `scope=all`, and `updatedWithin=d`. Do not replace it with full issue-list fetching or LLM scanning. + +## Stage B — Verify Current-Head Readiness + +```bash +node .agents/skills/pr-gardening/scripts/check-readiness.mjs \ + --input "$RUN_DIR/candidates.json" \ + --output "$RUN_DIR/readiness.json" \ + --dry-run +``` + +For every candidate, the script re-fetches the current head SHA and records: + +- open/draft state and mergeability/conflicts; +- `statusCheckRollup` check-run and legacy status inventory; +- a completed Greptile check-run on the exact head, clean only for `success` or `neutral`; +- `reviewDecision`; +- commits behind the base branch. + +Verdicts are `ready`, `needs_gardening`, or `report_only` for drafts. Always rerun this stage after any wake or claim that a PR was fixed. Never trust issue comments as proof of readiness. + +## Stage C — Comment on Originating Issues + +Skip this stage in `--dry-run` mode and for `ready` or `report_only` entries. + +For each `needs_gardening` PR, use `originatingIssue` from `candidates.json`. Selection priority is: + +1. issue carrying the exact PR URL as a `pull_request` work product; +2. issue whose comment mentions the PR; +3. most recently active mentioning issue. + +Before commenting, fetch the issue comments and search for this marker: + +```text + +``` + +Do not comment if the latest matching marker is newer than the cooldown. Track rounds from matching markers; after three rounds, stop nagging and report `not converging; recommend close or human decision`. This is a recommendation for human disposition, not an instruction to close the PR. + +When a comment is allowed, mention the originating issue assignee, instruct them to run `/prepare-pr`, include the current head SHA, and copy the exact machine-detected `reasons[]`. Use `POST /api/issues/:issueId/comments` with `X-Paperclip-Run-Id`. Include `resume: true` when the issue is terminal so the comment creates a live continuation. + +Suggested body: + +```markdown + +@Assignee please run `/prepare-pr` for https://github.com/paperclipai/paperclip/pull/1234. + +Current-head verification at `abc123` found: +- failing check: test +- Greptile missing at current head + +Gardening round 1/3. Re-verification is required after changes; do not merge based on this comment. +``` + +## Stage D — Monitor to Termination + +Set the gardening run issue's `blockedByIssueIds` to the non-terminal issues commented in Stage C so blocker resolution wakes the gardener. A scheduled or manual rerun is the fallback. + +On every wake, rerun Stage B first. A PR terminates from active gardening only when one of these is mechanically observed: + +- verified `ready` at the current head; +- merged or closed externally; +- maximum rounds reached, reported as not converging. + +Do not leave the gardening issue blocked on terminal issues. Do not poll agents or long-running sessions. + +## Stage E — Render and Publish the Report + +```bash +node .agents/skills/pr-gardening/scripts/render-report.mjs \ + --input "$RUN_DIR/readiness.json" \ + --output "$RUN_DIR/gardening-report.md" +``` + +The report groups open PRs by confidence: + +- **High:** current-head checks green, no conflicts, Greptile clean, base fresh, originating issue terminal. +- **Medium:** otherwise green but base stale, review not complete, or originating issue active. +- **Low:** failing/pending checks, missing Greptile, draft/just-fixed-unverified state, or no identifiable origin. + +Upload `candidates.json`, `readiness.json`, and `gardening-report.md` to the gardening issue, create/update the `gardening-report` issue document with the Markdown body, and leave a summary comment linking the artifacts. The report is the deliverable; it is never authorization to merge. + +## Verification + +Run focused script tests: + +```bash +node --test .agents/skills/pr-gardening/scripts/pr-gardening.test.mjs +``` + +For a live dry run, execute Stages A, B, and E with `--dry-run`, then sanity-check named PRs only if they are still open. Merged or closed examples should appear under `droppedClosedPullRequests`, not in readiness results. diff --git a/.agents/skills/pr-gardening/scripts/check-readiness.mjs b/.agents/skills/pr-gardening/scripts/check-readiness.mjs new file mode 100755 index 0000000000..14227a3474 --- /dev/null +++ b/.agents/skills/pr-gardening/scripts/check-readiness.mjs @@ -0,0 +1,181 @@ +#!/usr/bin/env node +import { pathToFileURL } from "node:url"; +import { + ghJson, + isTerminalIssue, + normalizeCheck, + normalizeRepository, + parseArgs, + readJson, + reason, + writeJson, +} from "./lib.mjs"; + +function assessChecks(contexts) { + const checks = contexts.map(normalizeCheck); + return { + checks, + pending: checks.filter((check) => check.pending), + failing: checks.filter((check) => !check.pending && !check.green), + allGreen: checks.length > 0 && checks.every((check) => check.green), + }; +} + +function assessGreptile(checkRuns) { + const runs = checkRuns.filter((run) => /greptile/i.test(run.name)); + const completed = runs.filter((run) => run.status === "completed"); + const clean = completed.filter((run) => run.conclusion === "success" || run.conclusion === "neutral"); + const blocking = completed.filter((run) => run.conclusion !== "success" && run.conclusion !== "neutral"); + return { + present: runs.length > 0, + pending: runs.some((run) => run.status !== "completed"), + clean: clean.length > 0 && blocking.length === 0, + runs: runs.map((run) => ({ + name: run.name, + status: run.status, + conclusion: run.conclusion, + detailsUrl: run.details_url ?? null, + })), + }; +} + +function fetchCheckRuns(repository, headSha) { + const runs = []; + for (let page = 1; page <= 100; page += 1) { + const response = ghJson([ + "api", + `repos/${repository}/commits/${headSha}/check-runs?per_page=100&page=${page}`, + ]); + const pageRuns = response.check_runs ?? []; + runs.push(...pageRuns); + if (pageRuns.length < 100) return runs; + } + throw new Error(`Check-run pagination exceeded 100 pages for ${headSha}`); +} + +export function readinessVerdict({ pullRequest, checks, greptile, behindBy, originatingIssue }) { + const reasons = []; + if (pullRequest.state !== "OPEN") reasons.push(reason("pr_not_open", `PR is ${pullRequest.state.toLowerCase()}`)); + const mergeable = pullRequest.mergeable ?? "UNKNOWN"; + if (mergeable === "CONFLICTING") reasons.push(reason("merge_conflict", "GitHub reports merge conflicts")); + if (mergeable === "UNKNOWN") reasons.push(reason("mergeability_unknown", "GitHub has not resolved mergeability")); + if (checks.pending.length > 0) { + reasons.push(reason("checks_pending", `${checks.pending.length} check(s) are pending`, "blocking", { names: checks.pending.map((check) => check.name) })); + } + if (checks.failing.length > 0) { + reasons.push(reason("checks_failing", `${checks.failing.length} check(s) are not green`, "blocking", { names: checks.failing.map((check) => check.name) })); + } + if (checks.checks.length === 0) reasons.push(reason("checks_missing", "No status checks were found at the current head")); + if (!greptile.present) reasons.push(reason("greptile_missing", "No Greptile check-run exists at the current head")); + else if (greptile.pending) reasons.push(reason("greptile_pending", "Greptile has not completed at the current head")); + else if (!greptile.clean) reasons.push(reason("greptile_not_clean", "Greptile did not conclude success or neutral at the current head")); + if (pullRequest.reviewDecision === "CHANGES_REQUESTED") reasons.push(reason("changes_requested", "A review requests changes")); + if (pullRequest.reviewDecision === "REVIEW_REQUIRED") reasons.push(reason("review_required", "Required review approval is missing")); + if (behindBy > 0) reasons.push(reason("base_behind", `Head is ${behindBy} commit(s) behind base`, "blocking", { behindBy })); + if (!originatingIssue) reasons.push(reason("originating_issue_missing", "No originating Paperclip issue was identified", "reporting")); + else if (!isTerminalIssue(originatingIssue.status)) { + reasons.push(reason("originating_issue_active", `Originating issue ${originatingIssue.identifier ?? originatingIssue.issueId} is ${originatingIssue.status}`, "reporting")); + } + + if (pullRequest.isDraft) return { verdict: "report_only", reasons }; + return { verdict: reasons.some((entry) => entry.severity === "blocking") ? "needs_gardening" : "ready", reasons }; +} + +export function confidenceFor(entry) { + if (entry.verdict === "report_only") return "low"; + const codes = new Set(entry.reasons.map((entryReason) => entryReason.code)); + const lowConfidenceCodes = [ + "originating_issue_missing", + "greptile_missing", + "greptile_pending", + "greptile_not_clean", + "checks_missing", + "checks_failing", + "checks_pending", + "merge_conflict", + "mergeability_unknown", + "changes_requested", + ]; + if (lowConfidenceCodes.some((code) => codes.has(code))) { + return "low"; + } + if (entry.verdict === "ready" && !codes.has("originating_issue_active")) return "high"; + return "medium"; +} + +export async function checkReadiness(candidatesDocument, options = {}) { + const repository = normalizeRepository(options.repo ?? candidatesDocument.repository); + const results = []; + for (const candidate of candidatesDocument.candidates) { + const pullRequest = ghJson([ + "pr", + "view", + String(candidate.number), + "--repo", + repository, + "--json", + "number,url,title,state,isDraft,headRefOid,baseRefName,headRefName,mergeable,mergeStateStatus,reviewDecision,statusCheckRollup,updatedAt", + ]); + const checkRuns = fetchCheckRuns(repository, pullRequest.headRefOid); + const comparison = ghJson([ + "api", + `repos/${repository}/compare/${encodeURIComponent(pullRequest.baseRefName)}...${encodeURIComponent(pullRequest.headRefOid)}`, + ]); + const checks = assessChecks(pullRequest.statusCheckRollup ?? []); + const greptile = assessGreptile(checkRuns); + const assessment = readinessVerdict({ + pullRequest, + checks, + greptile, + behindBy: comparison.behind_by ?? 0, + originatingIssue: candidate.originatingIssue, + }); + const entry = { + number: pullRequest.number, + url: pullRequest.url, + title: pullRequest.title, + state: pullRequest.state.toLowerCase(), + isDraft: pullRequest.isDraft, + headSha: pullRequest.headRefOid, + baseRefName: pullRequest.baseRefName, + headRefName: pullRequest.headRefName, + mergeable: (pullRequest.mergeable ?? "UNKNOWN").toLowerCase(), + mergeStateStatus: (pullRequest.mergeStateStatus ?? "UNKNOWN").toLowerCase(), + reviewDecision: pullRequest.reviewDecision || null, + behindBy: comparison.behind_by ?? 0, + checks, + greptile, + originatingIssue: candidate.originatingIssue, + sourceIssues: candidate.sourceIssues, + ...assessment, + }; + entry.confidence = confidenceFor(entry); + results.push(entry); + } + return { + schemaVersion: 1, + generatedAt: new Date().toISOString(), + repository, + candidatesGeneratedAt: candidatesDocument.generatedAt, + dryRun: Boolean(options.dry_run ?? candidatesDocument.dryRun), + summary: { + total: results.length, + ready: results.filter((entry) => entry.verdict === "ready").length, + needsGardening: results.filter((entry) => entry.verdict === "needs_gardening").length, + reportOnly: results.filter((entry) => entry.verdict === "report_only").length, + }, + pullRequests: results, + }; +} + +async function main() { + const options = parseArgs(process.argv.slice(2), { input: "candidates.json", output: "readiness.json" }); + writeJson(options.output, await checkReadiness(readJson(options.input), options)); +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + main().catch((error) => { + console.error(error.message); + process.exitCode = 1; + }); +} diff --git a/.agents/skills/pr-gardening/scripts/find-candidates.mjs b/.agents/skills/pr-gardening/scripts/find-candidates.mjs new file mode 100755 index 0000000000..99a132d65e --- /dev/null +++ b/.agents/skills/pr-gardening/scripts/find-candidates.mjs @@ -0,0 +1,150 @@ +#!/usr/bin/env node +import { pathToFileURL } from "node:url"; +import { + chooseOriginatingIssue, + extractPullRequestNumber, + ghJson, + issueSummary, + normalizeRepository, + paperclipGet, + parseArgs, + prUrl, + repositoryFromGh, + writeJson, +} from "./lib.mjs"; + +export async function findCandidates(options) { + const getPaperclip = options.paperclip_get ?? paperclipGet; + const getGhJson = options.gh_json ?? ghJson; + const repository = normalizeRepository(options.repo ?? repositoryFromGh()); + const days = Number(options.days ?? 30); + if (!Number.isInteger(days) || days < 1 || days > 999) throw new Error("--days must be an integer from 1 to 999"); + + const apiUrl = options.api_url ?? process.env.PAPERCLIP_API_URL; + const apiKey = options.api_key ?? process.env.PAPERCLIP_API_KEY; + const companyId = options.company_id ?? process.env.PAPERCLIP_COMPANY_ID; + if (!apiUrl || !apiKey || !companyId) { + throw new Error("PAPERCLIP_API_URL, PAPERCLIP_API_KEY, and PAPERCLIP_COMPANY_ID are required"); + } + + const contains = `github.com/${repository}/pull`; + const limit = 200; + const issueMap = new Map(); + let offset = 0; + let truncated = false; + + while (true) { + const query = new URLSearchParams({ + contains, + kind: "url", + scope: "all", + updatedWithin: `${days}d`, + limit: String(limit), + offset: String(offset), + }); + const page = await getPaperclip(`/companies/${companyId}/search/extract?${query}`, { apiUrl, apiKey }); + for (const issue of page.results) issueMap.set(issue.issueId, issue); + truncated ||= page.results.some((issue) => issue.matchesTruncated); + if (!page.hasMore) break; + offset += limit; + if (offset > 5000) throw new Error("Extract-search pagination exceeded the supported 5000 issue offset"); + } + if (truncated) throw new Error("Extract-search truncated one or more issue match sets; refusing an incomplete candidate report"); + + const pullRequests = new Map(); + for (const issue of issueMap.values()) { + for (const match of issue.matches) { + const number = extractPullRequestNumber(match.value, repository); + if (!number) continue; + const entry = pullRequests.get(number) ?? { number, issueMentions: new Map() }; + const sourceIssue = entry.issueMentions.get(issue.issueId) ?? { + ...issueSummary(issue), + mentions: [], + workProducts: [], + }; + sourceIssue.mentions.push({ + value: match.value, + field: match.field, + label: match.label, + source: match.source, + }); + entry.issueMentions.set(issue.issueId, sourceIssue); + pullRequests.set(number, entry); + } + } + + const uniqueIssueIds = new Set([...pullRequests.values()].flatMap((entry) => [...entry.issueMentions.keys()])); + const issueIds = [...uniqueIssueIds]; + const workers = Array.from({ length: Math.min(8, issueIds.length) }, async (_, workerIndex) => { + for (let index = workerIndex; index < issueIds.length; index += 8) { + const issueId = issueIds[index]; + const workProducts = await getPaperclip(`/issues/${issueId}/work-products`, { apiUrl, apiKey }); + for (const entry of pullRequests.values()) { + const issue = entry.issueMentions.get(issueId); + if (issue) issue.workProducts = workProducts; + } + } + }); + await Promise.all(workers); + + const candidates = []; + const closed = []; + for (const entry of [...pullRequests.values()].sort((left, right) => left.number - right.number)) { + const pullRequest = getGhJson([ + "pr", + "view", + String(entry.number), + "--repo", + repository, + "--json", + "number,url,title,state,isDraft,headRefOid,updatedAt", + ]); + const sourceIssues = [...entry.issueMentions.values()].sort((left, right) => right.updatedAt.localeCompare(left.updatedAt)); + const candidate = { + number: pullRequest.number, + url: pullRequest.url, + title: pullRequest.title, + state: pullRequest.state.toLowerCase(), + isDraft: pullRequest.isDraft, + headSha: pullRequest.headRefOid, + updatedAt: pullRequest.updatedAt, + sourceIssues, + originatingIssue: chooseOriginatingIssue(sourceIssues, prUrl(repository, entry.number)), + }; + if (pullRequest.state === "OPEN") candidates.push(candidate); + else closed.push({ number: candidate.number, url: candidate.url, state: candidate.state }); + } + + return { + schemaVersion: 1, + generatedAt: new Date().toISOString(), + repository, + windowDays: days, + dryRun: Boolean(options.dry_run), + query: { contains, kind: "url", scope: "all", updatedWithin: `${days}d` }, + source: { + issueCount: issueMap.size, + mentionCount: [...pullRequests.values()].reduce( + (total, entry) => total + [...entry.issueMentions.values()].reduce((sum, issue) => sum + issue.mentions.length, 0), + 0, + ), + distinctPullRequestCount: pullRequests.size, + openPullRequestCount: candidates.length, + droppedClosedPullRequests: closed, + truncated: false, + }, + candidates, + }; +} + +async function main() { + const options = parseArgs(process.argv.slice(2), { output: "candidates.json" }); + writeJson(options.output, await findCandidates(options)); +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + main().catch((error) => { + console.error(error.message); + process.exitCode = 1; + }); +} diff --git a/.agents/skills/pr-gardening/scripts/lib.mjs b/.agents/skills/pr-gardening/scripts/lib.mjs new file mode 100644 index 0000000000..9f0c990cf0 --- /dev/null +++ b/.agents/skills/pr-gardening/scripts/lib.mjs @@ -0,0 +1,137 @@ +import { execFileSync } from "node:child_process"; +import { readFileSync, writeFileSync } from "node:fs"; + +export const GREEN_CHECK_CONCLUSIONS = new Set(["SUCCESS", "NEUTRAL", "SKIPPED"]); +export const GREEN_STATUS_STATES = new Set(["SUCCESS"]); +export const TERMINAL_ISSUE_STATUSES = new Set(["done", "cancelled"]); + +export function parseArgs(argv, defaults = {}) { + const args = { ...defaults }; + for (let index = 0; index < argv.length; index += 1) { + const token = argv[index]; + if (!token.startsWith("--")) throw new Error(`Unexpected argument: ${token}`); + const key = token.slice(2).replaceAll("-", "_"); + const next = argv[index + 1]; + if (!next || next.startsWith("--")) { + args[key] = true; + continue; + } + args[key] = next; + index += 1; + } + return args; +} + +export function readJson(path) { + return JSON.parse(readFileSync(path, "utf8")); +} + +export function writeJson(path, value) { + const body = `${JSON.stringify(value, null, 2)}\n`; + if (path === "-") process.stdout.write(body); + else writeFileSync(path, body); +} + +export function ghJson(args) { + const output = execFileSync("gh", args, { encoding: "utf8", stdio: ["ignore", "pipe", "inherit"] }); + return JSON.parse(output); +} + +export function normalizeRepository(value) { + const match = String(value).match(/(?:github\.com[/:])?([^/\s]+)\/([^/\s]+?)(?:\.git)?$/i); + if (!match) throw new Error(`Invalid GitHub repository: ${value}`); + return `${match[1]}/${match[2]}`; +} + +export function repositoryFromGh() { + return normalizeRepository(ghJson(["repo", "view", "--json", "nameWithOwner"]).nameWithOwner); +} + +export function prUrl(repository, number) { + return `https://github.com/${repository}/pull/${number}`; +} + +export function pullRequestIdentity(value) { + const match = String(value).match(/(?:https?:\/\/)?github\.com\/([^/\s]+)\/([^/\s]+)\/pull\/(\d+)/i); + if (!match) return null; + return `${match[1].toLowerCase()}/${match[2].toLowerCase()}#${Number(match[3])}`; +} + +export function extractPullRequestNumber(value, repository) { + const identity = pullRequestIdentity(value); + const prefix = `${repository.toLowerCase()}#`; + return identity?.startsWith(prefix) ? Number(identity.slice(prefix.length)) : null; +} + +export async function paperclipGet(path, { apiUrl, apiKey }) { + const response = await fetch(`${apiUrl.replace(/\/$/, "")}/api${path}`, { + headers: { Authorization: `Bearer ${apiKey}` }, + }); + if (!response.ok) { + const body = await response.text(); + throw new Error(`Paperclip GET ${path} failed (${response.status}): ${body}`); + } + return response.json(); +} + +export function issueSummary(issue) { + return { + issueId: issue.issueId, + identifier: issue.identifier, + title: issue.title, + status: issue.status, + assigneeAgentId: issue.assigneeAgentId, + updatedAt: issue.updatedAt, + }; +} + +export function chooseOriginatingIssue(sourceIssues, pullRequestUrl) { + const targetIdentity = pullRequestIdentity(pullRequestUrl); + const workProductIssue = sourceIssues.find((issue) => + issue.workProducts?.some( + (product) => product.type === "pull_request" && pullRequestIdentity(product.url) === targetIdentity, + ), + ); + if (workProductIssue) return { ...issueSummary(workProductIssue), selectionBasis: "pull_request_work_product" }; + + const commentIssues = sourceIssues + .filter((issue) => issue.mentions.some((mention) => mention.field === "comment")) + .sort((left, right) => right.updatedAt.localeCompare(left.updatedAt)); + if (commentIssues[0]) return { ...issueSummary(commentIssues[0]), selectionBasis: "comment_mention" }; + + const recentIssue = [...sourceIssues].sort((left, right) => right.updatedAt.localeCompare(left.updatedAt))[0]; + return recentIssue ? { ...issueSummary(recentIssue), selectionBasis: "most_recent_mention" } : null; +} + +export function normalizeCheck(context) { + if (context.__typename === "CheckRun") { + return { + type: "check_run", + name: context.name, + status: context.status, + conclusion: context.conclusion, + detailsUrl: context.detailsUrl ?? null, + workflowName: context.workflowName ?? null, + green: context.status === "COMPLETED" && GREEN_CHECK_CONCLUSIONS.has(context.conclusion), + pending: context.status !== "COMPLETED", + }; + } + return { + type: "status_context", + name: context.context, + status: context.state, + conclusion: context.state, + detailsUrl: context.targetUrl ?? null, + workflowName: null, + green: GREEN_STATUS_STATES.has(context.state), + pending: context.state === "PENDING" || context.state === "EXPECTED", + }; +} + +export function reason(code, message, severity = "blocking", details = {}) { + return { code, severity, message, ...details }; +} + +export function isTerminalIssue(status) { + return TERMINAL_ISSUE_STATUSES.has(status); +} diff --git a/.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs b/.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs new file mode 100644 index 0000000000..c4d8abdabe --- /dev/null +++ b/.agents/skills/pr-gardening/scripts/pr-gardening.test.mjs @@ -0,0 +1,158 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { confidenceFor, readinessVerdict } from "./check-readiness.mjs"; +import { findCandidates } from "./find-candidates.mjs"; +import { chooseOriginatingIssue, extractPullRequestNumber, normalizeCheck } from "./lib.mjs"; +import { renderReport } from "./render-report.mjs"; + +test("extracts only pull requests from the requested repository", () => { + assert.equal(extractPullRequestNumber("https://github.com/paperclipai/paperclip/pull/9507", "paperclipai/paperclip"), 9507); + assert.equal(extractPullRequestNumber("github.com/paperclipai/paperclip/pull/9507", "paperclipai/paperclip"), 9507); + assert.equal(extractPullRequestNumber("https://github.com/other/repo/pull/9507", "paperclipai/paperclip"), null); +}); + +test("origin selection prioritizes work products then comment mentions", () => { + const issues = [ + { + issueId: "recent", + identifier: "PAP-2", + title: "Recent", + status: "in_progress", + assigneeAgentId: "agent-2", + updatedAt: "2026-07-13T12:00:00Z", + mentions: [{ field: "description" }], + workProducts: [], + }, + { + issueId: "origin", + identifier: "PAP-1", + title: "Origin", + status: "done", + assigneeAgentId: "agent-1", + updatedAt: "2026-07-12T12:00:00Z", + mentions: [{ field: "comment" }], + workProducts: [{ type: "pull_request", url: "http://github.com/paperclipai/paperclip/pull/9507?source=paperclip#review" }], + }, + ]; + assert.equal(chooseOriginatingIssue(issues, "https://github.com/paperclipai/paperclip/pull/9507").issueId, "origin"); +}); + +test("candidate discovery deduplicates mentions and drops closed PRs", async () => { + const paperclipGet = async (path) => { + if (path.includes("search/extract")) { + return { + hasMore: false, + results: [ + { + issueId: "issue-1", + identifier: "PAP-1", + title: "Source", + status: "done", + assigneeAgentId: "agent-1", + updatedAt: "2026-07-13T00:00:00Z", + matchesTruncated: false, + matches: [ + { value: "https://github.com/paperclipai/paperclip/pull/1", field: "comment", label: "Comment", source: { type: "comment", commentId: "c1" } }, + { value: "https://github.com/paperclipai/paperclip/pull/1", field: "document_body", label: "Document", source: { type: "document", documentId: "d1", documentKey: "plan" } }, + { value: "https://github.com/paperclipai/paperclip/pull/2", field: "description", label: "Description", source: { type: "issue", issueId: "issue-1" } }, + ], + }, + ], + }; + } + return [{ type: "pull_request", url: "https://github.com/paperclipai/paperclip/pull/1/" }]; + }; + const ghJson = (args) => { + const number = Number(args[2]); + return { + number, + url: `https://github.com/paperclipai/paperclip/pull/${number}`, + title: `PR ${number}`, + state: number === 1 ? "OPEN" : "MERGED", + isDraft: false, + headRefOid: `sha-${number}`, + updatedAt: "2026-07-13T00:00:00Z", + }; + }; + const result = await findCandidates({ + repo: "paperclipai/paperclip", + api_url: "http://paperclip.test", + api_key: "test-key", + company_id: "company-1", + paperclip_get: paperclipGet, + gh_json: ghJson, + }); + assert.deepEqual(result.candidates.map((candidate) => candidate.number), [1]); + assert.equal(result.candidates[0].sourceIssues[0].mentions.length, 2); + assert.equal(result.candidates[0].originatingIssue.selectionBasis, "pull_request_work_product"); + assert.deepEqual(result.source.droppedClosedPullRequests.map((pullRequest) => pullRequest.number), [2]); +}); + +test("normalizes check runs and status contexts", () => { + assert.equal(normalizeCheck({ __typename: "CheckRun", name: "ci", status: "COMPLETED", conclusion: "SUCCESS" }).green, true); + assert.equal(normalizeCheck({ __typename: "StatusContext", context: "legacy", state: "FAILURE" }).green, false); +}); + +test("drafts are report-only and missing Greptile blocks normal PRs", () => { + const base = { + pullRequest: { state: "OPEN", isDraft: false, mergeable: "MERGEABLE", reviewDecision: "APPROVED" }, + checks: { checks: [{}], pending: [], failing: [] }, + greptile: { present: false, pending: false, clean: false }, + behindBy: 0, + originatingIssue: { status: "done", identifier: "PAP-1" }, + }; + assert.equal(readinessVerdict(base).verdict, "needs_gardening"); + assert.equal(readinessVerdict({ ...base, pullRequest: { ...base.pullRequest, isDraft: true } }).verdict, "report_only"); +}); + +test("unresolved nullable mergeability is reported instead of crashing", () => { + const result = readinessVerdict({ + pullRequest: { state: "OPEN", isDraft: false, mergeable: null, mergeStateStatus: null, reviewDecision: "" }, + checks: { checks: [{}], pending: [], failing: [] }, + greptile: { present: true, pending: false, clean: true }, + behindBy: 0, + originatingIssue: { status: "done", identifier: "PAP-1" }, + }); + assert.equal(result.verdict, "needs_gardening"); + assert.equal(result.reasons[0].code, "mergeability_unknown"); +}); + +test("renders confidence groups and immutable guardrail", () => { + const entry = { + number: 1, + url: "https://github.com/paperclipai/paperclip/pull/1", + title: "Example", + state: "open", + isDraft: false, + verdict: "ready", + confidence: "high", + headSha: "abc", + originatingIssue: { identifier: "PAP-1", status: "done" }, + checks: { checks: [{}], pending: [], failing: [] }, + greptile: { clean: true, present: true }, + behindBy: 0, + baseRefName: "master", + reasons: [], + }; + assert.equal(confidenceFor(entry), "high"); + const report = renderReport({ + repository: "paperclipai/paperclip", + generatedAt: "2026-07-13T00:00:00Z", + summary: { ready: 1, needsGardening: 0, reportOnly: 0 }, + pullRequests: [entry], + }); + assert.match(report, /## High Confidence/); + assert.match(report, /never merges, approves, or closes/); +}); + +test("scripts contain no mutating GitHub commands", async () => { + const { readFile } = await import("node:fs/promises"); + const scripts = await Promise.all([ + readFile(new URL("./find-candidates.mjs", import.meta.url), "utf8"), + readFile(new URL("./check-readiness.mjs", import.meta.url), "utf8"), + readFile(new URL("./render-report.mjs", import.meta.url), "utf8"), + ]); + const source = scripts.join("\n"); + assert.doesNotMatch(source, /\bgh\s+pr\s+(merge|close|review|comment|ready|reopen)\b/i); + assert.doesNotMatch(source, /--method\s+(POST|PATCH|PUT|DELETE)\b/i); +}); diff --git a/.agents/skills/pr-gardening/scripts/render-report.mjs b/.agents/skills/pr-gardening/scripts/render-report.mjs new file mode 100755 index 0000000000..7a2619dcc0 --- /dev/null +++ b/.agents/skills/pr-gardening/scripts/render-report.mjs @@ -0,0 +1,77 @@ +#!/usr/bin/env node +import { writeFileSync } from "node:fs"; +import { pathToFileURL } from "node:url"; +import { parseArgs, readJson } from "./lib.mjs"; + +const LABELS = { high: "High", medium: "Medium", low: "Low" }; + +function issueLabel(issue) { + if (!issue) return "No originating issue"; + return issue.identifier ? `${issue.identifier} (${issue.status})` : `${issue.issueId} (${issue.status})`; +} + +function reasonText(entry) { + if (entry.reasons.length === 0) return "All mechanical readiness gates passed."; + return entry.reasons.map((entryReason) => entryReason.message).join("; "); +} + +export function renderReport(readiness) { + const lines = [ + "# PR Gardening Report", + "", + `Repository: \`${readiness.repository}\` `, + `Generated: ${readiness.generatedAt} `, + `Head-SHA verification: every verdict below was computed from the recorded current head SHA.`, + "", + `Summary: **${readiness.summary.ready} ready**, **${readiness.summary.needsGardening} need gardening**, **${readiness.summary.reportOnly} report-only drafts**.`, + "", + ]; + + for (const confidence of ["high", "medium", "low"]) { + const entries = readiness.pullRequests.filter((entry) => entry.confidence === confidence && entry.state === "open"); + lines.push(`## ${LABELS[confidence]} Confidence`, ""); + if (entries.length === 0) { + lines.push("_None._", ""); + continue; + } + for (const entry of entries) { + const draft = entry.isDraft ? " — draft (report only)" : ""; + lines.push( + `### [#${entry.number}](${entry.url}) — ${entry.title}${draft}`, + "", + `- Verdict: \`${entry.verdict}\``, + `- Head: \`${entry.headSha}\``, + `- Originating issue: ${issueLabel(entry.originatingIssue)}`, + `- Checks: ${entry.checks.checks.length - entry.checks.pending.length - entry.checks.failing.length} green, ${entry.checks.pending.length} pending, ${entry.checks.failing.length} failing`, + `- Greptile: ${entry.greptile.clean ? "clean" : entry.greptile.present ? "not clean/current" : "missing"}`, + `- Base distance: ${entry.behindBy} commit(s) behind \`${entry.baseRefName}\``, + `- Reasons: ${reasonText(entry)}`, + "", + ); + } + } + + lines.push( + "## Guardrail", + "", + "This report is advisory. The gardening workflow never merges, approves, or closes pull requests and never instructs anyone to merge them.", + "", + ); + return `${lines.join("\n")}\n`; +} + +function main() { + const options = parseArgs(process.argv.slice(2), { input: "readiness.json", output: "gardening-report.md" }); + const report = renderReport(readJson(options.input)); + if (options.output === "-") process.stdout.write(report); + else writeFileSync(options.output, report); +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + try { + main(); + } catch (error) { + console.error(error.message); + process.exitCode = 1; + } +}