This commit is contained in:
redle82 2026-08-16 07:20:36 +02:00 committed by GitHub
commit d81b1a5169
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 402 additions and 34 deletions

View File

@ -1027,10 +1027,30 @@ Per-Page Results:
Alerts Fired: [N] (X critical, Y high, Z medium)
Screenshots: .gstack/canary-reports/screenshots/
Human smoke: [CONFIRMED (user said pass) / AWAITING (screen opened, no reply yet)]
VERDICT: [DEPLOY IS HEALTHY / DEPLOY HAS ISSUES — details above]
```
**A HEALTHY status from Phases 1-5 is the automated read, not the final word.**
Every alert path already routes through AskUserQuestion (Phase 5) — but the
quiet, all-green path never puts anything in front of the user, which means
the most common outcome (nothing broke) is also the one nobody ever actually
looks at. Before finalizing the report:
```bash
open "<url>" 2>/dev/null || xdg-open "<url>" 2>/dev/null || echo "Couldn't auto-open — here's the URL: <url>"
```
Tell the user: "Automated monitoring found nothing wrong across {N} checks over
{duration} — but that's a script's opinion, not a look. I've opened the live
page. Take a glance, then tell me pass or fail so I can close this out as
actually verified." If the user confirms, mark `Human smoke: CONFIRMED` in the
report. If this is a non-interactive/spawned session, or the user doesn't
reply before you need to finish the report, write `Human smoke: AWAITING` and
say so plainly — do not silently treat "no alerts fired" as "a human looked
and it's fine."
Save report to `.gstack/canary-reports/{date}-canary.md` and `.gstack/canary-reports/{date}-canary.json`.
Log the result for the review dashboard:
@ -1040,17 +1060,22 @@ eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)"
mkdir -p ~/.gstack/projects/$SLUG
```
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","url":"<url>","duration_min":<N>,"alerts":<N>}`
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","human_smoke":"<CONFIRMED/AWAITING>","url":"<url>","duration_min":<N>,"alerts":<N>}`
### Phase 7: Baseline Update
If the deploy is healthy, offer to update the baseline:
If the deploy is healthy AND `Human smoke: CONFIRMED`, offer to update the baseline:
- **Context:** Canary monitoring completed. The deploy is healthy.
- **Context:** Canary monitoring completed. The deploy is healthy and you confirmed it looks right.
- **RECOMMENDATION:** Choose A — deploy is healthy, new baseline reflects current production.
- A) Update baseline with current screenshots
- B) Keep old baseline
If the deploy is healthy but `Human smoke: AWAITING` (nobody confirmed yet), skip
this offer — don't bake an unconfirmed state into the baseline other runs will
compare against. Mention it once: "Once you've had a look, run `/canary <url>
--baseline` if you want this state as the new reference point."
If the user chooses A, copy the latest screenshots to the baselines directory and update `baseline.json`.
## Important Rules

View File

@ -188,10 +188,30 @@ Per-Page Results:
Alerts Fired: [N] (X critical, Y high, Z medium)
Screenshots: .gstack/canary-reports/screenshots/
Human smoke: [CONFIRMED (user said pass) / AWAITING (screen opened, no reply yet)]
VERDICT: [DEPLOY IS HEALTHY / DEPLOY HAS ISSUES — details above]
```
**A HEALTHY status from Phases 1-5 is the automated read, not the final word.**
Every alert path already routes through AskUserQuestion (Phase 5) — but the
quiet, all-green path never puts anything in front of the user, which means
the most common outcome (nothing broke) is also the one nobody ever actually
looks at. Before finalizing the report:
```bash
open "<url>" 2>/dev/null || xdg-open "<url>" 2>/dev/null || echo "Couldn't auto-open — here's the URL: <url>"
```
Tell the user: "Automated monitoring found nothing wrong across {N} checks over
{duration} — but that's a script's opinion, not a look. I've opened the live
page. Take a glance, then tell me pass or fail so I can close this out as
actually verified." If the user confirms, mark `Human smoke: CONFIRMED` in the
report. If this is a non-interactive/spawned session, or the user doesn't
reply before you need to finish the report, write `Human smoke: AWAITING` and
say so plainly — do not silently treat "no alerts fired" as "a human looked
and it's fine."
Save report to `.gstack/canary-reports/{date}-canary.md` and `.gstack/canary-reports/{date}-canary.json`.
Log the result for the review dashboard:
@ -201,17 +221,22 @@ Log the result for the review dashboard:
mkdir -p ~/.gstack/projects/$SLUG
```
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","url":"<url>","duration_min":<N>,"alerts":<N>}`
Write a JSONL entry: `{"skill":"canary","timestamp":"<ISO>","status":"<HEALTHY/DEGRADED/BROKEN>","human_smoke":"<CONFIRMED/AWAITING>","url":"<url>","duration_min":<N>,"alerts":<N>}`
### Phase 7: Baseline Update
If the deploy is healthy, offer to update the baseline:
If the deploy is healthy AND `Human smoke: CONFIRMED`, offer to update the baseline:
- **Context:** Canary monitoring completed. The deploy is healthy.
- **Context:** Canary monitoring completed. The deploy is healthy and you confirmed it looks right.
- **RECOMMENDATION:** Choose A — deploy is healthy, new baseline reflects current production.
- A) Update baseline with current screenshots
- B) Keep old baseline
If the deploy is healthy but `Human smoke: AWAITING` (nobody confirmed yet), skip
this offer — don't bake an unconfirmed state into the baseline other runs will
compare against. Mention it once: "Once you've had a look, run `/canary <url>
--baseline` if you want this state as the new reference point."
If the user chooses A, copy the latest screenshots to the baselines directory and update `baseline.json`.
## Important Rules

View File

@ -1199,16 +1199,83 @@ Continue to Step 2.
Tell the user: "Checking CI status and merge readiness..."
Check CI status and merge readiness:
### 2a: Resolve which checks actually block merge
Not every check on a PR blocks merging — many repos run a fast required tier
plus a slower advisory/extended tier that reports status but isn't required
by branch protection. Treating every reported check as equally blocking makes
`/land-and-deploy` wait on jobs that were never going to gate the merge.
Fetch the full check list, then resolve which ones are actually required:
```bash
gh pr checks --json name,state,status,conclusion
```
Parse the output:
1. If any required checks are **FAILING**: **STOP.** "CI is failing on this PR. Here are the failing checks: {list}. Fix these before deploying — I won't merge code that hasn't passed CI."
2. If required checks are **PENDING**: Tell the user "CI is still running. I'll wait for it to finish." Proceed to Step 3.
3. If all checks pass (or no required checks): Tell the user "CI passed." Skip Step 3, go to Step 4.
```bash
BASE=$(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || echo main)
REQUIRED_CONTEXTS=$(gh api repos/{owner}/{repo}/branches/$BASE/protection --jq '.required_status_checks.contexts[]?' 2>/dev/null)
echo "$REQUIRED_CONTEXTS"
```
- If `gh api .../protection` succeeds and returns a non-empty list: this is
the authoritative required set. Only checks whose `name` appears in this
list can BLOCK or gate the wait in Step 3. Any check NOT in this list
(e.g. an extended/Mac/nightly tier) is advisory — note its state in the
readiness report (Step 3.5e) but never STOP or wait on it alone.
- If the API call fails (no admin read on protection, or branch protection
isn't configured): fall back to treating every check `gh pr checks` reports
as required — the previous behavior. This is a permissions/config
limitation, not a design choice, so don't silently narrow scope without it.
Parse the (now-scoped) required checks:
1. If any REQUIRED check is **FAILING**: go to 2b before deciding to STOP —
a failure isn't always a real failure.
2. If any REQUIRED check is **PENDING**: Tell the user "CI is still running.
I'll wait for it to finish." Proceed to Step 3.
3. If all REQUIRED checks pass: Tell the user "CI passed." If advisory checks
are still pending or failing, mention them once in passing ("the extended
suite is still running/red — that's advisory, not blocking merge") but do
not wait on them. Skip Step 3, go to Step 4.
### 2b: Recognize budget-exhaustion failures before treating them as real
A required check can come back FAILING for two very different reasons: the
code actually broke something, or the CI provider ran out of budget/minutes
and the job never really executed. These look identical in `conclusion:
failure` but need different responses — one blocks the merge, the other
means "run the equivalent checks locally and report status, don't just STOP."
For each FAILING required check, look at its run duration and step count
before writing the STOP message:
```bash
RUN_ID=$(gh pr checks --json name,link -q '.[] | select(.name=="<failing-check-name>") | .link' 2>/dev/null | grep -oE '[0-9]+$')
gh run view "$RUN_ID" --json status,conclusion,jobs -q '.jobs[] | {name, steps: (.steps | length), startedAt, completedAt}' 2>/dev/null
```
Budget-exhaustion signature: `steps` count is 0 (or 1 — just a runner-claim
step) AND the run completed in a few seconds rather than the job's normal
duration. If you've seen this check pass with N steps and a multi-minute
duration before, and it now shows 0 steps / seconds, that's the signature —
not a code regression.
- If the signature matches: don't report this as "CI is failing on this
PR" with the implication the code is broken. Instead: "The `<check>` check
didn't actually run — it has the signature of exhausted CI budget/minutes
(0 steps, {N}s), not a real failure. I can't fix a budget problem from
here. If this project has a local equivalent (a local-ci script, or the
individual commands the check would have run), I'll run that now so you
have a real pass/fail signal instead of a red X that means nothing."
Then look for and offer to run a local CI script if the repo has one
(common names: `local-ci.sh`, `ci-local.sh`, a `Makefile` target, or a
documented equivalent in CONTRIBUTING.md/CLAUDE.md). Report PASS/FAIL/
SKIPPED/NOT_AVAILABLE per gate you ran — don't claim "CI passed" from a
local run standing in for a hosted one that never executed.
- If the signature does NOT match (real duration, real steps, then failure):
this is a genuine failure. **STOP.** "CI is failing on this PR. Here are
the failing checks: {list}. Fix these before deploying — I won't merge
code that hasn't passed CI."
Also check for merge conflicts:
```bash
@ -1220,7 +1287,7 @@ If `CONFLICTING`: **STOP.** "This PR has merge conflicts with the base branch. R
## Step 3: Wait for CI (if pending)
If required checks are still pending, wait for them to complete. Use a timeout of 15 minutes:
If required checks (as scoped in 2a) are still pending, wait for them to complete. Use a timeout of 15 minutes:
```bash
gh pr checks --watch --fail-fast
@ -1229,7 +1296,7 @@ gh pr checks --watch --fail-fast
Record the CI wait time for the deploy report.
If CI passes within the timeout: Tell the user "CI passed after {duration}. Moving to readiness checks." Continue to Step 4.
If CI fails: **STOP.** "CI failed. Here's what broke: {failures}. This needs to pass before I can merge."
If CI fails: apply the 2b budget-exhaustion check to the failing run before writing the STOP message — don't repeat the raw "CI failed" framing on a run that never actually executed. If it's a real failure: **STOP.** "CI failed. Here's what broke: {failures}. This needs to pass before I can merge."
If timeout (15 min): **STOP.** "CI has been running for over 15 minutes — that's unusual. Check the GitHub Actions tab to see if something is stuck."
---
@ -1411,6 +1478,37 @@ Compare the PR body against the actual commits. Check for:
If the PR body looks stale or incomplete: **WARNING — PR body may not reflect current
changes.** List what's missing or stale.
### 3.5c-bis: Project-required PR report check
Some projects define a mandatory report format the PR body must carry before a
merge can be recommended — a specific checklist or table (e.g. "every check
run, its result, and evidence") documented in CLAUDE.md or CONTRIBUTING.md.
`/ship` may or may not populate this, so verify it's actually there rather
than assuming.
```bash
grep -n -iE "mandatory|required.*(report|checklist|table).*(PR|pull request)" CLAUDE.md CONTRIBUTING.md 2>/dev/null | head -5
```
- If nothing matches: this project has no such requirement. Skip this
sub-step, no warning.
- If a match is found: read the surrounding section (a few lines above and
below the match) to learn what the required format actually is — the
column names, the checks it must cover, whether every row needs PASS/FAIL/
SKIPPED/NOT_AVAILABLE rather than being silently omitted. Then compare that
against the current PR body:
- Missing entirely (the PR body has no such table/checklist at all):
**WARNING — required PR report missing.** Name the file/section that
documents the requirement and what's absent.
- Present but incomplete (rows missing, blank results, a check the project
requires that isn't listed): **WARNING — required PR report incomplete.**
List which rows/checks are missing or unresolved.
- Present and every required row has an explicit result: no warning.
- This check only ever produces a WARNING, never a silent skip of a real
requirement and never a hard BLOCKER on its own — 3.5e's existing
blocker/warning tiering (tests failing = blocker, everything else = warning
the user can override) already governs how hard to stop here.
### 3.5d: Document-release check
Check if documentation was updated on this branch:
@ -1461,7 +1559,8 @@ Build the full readiness report:
║ └─ Doc release: Run / NOT RUN (warning) ║
║ ║
║ PR BODY ║
║ └─ Accuracy: Current / STALE (warning) ║
║ ├─ Accuracy: Current / STALE (warning) ║
║ └─ Required report: Complete / Missing / N/A (warning) ║
║ ║
║ WARNINGS: N | BLOCKERS: N ║
╚══════════════════════════════════════════════════════════╝
@ -1491,6 +1590,7 @@ If the user chooses B: **STOP.** Give specific next steps:
- If E2E not run: "Run your E2E tests to make sure nothing is broken, then come back."
- If docs not updated: "Run `/document-release` to update CHANGELOG and docs."
- If PR body stale: "The PR description doesn't match what's actually in the diff — update it on GitHub."
- If the required PR report is missing or incomplete: "This project requires a specific report in the PR body (see {file/section from 3.5c-bis}) — fill in the missing rows/checks, then run `/land-and-deploy` again."
If the user chooses A or C: Tell the user "Merging now." Continue to Step 4.
@ -1848,15 +1948,63 @@ Take an annotated screenshot as evidence.
- Page has real content (not blank or error screen) → PASS
- Loads in under 10 seconds → PASS
If all pass: Tell the user "Site is healthy. Page loaded in {X}s, no console errors, content looks good. Screenshot saved to {path}." Mark as HEALTHY, continue to Step 9.
If all pass: Tell the user "Automated checks are green — page loaded in {X}s, no console errors, content looks good. Screenshot saved to {path}." Then go to **Step 7a** before marking HEALTHY — the automated canary is a machine's opinion, not a human's confirmation.
If any fail: show the evidence (screenshot path, console errors, perf numbers). Use AskUserQuestion:
- **Re-ground:** "I found some issues on the live site after the deploy. Here's what I see: {specific issues}. This might be temporary (caches clearing, CDN propagating) or it might be a real problem."
- **RECOMMENDATION:** Choose based on severity — B for critical (site down), A for minor (console errors).
- A) That's expected — the site is still warming up. Mark it as healthy.
- A) That's expected — the site is still warming up. Mark it as healthy, then go to Step 7a.
- B) That's broken — revert the merge and roll back to the previous version
- C) Let me investigate more — open the site and look at logs before deciding
### 7a: Open the screen for human validation — never just report
**This sub-step exists because a text summary of "it's healthy" is not the
same thing as a human seeing the change work.** An automated canary catches
crashes and console errors; it does not catch "the button is in the wrong
place," "the copy doesn't match what we agreed," or a business-logic edge
case the LLM judging its own screenshot can rationalize away. If nobody with
eyes on the actual product confirms the change, the task is not done — it
just has a report that says it's done, which is a worse failure mode than no
report at all, because it looks finished.
Check whether this project documents its own manual-validation entry point
(a script, a runbook, a specific convention for "how a human checks this"):
```bash
grep -rniE "validat|smoke.?test" CLAUDE.md CONTRIBUTING.md 2>/dev/null | grep -iE "script|\.sh|runbook|manual" | head -5
```
- **If a project-specific validation entry point is found** (a script path,
a documented command, a runbook section): that convention wins. Follow
what it says exactly — if it says to write a manifest file, or run a
specific script, do that, not the generic fallback below.
- **If nothing project-specific is found (the common case):** use the
generic fallback — open the verified URL directly for the user rather than
only telling them a URL exists:
```bash
open "<url>" 2>/dev/null || xdg-open "<url>" 2>/dev/null || echo "Couldn't auto-open — here's the URL: <url>"
```
Tell the user, plainly: "Automated checks passed, but that's a machine's
opinion — I've opened the live page so you can look at it yourself. This is
the part I can't verify for you: does it actually look and work right? Take
a look, then tell me pass or fail." If a `screenshot`/annotated image was
captured in the full canary sequence above, mention its path too, but the
opened live URL is the primary artifact — a screenshot proves what the
browser rendered at one instant, not that a human confirmed the thing.
**Do not mark the task DONE from Step 7a alone.** Two acceptable end states:
- The user replies with an explicit pass ("looks good", "confirmed",
equivalent) — mark VERIFIED (human-confirmed) in the Step 9 report and
continue.
- The user is not present to answer right now (headless/spawned session, or
they said "I'll check it later") — mark the Step 9 verdict as **DEPLOYED
(AUTOMATED CHECKS ONLY — awaiting human smoke test)**, not HEALTHY. Do not
silently upgrade an unconfirmed automated pass into a "verified" or
"healthy" verdict; say plainly that a human hasn't looked yet.
---
## Step 8: Revert (if needed)
@ -1922,10 +2070,18 @@ Verification: <HEALTHY / DEGRADED / SKIPPED / REVERTED>
Console: <N errors or "clean">
Load time: <Xs>
Screenshot: <path or "none">
Human smoke: <CONFIRMED (user said pass) / AWAITING (screen opened, no reply yet) / N/A (reverted or skipped)>
VERDICT: <DEPLOYED AND VERIFIED / DEPLOYED (UNVERIFIED) / STAGING VERIFIED / REVERTED>
VERDICT: <DEPLOYED AND VERIFIED / DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST) / DEPLOYED (UNVERIFIED) / STAGING VERIFIED / REVERTED>
```
**"DEPLOYED AND VERIFIED" requires `Human smoke: CONFIRMED`.** If the automated
canary passed but Step 7a's screen-open never got a human reply, the verdict
is `DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST)`, not
"VERIFIED" — a report claiming more certainty than actually exists is worse
than one that's honest about what's still open, because the task looks
closed when it isn't.
Save report to `.gstack/deploy-reports/{date}-pr{number}-deploy.md`.
Log to the review dashboard:
@ -1937,7 +2093,7 @@ mkdir -p ~/.gstack/projects/$SLUG
Write a JSONL entry with timing data:
```json
{"skill":"land-and-deploy","timestamp":"<ISO>","status":"<SUCCESS/REVERTED>","pr":<number>,"merge_sha":"<sha>","merge_path":"<auto/direct/queue>","first_run":<true/false>,"deploy_status":"<HEALTHY/DEGRADED/SKIPPED>","staging_status":"<VERIFIED/SKIPPED>","review_status":"<CURRENT/STALE/NOT_RUN/INLINE_FIX>","ci_wait_s":<N>,"queue_s":<N>,"deploy_s":<N>,"staging_s":<N>,"canary_s":<N>,"total_s":<N>}
{"skill":"land-and-deploy","timestamp":"<ISO>","status":"<SUCCESS/REVERTED>","pr":<number>,"merge_sha":"<sha>","merge_path":"<auto/direct/queue>","first_run":<true/false>,"deploy_status":"<HEALTHY/DEGRADED/SKIPPED>","staging_status":"<VERIFIED/SKIPPED>","review_status":"<CURRENT/STALE/NOT_RUN/INLINE_FIX>","human_smoke_status":"<CONFIRMED/AWAITING/NA>","ci_wait_s":<N>,"queue_s":<N>,"deploy_s":<N>,"staging_s":<N>,"canary_s":<N>,"total_s":<N>}
```
---
@ -1946,7 +2102,9 @@ Write a JSONL entry with timing data:
After the deploy report:
If verdict is DEPLOYED AND VERIFIED: Tell the user "Your changes are live and verified. Nice ship."
If verdict is DEPLOYED AND VERIFIED: Tell the user "Your changes are live and a human confirmed them. Nice ship."
If verdict is DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST): Tell the user "Your changes are live and passed the automated canary — I've opened the page for you. This one's on you to confirm: take a look, and tell me if it's a pass so I can close this out as verified rather than just deployed."
If verdict is DEPLOYED (UNVERIFIED): Tell the user "Your changes are merged and should be deploying. I wasn't able to verify the site — check it manually when you get a chance."
@ -1967,7 +2125,8 @@ Then suggest relevant follow-ups:
- **Auto-detect everything.** PR number, merge method, deploy strategy, project type, merge queues, staging environments. Only ask when information genuinely can't be inferred.
- **Poll with backoff.** Don't hammer GitHub API. 30-second intervals for CI/deploy, with reasonable timeouts.
- **Revert is always an option.** At every failure point, offer revert as an escape hatch. Explain what reverting does in plain English.
- **Single-pass verification, not continuous monitoring.** `/land-and-deploy` checks once. `/canary` does the extended monitoring loop.
- **Single-pass automated verification, not continuous monitoring.** The automated canary in Step 7 runs once — `/canary` does the extended monitoring loop. This is orthogonal to Step 7a's human confirmation: opening the screen once and waiting for one reply is not a polling loop, it's the one check a machine cannot do for itself.
- **A report never claims more certainty than what actually happened.** "Verified" means a human confirmed it, not that a script returned exit 0. If the human hasn't replied yet, the verdict says AWAITING, not HEALTHY or VERIFIED — see Step 9.
- **Clean up.** Delete the feature branch after merge (via `--delete-branch`).
- **First run = teacher mode.** Walk the user through everything. Explain what each check does and why it matters. Show them their infrastructure. Let them confirm before proceeding. Build trust through transparency.
- **Subsequent runs = efficient mode.** Brief status updates, no re-explanations. The user already trusts the tool — just do the job and report results.

View File

@ -295,16 +295,83 @@ Continue to Step 2.
Tell the user: "Checking CI status and merge readiness..."
Check CI status and merge readiness:
### 2a: Resolve which checks actually block merge
Not every check on a PR blocks merging — many repos run a fast required tier
plus a slower advisory/extended tier that reports status but isn't required
by branch protection. Treating every reported check as equally blocking makes
`/land-and-deploy` wait on jobs that were never going to gate the merge.
Fetch the full check list, then resolve which ones are actually required:
```bash
gh pr checks --json name,state,status,conclusion
```
Parse the output:
1. If any required checks are **FAILING**: **STOP.** "CI is failing on this PR. Here are the failing checks: {list}. Fix these before deploying — I won't merge code that hasn't passed CI."
2. If required checks are **PENDING**: Tell the user "CI is still running. I'll wait for it to finish." Proceed to Step 3.
3. If all checks pass (or no required checks): Tell the user "CI passed." Skip Step 3, go to Step 4.
```bash
BASE=$(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || echo main)
REQUIRED_CONTEXTS=$(gh api repos/{owner}/{repo}/branches/$BASE/protection --jq '.required_status_checks.contexts[]?' 2>/dev/null)
echo "$REQUIRED_CONTEXTS"
```
- If `gh api .../protection` succeeds and returns a non-empty list: this is
the authoritative required set. Only checks whose `name` appears in this
list can BLOCK or gate the wait in Step 3. Any check NOT in this list
(e.g. an extended/Mac/nightly tier) is advisory — note its state in the
readiness report (Step 3.5e) but never STOP or wait on it alone.
- If the API call fails (no admin read on protection, or branch protection
isn't configured): fall back to treating every check `gh pr checks` reports
as required — the previous behavior. This is a permissions/config
limitation, not a design choice, so don't silently narrow scope without it.
Parse the (now-scoped) required checks:
1. If any REQUIRED check is **FAILING**: go to 2b before deciding to STOP —
a failure isn't always a real failure.
2. If any REQUIRED check is **PENDING**: Tell the user "CI is still running.
I'll wait for it to finish." Proceed to Step 3.
3. If all REQUIRED checks pass: Tell the user "CI passed." If advisory checks
are still pending or failing, mention them once in passing ("the extended
suite is still running/red — that's advisory, not blocking merge") but do
not wait on them. Skip Step 3, go to Step 4.
### 2b: Recognize budget-exhaustion failures before treating them as real
A required check can come back FAILING for two very different reasons: the
code actually broke something, or the CI provider ran out of budget/minutes
and the job never really executed. These look identical in `conclusion:
failure` but need different responses — one blocks the merge, the other
means "run the equivalent checks locally and report status, don't just STOP."
For each FAILING required check, look at its run duration and step count
before writing the STOP message:
```bash
RUN_ID=$(gh pr checks --json name,link -q '.[] | select(.name=="<failing-check-name>") | .link' 2>/dev/null | grep -oE '[0-9]+$')
gh run view "$RUN_ID" --json status,conclusion,jobs -q '.jobs[] | {name, steps: (.steps | length), startedAt, completedAt}' 2>/dev/null
```
Budget-exhaustion signature: `steps` count is 0 (or 1 — just a runner-claim
step) AND the run completed in a few seconds rather than the job's normal
duration. If you've seen this check pass with N steps and a multi-minute
duration before, and it now shows 0 steps / seconds, that's the signature —
not a code regression.
- If the signature matches: don't report this as "CI is failing on this
PR" with the implication the code is broken. Instead: "The `<check>` check
didn't actually run — it has the signature of exhausted CI budget/minutes
(0 steps, {N}s), not a real failure. I can't fix a budget problem from
here. If this project has a local equivalent (a local-ci script, or the
individual commands the check would have run), I'll run that now so you
have a real pass/fail signal instead of a red X that means nothing."
Then look for and offer to run a local CI script if the repo has one
(common names: `local-ci.sh`, `ci-local.sh`, a `Makefile` target, or a
documented equivalent in CONTRIBUTING.md/CLAUDE.md). Report PASS/FAIL/
SKIPPED/NOT_AVAILABLE per gate you ran — don't claim "CI passed" from a
local run standing in for a hosted one that never executed.
- If the signature does NOT match (real duration, real steps, then failure):
this is a genuine failure. **STOP.** "CI is failing on this PR. Here are
the failing checks: {list}. Fix these before deploying — I won't merge
code that hasn't passed CI."
Also check for merge conflicts:
```bash
@ -316,7 +383,7 @@ If `CONFLICTING`: **STOP.** "This PR has merge conflicts with the base branch. R
## Step 3: Wait for CI (if pending)
If required checks are still pending, wait for them to complete. Use a timeout of 15 minutes:
If required checks (as scoped in 2a) are still pending, wait for them to complete. Use a timeout of 15 minutes:
```bash
gh pr checks --watch --fail-fast
@ -325,7 +392,7 @@ gh pr checks --watch --fail-fast
Record the CI wait time for the deploy report.
If CI passes within the timeout: Tell the user "CI passed after {duration}. Moving to readiness checks." Continue to Step 4.
If CI fails: **STOP.** "CI failed. Here's what broke: {failures}. This needs to pass before I can merge."
If CI fails: apply the 2b budget-exhaustion check to the failing run before writing the STOP message — don't repeat the raw "CI failed" framing on a run that never actually executed. If it's a real failure: **STOP.** "CI failed. Here's what broke: {failures}. This needs to pass before I can merge."
If timeout (15 min): **STOP.** "CI has been running for over 15 minutes — that's unusual. Check the GitHub Actions tab to see if something is stuck."
---
@ -507,6 +574,37 @@ Compare the PR body against the actual commits. Check for:
If the PR body looks stale or incomplete: **WARNING — PR body may not reflect current
changes.** List what's missing or stale.
### 3.5c-bis: Project-required PR report check
Some projects define a mandatory report format the PR body must carry before a
merge can be recommended — a specific checklist or table (e.g. "every check
run, its result, and evidence") documented in CLAUDE.md or CONTRIBUTING.md.
`/ship` may or may not populate this, so verify it's actually there rather
than assuming.
```bash
grep -n -iE "mandatory|required.*(report|checklist|table).*(PR|pull request)" CLAUDE.md CONTRIBUTING.md 2>/dev/null | head -5
```
- If nothing matches: this project has no such requirement. Skip this
sub-step, no warning.
- If a match is found: read the surrounding section (a few lines above and
below the match) to learn what the required format actually is — the
column names, the checks it must cover, whether every row needs PASS/FAIL/
SKIPPED/NOT_AVAILABLE rather than being silently omitted. Then compare that
against the current PR body:
- Missing entirely (the PR body has no such table/checklist at all):
**WARNING — required PR report missing.** Name the file/section that
documents the requirement and what's absent.
- Present but incomplete (rows missing, blank results, a check the project
requires that isn't listed): **WARNING — required PR report incomplete.**
List which rows/checks are missing or unresolved.
- Present and every required row has an explicit result: no warning.
- This check only ever produces a WARNING, never a silent skip of a real
requirement and never a hard BLOCKER on its own — 3.5e's existing
blocker/warning tiering (tests failing = blocker, everything else = warning
the user can override) already governs how hard to stop here.
### 3.5d: Document-release check
Check if documentation was updated on this branch:
@ -557,7 +655,8 @@ Build the full readiness report:
║ └─ Doc release: Run / NOT RUN (warning) ║
║ ║
║ PR BODY ║
║ └─ Accuracy: Current / STALE (warning) ║
║ ├─ Accuracy: Current / STALE (warning) ║
║ └─ Required report: Complete / Missing / N/A (warning) ║
║ ║
║ WARNINGS: N | BLOCKERS: N ║
╚══════════════════════════════════════════════════════════╝
@ -587,6 +686,7 @@ If the user chooses B: **STOP.** Give specific next steps:
- If E2E not run: "Run your E2E tests to make sure nothing is broken, then come back."
- If docs not updated: "Run `/document-release` to update CHANGELOG and docs."
- If PR body stale: "The PR description doesn't match what's actually in the diff — update it on GitHub."
- If the required PR report is missing or incomplete: "This project requires a specific report in the PR body (see {file/section from 3.5c-bis}) — fill in the missing rows/checks, then run `/land-and-deploy` again."
If the user chooses A or C: Tell the user "Merging now." Continue to Step 4.
@ -908,15 +1008,63 @@ Take an annotated screenshot as evidence.
- Page has real content (not blank or error screen) → PASS
- Loads in under 10 seconds → PASS
If all pass: Tell the user "Site is healthy. Page loaded in {X}s, no console errors, content looks good. Screenshot saved to {path}." Mark as HEALTHY, continue to Step 9.
If all pass: Tell the user "Automated checks are green — page loaded in {X}s, no console errors, content looks good. Screenshot saved to {path}." Then go to **Step 7a** before marking HEALTHY — the automated canary is a machine's opinion, not a human's confirmation.
If any fail: show the evidence (screenshot path, console errors, perf numbers). Use AskUserQuestion:
- **Re-ground:** "I found some issues on the live site after the deploy. Here's what I see: {specific issues}. This might be temporary (caches clearing, CDN propagating) or it might be a real problem."
- **RECOMMENDATION:** Choose based on severity — B for critical (site down), A for minor (console errors).
- A) That's expected — the site is still warming up. Mark it as healthy.
- A) That's expected — the site is still warming up. Mark it as healthy, then go to Step 7a.
- B) That's broken — revert the merge and roll back to the previous version
- C) Let me investigate more — open the site and look at logs before deciding
### 7a: Open the screen for human validation — never just report
**This sub-step exists because a text summary of "it's healthy" is not the
same thing as a human seeing the change work.** An automated canary catches
crashes and console errors; it does not catch "the button is in the wrong
place," "the copy doesn't match what we agreed," or a business-logic edge
case the LLM judging its own screenshot can rationalize away. If nobody with
eyes on the actual product confirms the change, the task is not done — it
just has a report that says it's done, which is a worse failure mode than no
report at all, because it looks finished.
Check whether this project documents its own manual-validation entry point
(a script, a runbook, a specific convention for "how a human checks this"):
```bash
grep -rniE "validat|smoke.?test" CLAUDE.md CONTRIBUTING.md 2>/dev/null | grep -iE "script|\.sh|runbook|manual" | head -5
```
- **If a project-specific validation entry point is found** (a script path,
a documented command, a runbook section): that convention wins. Follow
what it says exactly — if it says to write a manifest file, or run a
specific script, do that, not the generic fallback below.
- **If nothing project-specific is found (the common case):** use the
generic fallback — open the verified URL directly for the user rather than
only telling them a URL exists:
```bash
open "<url>" 2>/dev/null || xdg-open "<url>" 2>/dev/null || echo "Couldn't auto-open — here's the URL: <url>"
```
Tell the user, plainly: "Automated checks passed, but that's a machine's
opinion — I've opened the live page so you can look at it yourself. This is
the part I can't verify for you: does it actually look and work right? Take
a look, then tell me pass or fail." If a `screenshot`/annotated image was
captured in the full canary sequence above, mention its path too, but the
opened live URL is the primary artifact — a screenshot proves what the
browser rendered at one instant, not that a human confirmed the thing.
**Do not mark the task DONE from Step 7a alone.** Two acceptable end states:
- The user replies with an explicit pass ("looks good", "confirmed",
equivalent) — mark VERIFIED (human-confirmed) in the Step 9 report and
continue.
- The user is not present to answer right now (headless/spawned session, or
they said "I'll check it later") — mark the Step 9 verdict as **DEPLOYED
(AUTOMATED CHECKS ONLY — awaiting human smoke test)**, not HEALTHY. Do not
silently upgrade an unconfirmed automated pass into a "verified" or
"healthy" verdict; say plainly that a human hasn't looked yet.
---
## Step 8: Revert (if needed)
@ -982,10 +1130,18 @@ Verification: <HEALTHY / DEGRADED / SKIPPED / REVERTED>
Console: <N errors or "clean">
Load time: <Xs>
Screenshot: <path or "none">
Human smoke: <CONFIRMED (user said pass) / AWAITING (screen opened, no reply yet) / N/A (reverted or skipped)>
VERDICT: <DEPLOYED AND VERIFIED / DEPLOYED (UNVERIFIED) / STAGING VERIFIED / REVERTED>
VERDICT: <DEPLOYED AND VERIFIED / DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST) / DEPLOYED (UNVERIFIED) / STAGING VERIFIED / REVERTED>
```
**"DEPLOYED AND VERIFIED" requires `Human smoke: CONFIRMED`.** If the automated
canary passed but Step 7a's screen-open never got a human reply, the verdict
is `DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST)`, not
"VERIFIED" — a report claiming more certainty than actually exists is worse
than one that's honest about what's still open, because the task looks
closed when it isn't.
Save report to `.gstack/deploy-reports/{date}-pr{number}-deploy.md`.
Log to the review dashboard:
@ -997,7 +1153,7 @@ mkdir -p ~/.gstack/projects/$SLUG
Write a JSONL entry with timing data:
```json
{"skill":"land-and-deploy","timestamp":"<ISO>","status":"<SUCCESS/REVERTED>","pr":<number>,"merge_sha":"<sha>","merge_path":"<auto/direct/queue>","first_run":<true/false>,"deploy_status":"<HEALTHY/DEGRADED/SKIPPED>","staging_status":"<VERIFIED/SKIPPED>","review_status":"<CURRENT/STALE/NOT_RUN/INLINE_FIX>","ci_wait_s":<N>,"queue_s":<N>,"deploy_s":<N>,"staging_s":<N>,"canary_s":<N>,"total_s":<N>}
{"skill":"land-and-deploy","timestamp":"<ISO>","status":"<SUCCESS/REVERTED>","pr":<number>,"merge_sha":"<sha>","merge_path":"<auto/direct/queue>","first_run":<true/false>,"deploy_status":"<HEALTHY/DEGRADED/SKIPPED>","staging_status":"<VERIFIED/SKIPPED>","review_status":"<CURRENT/STALE/NOT_RUN/INLINE_FIX>","human_smoke_status":"<CONFIRMED/AWAITING/NA>","ci_wait_s":<N>,"queue_s":<N>,"deploy_s":<N>,"staging_s":<N>,"canary_s":<N>,"total_s":<N>}
```
---
@ -1006,7 +1162,9 @@ Write a JSONL entry with timing data:
After the deploy report:
If verdict is DEPLOYED AND VERIFIED: Tell the user "Your changes are live and verified. Nice ship."
If verdict is DEPLOYED AND VERIFIED: Tell the user "Your changes are live and a human confirmed them. Nice ship."
If verdict is DEPLOYED (AUTOMATED CHECKS ONLY — AWAITING HUMAN SMOKE TEST): Tell the user "Your changes are live and passed the automated canary — I've opened the page for you. This one's on you to confirm: take a look, and tell me if it's a pass so I can close this out as verified rather than just deployed."
If verdict is DEPLOYED (UNVERIFIED): Tell the user "Your changes are merged and should be deploying. I wasn't able to verify the site — check it manually when you get a chance."
@ -1027,7 +1185,8 @@ Then suggest relevant follow-ups:
- **Auto-detect everything.** PR number, merge method, deploy strategy, project type, merge queues, staging environments. Only ask when information genuinely can't be inferred.
- **Poll with backoff.** Don't hammer GitHub API. 30-second intervals for CI/deploy, with reasonable timeouts.
- **Revert is always an option.** At every failure point, offer revert as an escape hatch. Explain what reverting does in plain English.
- **Single-pass verification, not continuous monitoring.** `/land-and-deploy` checks once. `/canary` does the extended monitoring loop.
- **Single-pass automated verification, not continuous monitoring.** The automated canary in Step 7 runs once — `/canary` does the extended monitoring loop. This is orthogonal to Step 7a's human confirmation: opening the screen once and waiting for one reply is not a polling loop, it's the one check a machine cannot do for itself.
- **A report never claims more certainty than what actually happened.** "Verified" means a human confirmed it, not that a script returned exit 0. If the human hasn't replied yet, the verdict says AWAITING, not HEALTHY or VERIFIED — see Step 9.
- **Clean up.** Delete the feature branch after merge (via `--delete-branch`).
- **First run = teacher mode.** Walk the user through everything. Explain what each check does and why it matters. Show them their infrastructure. Let them confirm before proceeding. Build trust through transparency.
- **Subsequent runs = efficient mode.** Brief status updates, no re-explanations. The user already trusts the tool — just do the job and report results.