From 05be4dbb8a9dbe17c75786f63e62f7818a27f017 Mon Sep 17 00:00:00 2001
From: Garry Tan <garrytan@gmail.com>
Date: Fri, 14 Aug 2026 15:42:32 -0700
Subject: [PATCH] fix(ci): skill-docs freshness gate covers all 10 hosts and
 can actually fail
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The Codex/Factory gates ran 'git diff --exit-code -- .agents/' / '-- .factory/',
but both paths are gitignored (.gitignore:16-17) — git diff on ignored untracked
paths is always empty, so those two gates were structurally incapable of failing
and 7 of 10 hosts had no gate at all.

New shape: one 'gen:skill-docs --host all' pass (the generator hard-fails on any
per-host error, gating all 10 hosts on generates-cleanly), byte-freshness via
git diff for tracked output, plus a porcelain check that fails on untracked
generated strays (git diff can't see brand-new files). The gitignored-hosts
byte-freshness limitation is documented in the workflow comment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 .github/workflows/skill-docs.yml | 41 ++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/.github/workflows/skill-docs.yml b/.github/workflows/skill-docs.yml
index 700a8222a..69cabbe1d 100644
--- a/.github/workflows/skill-docs.yml
+++ b/.github/workflows/skill-docs.yml
@@ -7,27 +7,32 @@ jobs:
       - uses: actions/checkout@v4
       - uses: oven-sh/setup-bun@v2
       - run: bun install
-      - name: Check Claude host freshness
-        run: bun run gen:skill-docs
-      - name: Verify Claude skill docs are fresh
+      # One generation pass for ALL 10 hosts. gen-skill-docs --host all
+      # hard-fails on any per-host generation error (scripts/gen-skill-docs.ts
+      # aggregates failures and exits non-zero), so every host is gated on
+      # "generates cleanly." Known limitation, on purpose: the 9 gitignored
+      # host outputs (.agents/, .factory/, .kiro/, ...) are NOT byte-freshness
+      # checked — `git diff` on ignored untracked paths is always empty (the
+      # previous per-host `git diff -- .agents/` gates could never fail for
+      # exactly that reason). Byte-freshness is enforced only for tracked
+      # output (the Claude SKILL.md files), which the two steps below cover.
+      - name: Generate all host skill docs
+        run: bun run gen:skill-docs --host all
+      - name: Verify tracked skill docs are fresh
         run: |
           git diff --exit-code || {
-            echo "Generated SKILL.md files are stale. Run: bun run gen:skill-docs"
+            echo "Generated SKILL.md files are stale. Run: bun run gen:skill-docs --host all"
             exit 1
           }
-      - name: Check Codex host freshness
-        run: bun run gen:skill-docs --host codex
-      - name: Verify Codex skill docs are fresh
+      # git diff misses NEW untracked files (e.g. a freshly added skill whose
+      # generated SKILL.md was never committed). Fail on any untracked stray
+      # the generator produced outside the gitignored host dirs.
+      - name: Verify no untracked generated files
         run: |
-          git diff --exit-code -- .agents/ || {
-            echo "Generated Codex SKILL.md files are stale. Run: bun run gen:skill-docs --host codex"
+          STRAYS=$(git status --porcelain --untracked-files=all | grep '^??' || true)
+          if [ -n "$STRAYS" ]; then
+            echo "Generator produced untracked files that are neither committed nor gitignored:"
+            echo "$STRAYS"
+            echo "Commit them (bun run gen:skill-docs --host all) or gitignore them."
             exit 1
-          }
-      - name: Generate Factory skill docs
-        run: bun run gen:skill-docs --host factory
-      - name: Verify Factory skill docs are fresh
-        run: |
-          git diff --exit-code -- .factory/ || {
-            echo "Generated Factory SKILL.md files are stale. Run: bun run gen:skill-docs --host factory"
-            exit 1
-          }
+          fi