From 12244c4e475192e4575b1628f34d296f68d356b9 Mon Sep 17 00:00:00 2001
From: Garry Tan <garrytan@gmail.com>
Date: Thu, 18 Jun 2026 11:22:48 -0700
Subject: [PATCH] ci(evals): install gstack skill registry for the PTY smoke
 suite
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The first dry-run of e2e-pty-plan-smoke failed: the spawned interactive `claude`
printed "Unknown command: /plan-ceo-review". .claude/skills is gitignored, so a
fresh CI checkout has no gstack skill registry and the TUI can't resolve
/office-hours or /plan-ceo-review.

Add a Register step (scoped to the suite, after Seed, before Run) that mirrors
setup's --no-prefix user-scoped registry minimally: $HOME/.claude/skills/gstack
-> repo (resolves the preambles' absolute ~/.claude/skills/gstack/bin/* and
<skill>/sections/* paths) + per-skill SKILL.md/sections symlinks for the two
skills these tests invoke. HOME is /github/home in this container and the runner
adds no HOME/CLAUDE_CONFIG_DIR override (no hermetic mode), so $HOME is the right
anchor — the Seed step already proved claude reads it. No ./setup (binary build
+ Chromium + fonts + /dev/tty prompt); SKILL.md + bin/ + sections/ are committed.

Self-validating: fails the step loudly on a dangling symlink or missing
`name:` frontmatter, so a moved target surfaces here instead of as a silent
35-min "Unknown command" timeout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 .github/workflows/evals.yml | 47 +++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/.github/workflows/evals.yml b/.github/workflows/evals.yml
index 3d4a192bd..f2c5cb232 100644
--- a/.github/workflows/evals.yml
+++ b/.github/workflows/evals.yml
@@ -169,6 +169,53 @@ jobs:
             console.log("seeded", p);
           '
 
+      # PTY smokes drive the interactive `claude` TUI and send /office-hours and
+      # /plan-ceo-review. Claude Code discovers user-scoped skills from
+      # $HOME/.claude/skills/<name>/SKILL.md, but .claude/skills is gitignored, so
+      # a fresh CI checkout has NO registry — claude prints "Unknown command:
+      # /plan-ceo-review". Mirror setup's --no-prefix registry minimally: a gstack
+      # root symlink (resolves the preamble's absolute ~/.claude/skills/gstack/bin/*
+      # and ~/.claude/skills/gstack/<skill>/sections/* paths) plus a per-skill
+      # top-level dir holding SKILL.md (+ sections) symlinks for the two skills
+      # these tests invoke. No ./setup (it builds binaries, launches Chromium,
+      # installs fonts, reads a /dev/tty prompt) and no binary build (SKILL.md +
+      # bin/ + sections/ are committed). $HOME is /github/home here; the spawned
+      # claude inherits it (this runner adds no HOME/CLAUDE_CONFIG_DIR override,
+      # no hermetic mode) and the Seed step already proved claude reads $HOME.
+      - name: Register gstack skills for PTY smoke
+        if: matrix.suite.name == 'e2e-pty-plan-smoke'
+        run: |
+          set -eu
+          SKILLS_DIR="$HOME/.claude/skills"
+          REPO="$GITHUB_WORKSPACE"   # /__w/gstack/gstack
+          mkdir -p "$SKILLS_DIR"
+          ln -snf "$REPO" "$SKILLS_DIR/gstack"
+          for s in office-hours plan-ceo-review; do
+            mkdir -p "$SKILLS_DIR/$s"
+            ln -snf "$REPO/$s/SKILL.md" "$SKILLS_DIR/$s/SKILL.md"
+            ln -snf "$REPO/$s/sections" "$SKILLS_DIR/$s/sections"
+          done
+          echo "--- registry under $SKILLS_DIR ---"
+          ls -la "$SKILLS_DIR/gstack" "$SKILLS_DIR/office-hours" "$SKILLS_DIR/plan-ceo-review"
+          # Fail fast if any committed target moved/renamed — a dangling symlink
+          # would otherwise resurface as a silent "Unknown command" + 35-min timeout.
+          for f in \
+            "$SKILLS_DIR/office-hours/SKILL.md" \
+            "$SKILLS_DIR/plan-ceo-review/SKILL.md" \
+            "$SKILLS_DIR/gstack/bin/gstack-update-check" \
+            "$SKILLS_DIR/gstack/office-hours/sections/design-and-handoff.md" \
+            "$SKILLS_DIR/gstack/plan-ceo-review/sections/review-sections.md"; do
+            if [ ! -e "$f" ]; then
+              echo "ERROR: skill-registry target missing (symlink dangles): $f" >&2
+              exit 1
+            fi
+          done
+          grep -m1 '^name: office-hours$'    "$SKILLS_DIR/office-hours/SKILL.md"    >/dev/null \
+            || { echo "ERROR: office-hours SKILL.md missing 'name: office-hours' frontmatter" >&2; exit 1; }
+          grep -m1 '^name: plan-ceo-review$' "$SKILLS_DIR/plan-ceo-review/SKILL.md" >/dev/null \
+            || { echo "ERROR: plan-ceo-review SKILL.md missing 'name: plan-ceo-review' frontmatter" >&2; exit 1; }
+          echo "skill registry OK"
+
       - name: Run ${{ matrix.suite.name }}
         env:
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}