mirror of https://github.com/garrytan/gstack.git
fix: name web performance benchmark
This commit is contained in:
Nader Helmy
parent
b76a2d9555
commit
6148354529
|
|
@ -61,7 +61,7 @@ Invoke them by name (e.g., `/office-hours`).
|
|||
| `/learn` | Manage what gstack learned across sessions. |
|
||||
| `/retro` | Weekly retro with per-person breakdowns and shipping streaks. |
|
||||
| `/health` | Code quality dashboard (type checker, linter, tests, dead code). |
|
||||
| `/benchmark` | Performance regression detection (page load, Core Web Vitals). |
|
||||
| `/web-performance-benchmark` | Performance regression detection (page load, Core Web Vitals). |
|
||||
| `/benchmark-models` | Cross-model benchmark for skills (Claude, GPT, Gemini side-by-side). |
|
||||
| `/cso` | OWASP Top 10 + STRIDE security audit. |
|
||||
| `/setup-gbrain` | Set up gbrain for cross-machine session memory sync. |
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ WebSocket — Claude's Bash tool already exists, so we use it.
|
|||
Three escalating modes:
|
||||
|
||||
- **Headless** (default). Daemon runs Chromium with no visible window. Fastest,
|
||||
cheapest, what skills like `/qa`, `/design-review`, `/benchmark` use by
|
||||
cheapest, what skills like `/qa`, `/design-review`, `/web-performance-benchmark` use by
|
||||
default.
|
||||
- **Headed via `$B connect`**. Same daemon, but Chromium is visible (rebranded
|
||||
as "GStack Browser") with the Side Panel extension auto-loaded. You watch
|
||||
|
|
|
|||
|
|
@ -120,7 +120,7 @@ gstack/
|
|||
├── plan-ceo-review/ # /plan-ceo-review skill
|
||||
├── plan-eng-review/ # /plan-eng-review skill
|
||||
├── autoplan/ # /autoplan skill (auto-review pipeline: CEO → design → eng)
|
||||
├── benchmark/ # /benchmark skill (performance regression detection)
|
||||
├── benchmark/ # /web-performance-benchmark skill (performance regression detection)
|
||||
├── canary/ # /canary skill (post-deploy monitoring loop)
|
||||
├── codex/ # /codex skill (multi-AI second opinion via OpenAI Codex CLI)
|
||||
├── land-and-deploy/ # /land-and-deploy skill (merge → deploy → canary verify)
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ Fork it. Improve it. Make it yours. And if you want to hate on free open source
|
|||
|
||||
Open Claude Code and paste this. Claude does the rest.
|
||||
|
||||
> Install gstack: run **`git clone --single-branch --depth 1 https://github.com/garrytan/gstack.git ~/.claude/skills/gstack && cd ~/.claude/skills/gstack && ./setup`** then add a "gstack" section to CLAUDE.md that says to use the /browse skill from gstack for all web browsing, never use mcp\_\_claude-in-chrome\_\_\* tools, and lists the available skills: /office-hours, /plan-ceo-review, /plan-eng-review, /plan-design-review, /design-consultation, /design-shotgun, /design-html, /review, /ship, /land-and-deploy, /canary, /benchmark, /browse, /connect-chrome, /qa, /qa-only, /design-review, /setup-browser-cookies, /setup-deploy, /setup-gbrain, /retro, /investigate, /document-release, /document-generate, /codex, /cso, /autoplan, /plan-devex-review, /devex-review, /careful, /freeze, /guard, /unfreeze, /gstack-upgrade, /learn. Then ask the user if they also want to add gstack to the current project so teammates get it.
|
||||
> Install gstack: run **`git clone --single-branch --depth 1 https://github.com/garrytan/gstack.git ~/.claude/skills/gstack && cd ~/.claude/skills/gstack && ./setup`** then add a "gstack" section to CLAUDE.md that says to use the /browse skill from gstack for all web browsing, never use mcp\_\_claude-in-chrome\_\_\* tools, and lists the available skills: /office-hours, /plan-ceo-review, /plan-eng-review, /plan-design-review, /design-consultation, /design-shotgun, /design-html, /review, /ship, /land-and-deploy, /canary, /web-performance-benchmark, /browse, /connect-chrome, /qa, /qa-only, /design-review, /setup-browser-cookies, /setup-deploy, /setup-gbrain, /retro, /investigate, /document-release, /document-generate, /codex, /cso, /autoplan, /plan-devex-review, /devex-review, /careful, /freeze, /guard, /unfreeze, /gstack-upgrade, /learn. Then ask the user if they also want to add gstack to the current project so teammates get it.
|
||||
|
||||
### Step 2: Team mode — auto-update for shared repos (recommended)
|
||||
|
||||
|
|
@ -197,7 +197,7 @@ Each skill feeds into the next. `/office-hours` writes a design doc that `/plan-
|
|||
| `/ship` | **Release Engineer** | Sync main, run tests, audit coverage, push, open PR. Bootstraps test frameworks if you don't have one. |
|
||||
| `/land-and-deploy` | **Release Engineer** | Merge the PR, wait for CI and deploy, verify production health. One command from "approved" to "verified in production." |
|
||||
| `/canary` | **SRE** | Post-deploy monitoring loop. Watches for console errors, performance regressions, and page failures. |
|
||||
| `/benchmark` | **Performance Engineer** | Baseline page load times, Core Web Vitals, and resource sizes. Compare before/after on every PR. |
|
||||
| `/web-performance-benchmark` | **Performance Engineer** | Baseline page load times, Core Web Vitals, and resource sizes. Compare before/after on every PR. |
|
||||
| `/document-release` | **Technical Writer** | Update all project docs to match what you just shipped. Catches stale READMEs automatically. Builds a Diataxis coverage map (reference / how-to / tutorial / explanation) so gaps are visible in the PR body. |
|
||||
| `/document-generate` | **Documentation Author** | Generate missing docs from scratch using the Diataxis framework. Researches the codebase first, then writes reference / how-to / tutorial / explanation docs that actually match the code. Invokable standalone or chained from `/document-release` when the coverage map finds gaps. Learn more: [tutorial](docs/tutorial-document-generate.md) • [how-to](docs/howto-document-a-shipped-feature.md) • [why Diataxis](docs/explanation-diataxis-in-gstack.md). |
|
||||
| `/retro` | **Eng Manager** | Team-aware weekly retro. Per-person breakdowns, shipping streaks, test health trends, growth opportunities. `/retro global` runs across all your projects and AI tools (Claude Code, Codex, Gemini). |
|
||||
|
|
@ -480,7 +480,7 @@ On Windows without Developer Mode (MSYS2 / Git Bash), `setup` falls back to file
|
|||
Use /browse from gstack for all web browsing. Never use mcp__claude-in-chrome__* tools.
|
||||
Available skills: /office-hours, /plan-ceo-review, /plan-eng-review, /plan-design-review,
|
||||
/design-consultation, /design-shotgun, /design-html, /review, /ship, /land-and-deploy,
|
||||
/canary, /benchmark, /browse, /open-gstack-browser, /qa, /qa-only, /design-review,
|
||||
/canary, /web-performance-benchmark, /browse, /open-gstack-browser, /qa, /qa-only, /design-review,
|
||||
/setup-browser-cookies, /setup-deploy, /setup-gbrain, /sync-gbrain, /retro, /investigate,
|
||||
/document-release, /document-generate, /codex, /cso, /autoplan, /pair-agent, /careful, /freeze,
|
||||
/guard, /unfreeze, /gstack-upgrade, /learn.
|
||||
|
|
|
|||
2
SKILL.md
2
SKILL.md
|
|
@ -587,7 +587,7 @@ quality gates that produce better results than answering inline.
|
|||
- User asks to make a PDF, document, publication → invoke `/make-pdf`
|
||||
- User asks to launch a real browser for QA, "open the browser" → invoke `/open-gstack-browser`
|
||||
- User asks to import cookies for authenticated testing → invoke `/setup-browser-cookies`
|
||||
- User asks about page speed, performance regression, benchmarks → invoke `/benchmark`
|
||||
- User asks about page speed, Core Web Vitals, or a web performance regression → invoke `/web-performance-benchmark`
|
||||
- User asks what gstack has learned, "show learnings" → invoke `/learn`
|
||||
- User asks to tune question sensitivity, "stop asking me that" → invoke `/plan-tune`
|
||||
- User asks for code quality dashboard, "health check" → invoke `/health`
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ quality gates that produce better results than answering inline.
|
|||
- User asks to make a PDF, document, publication → invoke `/make-pdf`
|
||||
- User asks to launch a real browser for QA, "open the browser" → invoke `/open-gstack-browser`
|
||||
- User asks to import cookies for authenticated testing → invoke `/setup-browser-cookies`
|
||||
- User asks about page speed, performance regression, benchmarks → invoke `/benchmark`
|
||||
- User asks about page speed, Core Web Vitals, or a web performance regression → invoke `/web-performance-benchmark`
|
||||
- User asks what gstack has learned, "show learnings" → invoke `/learn`
|
||||
- User asks to tune question sensitivity, "stop asking me that" → invoke `/plan-tune`
|
||||
- User asks for code quality dashboard, "health check" → invoke `/health`
|
||||
|
|
|
|||
2
TODOS.md
2
TODOS.md
|
|
@ -2191,7 +2191,7 @@ Shipped in v0.6.5. TemplateContext in gen-skill-docs.ts bakes skill name into pr
|
|||
### Deploy pipeline (v0.9.8.0)
|
||||
- /land-and-deploy — merge PR, wait for CI/deploy, canary verification
|
||||
- /canary — post-deploy monitoring loop with anomaly detection
|
||||
- /benchmark — performance regression detection with Core Web Vitals
|
||||
- /web-performance-benchmark — performance regression detection with Core Web Vitals
|
||||
- /setup-deploy — one-time deploy platform configuration
|
||||
- /review Performance & Bundle Impact pass
|
||||
- E2E model pinning (Sonnet default, Opus for quality tests)
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ allowed-tools:
|
|||
Runs the same prompt through Claude,
|
||||
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
|
||||
and optionally quality via LLM judge. Answers "which model is actually best
|
||||
for this skill?" with data instead of vibes. Separate from /benchmark, which
|
||||
for this skill?" with data instead of vibes. Separate from /web-performance-benchmark, which
|
||||
measures web page performance. Use when: "benchmark models", "compare models",
|
||||
"which model is best for X", "cross-model comparison", "model shootout".
|
||||
|
||||
|
|
@ -539,7 +539,7 @@ Skills that run plan reviews (`/plan-*-review`, `/codex review`) include the EXI
|
|||
|
||||
You are running the `/benchmark-models` workflow. Wraps the `gstack-model-benchmark` binary with an interactive flow that picks a prompt, confirms providers, previews auth, and runs the benchmark.
|
||||
|
||||
Different from `/benchmark` — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.
|
||||
Different from `/web-performance-benchmark` — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ description: |
|
|||
Cross-model benchmark for gstack skills. Runs the same prompt through Claude,
|
||||
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
|
||||
and optionally quality via LLM judge. Answers "which model is actually best
|
||||
for this skill?" with data instead of vibes. Separate from /benchmark, which
|
||||
for this skill?" with data instead of vibes. Separate from /web-performance-benchmark, which
|
||||
measures web page performance. Use when: "benchmark models", "compare models",
|
||||
"which model is best for X", "cross-model comparison", "model shootout". (gstack)
|
||||
voice-triggers:
|
||||
|
|
@ -30,7 +30,7 @@ allowed-tools:
|
|||
|
||||
You are running the `/benchmark-models` workflow. Wraps the `gstack-model-benchmark` binary with an interactive flow that picks a prompt, confirms providers, previews auth, and runs the benchmark.
|
||||
|
||||
Different from `/benchmark` — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.
|
||||
Different from `/web-performance-benchmark` — that skill measures web page performance (Core Web Vitals, load times). This skill measures AI model performance on gstack skills or arbitrary prompts.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: gstack-benchmark
|
||||
name: web-performance-benchmark
|
||||
preamble-tier: 1
|
||||
version: 1.0.0
|
||||
description: Web page performance regression detection using the browse daemon. (gstack)
|
||||
|
|
@ -85,7 +85,7 @@ _QUESTION_TUNING=$(~/.claude/skills/gstack/bin/gstack-config get question_tuning
|
|||
echo "QUESTION_TUNING: $_QUESTION_TUNING"
|
||||
mkdir -p ~/.gstack/analytics
|
||||
if [ "$_TEL" != "off" ]; then
|
||||
echo '{"skill":"gstack-benchmark","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","repo":"'$(_repo=$(basename "$(git rev-parse --show-toplevel 2>/dev/null)" 2>/dev/null | tr -cd 'a-zA-Z0-9._-'); echo "${_repo:-unknown}")'"}' >> ~/.gstack/analytics/skill-usage.jsonl 2>/dev/null || true
|
||||
echo '{"skill":"web-performance-benchmark","ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","repo":"'$(_repo=$(basename "$(git rev-parse --show-toplevel 2>/dev/null)" 2>/dev/null | tr -cd 'a-zA-Z0-9._-'); echo "${_repo:-unknown}")'"}' >> ~/.gstack/analytics/skill-usage.jsonl 2>/dev/null || true
|
||||
fi
|
||||
for _PF in $(find ~/.gstack/analytics -maxdepth 1 -name '.pending-*' 2>/dev/null); do
|
||||
if [ -f "$_PF" ]; then
|
||||
|
|
@ -107,7 +107,7 @@ if [ -f "$_LEARN_FILE" ]; then
|
|||
else
|
||||
echo "LEARNINGS: 0"
|
||||
fi
|
||||
~/.claude/skills/gstack/bin/gstack-timeline-log '{"skill":"gstack-benchmark","event":"started","branch":"'"$_BRANCH"'","session":"'"$_SESSION_ID"'"}' 2>/dev/null &
|
||||
~/.claude/skills/gstack/bin/gstack-timeline-log '{"skill":"web-performance-benchmark","event":"started","branch":"'"$_BRANCH"'","session":"'"$_SESSION_ID"'"}' 2>/dev/null &
|
||||
_HAS_ROUTING="no"
|
||||
if [ -f CLAUDE.md ] && grep -q "## Skill routing" CLAUDE.md 2>/dev/null; then
|
||||
_HAS_ROUTING="yes"
|
||||
|
|
@ -572,29 +572,33 @@ If `NEEDS_SETUP`:
|
|||
fi
|
||||
```
|
||||
|
||||
# /gstack-benchmark - Web Page Performance Regression Detection
|
||||
# /web-performance-benchmark - Web Page Performance Regression Detection
|
||||
|
||||
You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow.
|
||||
|
||||
Your job is to measure, baseline, compare, and alert. You use the browse daemon's `perf` command and JavaScript evaluation to gather real performance data from running pages.
|
||||
|
||||
## Not For
|
||||
## Scope
|
||||
|
||||
Do not run this skill for the bare word "benchmark" unless the request is clearly
|
||||
about web page performance. Skip this skill for model benchmarks, agent evals,
|
||||
business benchmark research, product metric design, or atrib behavior-impact
|
||||
measurement. For cross-model gstack skill comparisons, use `/benchmark-models`.
|
||||
Use this skill for web page speed, Core Web Vitals, resource sizes, and performance
|
||||
regressions. Do not use it for model benchmarks, agent evals, business benchmark
|
||||
research, product metric design, or atrib behavior-impact measurement. For
|
||||
cross-model gstack skill comparisons, use `/benchmark-models`.
|
||||
|
||||
## User-invocable
|
||||
When the user types `/gstack-benchmark`, run this skill.
|
||||
Use `/web-performance-benchmark` for new work.
|
||||
|
||||
Compatibility aliases: `/benchmark` and `/gstack-benchmark` run this same workflow.
|
||||
Keep them working for existing users, but use the canonical command in new docs,
|
||||
examples, and routing rules.
|
||||
|
||||
## Arguments
|
||||
- `/gstack-benchmark <url>` — full performance audit with baseline comparison
|
||||
- `/gstack-benchmark <url> --baseline` — capture baseline (run before making changes)
|
||||
- `/gstack-benchmark <url> --quick` — single-pass timing check (no baseline needed)
|
||||
- `/gstack-benchmark <url> --pages /,/dashboard,/api/health` — specify pages
|
||||
- `/gstack-benchmark --diff` — benchmark only pages affected by current branch
|
||||
- `/gstack-benchmark --trend` — show performance trends from historical data
|
||||
- `/web-performance-benchmark <url>` — full performance audit with baseline comparison
|
||||
- `/web-performance-benchmark <url> --baseline` — capture baseline (run before making changes)
|
||||
- `/web-performance-benchmark <url> --quick` — single-pass timing check (no baseline needed)
|
||||
- `/web-performance-benchmark <url> --pages /,/dashboard,/api/health` — specify pages
|
||||
- `/web-performance-benchmark --diff` — benchmark only pages affected by current branch
|
||||
- `/web-performance-benchmark --trend` — show performance trends from historical data
|
||||
|
||||
## Instructions
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
name: gstack-benchmark
|
||||
name: web-performance-benchmark
|
||||
preamble-tier: 1
|
||||
version: 1.0.0
|
||||
description: |
|
||||
|
|
@ -28,29 +28,33 @@ allowed-tools:
|
|||
|
||||
{{BROWSE_SETUP}}
|
||||
|
||||
# /gstack-benchmark - Web Page Performance Regression Detection
|
||||
# /web-performance-benchmark - Web Page Performance Regression Detection
|
||||
|
||||
You are a **Performance Engineer** who has optimized apps serving millions of requests. You know that performance doesn't degrade in one big regression — it dies by a thousand paper cuts. Each PR adds 50ms here, 20KB there, and one day the app takes 8 seconds to load and nobody knows when it got slow.
|
||||
|
||||
Your job is to measure, baseline, compare, and alert. You use the browse daemon's `perf` command and JavaScript evaluation to gather real performance data from running pages.
|
||||
|
||||
## Not For
|
||||
## Scope
|
||||
|
||||
Do not run this skill for the bare word "benchmark" unless the request is clearly
|
||||
about web page performance. Skip this skill for model benchmarks, agent evals,
|
||||
business benchmark research, product metric design, or atrib behavior-impact
|
||||
measurement. For cross-model gstack skill comparisons, use `/benchmark-models`.
|
||||
Use this skill for web page speed, Core Web Vitals, resource sizes, and performance
|
||||
regressions. Do not use it for model benchmarks, agent evals, business benchmark
|
||||
research, product metric design, or atrib behavior-impact measurement. For
|
||||
cross-model gstack skill comparisons, use `/benchmark-models`.
|
||||
|
||||
## User-invocable
|
||||
When the user types `/gstack-benchmark`, run this skill.
|
||||
Use `/web-performance-benchmark` for new work.
|
||||
|
||||
Compatibility aliases: `/benchmark` and `/gstack-benchmark` run this same workflow.
|
||||
Keep them working for existing users, but use the canonical command in new docs,
|
||||
examples, and routing rules.
|
||||
|
||||
## Arguments
|
||||
- `/gstack-benchmark <url>` — full performance audit with baseline comparison
|
||||
- `/gstack-benchmark <url> --baseline` — capture baseline (run before making changes)
|
||||
- `/gstack-benchmark <url> --quick` — single-pass timing check (no baseline needed)
|
||||
- `/gstack-benchmark <url> --pages /,/dashboard,/api/health` — specify pages
|
||||
- `/gstack-benchmark --diff` — benchmark only pages affected by current branch
|
||||
- `/gstack-benchmark --trend` — show performance trends from historical data
|
||||
- `/web-performance-benchmark <url>` — full performance audit with baseline comparison
|
||||
- `/web-performance-benchmark <url> --baseline` — capture baseline (run before making changes)
|
||||
- `/web-performance-benchmark <url> --quick` — single-pass timing check (no baseline needed)
|
||||
- `/web-performance-benchmark <url> --pages /,/dashboard,/api/health` — specify pages
|
||||
- `/web-performance-benchmark --diff` — benchmark only pages affected by current branch
|
||||
- `/web-performance-benchmark --trend` — show performance trends from historical data
|
||||
|
||||
## Instructions
|
||||
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ Detailed guides for every gstack skill — philosophy, workflow, and examples.
|
|||
| [`/ship`](#ship) | **Release Engineer** | Sync main, run tests, audit coverage, push, open PR. Bootstraps test frameworks if you don't have one. One command. |
|
||||
| [`/land-and-deploy`](#land-and-deploy) | **Release Engineer** | Merge the PR, wait for CI and deploy, verify production health. One command from "approved" to "verified in production." |
|
||||
| [`/canary`](#canary) | **SRE** | Post-deploy monitoring loop. Watches for console errors, performance regressions, and page failures using the browse daemon. |
|
||||
| [`/benchmark`](#benchmark) | **Performance Engineer** | Baseline page load times, Core Web Vitals, and resource sizes. Compare before/after on every PR. Track trends over time. |
|
||||
| [`/web-performance-benchmark`](#web-performance-benchmark) | **Performance Engineer** | Baseline page load times, Core Web Vitals, and resource sizes. Compare before/after on every PR. Track trends over time. |
|
||||
| [`/cso`](#cso) | **Chief Security Officer** | OWASP Top 10 + STRIDE threat modeling security audit. Scans for injection, auth, crypto, and access control issues. |
|
||||
| [`/document-release`](#document-release) | **Technical Writer** | Update all project docs to match what you just shipped. Catches stale READMEs automatically. |
|
||||
| [`/document-generate`](#document-generate) | **Technical Writer** | Generate Diataxis docs (tutorial / how-to / reference / explanation) for a feature from code. |
|
||||
|
|
@ -715,16 +715,16 @@ Claude: Monitoring 8 pages every 2 minutes...
|
|||
|
||||
---
|
||||
|
||||
## `/benchmark`
|
||||
## `/web-performance-benchmark`
|
||||
|
||||
This is my **performance engineer mode**.
|
||||
|
||||
`/benchmark` establishes performance baselines for your pages: load time, Core Web Vitals (LCP, CLS, INP), resource counts, and total transfer size. Run it before and after a PR to catch regressions.
|
||||
`/web-performance-benchmark` establishes performance baselines for your pages: load time, Core Web Vitals (LCP, CLS, INP), resource counts, and total transfer size. Run it before and after a PR to catch regressions.
|
||||
|
||||
It uses the browse daemon for real Chromium measurements, not synthetic estimates. Multiple runs averaged. Results persist so you can track trends across PRs.
|
||||
|
||||
```
|
||||
You: /benchmark https://myapp.com
|
||||
You: /web-performance-benchmark https://myapp.com
|
||||
|
||||
Claude: Benchmarking 5 pages (3 runs each)...
|
||||
|
||||
|
|
|
|||
|
|
@ -11,7 +11,6 @@ Conventions:
|
|||
## Skills
|
||||
|
||||
- [/autoplan](autoplan/SKILL.md): Auto-review pipeline — reads the full CEO, design, eng, and DX review skills from disk and runs them sequentially with auto-decisions using 6 decision principles.
|
||||
- [/benchmark](benchmark/SKILL.md): Performance regression detection using the browse daemon.
|
||||
- [/benchmark-models](benchmark-models/SKILL.md): Cross-model benchmark for gstack skills.
|
||||
- [/browse](browse/SKILL.md): Fast headless browser for QA testing and site dogfooding.
|
||||
- [/canary](canary/SKILL.md): Post-deploy canary monitoring.
|
||||
|
|
@ -65,6 +64,7 @@ Conventions:
|
|||
- [/spec](spec/SKILL.md): Turn vague intent into a precise, executable spec in five phases.
|
||||
- [/sync-gbrain](sync-gbrain/SKILL.md): Keep gbrain current with this repo's code and refresh agent search guidance in CLAUDE.md.
|
||||
- [/unfreeze](unfreeze/SKILL.md): Clear the freeze boundary set by /freeze, allowing edits to all directories again.
|
||||
- [/web-performance-benchmark](web-performance-benchmark/SKILL.md): Web page performance regression detection using the browse daemon.
|
||||
|
||||
## Browse Commands
|
||||
|
||||
|
|
|
|||
|
|
@ -1898,7 +1898,7 @@ If verdict is REVERTED: Tell the user "The merge was reverted. Your changes are
|
|||
|
||||
Then suggest relevant follow-ups:
|
||||
- If a production URL was verified: "Want extended monitoring? Run `/canary <url>` to watch the site for the next 10 minutes."
|
||||
- If performance data was collected: "Want a deeper performance analysis? Run `/benchmark <url>`."
|
||||
- If performance data was collected: "Want a deeper performance analysis? Run `/web-performance-benchmark <url>`."
|
||||
- "Need to update docs? Run `/document-release` to sync README, CHANGELOG, and other docs with what you just shipped."
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -988,7 +988,7 @@ If verdict is REVERTED: Tell the user "The merge was reverted. Your changes are
|
|||
|
||||
Then suggest relevant follow-ups:
|
||||
- If a production URL was verified: "Want extended monitoring? Run `/canary <url>` to watch the site for the next 10 minutes."
|
||||
- If performance data was collected: "Want a deeper performance analysis? Run `/benchmark <url>`."
|
||||
- If performance data was collected: "Want a deeper performance analysis? Run `/web-performance-benchmark <url>`."
|
||||
- "Need to update docs? Run `/document-release` to sync README, CHANGELOG, and other docs with what you just shipped."
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -992,7 +992,7 @@ Tell the user:
|
|||
> You're all set! Here's what you can do with the connected Chrome:
|
||||
>
|
||||
> **Watch Claude work in real time:**
|
||||
> - Run any gstack skill (`/qa`, `/design-review`, `/benchmark`) and watch
|
||||
> - Run any gstack skill (`/qa`, `/design-review`, `/web-performance-benchmark`) and watch
|
||||
> every action happen in the visible Chrome window + Side Panel feed
|
||||
> - No cookie import needed — the Playwright browser shares its own session
|
||||
>
|
||||
|
|
@ -1010,7 +1010,7 @@ Tell the user:
|
|||
> - `/qa` runs its full test suite in the visible browser — you see every page
|
||||
> load, every click, every assertion
|
||||
> - `/design-review` takes screenshots in the real browser — same pixels you see
|
||||
> - `/benchmark` measures performance in the headed browser
|
||||
> - `/web-performance-benchmark` measures performance in the headed browser
|
||||
|
||||
Then proceed with whatever the user asked to do. If they didn't specify a task,
|
||||
ask what they'd like to test or browse.
|
||||
|
|
|
|||
|
|
@ -185,7 +185,7 @@ Tell the user:
|
|||
> You're all set! Here's what you can do with the connected Chrome:
|
||||
>
|
||||
> **Watch Claude work in real time:**
|
||||
> - Run any gstack skill (`/qa`, `/design-review`, `/benchmark`) and watch
|
||||
> - Run any gstack skill (`/qa`, `/design-review`, `/web-performance-benchmark`) and watch
|
||||
> every action happen in the visible Chrome window + Side Panel feed
|
||||
> - No cookie import needed — the Playwright browser shares its own session
|
||||
>
|
||||
|
|
@ -203,7 +203,7 @@ Tell the user:
|
|||
> - `/qa` runs its full test suite in the visible browser — you see every page
|
||||
> load, every click, every assertion
|
||||
> - `/design-review` takes screenshots in the real browser — same pixels you see
|
||||
> - `/benchmark` measures performance in the headed browser
|
||||
> - `/web-performance-benchmark` measures performance in the headed browser
|
||||
|
||||
Then proceed with whatever the user asked to do. If they didn't specify a task,
|
||||
ask what they'd like to test or browse.
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@ When asked for coding work, pick the dispatch tier:
|
|||
|
||||
**HEAVY:** needs a specific gstack methodology
|
||||
→ sessions_spawn(runtime: "acp", prompt: "Load gstack. Run /qa https://...")
|
||||
Skills: /cso, /review, /qa, /ship, /investigate, /design-review, /benchmark, /gstack-upgrade
|
||||
Skills: /cso, /review, /qa, /ship, /investigate, /design-review, /web-performance-benchmark, /gstack-upgrade
|
||||
|
||||
**FULL:** build a complete feature, multi-day scope, needs planning + review
|
||||
→ sessions_spawn(runtime: "acp", prompt: "<gstack-full content>\n\n<task>")
|
||||
|
|
|
|||
|
|
@ -24,12 +24,10 @@ import { COMMAND_DESCRIPTIONS as BROWSE_COMMANDS } from '../browse/src/commands'
|
|||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const OUTPUT = path.join(ROOT, 'gstack', 'llms.txt');
|
||||
const CLAUDE_SKIPPED_SKILL_DIRS = new Set(['claude']);
|
||||
|
||||
interface SkillEntry {
|
||||
name: string;
|
||||
description: string;
|
||||
href: string;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -115,11 +113,6 @@ function oneLine(text: string): string {
|
|||
return first.replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function skillDirForOutput(output: string): string {
|
||||
const dir = path.dirname(output);
|
||||
return dir === '.' ? '.' : dir;
|
||||
}
|
||||
|
||||
interface GenerateOptions {
|
||||
/** Override repo root (for tests). */
|
||||
root?: string;
|
||||
|
|
@ -142,9 +135,6 @@ export async function generateLlmsTxt(opts: GenerateOptions = {}): Promise<Gener
|
|||
const templates = discoverTemplates(root);
|
||||
const skills: SkillEntry[] = [];
|
||||
for (const t of templates) {
|
||||
if (CLAUDE_SKIPPED_SKILL_DIRS.has(skillDirForOutput(t.output))) {
|
||||
continue;
|
||||
}
|
||||
const filePath = path.join(root, t.tmpl);
|
||||
const entry = parseSkillFrontmatter(filePath);
|
||||
if (!entry) {
|
||||
|
|
@ -154,7 +144,7 @@ export async function generateLlmsTxt(opts: GenerateOptions = {}): Promise<Gener
|
|||
}
|
||||
continue;
|
||||
}
|
||||
skills.push({ ...entry, href: t.output });
|
||||
skills.push(entry);
|
||||
}
|
||||
skills.sort((a, b) => a.name.localeCompare(b.name));
|
||||
|
||||
|
|
@ -177,7 +167,7 @@ export async function generateLlmsTxt(opts: GenerateOptions = {}): Promise<Gener
|
|||
lines.push('');
|
||||
for (const skill of skills) {
|
||||
const summary = oneLine(skill.description);
|
||||
lines.push(`- [/${skill.name}](${skill.href}): ${summary}`);
|
||||
lines.push(`- [/${skill.name}](${skill.name}/SKILL.md): ${summary}`);
|
||||
}
|
||||
lines.push('');
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
},
|
||||
"benchmark-models": {
|
||||
"lead": "Cross-model benchmark for gstack skills.",
|
||||
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,\nand optionally quality via LLM judge. Answers \"which model is actually best\nfor this skill?\" with data instead of vibes. Separate from /benchmark, which\nmeasures web page performance. Use when: \"benchmark models\", \"compare models\",\n\"which model is best for X\", \"cross-model comparison\", \"model shootout\".",
|
||||
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,\nand optionally quality via LLM judge. Answers \"which model is actually best\nfor this skill?\" with data instead of vibes. Separate from /web-performance-benchmark, which\nmeasures web page performance. Use when: \"benchmark models\", \"compare models\",\n\"which model is best for X\", \"cross-model comparison\", \"model shootout\".",
|
||||
"voice_line": "Voice triggers (speech-to-text aliases): \"compare models\", \"model shootout\", \"which model is best\"."
|
||||
},
|
||||
"browse": {
|
||||
|
|
|
|||
8
setup
8
setup
|
|
@ -1008,6 +1008,14 @@ if [ "$INSTALL_CLAUDE" -eq 1 ]; then
|
|||
if [ -L "$_OGB_LINK" ] || [ ! -e "$_OGB_LINK" ]; then
|
||||
_link_or_copy "gstack/open-gstack-browser" "$_OGB_LINK"
|
||||
fi
|
||||
# Compatibility aliases for the former generic benchmark command. Keep
|
||||
# both paths available while new docs use /web-performance-benchmark.
|
||||
for _WEB_PERF_ALIAS in benchmark gstack-benchmark; do
|
||||
_WEB_PERF_LINK="$INSTALL_SKILLS_DIR/$_WEB_PERF_ALIAS"
|
||||
if [ -L "$_WEB_PERF_LINK" ] || [ ! -e "$_WEB_PERF_LINK" ]; then
|
||||
_link_or_copy "$SOURCE_GSTACK_DIR/benchmark" "$_WEB_PERF_LINK"
|
||||
fi
|
||||
done
|
||||
if [ "$LOCAL_INSTALL" -eq 1 ]; then
|
||||
log "gstack ready (project-local)."
|
||||
log " skills: $INSTALL_SKILLS_DIR"
|
||||
|
|
|
|||
|
|
@ -473,15 +473,22 @@ describe('gen-skill-docs', () => {
|
|||
|
||||
for (const content of [generated, template]) {
|
||||
const frontmatter = content.match(/^---\n([\s\S]*?)\n---/)?.[1] ?? '';
|
||||
expect(frontmatter).toContain('name: gstack-benchmark');
|
||||
expect(frontmatter).not.toContain('name: benchmark');
|
||||
expect(frontmatter).toContain('name: web-performance-benchmark');
|
||||
expect(frontmatter).not.toContain('name: benchmark\n');
|
||||
expect(frontmatter).toContain('web performance benchmark');
|
||||
expect(frontmatter).not.toMatch(/["']benchmark["']/);
|
||||
expect(content).toContain('Do not run this skill for the bare word "benchmark"');
|
||||
expect(content).toContain('For cross-model gstack skill comparisons, use `/benchmark-models`.');
|
||||
expect(content).toContain('/gstack-benchmark <url>');
|
||||
expect(content).not.toContain('When the user types `/benchmark`, run this skill.');
|
||||
expect(content).toContain('Use `/web-performance-benchmark` for new work.');
|
||||
expect(content).toContain('Compatibility aliases: `/benchmark` and `/gstack-benchmark` run this same workflow.');
|
||||
expect(content).toContain('/web-performance-benchmark <url>');
|
||||
}
|
||||
|
||||
expect(template).toContain('cross-model gstack skill comparisons, use `/benchmark-models`.');
|
||||
});
|
||||
|
||||
test('setup retains benchmark command compatibility aliases', () => {
|
||||
const setup = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
|
||||
|
||||
expect(setup).toContain('for _WEB_PERF_ALIAS in benchmark gstack-benchmark; do');
|
||||
expect(setup).toContain('_link_or_copy "$SOURCE_GSTACK_DIR/benchmark" "$_WEB_PERF_LINK"');
|
||||
});
|
||||
|
||||
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
|
||||
|
|
@ -1708,7 +1715,9 @@ describe('Codex generation (--host codex)', () => {
|
|||
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
||||
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
|
||||
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
||||
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
||||
const template = fs.readFileSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'), 'utf-8');
|
||||
const frontmatterName = template.match(/^name:\s*(.+)$/m)?.[1]?.trim() ?? entry.name;
|
||||
const codexName = frontmatterName.startsWith('gstack-') ? frontmatterName : `gstack-${frontmatterName}`;
|
||||
if (isSymlinkLoop(codexName)) continue;
|
||||
skills.push({ dir: entry.name, codexName });
|
||||
}
|
||||
|
|
@ -2027,7 +2036,9 @@ describe('Factory generation (--host factory)', () => {
|
|||
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
||||
if (entry.name === 'codex') continue;
|
||||
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
||||
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
||||
const template = fs.readFileSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'), 'utf-8');
|
||||
const frontmatterName = template.match(/^name:\s*(.+)$/m)?.[1]?.trim() ?? entry.name;
|
||||
const factoryName = frontmatterName.startsWith('gstack-') ? frontmatterName : `gstack-${frontmatterName}`;
|
||||
if (isSymlinkLoop(factoryName)) continue;
|
||||
skills.push({ dir: entry.name, factoryName });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -367,6 +367,7 @@ export const E2E_TOUCHFILES: Record<string, string[]> = {
|
|||
'journey-retro': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts'],
|
||||
'journey-design-system': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts'],
|
||||
'journey-visual-qa': ['*/SKILL.md.tmpl', 'SKILL.md.tmpl', 'scripts/gen-skill-docs.ts'],
|
||||
'journey-web-performance-benchmark': ['benchmark/**', 'scripts/gen-skill-docs.ts'],
|
||||
|
||||
// Opus 4.7 behavior evals — keys match testName: values in the test file.
|
||||
// Routing sub-tests use template literal `routing-${c.name}` testNames,
|
||||
|
|
@ -735,6 +736,7 @@ export const E2E_TIERS: Record<string, 'gate' | 'periodic'> = {
|
|||
'journey-retro': 'periodic',
|
||||
'journey-design-system': 'periodic',
|
||||
'journey-visual-qa': 'periodic',
|
||||
'journey-web-performance-benchmark': 'periodic',
|
||||
|
||||
// Opus 4.7 overlay evals — periodic (non-deterministic LLM behavior + Opus cost)
|
||||
'fanout-arm-overlay-on': 'periodic',
|
||||
|
|
|
|||
|
|
@ -25,29 +25,17 @@ describe('gen-llms-txt — shape', () => {
|
|||
expect(generated.content).toContain('auto-generated');
|
||||
});
|
||||
|
||||
test('every generated Claude skill appears in the index', () => {
|
||||
test('every skill .tmpl in the repo appears in the index', () => {
|
||||
const templates = discoverTemplates(ROOT);
|
||||
const claudeGeneratedTemplates = templates.filter((t) => path.dirname(t.output) !== 'claude');
|
||||
// Filter to those that successfully parsed (have name + description).
|
||||
expect(generated.skills.length).toBeGreaterThan(0);
|
||||
expect(generated.skills.length).toBeLessThanOrEqual(claudeGeneratedTemplates.length);
|
||||
expect(generated.content).not.toContain('[/claude]');
|
||||
expect(generated.skills.length).toBeLessThanOrEqual(templates.length);
|
||||
|
||||
for (const skill of generated.skills) {
|
||||
expect(generated.content).toMatch(new RegExp(`/${skill.name}\\b`));
|
||||
}
|
||||
});
|
||||
|
||||
test('every skill link points at an existing generated file', () => {
|
||||
const skillsSection = generated.content.split('## Skills')[1].split('## Browse Commands')[0];
|
||||
const links = [...skillsSection.matchAll(/- \[\/[^\]]+\]\(([^)]+)\):/g)].map((m) => m[1]);
|
||||
expect(links.length).toBe(generated.skills.length);
|
||||
|
||||
for (const href of links) {
|
||||
expect(fs.existsSync(path.join(ROOT, href)), href).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test('every browse command in COMMAND_DESCRIPTIONS appears in the index', () => {
|
||||
expect(generated.browseCommands.length).toBeGreaterThan(0);
|
||||
for (const cmd of generated.browseCommands) {
|
||||
|
|
|
|||
|
|
@ -328,7 +328,7 @@ describeIfSelected('Benchmark skill E2E', ['benchmark-workflow'], () => {
|
|||
|
||||
testConcurrentIfSelected('benchmark-workflow', async () => {
|
||||
const result = await runSkillTest({
|
||||
prompt: `Read benchmark/SKILL.md for the /benchmark skill instructions.
|
||||
prompt: `Read benchmark/SKILL.md for the /web-performance-benchmark skill instructions.
|
||||
|
||||
You are simulating a benchmark run. There is NO browse daemon available and NO production URL.
|
||||
|
||||
|
|
@ -353,8 +353,8 @@ Just create the files showing the correct schema and report format.`,
|
|||
runId,
|
||||
});
|
||||
|
||||
logCost('/benchmark', result);
|
||||
recordE2E(evalCollector, '/benchmark workflow', 'Benchmark skill E2E', result);
|
||||
logCost('/web-performance-benchmark', result);
|
||||
recordE2E(evalCollector, '/web-performance-benchmark workflow', 'Web performance benchmark skill E2E', result);
|
||||
expect(result.exitReason).toBe('success');
|
||||
|
||||
expect(fs.existsSync(path.join(benchDir, '.gstack', 'benchmark-reports'))).toBe(true);
|
||||
|
|
|
|||
|
|
@ -65,20 +65,23 @@ if (evalsEnabled && process.env.EVALS_TIER) {
|
|||
* install that point to different worktrees or dangling targets. */
|
||||
function installSkills(tmpDir: string) {
|
||||
const skillDirs = [
|
||||
'', // root gstack SKILL.md
|
||||
'qa', 'qa-only', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review',
|
||||
'plan-design-review', 'design-review', 'design-consultation', 'retro',
|
||||
'document-release', 'investigate', 'office-hours', 'browse', 'setup-browser-cookies',
|
||||
'gstack-upgrade', 'humanizer',
|
||||
{ source: '', command: 'gstack' },
|
||||
{ source: 'qa' }, { source: 'qa-only' }, { source: 'ship' }, { source: 'review' },
|
||||
{ source: 'plan-ceo-review' }, { source: 'plan-eng-review' },
|
||||
{ source: 'plan-design-review' }, { source: 'design-review' },
|
||||
{ source: 'design-consultation' }, { source: 'retro' },
|
||||
{ source: 'document-release' }, { source: 'investigate' }, { source: 'office-hours' },
|
||||
{ source: 'browse' }, { source: 'benchmark', command: 'web-performance-benchmark' },
|
||||
{ source: 'setup-browser-cookies' }, { source: 'gstack-upgrade' }, { source: 'humanizer' },
|
||||
];
|
||||
|
||||
const targetBase = path.join(tmpDir, '.claude', 'skills');
|
||||
|
||||
for (const skill of skillDirs) {
|
||||
const srcPath = path.join(ROOT, skill, 'SKILL.md');
|
||||
const srcPath = path.join(ROOT, skill.source, 'SKILL.md');
|
||||
if (!fs.existsSync(srcPath)) continue;
|
||||
|
||||
const skillName = skill || 'gstack';
|
||||
const skillName = skill.command || skill.source;
|
||||
const destDir = path.join(targetBase, skillName);
|
||||
fs.mkdirSync(destDir, { recursive: true });
|
||||
fs.copyFileSync(srcPath, path.join(destDir, 'SKILL.md'));
|
||||
|
|
@ -107,6 +110,7 @@ Key routing rules:
|
|||
- Design system, brand → invoke design-consultation
|
||||
- Visual audit, design polish → invoke design-review
|
||||
- Architecture review → invoke plan-eng-review
|
||||
- Web page speed, Core Web Vitals, or performance regression → invoke web-performance-benchmark
|
||||
`);
|
||||
}
|
||||
|
||||
|
|
@ -181,6 +185,34 @@ describeE2E('Skill Routing E2E — Developer Journey', () => {
|
|||
evalCollector?.finalize();
|
||||
});
|
||||
|
||||
testIfSelected('journey-web-performance-benchmark', async () => {
|
||||
const tmpDir = createRoutingWorkDir('web-performance-benchmark');
|
||||
try {
|
||||
const testName = 'journey-web-performance-benchmark';
|
||||
const expectedSkill = 'web-performance-benchmark';
|
||||
const result = await runSkillTest({
|
||||
prompt: 'Our checkout page got slower after the latest release. Measure Core Web Vitals, compare the current page load against a baseline, and identify any web performance regression.',
|
||||
workingDirectory: tmpDir,
|
||||
maxTurns: 5,
|
||||
allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
|
||||
timeout: 60_000,
|
||||
testName,
|
||||
runId,
|
||||
});
|
||||
|
||||
const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
|
||||
const actualSkill = skillCalls[0]?.input?.skill;
|
||||
|
||||
logCost(`journey: ${testName}`, result);
|
||||
recordRouting(testName, result, expectedSkill, actualSkill);
|
||||
|
||||
expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
|
||||
expect(actualSkill).toBe(expectedSkill);
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
}, 150_000);
|
||||
|
||||
testIfSelected('journey-ideation', async () => {
|
||||
const tmpDir = createRoutingWorkDir('ideation');
|
||||
try {
|
||||
|
|
|
|||
Loading…
Reference in New Issue