mirror of https://github.com/garrytan/gstack.git
Merge d71434287a into 2beb636f7c
This commit is contained in:
commit
bbfc0969e2
|
|
@ -19,6 +19,7 @@ bin/gstack-global-discover*
|
|||
.opencode/
|
||||
.slate/
|
||||
.cursor/
|
||||
.grok/
|
||||
.openclaw/
|
||||
.hermes/
|
||||
.gbrain/
|
||||
|
|
|
|||
49
TODOS.md
49
TODOS.md
|
|
@ -2,6 +2,55 @@
|
|||
|
||||
## NEXT PRIORITY
|
||||
|
||||
### P2: Grok Build full-compat follow-ups (after `feat/grok-build-host`)
|
||||
|
||||
**What:** Finish / harden the Grok-native packaging work landed on
|
||||
`feat/grok-build-host` (runtime root, SETUP path fix, host-bleed cleanup,
|
||||
bridges, `bin/gstack-grok-compat-audit`, section pointers for token ceiling).
|
||||
These are the remaining product/ship items — not blockers for local Grok use.
|
||||
|
||||
**Context (landed on branch):**
|
||||
- U1–U3 packaging + U4–U7 bridges + U9 audit gate
|
||||
- Setup host-scoping (no always-on Codex `.agents/` regen / Claude plan-tune
|
||||
prompt on `--host grok-build`)
|
||||
- Option A: Grok carved skills use STOP-Read section pointers
|
||||
(`~/.grok/skills/gstack-*/sections/`) — `/ship` always-loaded ~20k tokens
|
||||
(was ~45k)
|
||||
|
||||
**Follow-ups to track:**
|
||||
|
||||
1. **Upstream PR** — open/update PR against `garrytan/gstack` for
|
||||
`feat/grok-build-host` (VERSION/CHANGELOG/title deferred until local DoD
|
||||
green on main merge path).
|
||||
2. **U4 behavioral smoke** — if claiming `/spec --execute` COMPATIBLE (not
|
||||
file-only), record multi-turn tool-use / cwd isolation / fail-closed auth
|
||||
smoke or fixture; flag-string greps alone are not enough.
|
||||
3. **Section pointers for other external hosts (optional)** — Codex ship is
|
||||
still monolith ~39k (near soft ceiling). Reuse
|
||||
`hostUsesSectionPointers` for Codex/Cursor/Factory when package paths are
|
||||
portable (Codex: `.agents/skills/gstack-*` or global
|
||||
`~/.codex/skills/gstack-*`).
|
||||
4. **R10 / U8 optional `gstack-codex`** — only if product wants suite
|
||||
completeness; keep bare `/codex` = OpenAI plugin; deny bare `codex` alias
|
||||
on Grok install (partial deny already in `link_grok_skill_dirs`).
|
||||
5. **Gbrain Grok MCP stretch** — optional `~/.grok/config.toml` gbrain stanza
|
||||
(absolute path, TOML RMW, atomic write + backup; non-interactive refuse
|
||||
overwrite without `--force`). Success path remains CLI + AGENTS.md.
|
||||
6. **Audit harness productization** — keep `bin/gstack-grok-compat-audit` as
|
||||
long-lived multi-host CI later, or document as branch/DoD gate only
|
||||
(open question from plan ce-doc-review).
|
||||
7. **Name collision note** — document `/review` vs compound-engineering
|
||||
`code-review` routing in `~/.grok/AGENTS.md` (do not delete CE skill).
|
||||
|
||||
**Why:** Branch is implementation-ready for local Grok; these items close
|
||||
honesty gaps, upstream ship, and optional suite completeness without
|
||||
reopening Phase A packaging.
|
||||
|
||||
**Depends on / blocked by:** Nothing for local use. Upstream PR waits on
|
||||
clean validation + VERSION/CHANGELOG if that is the release process.
|
||||
|
||||
---
|
||||
|
||||
### P1: #1882 — portable skill-install prefix (non-`gstack` install dirs break silently)
|
||||
|
||||
**What:** Every generated SKILL.md hardcodes the literal `~/.claude/skills/gstack/...`
|
||||
|
|
|
|||
|
|
@ -20,11 +20,12 @@ allowed-tools:
|
|||
## When to invoke this skill
|
||||
|
||||
Runs the same prompt through Claude,
|
||||
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
|
||||
and optionally quality via LLM judge. Answers "which model is actually best
|
||||
for this skill?" with data instead of vibes. Separate from /benchmark, which
|
||||
measures web page performance. Use when: "benchmark models", "compare models",
|
||||
"which model is best for X", "cross-model comparison", "model shootout".
|
||||
GPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,
|
||||
cost, and optionally quality via LLM judge. Answers "which model is actually
|
||||
best for this skill?" with data instead of vibes. Separate from /benchmark,
|
||||
which measures web page performance. Use when: "benchmark models", "compare
|
||||
models", "which model is best for X", "cross-model comparison", "model
|
||||
shootout".
|
||||
|
||||
Voice triggers (speech-to-text aliases): "compare models", "model shootout", "which model is best".
|
||||
|
||||
|
|
@ -578,12 +579,12 @@ If C: ask for the path. Verify it exists. Use as positional argument.
|
|||
## Step 2: Choose providers
|
||||
|
||||
```bash
|
||||
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini --dry-run
|
||||
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini,grok --dry-run
|
||||
```
|
||||
|
||||
Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).
|
||||
|
||||
If ALL three show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, or `gemini login` / `export GOOGLE_API_KEY`.
|
||||
If ALL show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, `gemini login` / `export GOOGLE_API_KEY`, or `grok` login / `export XAI_API_KEY`.
|
||||
|
||||
If at least one is OK: AskUserQuestion:
|
||||
- **Simplify:** "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
|
||||
|
|
@ -591,7 +592,8 @@ If at least one is OK: AskUserQuestion:
|
|||
- **Options:**
|
||||
- A) All authed providers. Completeness: 10/10.
|
||||
- B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
|
||||
- C) Pick two — specify on next turn. Completeness: 8/10.
|
||||
- C) Only Grok (Grok-only machine). Completeness: 7/10.
|
||||
- D) Pick two — specify on next turn. Completeness: 8/10.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -608,7 +610,8 @@ If judge is available, AskUserQuestion:
|
|||
- A) Enable judge (adds ~$0.05). Completeness: 10/10.
|
||||
- B) Skip judge — speed/cost/tokens only. Completeness: 7/10.
|
||||
|
||||
If judge is NOT available, skip this question and omit the `--judge` flag.
|
||||
If judge is NOT available (Grok-only / no ANTHROPIC_API_KEY): skip this question
|
||||
and omit `--judge` (Grok-or-skip). Speed/cost/tokens still work.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -4,11 +4,12 @@ preamble-tier: 1
|
|||
version: 1.0.0
|
||||
description: |
|
||||
Cross-model benchmark for gstack skills. Runs the same prompt through Claude,
|
||||
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
|
||||
and optionally quality via LLM judge. Answers "which model is actually best
|
||||
for this skill?" with data instead of vibes. Separate from /benchmark, which
|
||||
measures web page performance. Use when: "benchmark models", "compare models",
|
||||
"which model is best for X", "cross-model comparison", "model shootout". (gstack)
|
||||
GPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,
|
||||
cost, and optionally quality via LLM judge. Answers "which model is actually
|
||||
best for this skill?" with data instead of vibes. Separate from /benchmark,
|
||||
which measures web page performance. Use when: "benchmark models", "compare
|
||||
models", "which model is best for X", "cross-model comparison", "model
|
||||
shootout". (gstack)
|
||||
voice-triggers:
|
||||
- "compare models"
|
||||
- "model shootout"
|
||||
|
|
@ -69,12 +70,12 @@ If C: ask for the path. Verify it exists. Use as positional argument.
|
|||
## Step 2: Choose providers
|
||||
|
||||
```bash
|
||||
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini --dry-run
|
||||
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini,grok --dry-run
|
||||
```
|
||||
|
||||
Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).
|
||||
|
||||
If ALL three show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, or `gemini login` / `export GOOGLE_API_KEY`.
|
||||
If ALL show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, `gemini login` / `export GOOGLE_API_KEY`, or `grok` login / `export XAI_API_KEY`.
|
||||
|
||||
If at least one is OK: AskUserQuestion:
|
||||
- **Simplify:** "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
|
||||
|
|
@ -82,7 +83,8 @@ If at least one is OK: AskUserQuestion:
|
|||
- **Options:**
|
||||
- A) All authed providers. Completeness: 10/10.
|
||||
- B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
|
||||
- C) Pick two — specify on next turn. Completeness: 8/10.
|
||||
- C) Only Grok (Grok-only machine). Completeness: 7/10.
|
||||
- D) Pick two — specify on next turn. Completeness: 8/10.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -99,7 +101,8 @@ If judge is available, AskUserQuestion:
|
|||
- A) Enable judge (adds ~$0.05). Completeness: 10/10.
|
||||
- B) Skip judge — speed/cost/tokens only. Completeness: 7/10.
|
||||
|
||||
If judge is NOT available, skip this question and omit the `--judge` flag.
|
||||
If judge is NOT available (Grok-only / no ANTHROPIC_API_KEY): skip this question
|
||||
and omit `--judge` (Grok-or-skip). Speed/cost/tokens still work.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,362 @@
|
|||
#!/usr/bin/env bun
|
||||
/**
|
||||
* gstack-grok-compat-audit — Definition of Done gate for Grok Build packaging
|
||||
* + bridge honesty (plan U9 / R11).
|
||||
*
|
||||
* Exit 0 only when packaging checks pass for installed gstack membership
|
||||
* packages and bridge gates are not theater-greened.
|
||||
*
|
||||
* Usage:
|
||||
* bin/gstack-grok-compat-audit
|
||||
* bin/gstack-grok-compat-audit --skills-dir ~/.grok/skills
|
||||
* bin/gstack-grok-compat-audit --phase a # packaging only
|
||||
* bin/gstack-grok-compat-audit --phase ab # packaging + bridge gates (default)
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { execFileSync } from 'child_process';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
function arg(name: string, def?: string): string | undefined {
|
||||
const idx = args.findIndex(a => a === name || a.startsWith(name + '='));
|
||||
if (idx < 0) return def;
|
||||
const eq = args[idx].indexOf('=');
|
||||
if (eq >= 0) return args[idx].slice(eq + 1);
|
||||
return args[idx + 1] ?? def;
|
||||
}
|
||||
|
||||
const SKILLS_DIR = path.resolve(
|
||||
(arg('--skills-dir') ?? path.join(os.homedir(), '.grok', 'skills')).replace(/^~/, os.homedir()),
|
||||
);
|
||||
const PHASE = (arg('--phase') ?? 'ab').toLowerCase();
|
||||
const GSTACK_ROOT = path.join(SKILLS_DIR, 'gstack');
|
||||
|
||||
type Severity = 'fail' | 'warn' | 'ok';
|
||||
interface Finding {
|
||||
severity: Severity;
|
||||
area: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
const findings: Finding[] = [];
|
||||
|
||||
function fail(area: string, message: string) {
|
||||
findings.push({ severity: 'fail', area, message });
|
||||
}
|
||||
function warn(area: string, message: string) {
|
||||
findings.push({ severity: 'warn', area, message });
|
||||
}
|
||||
function ok(area: string, message: string) {
|
||||
findings.push({ severity: 'ok', area, message });
|
||||
}
|
||||
|
||||
function exists(p: string): boolean {
|
||||
try {
|
||||
fs.accessSync(p);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function readText(p: string): string {
|
||||
try {
|
||||
return fs.readFileSync(p, 'utf-8');
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function isGstackMembership(dirName: string): boolean {
|
||||
if (dirName === 'gstack' || dirName === 'connect-chrome' || dirName === 'browse') return true;
|
||||
if (dirName.startsWith('gstack-')) return true;
|
||||
// bare aliases that point at gstack packages (symlink or real SKILL with gstack markers)
|
||||
const skill = path.join(SKILLS_DIR, dirName, 'SKILL.md');
|
||||
if (!exists(skill)) return false;
|
||||
const body = readText(skill);
|
||||
return body.includes('$GSTACK_ROOT') || body.includes('gstack') || body.includes('AUTO-GENERATED');
|
||||
}
|
||||
|
||||
function listMembershipPackages(): string[] {
|
||||
if (!exists(SKILLS_DIR)) return [];
|
||||
return fs
|
||||
.readdirSync(SKILLS_DIR, { withFileTypes: true })
|
||||
.filter(d => d.isDirectory() || d.isSymbolicLink())
|
||||
.map(d => d.name)
|
||||
.filter(isGstackMembership)
|
||||
.sort();
|
||||
}
|
||||
|
||||
// ─── Packaging suite (Phase A) ────────────────────────────────
|
||||
|
||||
function auditRuntimeRoot() {
|
||||
const area = 'runtime-root';
|
||||
if (!exists(GSTACK_ROOT)) {
|
||||
fail(area, `missing runtime root: ${GSTACK_ROOT}`);
|
||||
return;
|
||||
}
|
||||
const requiredDirs = [
|
||||
'bin',
|
||||
'browse/dist',
|
||||
'browse/src',
|
||||
'review/specialists',
|
||||
'scripts',
|
||||
];
|
||||
for (const rel of requiredDirs) {
|
||||
const p = path.join(GSTACK_ROOT, rel);
|
||||
if (!exists(p)) fail(area, `missing required asset: ${rel}`);
|
||||
else ok(area, `present: ${rel}`);
|
||||
}
|
||||
// Conditional when monorepo had them at install — soft-check if src monorepo has them
|
||||
const monorepoHas = (rel: string) => exists(path.join(ROOT, rel));
|
||||
if (monorepoHas('design/dist') && !exists(path.join(GSTACK_ROOT, 'design/dist'))) {
|
||||
fail(area, 'monorepo has design/dist but runtime root does not');
|
||||
}
|
||||
if (monorepoHas('extension') && !exists(path.join(GSTACK_ROOT, 'extension'))) {
|
||||
fail(area, 'monorepo has extension/ but runtime root does not');
|
||||
}
|
||||
if (monorepoHas('make-pdf/dist') && !exists(path.join(GSTACK_ROOT, 'make-pdf/dist'))) {
|
||||
warn(area, 'monorepo has make-pdf/dist but runtime root does not (membership optional)');
|
||||
}
|
||||
for (const f of ['checklist.md', 'TODOS-format.md']) {
|
||||
const p = path.join(GSTACK_ROOT, 'review', f);
|
||||
if (!exists(p)) fail(area, `missing review/${f}`);
|
||||
else ok(area, `present: review/${f}`);
|
||||
}
|
||||
}
|
||||
|
||||
function auditNoDoubleHome() {
|
||||
const area = 'no-double-home';
|
||||
const pkgs = listMembershipPackages();
|
||||
let hits = 0;
|
||||
for (const name of pkgs) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
const body = readText(skill);
|
||||
if (/\$HOME\$GSTACK/.test(body)) {
|
||||
fail(area, `${name}: contains $HOME$GSTACK (double-home path bug)`);
|
||||
hits++;
|
||||
}
|
||||
}
|
||||
if (hits === 0) ok(area, `no $HOME$GSTACK in ${pkgs.length} packages`);
|
||||
}
|
||||
|
||||
function auditNoUngeneratedMonorepoPointers() {
|
||||
const area = 'no-monorepo-skill-pointer';
|
||||
// Skills under ~/.grok/skills must not be bare monorepo dirs with unrewritten Claude SKILL.md
|
||||
// Heuristic: ungenerated monorepo package often has MODEL_OVERLAY: claude section + ~/.claude paths without rewrite
|
||||
const pkgs = listMembershipPackages();
|
||||
for (const name of pkgs) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
const body = readText(skill);
|
||||
// Skip runtime root
|
||||
if (name === 'gstack' && !body.includes('name:')) continue;
|
||||
if (body.includes('~/.claude/skills/gstack') && !body.includes('$GSTACK_ROOT')) {
|
||||
// Still has unrewritten Claude global path without env var form
|
||||
fail(area, `${name}: unrewritten ~/.claude/skills/gstack pointer`);
|
||||
}
|
||||
}
|
||||
ok(area, 'membership packages scanned for unrewritten Claude monorepo pointers');
|
||||
}
|
||||
|
||||
function auditConnectChrome() {
|
||||
const area = 'connect-chrome';
|
||||
const link = path.join(SKILLS_DIR, 'connect-chrome');
|
||||
if (!exists(link)) {
|
||||
fail(area, 'connect-chrome missing under skills dir');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const target = fs.realpathSync(link);
|
||||
if (!/open-gstack-browser/.test(target) && !/gstack-open-gstack-browser/.test(target)) {
|
||||
fail(area, `connect-chrome does not resolve to open-gstack-browser (got ${target})`);
|
||||
} else {
|
||||
ok(area, `connect-chrome → ${target}`);
|
||||
}
|
||||
} catch (e) {
|
||||
fail(area, `connect-chrome realpath failed: ${(e as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function auditHostBleedSample() {
|
||||
const area = 'host-bleed';
|
||||
const samples = ['gstack-ship', 'ship', 'gstack-browse', 'browse', 'gstack-claude', 'claude'];
|
||||
for (const name of samples) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
const body = readText(skill);
|
||||
if (/MODEL_OVERLAY:\s*claude/.test(body)) {
|
||||
fail(area, `${name}: MODEL_OVERLAY: claude still present`);
|
||||
}
|
||||
if (/## Model-Specific Behavioral Patch \(claude\)/.test(body)) {
|
||||
fail(area, `${name}: Claude model overlay section body present`);
|
||||
}
|
||||
// Claude skill is allowed to mention claude CLI extensively
|
||||
if (name !== 'claude' && name !== 'gstack-claude') {
|
||||
if (/\bAskUserQuestion\b/.test(body) && !/ask_user_question/.test(body)) {
|
||||
// May still appear in examples — soft
|
||||
warn(area, `${name}: raw AskUserQuestion token (prefer ask_user_question rewrite)`);
|
||||
}
|
||||
}
|
||||
if (name === 'claude' || name === 'gstack-claude') {
|
||||
if (!/MULTI_CLI_BRIDGE/.test(body) && !/Step 0: Check Claude CLI/.test(body)) {
|
||||
fail(area, `${name}: missing MULTI_CLI_BRIDGE / Step 0 detect`);
|
||||
} else {
|
||||
ok(area, `${name}: Step 0 / multi-CLI packaging present`);
|
||||
}
|
||||
}
|
||||
}
|
||||
ok(area, 'host-bleed sample scan complete');
|
||||
}
|
||||
|
||||
function auditSkillifySdkPath() {
|
||||
const area = 'skillify-sdk';
|
||||
const src = path.join(GSTACK_ROOT, 'browse', 'src', 'browse-client.ts');
|
||||
if (exists(src)) ok(area, 'browse/src/browse-client.ts resolvable under $GSTACK_ROOT');
|
||||
else fail(area, 'browse/src/browse-client.ts missing — skillify SDK path broken');
|
||||
}
|
||||
|
||||
// ─── Bridge gates (Phase B) ───────────────────────────────────
|
||||
|
||||
function auditBridges() {
|
||||
// Install prose alone must not claim primary READY when CLI missing
|
||||
const area = 'bridge-policy';
|
||||
|
||||
// Prefer installed runtime root binary (what users run); fall back to monorepo.
|
||||
const installedBench = path.join(GSTACK_ROOT, 'bin', 'gstack-model-benchmark');
|
||||
const monorepoBench = path.join(ROOT, 'bin', 'gstack-model-benchmark');
|
||||
const benchBin = exists(installedBench) ? installedBench : monorepoBench;
|
||||
if (!exists(installedBench) && exists(monorepoBench)) {
|
||||
warn(area, 'benchmark: using monorepo binary (installed runtime bin missing) — packaging incomplete?');
|
||||
}
|
||||
try {
|
||||
// No shell: argv list avoids monorepo-path injection.
|
||||
const help = execFileSync(benchBin, ['--prompt', 'x', '--models', 'grok', '--dry-run'], {
|
||||
encoding: 'utf-8',
|
||||
timeout: 15000,
|
||||
maxBuffer: 2 * 1024 * 1024,
|
||||
});
|
||||
if (/unknown provider.*grok/i.test(help)) {
|
||||
fail(area, 'benchmark: grok still unknown provider');
|
||||
} else if (/grok:\s*(OK|NOT READY)/i.test(help)) {
|
||||
ok(area, `benchmark: grok provider wired via ${benchBin} (dry-run READY/NOT READY is boolean auth)`);
|
||||
} else {
|
||||
warn(area, `benchmark dry-run unexpected output: ${help.slice(0, 200)}`);
|
||||
}
|
||||
} catch (e) {
|
||||
const msg = (e as { stdout?: string; message?: string }).stdout
|
||||
?? (e as Error).message
|
||||
?? String(e);
|
||||
if (/unknown provider.*grok/i.test(msg)) fail(area, 'benchmark: grok unknown provider');
|
||||
else warn(area, `benchmark dry-run error: ${String(msg).slice(0, 200)}`);
|
||||
}
|
||||
|
||||
// spec: at least one installed package must exist and use Grok-native spawn
|
||||
let specSeen = 0;
|
||||
for (const name of ['gstack-spec', 'spec']) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
specSeen++;
|
||||
const body = readText(skill);
|
||||
if (/\$\(cat\s+["']?\$ARCHIVE/.test(body) || /grok[^\n]*\$\(cat/.test(body)) {
|
||||
fail(area, `${name}: banned $(cat …) into grok argv`);
|
||||
}
|
||||
if (/grok --prompt-file/.test(body) || /Spawn \*\*Grok\*\*/.test(body)) {
|
||||
ok(area, `${name}: Grok-native spawn present`);
|
||||
} else if (/Spawned:.*claude -p/.test(body) && !/--execute-claude/.test(body)) {
|
||||
fail(area, `${name}: default execute still only claude -p (no Grok spawn)`);
|
||||
} else {
|
||||
fail(area, `${name}: missing researched grok --prompt-file spawn`);
|
||||
}
|
||||
}
|
||||
if (specSeen === 0) {
|
||||
fail(area, 'spec: neither gstack-spec nor spec package installed under skills dir');
|
||||
}
|
||||
|
||||
// setup-gbrain: must not hard-require which claude for success
|
||||
let gbrainSeen = 0;
|
||||
for (const name of ['gstack-setup-gbrain', 'setup-gbrain']) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
gbrainSeen++;
|
||||
const body = readText(skill);
|
||||
if (/Grok Build success path/.test(body) || /no Claude required/i.test(body)) {
|
||||
ok(area, `${name}: Grok success path documented`);
|
||||
} else {
|
||||
warn(area, `${name}: missing explicit Grok success path (CLI+AGENTS)`);
|
||||
}
|
||||
}
|
||||
if (gbrainSeen === 0) {
|
||||
warn(area, 'setup-gbrain: package not installed (bridge not audited)');
|
||||
}
|
||||
|
||||
// pair-agent: local must not hard-require ngrok; remote has install+security+teardown
|
||||
let pairSeen = 0;
|
||||
for (const name of ['gstack-pair-agent', 'pair-agent']) {
|
||||
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
|
||||
if (!exists(skill)) continue;
|
||||
pairSeen++;
|
||||
const body = readText(skill);
|
||||
if (/teardown/i.test(body) && /ngrok/i.test(body) && /127\.0\.0\.1|security/i.test(body)) {
|
||||
ok(area, `${name}: remote ngrok install+security+teardown present`);
|
||||
} else {
|
||||
warn(area, `${name}: remote security/teardown checklist incomplete`);
|
||||
}
|
||||
if (/DEPENDENT/.test(body)) ok(area, `${name}: honest DEPENDENT labeling`);
|
||||
}
|
||||
if (pairSeen === 0) {
|
||||
warn(area, 'pair-agent: package not installed (bridge not audited)');
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Main ─────────────────────────────────────────────────────
|
||||
|
||||
function main() {
|
||||
console.log('== gstack-grok-compat-audit ==');
|
||||
console.log(` skills-dir: ${SKILLS_DIR}`);
|
||||
console.log(` phase: ${PHASE}`);
|
||||
console.log(` monorepo: ${ROOT}`);
|
||||
console.log('');
|
||||
|
||||
if (!exists(SKILLS_DIR)) {
|
||||
fail('install', `skills dir missing: ${SKILLS_DIR} — run: cd ${ROOT} && ./setup --host grok-build`);
|
||||
} else {
|
||||
const pkgs = listMembershipPackages();
|
||||
console.log(` membership packages: ${pkgs.length}`);
|
||||
auditRuntimeRoot();
|
||||
auditNoDoubleHome();
|
||||
auditNoUngeneratedMonorepoPointers();
|
||||
auditConnectChrome();
|
||||
auditHostBleedSample();
|
||||
auditSkillifySdkPath();
|
||||
|
||||
if (PHASE === 'ab' || PHASE === 'b' || PHASE === 'full') {
|
||||
auditBridges();
|
||||
}
|
||||
}
|
||||
|
||||
const fails = findings.filter(f => f.severity === 'fail');
|
||||
const warns = findings.filter(f => f.severity === 'warn');
|
||||
const oks = findings.filter(f => f.severity === 'ok');
|
||||
|
||||
for (const f of findings) {
|
||||
const tag = f.severity === 'fail' ? 'FAIL' : f.severity === 'warn' ? 'WARN' : 'OK ';
|
||||
console.log(`[${tag}] ${f.area}: ${f.message}`);
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(`Summary: ${oks.length} ok, ${warns.length} warn, ${fails.length} fail`);
|
||||
if (fails.length > 0) {
|
||||
console.log('VERDICT: INCOMPATIBLE (packaging/bridge gate failed)');
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('VERDICT: packaging COMPATIBLE' + (PHASE.includes('b') || PHASE === 'full' ? ' (+ bridge gates honest)' : ''));
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
main();
|
||||
|
|
@ -7,7 +7,7 @@
|
|||
* gstack-model-benchmark <skill-or-prompt-file> [options]
|
||||
*
|
||||
* Options:
|
||||
* --models claude,gpt,gemini Comma-separated provider list (default: claude)
|
||||
* --models claude,gpt,gemini,grok Comma-separated provider list (default: claude)
|
||||
* --prompt "<text>" Inline prompt instead of a file
|
||||
* --workdir <path> Working dir passed to each CLI (default: cwd)
|
||||
* --timeout-ms <n> Per-provider timeout (default: 300000)
|
||||
|
|
@ -15,27 +15,29 @@
|
|||
* --skip-unavailable Skip providers that fail available() check
|
||||
* (default: include them with unavailable marker)
|
||||
* --judge Run Anthropic SDK judge on outputs for quality score
|
||||
* (requires ANTHROPIC_API_KEY; adds ~$0.05 per call)
|
||||
* (requires ANTHROPIC_API_KEY; Grok-only skips judge)
|
||||
* --dry-run Validate flags + resolve auth, don't invoke providers
|
||||
*
|
||||
* Examples:
|
||||
* gstack-model-benchmark --prompt "Write a haiku about databases" --models claude,gpt
|
||||
* gstack-model-benchmark ./test-prompt.txt --models claude,gpt,gemini --judge
|
||||
* gstack-model-benchmark --prompt "hi" --models claude,gpt,gemini --dry-run
|
||||
* gstack-model-benchmark --prompt "hi" --models grok --dry-run
|
||||
*/
|
||||
|
||||
import '../lib/conductor-env-shim';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { runBenchmark, formatTable, formatJson, formatMarkdown, type BenchmarkInput } from '../test/helpers/benchmark-runner';
|
||||
import { runBenchmark, formatTable, formatJson, formatMarkdown, type BenchmarkInput, type ProviderName } from '../test/helpers/benchmark-runner';
|
||||
import { ClaudeAdapter } from '../test/helpers/providers/claude';
|
||||
import { GptAdapter } from '../test/helpers/providers/gpt';
|
||||
import { GeminiAdapter } from '../test/helpers/providers/gemini';
|
||||
import { GrokAdapter } from '../test/helpers/providers/grok';
|
||||
|
||||
const ADAPTER_FACTORIES = {
|
||||
const ADAPTER_FACTORIES: Record<ProviderName, () => { name: string; available: () => Promise<{ ok: boolean; reason?: string }> }> = {
|
||||
claude: () => new ClaudeAdapter(),
|
||||
gpt: () => new GptAdapter(),
|
||||
gemini: () => new GeminiAdapter(),
|
||||
grok: () => new GrokAdapter(),
|
||||
};
|
||||
|
||||
type OutputFormat = 'table' | 'json' | 'markdown';
|
||||
|
|
@ -76,13 +78,13 @@ function positionalArgs(args: string[]): string[] {
|
|||
return positional;
|
||||
}
|
||||
|
||||
function parseProviders(s: string | undefined): Array<'claude' | 'gpt' | 'gemini'> {
|
||||
function parseProviders(s: string | undefined): ProviderName[] {
|
||||
if (!s) return ['claude'];
|
||||
const seen = new Set<'claude' | 'gpt' | 'gemini'>();
|
||||
const seen = new Set<ProviderName>();
|
||||
for (const p of s.split(',').map(x => x.trim()).filter(Boolean)) {
|
||||
if (p === 'claude' || p === 'gpt' || p === 'gemini') seen.add(p);
|
||||
if (p === 'claude' || p === 'gpt' || p === 'gemini' || p === 'grok') seen.add(p);
|
||||
else {
|
||||
console.error(`WARN: unknown provider '${p}' — skipping. Valid: claude, gpt, gemini.`);
|
||||
console.error(`WARN: unknown provider '${p}' — skipping. Valid: claude, gpt, gemini, grok.`);
|
||||
}
|
||||
}
|
||||
return seen.size ? Array.from(seen) : ['claude'];
|
||||
|
|
@ -129,11 +131,16 @@ async function main(): Promise<void> {
|
|||
const report = await runBenchmark(input);
|
||||
|
||||
if (doJudge) {
|
||||
try {
|
||||
const { judgeEntries } = await import('../test/helpers/benchmark-judge');
|
||||
await judgeEntries(report);
|
||||
} catch (err) {
|
||||
console.error(`WARN: judge unavailable: ${(err as Error).message}`);
|
||||
// Grok-or-skip: judge requires Anthropic; Grok-only machines skip cleanly
|
||||
if (!process.env.ANTHROPIC_API_KEY) {
|
||||
console.error('WARN: judge skipped — ANTHROPIC_API_KEY not set (Grok-or-skip judge).');
|
||||
} else {
|
||||
try {
|
||||
const { judgeEntries } = await import('../test/helpers/benchmark-judge');
|
||||
await judgeEntries(report);
|
||||
} catch (err) {
|
||||
console.error(`WARN: judge unavailable: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -149,7 +156,7 @@ async function main(): Promise<void> {
|
|||
|
||||
async function dryRunReport(opts: {
|
||||
prompt: string;
|
||||
providers: Array<'claude' | 'gpt' | 'gemini'>;
|
||||
providers: ProviderName[];
|
||||
workdir: string;
|
||||
timeoutMs: number;
|
||||
output: OutputFormat;
|
||||
|
|
|
|||
|
|
@ -30,6 +30,11 @@ modify files.
|
|||
|
||||
The generated external invocation name is `gstack-claude`.
|
||||
|
||||
**MULTI_CLI_BRIDGE:** Primary modes always call the Claude Code CLI. Packaging and
|
||||
Step 0 detect/install are first-class on this host; this skill does **not** work
|
||||
without Claude installed and authenticated. Residual foreign-CLI dependency is
|
||||
expected and labeled — never claim "works without Claude."
|
||||
|
||||
---
|
||||
|
||||
## Step 0: Check Claude CLI
|
||||
|
|
|
|||
|
|
@ -0,0 +1,113 @@
|
|||
import type { HostConfig } from '../scripts/host-config';
|
||||
|
||||
/**
|
||||
* Grok Build (xAI) host — based on community PR #2028 (adamouabakar),
|
||||
* aligned with cursor/factory host patterns and Grok skill discovery.
|
||||
*
|
||||
* Skills generate into <repo>/.grok/skills/gstack-* and install under
|
||||
* ~/.grok/skills/ (flat skill packages + runtime root at ~/.grok/skills/gstack).
|
||||
*
|
||||
* Grok discovers user skills from ~/.grok/skills/<name>/SKILL.md.
|
||||
* Frontmatter `name:` stays unprefixed (browse, ship, …) so slash commands
|
||||
* remain /browse, /ship, etc. Directory names use gstack- prefix (external hosts).
|
||||
*/
|
||||
const grokBuild: HostConfig = {
|
||||
name: 'grok-build',
|
||||
displayName: 'Grok Build',
|
||||
cliCommand: 'grok',
|
||||
cliAliases: ['grok-build'],
|
||||
|
||||
// Relative to $HOME for global install path docs / preamble
|
||||
globalRoot: '.grok/skills/gstack',
|
||||
// Project-local runtime root (optional team mode)
|
||||
localSkillRoot: '.grok/skills/gstack',
|
||||
// Gitignored generated skill docs live here
|
||||
hostSubdir: '.grok',
|
||||
usesEnvVars: true,
|
||||
|
||||
frontmatter: {
|
||||
mode: 'allowlist',
|
||||
// Keep triggers so Grok model-invocation routing still works
|
||||
keepFields: ['name', 'description', 'triggers', 'allowed-tools'],
|
||||
descriptionLimit: null,
|
||||
},
|
||||
|
||||
generation: {
|
||||
generateMetadata: false,
|
||||
// Upstream convention: /codex skill is a Claude↔Codex bridge; all external
|
||||
// hosts skip it (host-config.test.ts). Grok users already have the openai-codex
|
||||
// plugin; keep /claude as an optional outside-voice skill.
|
||||
skipSkills: ['codex'],
|
||||
},
|
||||
|
||||
pathRewrites: [
|
||||
{ from: '~/.claude/skills/gstack', to: '$GSTACK_ROOT' },
|
||||
{ from: '.claude/skills/gstack', to: '.grok/skills/gstack' },
|
||||
{ from: '.claude/skills/review', to: '.grok/skills/gstack/review' },
|
||||
{ from: '.claude/skills', to: '.grok/skills' },
|
||||
{ from: '~/.claude/skills', to: '~/.grok/skills' },
|
||||
{ from: 'CLAUDE.md', to: 'AGENTS.md' },
|
||||
// Defense-in-depth if a Claude model overlay ever slips through (U3 / KTD6)
|
||||
{ from: 'MODEL_OVERLAY: claude', to: 'MODEL_OVERLAY: none' },
|
||||
{ from: 'use the Skill tool', to: 'invoke the skill via slash command or skill load' },
|
||||
],
|
||||
|
||||
toolRewrites: {
|
||||
'use the Bash tool': 'run this command in the shell',
|
||||
'use the Write tool': 'create this file',
|
||||
'use the Read tool': 'read the file',
|
||||
'use the Edit tool': 'edit the file',
|
||||
'use the Agent tool': 'dispatch a subagent',
|
||||
'use the Grep tool': 'search for',
|
||||
'use the Glob tool': 'find files matching',
|
||||
'use the Skill tool': 'invoke the skill via slash command or skill load',
|
||||
AskUserQuestion: 'ask_user_question',
|
||||
ExitPlanMode: 'exit_plan_mode',
|
||||
},
|
||||
|
||||
// Suppress Claude-only outside-voice orchestration that assumes Claude can
|
||||
// spawn Codex as itself. Keep plan/review skills; they still run on Grok.
|
||||
suppressedResolvers: [
|
||||
'GBRAIN_CONTEXT_LOAD',
|
||||
'GBRAIN_SAVE_RESULTS',
|
||||
],
|
||||
|
||||
// Thin runtime root: every asset skills resolve via $GSTACK_ROOT (R1 / U1).
|
||||
// Dual-write with setup create_grok_runtime_root — keep lists in sync.
|
||||
runtimeRoot: {
|
||||
globalSymlinks: [
|
||||
'bin',
|
||||
'browse/dist',
|
||||
'browse/bin',
|
||||
'browse/src',
|
||||
'design/dist',
|
||||
'make-pdf/dist',
|
||||
'extension',
|
||||
'scripts',
|
||||
'review/specialists',
|
||||
'gstack-upgrade',
|
||||
'ETHOS.md',
|
||||
],
|
||||
globalFiles: {
|
||||
review: [
|
||||
'checklist.md',
|
||||
'TODOS-format.md',
|
||||
'design-checklist.md',
|
||||
'greptile-triage.md',
|
||||
],
|
||||
},
|
||||
},
|
||||
|
||||
install: {
|
||||
prefixable: false,
|
||||
linkingStrategy: 'symlink-generated',
|
||||
},
|
||||
|
||||
learningsMode: 'basic',
|
||||
boundaryInstruction:
|
||||
'IMPORTANT: Prefer ~/.grok/skills/gstack and $GSTACK_ROOT over ~/.claude/skills/gstack. ' +
|
||||
'Do not assume Claude Code tools (TodoWrite, Skill tool, Claude-in-Chrome MCP). ' +
|
||||
'Use Grok shell/read/edit/web tools and ask_user_question. Prefer /browse over browser MCPs.',
|
||||
};
|
||||
|
||||
export default grokBuild;
|
||||
|
|
@ -16,9 +16,10 @@ import cursor from './cursor';
|
|||
import openclaw from './openclaw';
|
||||
import hermes from './hermes';
|
||||
import gbrain from './gbrain';
|
||||
import grokBuild from './grok-build';
|
||||
|
||||
/** All registered host configs. Add new hosts here. */
|
||||
export const ALL_HOST_CONFIGS: HostConfig[] = [claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain];
|
||||
export const ALL_HOST_CONFIGS: HostConfig[] = [claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain, grokBuild];
|
||||
|
||||
/** Map from host name to config. */
|
||||
export const HOST_CONFIG_MAP: Record<string, HostConfig> = Object.fromEntries(
|
||||
|
|
@ -65,4 +66,4 @@ export function getExternalHosts(): HostConfig[] {
|
|||
}
|
||||
|
||||
// Re-export individual configs for direct import
|
||||
export { claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain };
|
||||
export { claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain, grokBuild };
|
||||
|
|
|
|||
|
|
@ -994,6 +994,27 @@ browser to the internet securely).
|
|||
|
||||
STOP here. Wait for the user to install ngrok and re-invoke.
|
||||
|
||||
**Bridge status:** Remote pair is **DEPENDENT** on ngrok until the tunnel binary
|
||||
is installed and authenticated. Local same-machine pair remains zero-extra-dep
|
||||
and must never hard-require ngrok.
|
||||
|
||||
### Remote path security + teardown checklist
|
||||
|
||||
Remote pairing expands attack surface vs local pair. Always:
|
||||
|
||||
1. **Authenticated tunnel only** — never run an unauthenticated public expose.
|
||||
2. **Bind local pair endpoint to 127.0.0.1** — do not advertise LAN bind.
|
||||
3. **Authtoken out-of-band** — user pastes token into `ngrok config add-authtoken`;
|
||||
never echo the token or tunnel URL into session transcripts beyond what the
|
||||
CLI already printed for the user to copy.
|
||||
4. **Teardown after pair** — when pairing ends, stop the ngrok tunnel process
|
||||
(or tell the user: `pkill -f 'ngrok http'` / dashboard stop). Do not leave
|
||||
a long-lived public tunnel running.
|
||||
5. **Verify:** `ngrok config check` after auth; `which ngrok` for install.
|
||||
|
||||
If ngrok is missing, report bridge DEPENDENT (install path present) — do **not**
|
||||
mark the remote primary path READY/COMPATIBLE from install prose alone.
|
||||
|
||||
## Step 5: Verify connection
|
||||
|
||||
After the user pastes the instructions into the other agent, wait a moment then check:
|
||||
|
|
|
|||
|
|
@ -189,6 +189,27 @@ browser to the internet securely).
|
|||
|
||||
STOP here. Wait for the user to install ngrok and re-invoke.
|
||||
|
||||
**Bridge status:** Remote pair is **DEPENDENT** on ngrok until the tunnel binary
|
||||
is installed and authenticated. Local same-machine pair remains zero-extra-dep
|
||||
and must never hard-require ngrok.
|
||||
|
||||
### Remote path security + teardown checklist
|
||||
|
||||
Remote pairing expands attack surface vs local pair. Always:
|
||||
|
||||
1. **Authenticated tunnel only** — never run an unauthenticated public expose.
|
||||
2. **Bind local pair endpoint to 127.0.0.1** — do not advertise LAN bind.
|
||||
3. **Authtoken out-of-band** — user pastes token into `ngrok config add-authtoken`;
|
||||
never echo the token or tunnel URL into session transcripts beyond what the
|
||||
CLI already printed for the user to copy.
|
||||
4. **Teardown after pair** — when pairing ends, stop the ngrok tunnel process
|
||||
(or tell the user: `pkill -f 'ngrok http'` / dashboard stop). Do not leave
|
||||
a long-lived public tunnel running.
|
||||
5. **Verify:** `ngrok config check` after auth; `which ngrok` for install.
|
||||
|
||||
If ngrok is missing, report bridge DEPENDENT (install path present) — do **not**
|
||||
mark the remote primary path READY/COMPATIBLE from install prose alone.
|
||||
|
||||
## Step 5: Verify connection
|
||||
|
||||
After the user pastes the instructions into the other agent, wait a moment then check:
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ import * as path from 'path';
|
|||
import type { Host, TemplateContext } from './resolvers/types';
|
||||
import { HOST_PATHS, unwrapResolver } from './resolvers/types';
|
||||
import { RESOLVERS } from './resolvers/index';
|
||||
import { hostUsesSectionPointers } from './resolvers/sections';
|
||||
import { externalSkillName, extractHookSafetyProse as _extractHookSafetyProse, extractNameAndDescription as _extractNameAndDescription, condenseOpenAIShortDescription as _condenseOpenAIShortDescription, generateOpenAIYaml as _generateOpenAIYaml } from './resolvers/codex-helpers';
|
||||
import { generatePlanCompletionAuditShip, generatePlanCompletionAuditReview, generatePlanVerificationExec } from './resolvers/review';
|
||||
import { ALL_HOST_CONFIGS, ALL_HOST_NAMES, resolveHostArg, getHostConfig } from '../hosts/index';
|
||||
|
|
@ -91,9 +92,11 @@ let HOST: Host = HOST_ARG_VAL === 'all' ? 'claude' : HOST_ARG_VAL;
|
|||
|
||||
// ─── Model Overlay Selection ────────────────────────────────
|
||||
// --model is explicit. We do NOT auto-detect from host (host ≠ model).
|
||||
// Default is 'claude'. Missing overlay file → empty string (graceful).
|
||||
// Default is 'claude' for most hosts. Grok Build suppresses the Claude
|
||||
// model overlay body unless --model is passed explicitly (U3 / KTD6).
|
||||
import { ALL_MODEL_NAMES, resolveModel, type Model } from './models';
|
||||
const MODEL_ARG = process.argv.find(a => a.startsWith('--model'));
|
||||
const MODEL_EXPLICIT = !!MODEL_ARG;
|
||||
const MODEL_ARG_VAL: Model = (() => {
|
||||
if (!MODEL_ARG) return 'claude';
|
||||
const val = MODEL_ARG.includes('=') ? MODEL_ARG.split('=')[1] : process.argv[process.argv.indexOf(MODEL_ARG) + 1];
|
||||
|
|
@ -729,9 +732,12 @@ function buildContext(
|
|||
const preambleTier = tierMatch ? parseInt(tierMatch[1], 10) : undefined;
|
||||
const interactiveMatch = tmplContent.match(/^interactive:\s*(true|false)\s*$/m);
|
||||
const interactive = interactiveMatch ? interactiveMatch[1] === 'true' : undefined;
|
||||
// Grok-native packaging: no Claude MODEL_OVERLAY section unless user forced --model
|
||||
const modelForHost: Model | undefined =
|
||||
host === 'grok-build' && !MODEL_EXPLICIT ? undefined : MODEL_ARG_VAL;
|
||||
return {
|
||||
skillName, tmplPath, benefitsFrom, host, paths: HOST_PATHS[host],
|
||||
preambleTier, model: MODEL_ARG_VAL, interactive, explainLevel: EXPLAIN_LEVEL,
|
||||
preambleTier, model: modelForHost, interactive, explainLevel: EXPLAIN_LEVEL,
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -1019,14 +1025,15 @@ for (const currentHost of hostsToRun) {
|
|||
}
|
||||
}
|
||||
|
||||
// ─── Section generation (v2 plan T9, Claude-first carve) ───
|
||||
// On-demand sections/*.md for carved skills. Generated for CLAUDE ONLY:
|
||||
// every other host inlines section content via the {{SECTION:id}} resolver
|
||||
// (keeping the full monolith skill), so they need no section files and we
|
||||
// sidestep host-portable section paths until that plumbing lands. No-op for
|
||||
// any skill without a sections/ dir. Mirrors the SKILL.md DRY_RUN handling so
|
||||
// sections participate in the freshness gate.
|
||||
for (const sec of currentHost === 'claude' ? discoverSectionTemplates(ROOT) : []) {
|
||||
// ─── Section generation (v2 plan T9 carve) ───
|
||||
// On-demand sections/*.md for carved skills.
|
||||
// Pointer hosts (Claude, Grok Build): emit section files next to the skill
|
||||
// package; {{SECTION:id}} is a STOP-Read pointer (keeps the skeleton small).
|
||||
// Inline hosts (Codex, Factory, …): {{SECTION:id}} inlines content into the
|
||||
// monolith SKILL.md — no separate section files.
|
||||
// No-op for any skill without a sections/ dir. Mirrors the SKILL.md DRY_RUN
|
||||
// handling so sections participate in the freshness gate.
|
||||
for (const sec of hostUsesSectionPointers(currentHost) ? discoverSectionTemplates(ROOT) : []) {
|
||||
if (currentHostConfig.generation.includeSkills?.length &&
|
||||
!currentHostConfig.generation.includeSkills.includes(sec.skillDir)) continue;
|
||||
if (currentHostConfig.generation.skipSkills?.length &&
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
},
|
||||
"benchmark-models": {
|
||||
"lead": "Cross-model benchmark for gstack skills.",
|
||||
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,\nand optionally quality via LLM judge. Answers \"which model is actually best\nfor this skill?\" with data instead of vibes. Separate from /benchmark, which\nmeasures web page performance. Use when: \"benchmark models\", \"compare models\",\n\"which model is best for X\", \"cross-model comparison\", \"model shootout\".",
|
||||
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,\ncost, and optionally quality via LLM judge. Answers \"which model is actually\nbest for this skill?\" with data instead of vibes. Separate from /benchmark,\nwhich measures web page performance. Use when: \"benchmark models\", \"compare\nmodels\", \"which model is best for X\", \"cross-model comparison\", \"model\nshootout\".",
|
||||
"voice_line": "Voice triggers (speech-to-text aliases): \"compare models\", \"model shootout\", \"which model is best\"."
|
||||
},
|
||||
"browse": {
|
||||
|
|
|
|||
|
|
@ -99,14 +99,24 @@ export function generateSnapshotFlags(_ctx: TemplateContext): string {
|
|||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/** Resolve dist binary path: env-var hosts use $GSTACK_* (never $HOME+$GSTACK_*). */
|
||||
export function resolveDistBinary(dir: string, binary: string): string {
|
||||
if (dir.startsWith('$')) {
|
||||
// e.g. $GSTACK_BROWSE already points at .../browse/dist
|
||||
return `${dir}/${binary}`;
|
||||
}
|
||||
return `$HOME${dir.replace(/^~/, '')}/${binary}`;
|
||||
}
|
||||
|
||||
export function generateBrowseSetup(ctx: TemplateContext): string {
|
||||
const globalBrowse = resolveDistBinary(ctx.paths.browseDir, 'browse');
|
||||
return `## SETUP (run this check BEFORE any browse command)
|
||||
|
||||
\`\`\`bash
|
||||
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
|
||||
B=""
|
||||
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse" ] && B="$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse"
|
||||
[ -z "$B" ] && B="$HOME${ctx.paths.browseDir.replace(/^~/, '')}/browse"
|
||||
[ -z "$B" ] && B="${globalBrowse}"
|
||||
if [ -x "$B" ]; then
|
||||
echo "READY: $B"
|
||||
else
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import type { TemplateContext } from './types';
|
||||
import { AI_SLOP_BLACKLIST, OPENAI_HARD_REJECTIONS, OPENAI_LITMUS_CHECKS } from './constants';
|
||||
import { resolveDistBinary } from './browse';
|
||||
|
||||
export function generateDesignReviewLite(ctx: TemplateContext): string {
|
||||
const litmusList = OPENAI_LITMUS_CHECKS.map((item, i) => `${i + 1}. ${item}`).join(' ');
|
||||
|
|
@ -786,13 +787,15 @@ Source: [OpenAI "Designing Delightful Frontends with GPT-5.4"](https://developer
|
|||
}
|
||||
|
||||
export function generateDesignSetup(ctx: TemplateContext): string {
|
||||
const globalDesign = resolveDistBinary(ctx.paths.designDir, 'design');
|
||||
const globalBrowse = resolveDistBinary(ctx.paths.browseDir, 'browse');
|
||||
return `## DESIGN SETUP (run this check BEFORE any design mockup command)
|
||||
|
||||
\`\`\`bash
|
||||
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
|
||||
D=""
|
||||
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design" ] && D="$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design"
|
||||
[ -z "$D" ] && D="$HOME${ctx.paths.designDir.replace(/^~/, '')}/design"
|
||||
[ -z "$D" ] && D="${globalDesign}"
|
||||
if [ -x "$D" ]; then
|
||||
echo "DESIGN_READY: $D"
|
||||
else
|
||||
|
|
@ -800,7 +803,7 @@ else
|
|||
fi
|
||||
B=""
|
||||
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse" ] && B="$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse"
|
||||
[ -z "$B" ] && B="$HOME${ctx.paths.browseDir.replace(/^~/, '')}/browse"
|
||||
[ -z "$B" ] && B="${globalBrowse}"
|
||||
if [ -x "$B" ]; then
|
||||
echo "BROWSE_READY: $B"
|
||||
else
|
||||
|
|
@ -831,13 +834,14 @@ data, not project files. They persist across branches, conversations, and worksp
|
|||
}
|
||||
|
||||
export function generateDesignMockup(ctx: TemplateContext): string {
|
||||
const globalDesign = resolveDistBinary(ctx.paths.designDir, 'design');
|
||||
return `## Visual Design Exploration
|
||||
|
||||
\`\`\`bash
|
||||
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
|
||||
D=""
|
||||
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design" ] && D="$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design"
|
||||
[ -z "$D" ] && D="$HOME${ctx.paths.designDir.replace(/^~/, '')}/design"
|
||||
[ -z "$D" ] && D="${globalDesign}"
|
||||
[ -x "$D" ] && echo "DESIGN_READY" || echo "DESIGN_NOT_AVAILABLE"
|
||||
\`\`\`
|
||||
|
||||
|
|
|
|||
|
|
@ -36,6 +36,7 @@ import { generateMakePdfSetup } from './make-pdf';
|
|||
import { generateTasksSectionEmit, generateTasksSectionAggregate } from './tasks-section';
|
||||
import { SECTION, SECTION_INDEX } from './sections';
|
||||
import { generateRedactTaxonomyTable, generateRedactInvocationBlock } from './redact-doc';
|
||||
import { generateSpecSpawn, generateSpecExecuteFlag } from './spec-spawn';
|
||||
|
||||
export const RESOLVERS: Record<string, ResolverValue> = {
|
||||
SLUG_EVAL: generateSlugEval,
|
||||
|
|
@ -100,6 +101,8 @@ export const RESOLVERS: Record<string, ResolverValue> = {
|
|||
MAKE_PDF_SETUP: generateMakePdfSetup,
|
||||
TASKS_SECTION_EMIT: generateTasksSectionEmit,
|
||||
TASKS_SECTION_AGGREGATE: generateTasksSectionAggregate,
|
||||
SPEC_SPAWN: generateSpecSpawn,
|
||||
SPEC_EXECUTE_FLAG: generateSpecExecuteFlag,
|
||||
SECTION,
|
||||
SECTION_INDEX,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import type { TemplateContext } from './types';
|
||||
import { resolveDistBinary } from './browse';
|
||||
|
||||
/**
|
||||
* {{MAKE_PDF_SETUP}} — emits the shell preamble that resolves $P to the
|
||||
|
|
@ -8,10 +9,11 @@ import type { TemplateContext } from './types';
|
|||
*
|
||||
* Resolution order (matches src/browseClient.ts::resolveBrowseBin):
|
||||
* 1. Local skill root: $_ROOT/{localSkillRoot}/make-pdf/dist/pdf
|
||||
* 2. Global: ~/{globalRoot}/make-pdf/dist/pdf
|
||||
* 2. Global: $GSTACK_MAKE_PDF/pdf or ~/{globalRoot}/make-pdf/dist/pdf
|
||||
* 3. Env override (MAKE_PDF_BIN) — for contributor dev builds
|
||||
*/
|
||||
export function generateMakePdfSetup(ctx: TemplateContext): string {
|
||||
const globalPdf = resolveDistBinary(ctx.paths.makePdfDir, 'pdf');
|
||||
return `## MAKE-PDF SETUP (run this check BEFORE any make-pdf command)
|
||||
|
||||
\`\`\`bash
|
||||
|
|
@ -19,7 +21,7 @@ _ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
|
|||
P=""
|
||||
[ -n "$MAKE_PDF_BIN" ] && [ -x "$MAKE_PDF_BIN" ] && P="$MAKE_PDF_BIN"
|
||||
[ -z "$P" ] && [ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/make-pdf/dist/pdf" ] && P="$_ROOT/${ctx.paths.localSkillRoot}/make-pdf/dist/pdf"
|
||||
[ -z "$P" ] && P="$HOME${ctx.paths.makePdfDir.replace(/^~/, '')}/pdf"
|
||||
[ -z "$P" ] && P="${globalPdf}"
|
||||
if [ -x "$P" ]; then
|
||||
echo "MAKE_PDF_READY: $P"
|
||||
alias _p_="$P" # shellcheck alias helper (not exported)
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ GSTACK_ROOT="$HOME/${hostConfig.globalRoot}"
|
|||
GSTACK_BIN="$GSTACK_ROOT/bin"
|
||||
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
|
||||
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
|
||||
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
|
||||
`
|
||||
: '';
|
||||
|
||||
|
|
|
|||
|
|
@ -1,25 +1,28 @@
|
|||
/**
|
||||
* Section resolvers (v2 plan T9, Claude-first carve).
|
||||
* Section resolvers (v2 plan T9 carve).
|
||||
*
|
||||
* A carved skill keeps its prose-heavy steps in `<skill>/sections/<id>.md`, read
|
||||
* on demand. The SAME template ships to every host, so these resolvers make the
|
||||
* carve host-aware:
|
||||
*
|
||||
* - On CLAUDE: {{SECTION:id}} emits a STOP-Read pointer to the generated section
|
||||
* file (the skeleton), and the section .md is generated + installed separately.
|
||||
* - On every OTHER host: {{SECTION:id}} INLINES the section template's content,
|
||||
* so external hosts keep the full monolith ship skill (no section files, no
|
||||
* host-portable-path problem). Inlined content keeps its own {{RESOLVER}}
|
||||
* tokens, which the generator's multi-pass resolve expands.
|
||||
* - On CLAUDE: SECTION:id emits a STOP-Read pointer to the generated section
|
||||
* file under the nested monorepo install ({skillRoot}/{skill}/sections/).
|
||||
* - On GROK-BUILD: same pointer mode, but paths use the flat Grok package layout
|
||||
* (~/.grok/skills/gstack-{skill}/sections/). Section files are generated into
|
||||
* each package's sections/ dir and ride along with package install.
|
||||
* - On every OTHER host: SECTION placeholders INLINE the section template content,
|
||||
* so those hosts keep the full monolith skill (no section files, no
|
||||
* host-portable-path problem). Inlined content keeps its own resolver tokens,
|
||||
* which the generator's multi-pass resolve expands.
|
||||
*
|
||||
* {{SECTION_INDEX:skill}} renders the situation→section table from the PASSIVE
|
||||
* manifest on Claude (empty on other hosts — they have no sections). The manifest
|
||||
* SECTION_INDEX renders the situation-to-section table from the PASSIVE
|
||||
* manifest on pointer hosts (empty when sections are inlined). The manifest
|
||||
* is the single source of id/file/title/trigger text (CM2; v2_PLAN.md:663).
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import type { ResolverFn, TemplateContext } from './types';
|
||||
import type { Host, ResolverFn, TemplateContext } from './types';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..', '..');
|
||||
|
||||
|
|
@ -34,6 +37,21 @@ interface SectionManifest {
|
|||
sections: SectionEntry[];
|
||||
}
|
||||
|
||||
/** Hosts that load carved sections on demand (not monolith-inline). */
|
||||
export function hostUsesSectionPointers(host: Host | string): boolean {
|
||||
return host === 'claude' || host === 'grok-build';
|
||||
}
|
||||
|
||||
/**
|
||||
* External package dir name for flat hosts (gstack-ship, gstack-upgrade, …).
|
||||
* Mirrors gen-skill-docs externalSkillName for skill dirs.
|
||||
*/
|
||||
export function externalSkillPackageName(skillName: string): string {
|
||||
if (skillName === '.' || skillName === '' || skillName === 'gstack') return 'gstack';
|
||||
if (skillName.startsWith('gstack-')) return skillName;
|
||||
return `gstack-${skillName}`;
|
||||
}
|
||||
|
||||
function loadManifest(skill: string): SectionManifest {
|
||||
const p = path.join(ROOT, skill, 'sections', 'manifest.json');
|
||||
const raw = fs.readFileSync(p, 'utf-8');
|
||||
|
|
@ -49,35 +67,56 @@ function findSection(skill: string, id: string): SectionEntry {
|
|||
}
|
||||
|
||||
/**
|
||||
* {{SECTION:id}} — pointer on Claude, inline on other hosts.
|
||||
* Claude path uses the stable gstack-root install (`{skillRoot}/{skill}/sections/`),
|
||||
* which always exists, instead of a naked relative path (Codex outside-voice #7).
|
||||
* Absolute-style path the agent should Read for a section file.
|
||||
* Claude: nested monorepo install under skillRoot.
|
||||
* Grok: flat package next to the thin runtime root.
|
||||
*/
|
||||
export function sectionPointerPath(
|
||||
host: Host | string,
|
||||
skillName: string,
|
||||
sectionFile: string,
|
||||
skillRoot: string,
|
||||
): string {
|
||||
if (host === 'grok-build') {
|
||||
const pkg = externalSkillPackageName(skillName);
|
||||
return `~/.grok/skills/${pkg}/sections/${sectionFile}`;
|
||||
}
|
||||
// Claude (and any future nested-install pointer host)
|
||||
return `${skillRoot}/${skillName}/sections/${sectionFile}`;
|
||||
}
|
||||
|
||||
function stopReadDirective(sectionPath: string, trigger: string): string {
|
||||
return [
|
||||
`> **STOP.** Before ${trigger}, Read \`${sectionPath}\` and execute it`,
|
||||
`> in full. Do not work from memory — that section is the source of truth for this step.`,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* SECTION:id — pointer on Claude/Grok, inline on other hosts.
|
||||
*/
|
||||
export const SECTION: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
|
||||
const id = args?.[0];
|
||||
if (!id) throw new Error('{{SECTION:id}} requires a section id');
|
||||
const entry = findSection(ctx.skillName, id);
|
||||
|
||||
if (ctx.host === 'claude') {
|
||||
const sectionPath = `${ctx.paths.skillRoot}/${ctx.skillName}/sections/${entry.file}`;
|
||||
return [
|
||||
`> **STOP.** Before ${entry.trigger}, Read \`${sectionPath}\` and execute it`,
|
||||
`> in full. Do not work from memory — that section is the source of truth for this step.`,
|
||||
].join('\n');
|
||||
if (hostUsesSectionPointers(ctx.host)) {
|
||||
const sectionPath = sectionPointerPath(ctx.host, ctx.skillName, entry.file, ctx.paths.skillRoot);
|
||||
return stopReadDirective(sectionPath, entry.trigger);
|
||||
}
|
||||
|
||||
// Non-Claude hosts inline the section template content (monolith preserved).
|
||||
// Non-pointer hosts inline the section template content (monolith preserved).
|
||||
// Inner {{RESOLVER}} tokens are expanded by the generator's multi-pass resolve.
|
||||
const tmplPath = path.join(ROOT, ctx.skillName, 'sections', `${entry.file}.tmpl`);
|
||||
return fs.readFileSync(tmplPath, 'utf-8').trimEnd();
|
||||
};
|
||||
|
||||
/**
|
||||
* {{SECTION_INDEX:skill}} — situation→section table from the passive manifest.
|
||||
* Claude only; other hosts inline everything so an index would be noise.
|
||||
* SECTION_INDEX — situation-to-section table from the passive manifest.
|
||||
* Pointer hosts only; inline hosts have no separate section files.
|
||||
*/
|
||||
export const SECTION_INDEX: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
|
||||
if (ctx.host !== 'claude') return '';
|
||||
if (!hostUsesSectionPointers(ctx.host)) return '';
|
||||
const skill = args?.[0] ?? ctx.skillName;
|
||||
const manifest = loadManifest(skill);
|
||||
const lines: string[] = [
|
||||
|
|
@ -90,7 +129,11 @@ export const SECTION_INDEX: ResolverFn = (ctx: TemplateContext, args?: string[])
|
|||
'|------|-------------------|',
|
||||
];
|
||||
for (const s of manifest.sections) {
|
||||
lines.push(`| ${s.trigger} | \`sections/${s.file}\` |`);
|
||||
const sectionPath = sectionPointerPath(ctx.host, skill, s.file, ctx.paths.skillRoot);
|
||||
// Table shows the resolvable path (Grok: full ~/.grok/...; Claude: short sections/)
|
||||
const display =
|
||||
ctx.host === 'grok-build' ? `\`${sectionPath}\`` : `\`sections/${s.file}\``;
|
||||
lines.push(`| ${s.trigger} | ${display} |`);
|
||||
}
|
||||
return lines.join('\n');
|
||||
};
|
||||
|
|
|
|||
|
|
@ -0,0 +1,118 @@
|
|||
import type { TemplateContext } from './types';
|
||||
|
||||
/**
|
||||
* Host-native agent spawn for /spec --execute (U4 / R3).
|
||||
*
|
||||
* Grok research matrix (local CLI grok 0.2.x):
|
||||
* -p / --single <PROMPT> short single-turn
|
||||
* --prompt-file <PATH> single-turn from file (preferred for archives)
|
||||
* --cwd <CWD> working directory
|
||||
* --always-approve elevated auto-approve (opt-in / documented)
|
||||
* Never: $(cat …) into argv (ARG_MAX); never invent --permission-mode acceptEdits.
|
||||
*
|
||||
* Claude host generation unchanged: stdin pipe into claude -p.
|
||||
*/
|
||||
export function generateSpecSpawn(ctx: TemplateContext): string {
|
||||
if (ctx.host === 'grok-build') {
|
||||
return `If A and worktree created: spawn **Grok** headless with the archived
|
||||
spec as a prompt file (never \`$(cat …)\` into argv — ARG_MAX / quoting risk).
|
||||
|
||||
**Auth gate (fail closed):** before spawn, verify Grok is available and configured:
|
||||
|
||||
\`\`\`bash
|
||||
command -v grok >/dev/null 2>&1 || { echo "STOP: grok CLI not on PATH. Install Grok Build, or re-run with --no-execute / --file-only."; exit 1; }
|
||||
if [ ! -f "$HOME/.grok/auth.json" ] && [ -z "\${XAI_API_KEY:-}\${GROK_API_KEY:-}" ]; then
|
||||
echo "STOP: Grok not authenticated (no ~/.grok/auth.json and no XAI_API_KEY/GROK_API_KEY). Log in via \`grok\`, or use --no-execute."
|
||||
exit 1
|
||||
fi
|
||||
# ARCHIVE_PATH must stay under SPAWN_PATH or the gstack projects allowlist (fail closed).
|
||||
if [ ! -f "$ARCHIVE_PATH" ]; then
|
||||
echo "STOP: ARCHIVE_PATH missing: $ARCHIVE_PATH"; exit 1
|
||||
fi
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
ARCHIVE_REAL=$(realpath "$ARCHIVE_PATH")
|
||||
else
|
||||
ARCHIVE_REAL=$(cd "$(dirname "$ARCHIVE_PATH")" && pwd -P)/$(basename "$ARCHIVE_PATH")
|
||||
fi
|
||||
SPAWN_REAL=$(cd "$SPAWN_PATH" 2>/dev/null && pwd -P || echo "")
|
||||
if [ -z "$SPAWN_REAL" ]; then
|
||||
echo "STOP: SPAWN_PATH is not a real directory: $SPAWN_PATH"; exit 1
|
||||
fi
|
||||
STATE_PROJECTS="\${GSTACK_STATE_ROOT:-\$HOME/.gstack}/projects"
|
||||
case "$ARCHIVE_REAL" in
|
||||
"$STATE_PROJECTS"/*|"$SPAWN_REAL"/*) ;; # allowlisted
|
||||
*)
|
||||
echo "STOP: ARCHIVE_PATH realpath not under SPAWN_PATH or allowlisted archive dir ($STATE_PROJECTS)."; exit 1
|
||||
;;
|
||||
esac
|
||||
\`\`\`
|
||||
|
||||
**Security:** default spawn does **not** pass \`--always-approve\` (elevated
|
||||
auto-approve). Only spawn after the user confirmed the D16 gate. If the user
|
||||
explicitly opts into unattended tool use, append \`--always-approve\` to the
|
||||
command below — never enable it by default. Spec archives must not contain
|
||||
secrets. Third-party note: archive body is sent to xAI for processing.
|
||||
|
||||
\`\`\`bash
|
||||
# Prefer --prompt-file (researched). Elevated --always-approve is opt-in only.
|
||||
(cd "$SPAWN_PATH" && grok --prompt-file "$ARCHIVE_PATH" --cwd "$SPAWN_PATH" 2>&1) &
|
||||
SPAWN_PID=$!
|
||||
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
|
||||
echo "Follow with: cd $SPAWN_PATH && grok --continue"
|
||||
\`\`\`
|
||||
|
||||
**Optional Claude execute:** if the user asked for \`--execute-claude\` instead of
|
||||
default Grok execute, and \`claude\` is on PATH, you may spawn — only after the
|
||||
same ARCHIVE_PATH allowlist gate as the Grok path (reuse the block above; never
|
||||
pipe an unallowlisted archive):
|
||||
|
||||
\`\`\`bash
|
||||
# Re-run allowlist (same fail-closed rules as Grok path) before cat|claude.
|
||||
if [ ! -f "$ARCHIVE_PATH" ]; then
|
||||
echo "STOP: ARCHIVE_PATH missing: $ARCHIVE_PATH"; exit 1
|
||||
fi
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
ARCHIVE_REAL=$(realpath "$ARCHIVE_PATH")
|
||||
else
|
||||
ARCHIVE_REAL=$(cd "$(dirname "$ARCHIVE_PATH")" && pwd -P)/$(basename "$ARCHIVE_PATH")
|
||||
fi
|
||||
SPAWN_REAL=$(cd "$SPAWN_PATH" 2>/dev/null && pwd -P || echo "")
|
||||
if [ -z "$SPAWN_REAL" ]; then
|
||||
echo "STOP: SPAWN_PATH is not a real directory: $SPAWN_PATH"; exit 1
|
||||
fi
|
||||
STATE_PROJECTS="\${GSTACK_STATE_ROOT:-\$HOME/.gstack}/projects"
|
||||
case "$ARCHIVE_REAL" in
|
||||
"$STATE_PROJECTS"/*|"$SPAWN_REAL"/*) ;; # allowlisted
|
||||
*)
|
||||
echo "STOP: ARCHIVE_PATH realpath not under SPAWN_PATH or allowlisted archive dir ($STATE_PROJECTS)."; exit 1
|
||||
;;
|
||||
esac
|
||||
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
|
||||
\`\`\`
|
||||
|
||||
Do **not** silently fall through to Claude when Grok is missing — STOP instead.
|
||||
|
||||
If no safe multi-line file ingest is available on an older Grok CLI (no
|
||||
\`--prompt-file\`), demote to \`--no-execute\` / file-only and tell the user:
|
||||
"This Grok CLI lacks --prompt-file; filed the issue only. Upgrade Grok Build or
|
||||
use --execute-claude if Claude is installed."`;
|
||||
}
|
||||
|
||||
// Claude + all other hosts: classic claude -p stdin pipe
|
||||
return `If A and worktree created: spawn \`claude -p\` with the spec piped via stdin:
|
||||
|
||||
\`\`\`bash
|
||||
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
|
||||
SPAWN_PID=$!
|
||||
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
|
||||
echo "Follow with: cd $SPAWN_PATH && claude --resume"
|
||||
\`\`\``;
|
||||
}
|
||||
|
||||
/** Flag-table row for --execute (host-aware description). */
|
||||
export function generateSpecExecuteFlag(ctx: TemplateContext): string {
|
||||
if (ctx.host === 'grok-build') {
|
||||
return '| `--execute` | conditional default (see Phase 5) | Spawn `grok --prompt-file` headless in a fresh worktree after filing the issue. |';
|
||||
}
|
||||
return '| `--execute` | conditional default (see Phase 5) | Spawn `claude -p` in a fresh worktree after filing the issue. |';
|
||||
}
|
||||
303
setup
303
setup
|
|
@ -24,6 +24,8 @@ FACTORY_SKILLS="$HOME/.factory/skills"
|
|||
FACTORY_GSTACK="$FACTORY_SKILLS/gstack"
|
||||
OPENCODE_SKILLS="$HOME/.config/opencode/skills"
|
||||
OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"
|
||||
GROK_SKILLS="$HOME/.grok/skills"
|
||||
GROK_GSTACK="$GROK_SKILLS/gstack"
|
||||
|
||||
IS_WINDOWS=0
|
||||
case "$(uname -s)" in
|
||||
|
|
@ -85,7 +87,7 @@ NO_TEAM_MODE=0
|
|||
PLAN_TUNE_HOOKS_MODE="" # "" = resolve from env/config/prompt; "yes"/"no" = explicit
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--host) [ -z "$2" ] && echo "Missing value for --host (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, or auto)" >&2 && exit 1; HOST="$2"; shift 2 ;;
|
||||
--host) [ -z "$2" ] && echo "Missing value for --host (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, grok-build, grok, or auto)" >&2 && exit 1; HOST="$2"; shift 2 ;;
|
||||
--host=*) HOST="${1#--host=}"; shift ;;
|
||||
--local) LOCAL_INSTALL=1; shift ;;
|
||||
--prefix) SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1; shift ;;
|
||||
|
|
@ -100,8 +102,11 @@ while [ $# -gt 0 ]; do
|
|||
esac
|
||||
done
|
||||
|
||||
# Alias: --host grok → grok-build
|
||||
if [ "$HOST" = "grok" ]; then HOST="grok-build"; fi
|
||||
|
||||
case "$HOST" in
|
||||
claude|codex|kiro|factory|opencode|auto) ;;
|
||||
claude|codex|kiro|factory|opencode|grok-build|auto) ;;
|
||||
openclaw)
|
||||
echo ""
|
||||
echo "OpenClaw integration uses a different model — OpenClaw spawns Claude Code"
|
||||
|
|
@ -136,7 +141,7 @@ case "$HOST" in
|
|||
echo "GBrain setup and brain skills ship from the GBrain repo."
|
||||
echo ""
|
||||
exit 0 ;;
|
||||
*) echo "Unknown --host value: $HOST (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, or auto)" >&2; exit 1 ;;
|
||||
*) echo "Unknown --host value: $HOST (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, grok-build, grok, or auto)" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
# ─── Resolve skill prefix preference ─────────────────────────
|
||||
|
|
@ -200,14 +205,16 @@ INSTALL_CODEX=0
|
|||
INSTALL_KIRO=0
|
||||
INSTALL_FACTORY=0
|
||||
INSTALL_OPENCODE=0
|
||||
INSTALL_GROK=0
|
||||
if [ "$HOST" = "auto" ]; then
|
||||
command -v claude >/dev/null 2>&1 && INSTALL_CLAUDE=1
|
||||
command -v codex >/dev/null 2>&1 && INSTALL_CODEX=1
|
||||
command -v kiro-cli >/dev/null 2>&1 && INSTALL_KIRO=1
|
||||
command -v droid >/dev/null 2>&1 && INSTALL_FACTORY=1
|
||||
command -v opencode >/dev/null 2>&1 && INSTALL_OPENCODE=1
|
||||
command -v grok >/dev/null 2>&1 && INSTALL_GROK=1
|
||||
# If none found, default to claude
|
||||
if [ "$INSTALL_CLAUDE" -eq 0 ] && [ "$INSTALL_CODEX" -eq 0 ] && [ "$INSTALL_KIRO" -eq 0 ] && [ "$INSTALL_FACTORY" -eq 0 ] && [ "$INSTALL_OPENCODE" -eq 0 ]; then
|
||||
if [ "$INSTALL_CLAUDE" -eq 0 ] && [ "$INSTALL_CODEX" -eq 0 ] && [ "$INSTALL_KIRO" -eq 0 ] && [ "$INSTALL_FACTORY" -eq 0 ] && [ "$INSTALL_OPENCODE" -eq 0 ] && [ "$INSTALL_GROK" -eq 0 ]; then
|
||||
INSTALL_CLAUDE=1
|
||||
fi
|
||||
elif [ "$HOST" = "claude" ]; then
|
||||
|
|
@ -220,6 +227,8 @@ elif [ "$HOST" = "factory" ]; then
|
|||
INSTALL_FACTORY=1
|
||||
elif [ "$HOST" = "opencode" ]; then
|
||||
INSTALL_OPENCODE=1
|
||||
elif [ "$HOST" = "grok-build" ]; then
|
||||
INSTALL_GROK=1
|
||||
fi
|
||||
|
||||
migrate_direct_codex_install() {
|
||||
|
|
@ -438,16 +447,13 @@ if [ ! -x "$BROWSE_BIN" ]; then
|
|||
exit 1
|
||||
fi
|
||||
|
||||
# 1b. Generate .agents/ Codex skill docs — always regenerate to prevent stale descriptions.
|
||||
# 1b. Generate .agents/ Codex skill docs — only when installing for Codex.
|
||||
# .agents/ is no longer committed — generated at setup time from .tmpl templates.
|
||||
# bun run build already does this, but we need it when NEEDS_BUILD=0 (binary is fresh).
|
||||
# Always regenerate: generation is fast (<2s) and mtime-based staleness checks are fragile
|
||||
# (miss stale files when timestamps match after clone/checkout/upgrade).
|
||||
AGENTS_DIR="$SOURCE_GSTACK_DIR/.agents/skills"
|
||||
NEEDS_AGENTS_GEN=1
|
||||
|
||||
if [ "$NEEDS_AGENTS_GEN" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
|
||||
log "Generating .agents/ skill docs..."
|
||||
# Host-scoped: --host grok-build / claude / factory / etc. must not pay Codex regen cost
|
||||
# or imply a Codex install is in progress.
|
||||
if [ "$INSTALL_CODEX" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
|
||||
log "Generating .agents/ skill docs for Codex..."
|
||||
(
|
||||
cd "$SOURCE_GSTACK_DIR"
|
||||
bun_cmd install --frozen-lockfile 2>/dev/null || bun_cmd install
|
||||
|
|
@ -475,6 +481,16 @@ if [ "$INSTALL_OPENCODE" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
|
|||
)
|
||||
fi
|
||||
|
||||
# 1e. Generate .grok/ Grok Build skill docs
|
||||
if [ "$INSTALL_GROK" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
|
||||
log "Generating .grok/ skill docs for Grok Build..."
|
||||
(
|
||||
cd "$SOURCE_GSTACK_DIR"
|
||||
bun_cmd install --frozen-lockfile 2>/dev/null || bun_cmd install
|
||||
bun_cmd run gen:skill-docs --host grok-build
|
||||
)
|
||||
fi
|
||||
|
||||
# 2. Ensure Playwright's Chromium is available
|
||||
if ! ensure_playwright_browser; then
|
||||
echo "Installing Playwright Chromium..."
|
||||
|
|
@ -972,6 +988,247 @@ link_opencode_skill_dirs() {
|
|||
fi
|
||||
}
|
||||
|
||||
# ─── Grok Build (xAI) runtime root + skill links ───────────────────────────
|
||||
# Grok discovers skills from ~/.grok/skills/<pkg>/SKILL.md.
|
||||
# Generated packages live in <repo>/.grok/skills/gstack-* (path-rewritten).
|
||||
# Runtime assets (bin, browse, design, specialists, …) live at ~/.grok/skills/gstack.
|
||||
#
|
||||
# Security: every required link target must realpath under monorepo install root.
|
||||
# Dual-write with hosts/grok-build.ts runtimeRoot — keep lists in sync (R1 / U1).
|
||||
_grok_link_under_monorepo() {
|
||||
local monorepo_root="$1"
|
||||
local src="$2"
|
||||
local dst="$3"
|
||||
local required="${4:-0}" # 1 = fail closed if missing when expected
|
||||
|
||||
if [ ! -e "$src" ]; then
|
||||
if [ "$required" = "1" ]; then
|
||||
echo "error: required Grok runtime asset missing: $src" >&2
|
||||
echo " monorepo root: $monorepo_root" >&2
|
||||
echo " Re-run from a full gstack checkout after builds complete." >&2
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
local src_real monorepo_real
|
||||
monorepo_real=$(cd "$monorepo_root" 2>/dev/null && pwd -P)
|
||||
# Fail closed: empty monorepo_real makes "$monorepo_real"/* expand to /* and
|
||||
# match every absolute path — never allow that.
|
||||
if [ -z "$monorepo_real" ] || [ ! -d "$monorepo_real" ]; then
|
||||
echo "error: refusing Grok runtime link — monorepo root unresolved: $monorepo_root" >&2
|
||||
return 1
|
||||
fi
|
||||
# Prefer full realpath when available so a final-component symlink cannot escape.
|
||||
if command -v realpath >/dev/null 2>&1; then
|
||||
src_real=$(realpath "$src" 2>/dev/null) || src_real=""
|
||||
else
|
||||
src_real=$(cd "$(dirname "$src")" 2>/dev/null && pwd -P)/$(basename "$src")
|
||||
fi
|
||||
if [ -z "$src_real" ]; then
|
||||
echo "error: refusing Grok runtime link — cannot realpath src: $src" >&2
|
||||
return 1
|
||||
fi
|
||||
case "$src_real" in
|
||||
"$monorepo_real"|"$monorepo_real"/*) ;;
|
||||
*)
|
||||
echo "error: refusing Grok runtime link outside monorepo:" >&2
|
||||
echo " src=$src_real" >&2
|
||||
echo " monorepo=$monorepo_real" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
|
||||
_link_or_copy "$src" "$dst"
|
||||
}
|
||||
|
||||
create_grok_runtime_root() {
|
||||
local gstack_dir="$1"
|
||||
local grok_gstack="$2"
|
||||
local generated_root="$gstack_dir/.grok/skills/gstack"
|
||||
local staging=""
|
||||
local req
|
||||
|
||||
# Preflight required monorepo assets BEFORE wiping the live install.
|
||||
# Without this, a mid-install failure under set -e leaves an empty/half tree
|
||||
# after rm -rf (review finding #6).
|
||||
for req in bin browse/dist browse/src scripts review/specialists; do
|
||||
if [ ! -e "$gstack_dir/$req" ]; then
|
||||
echo "error: preflight — required monorepo asset missing: $gstack_dir/$req" >&2
|
||||
echo " Run a full monorepo build (./setup or bun run build) before installing Grok runtime." >&2
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
# Core review files audited by gstack-grok-compat-audit — fail closed when monorepo has review/
|
||||
if [ -d "$gstack_dir/review" ]; then
|
||||
for req in checklist.md TODOS-format.md; do
|
||||
if [ ! -f "$gstack_dir/review/$req" ]; then
|
||||
echo "error: preflight — core review file missing: $gstack_dir/review/$req" >&2
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
# design/dist required when monorepo ships the design package
|
||||
if [ -d "$gstack_dir/design" ] && [ ! -d "$gstack_dir/design/dist" ]; then
|
||||
echo "error: preflight — monorepo has design/ but design/dist is missing — run design build" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Stage into a sibling .next tree, then atomic rename into place so a partial
|
||||
# link failure never leaves the live ~/.grok/skills/gstack empty.
|
||||
staging="${grok_gstack}.next.$$"
|
||||
rm -rf "$staging"
|
||||
mkdir -p "$staging" \
|
||||
"$staging/browse" \
|
||||
"$staging/design" \
|
||||
"$staging/make-pdf" \
|
||||
"$staging/gstack-upgrade" \
|
||||
"$staging/review" || return 1
|
||||
|
||||
# Cleanup staging on any failure from here until atomic swap
|
||||
# shellcheck disable=SC2064
|
||||
trap 'rm -rf "$staging"' RETURN
|
||||
|
||||
if [ -f "$generated_root/SKILL.md" ]; then
|
||||
_link_or_copy "$generated_root/SKILL.md" "$staging/SKILL.md"
|
||||
elif [ -f "$gstack_dir/SKILL.md" ]; then
|
||||
_link_or_copy "$gstack_dir/SKILL.md" "$staging/SKILL.md"
|
||||
fi
|
||||
|
||||
# Required core assets (fail closed) — dual-write with hosts/grok-build.ts runtimeRoot
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/bin" "$staging/bin" 1 || return 1
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/dist" "$staging/browse/dist" 1 || return 1
|
||||
if [ -d "$gstack_dir/browse/bin" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/bin" "$staging/browse/bin" 0 || return 1
|
||||
fi
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/src" "$staging/browse/src" 1 || return 1
|
||||
|
||||
if [ -d "$gstack_dir/design/dist" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/design/dist" "$staging/design/dist" 1 || return 1
|
||||
fi
|
||||
|
||||
# make-pdf/dist — link when present (membership skill)
|
||||
if [ -d "$gstack_dir/make-pdf/dist" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/make-pdf/dist" "$staging/make-pdf/dist" 0 || return 1
|
||||
fi
|
||||
|
||||
# extension/ — required when monorepo has it
|
||||
if [ -d "$gstack_dir/extension" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/extension" "$staging/extension" 1 || return 1
|
||||
fi
|
||||
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/scripts" "$staging/scripts" 1 || return 1
|
||||
|
||||
# Core review files — required=1 so install policy matches gstack-grok-compat-audit (#7)
|
||||
for f in checklist.md TODOS-format.md; do
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/$f" "$staging/review/$f" 1 || return 1
|
||||
done
|
||||
# Optional review assets (soft when monorepo lacks them)
|
||||
for f in design-checklist.md greptile-triage.md; do
|
||||
if [ -f "$gstack_dir/review/$f" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/$f" "$staging/review/$f" 0 || return 1
|
||||
fi
|
||||
done
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/specialists" "$staging/review/specialists" 1 || return 1
|
||||
|
||||
if [ -f "$gstack_dir/gstack-upgrade/SKILL.md" ] || [ -f "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" ]; then
|
||||
if [ -f "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" ]; then
|
||||
_link_or_copy "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" "$staging/gstack-upgrade/SKILL.md"
|
||||
else
|
||||
_link_or_copy "$gstack_dir/gstack-upgrade/SKILL.md" "$staging/gstack-upgrade/SKILL.md"
|
||||
fi
|
||||
fi
|
||||
if [ -f "$gstack_dir/ETHOS.md" ]; then
|
||||
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/ETHOS.md" "$staging/ETHOS.md" 0 || return 1
|
||||
fi
|
||||
|
||||
# Atomic swap: only wipe live install after staging is fully linked
|
||||
trap - RETURN
|
||||
if [ -L "$grok_gstack" ]; then
|
||||
rm -f "$grok_gstack"
|
||||
elif [ -d "$grok_gstack" ] && [ "$grok_gstack" != "$gstack_dir" ]; then
|
||||
rm -rf "$grok_gstack"
|
||||
fi
|
||||
# Prefer rename; fall back to mv -T when available for non-empty dest edge cases
|
||||
if ! mv "$staging" "$grok_gstack" 2>/dev/null; then
|
||||
echo "error: failed to promote staged Grok runtime root to $grok_gstack" >&2
|
||||
rm -rf "$staging"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
link_grok_skill_dirs() {
|
||||
local gstack_dir="$1"
|
||||
local skills_dir="$2"
|
||||
local grok_dir="$gstack_dir/.grok/skills"
|
||||
local linked=()
|
||||
|
||||
if [ ! -d "$grok_dir" ]; then
|
||||
echo " Generating .grok/ skill docs..."
|
||||
( cd "$gstack_dir" && bun run gen:skill-docs --host grok-build )
|
||||
fi
|
||||
|
||||
if [ ! -d "$grok_dir" ]; then
|
||||
echo " warning: .grok/skills/ generation failed — run 'bun run gen:skill-docs --host grok-build' manually" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
mkdir -p "$skills_dir"
|
||||
|
||||
for skill_dir in "$grok_dir"/gstack*/; do
|
||||
if [ -f "$skill_dir/SKILL.md" ]; then
|
||||
skill_name="$(basename "$skill_dir")"
|
||||
# Runtime root is installed separately as ~/.grok/skills/gstack
|
||||
[ "$skill_name" = "gstack" ] && continue
|
||||
target="$skills_dir/$skill_name"
|
||||
# Replace stale monorepo flat symlinks (browse → ~/gstack/browse) with generated packages
|
||||
if [ -L "$target" ] || [ ! -e "$target" ]; then
|
||||
_link_or_copy "$skill_dir" "$target"
|
||||
linked+=("$skill_name")
|
||||
elif [ -d "$target" ] && [ ! -f "$target/SKILL.md" ]; then
|
||||
rm -rf "$target"
|
||||
_link_or_copy "$skill_dir" "$target"
|
||||
linked+=("$skill_name")
|
||||
else
|
||||
# Force refresh: generated content is authoritative for this host
|
||||
rm -rf "$target"
|
||||
_link_or_copy "$skill_dir" "$target"
|
||||
linked+=("$skill_name")
|
||||
fi
|
||||
|
||||
# Also expose unprefixed alias (browse) so slash names match frontmatter
|
||||
# without depending on Grok's name: field alone.
|
||||
# Deny bare `codex` alias (OpenAI plugin owns /codex); optional gstack-codex only.
|
||||
bare_name="${skill_name#gstack-}"
|
||||
if [ -n "$bare_name" ] && [ "$bare_name" != "$skill_name" ] && [ "$bare_name" != "codex" ]; then
|
||||
bare_target="$skills_dir/$bare_name"
|
||||
# Only alias if missing or already a symlink (don't clobber real non-gstack skills)
|
||||
if [ -L "$bare_target" ] || [ ! -e "$bare_target" ]; then
|
||||
_link_or_copy "$skill_dir" "$bare_target"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Backwards-compat alias: /connect-chrome → generated open-gstack-browser (U2 / KTD7)
|
||||
# Force-refresh stale monorepo targets that pointed at source tree packages.
|
||||
local _ogb_src=""
|
||||
if [ -d "$skills_dir/gstack-open-gstack-browser" ]; then
|
||||
_ogb_src="$skills_dir/gstack-open-gstack-browser"
|
||||
elif [ -d "$grok_dir/gstack-open-gstack-browser" ]; then
|
||||
_ogb_src="$grok_dir/gstack-open-gstack-browser"
|
||||
fi
|
||||
if [ -n "$_ogb_src" ]; then
|
||||
rm -rf "$skills_dir/connect-chrome"
|
||||
_link_or_copy "$_ogb_src" "$skills_dir/connect-chrome"
|
||||
linked+=("connect-chrome→open-gstack-browser")
|
||||
fi
|
||||
|
||||
if [ ${#linked[@]} -gt 0 ]; then
|
||||
echo " linked skills: ${linked[*]}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 4. Install for Claude (default)
|
||||
SKILLS_BASENAME="$(basename "$INSTALL_SKILLS_DIR")"
|
||||
SKILLS_PARENT_BASENAME="$(basename "$(dirname "$INSTALL_SKILLS_DIR")")"
|
||||
|
|
@ -1193,6 +1450,17 @@ if [ "$INSTALL_OPENCODE" -eq 1 ]; then
|
|||
echo " opencode skills: $OPENCODE_SKILLS"
|
||||
fi
|
||||
|
||||
# 6d. Install for Grok Build (xAI)
|
||||
if [ "$INSTALL_GROK" -eq 1 ]; then
|
||||
mkdir -p "$GROK_SKILLS"
|
||||
create_grok_runtime_root "$SOURCE_GSTACK_DIR" "$GROK_GSTACK"
|
||||
link_grok_skill_dirs "$SOURCE_GSTACK_DIR" "$GROK_SKILLS"
|
||||
echo "gstack ready (grok-build)."
|
||||
echo " browse: $BROWSE_BIN"
|
||||
echo " grok skills: $GROK_SKILLS"
|
||||
echo " runtime root: $GROK_GSTACK"
|
||||
fi
|
||||
|
||||
# 7. Create .agents/ sidecar symlinks for the real Codex skill target.
|
||||
# The root Codex skill ends up pointing at $SOURCE_GSTACK_DIR/.agents/skills/gstack,
|
||||
# so the runtime assets must live there for both global and repo-local installs.
|
||||
|
|
@ -1337,6 +1605,10 @@ fi
|
|||
# something at runtime instead of being agent-convention. Explicit consent UX
|
||||
# per D4 + Codex: never mutate settings.json silently.
|
||||
#
|
||||
# Host-scoped: these hooks only affect Claude Code. Skip the prompt entirely for
|
||||
# non-Claude host installs (e.g. --host grok-build) unless the user explicitly
|
||||
# passed --plan-tune-hooks. Grok /plan-tune still works via skill-text path.
|
||||
#
|
||||
# Idempotent via _gstack_source tag = 'plan-tune-cathedral'. If both hooks
|
||||
# already registered under that tag, the install is a no-op (no prompt).
|
||||
PLAN_TUNE_LOG_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/question-log-hook"
|
||||
|
|
@ -1344,7 +1616,14 @@ PLAN_TUNE_PREF_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/question-preference-h
|
|||
AUQ_ERROR_FALLBACK_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/auq-error-fallback-hook"
|
||||
PLAN_TUNE_INSTALL_MARKER="$HOME/.gstack/.plan-tune-hooks-prompted"
|
||||
|
||||
# Explicit --plan-tune-hooks still allowed from any host install (user intent).
|
||||
_PT_EXPLICIT_YES=0
|
||||
case "$(printf '%s' "${PLAN_TUNE_HOOKS_MODE:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')" in
|
||||
y|yes|true|install|on|1) _PT_EXPLICIT_YES=1 ;;
|
||||
esac
|
||||
|
||||
if [ "$NO_TEAM_MODE" -ne 1 ] \
|
||||
&& { [ "$INSTALL_CLAUDE" -eq 1 ] || [ "$_PT_EXPLICIT_YES" -eq 1 ]; } \
|
||||
&& [ -x "$SETTINGS_HOOK" ] \
|
||||
&& [ -x "$PLAN_TUNE_LOG_HOOK" ] \
|
||||
&& [ -x "$PLAN_TUNE_PREF_HOOK" ]; then
|
||||
|
|
|
|||
|
|
@ -788,8 +788,22 @@ Claude Code) can call it as both a CLI and an MCP tool.
|
|||
|
||||
**Scope honesty:** This skill's MCP registration step (5a) uses
|
||||
`claude mcp add` and targets Claude Code specifically. Other local hosts
|
||||
(Cursor, Codex CLI, etc.) will still get the gbrain CLI on PATH — they can
|
||||
register `gbrain serve` in their own MCP config manually after setup.
|
||||
(Cursor, Codex CLI, **Grok Build**, etc.) will still get the gbrain CLI on
|
||||
PATH — they can register `gbrain serve` in their own MCP config manually
|
||||
after setup.
|
||||
|
||||
**Grok Build success path (no Claude required):** setup is **COMPATIBLE** when
|
||||
(1) `gbrain` is on PATH, (2) the engine is healthy (`gbrain doctor`), and
|
||||
(3) AGENTS.md (or project docs) carry gbrain guidance via `/sync-gbrain`.
|
||||
Claude MCP registration is optional. Never mark setup FAILED only because
|
||||
`claude` is missing. Optional Grok MCP into `~/.grok/config.toml` is a stretch
|
||||
(absolute path, structured TOML RMW, gbrain stanza only, atomic write +
|
||||
backup; non-interactive refuses non-equivalent overwrite unless `--force` or
|
||||
a real TTY confirm).
|
||||
|
||||
**Bridge policy:** install prose alone does not green the primary path when
|
||||
the CLI is missing — detect/install guidance may be READY; overall bridge is
|
||||
DEPENDENT until `gbrain` is present.
|
||||
|
||||
**Audience:** local-Mac users. openclaw/hermes agents typically run in cloud
|
||||
docker containers with their own gbrain; "sharing" a brain between them and
|
||||
|
|
@ -1258,9 +1272,14 @@ doctor output and STOP.
|
|||
|
||||
---
|
||||
|
||||
## Step 5a: Register gbrain as Claude Code MCP (D18)
|
||||
## Step 5a: Register gbrain as Claude Code MCP (D18) — optional
|
||||
|
||||
Only if `which claude` resolves. Ask: "Give Claude Code a typed tool surface
|
||||
Only if `which claude` resolves. If `claude` is **not** on PATH (typical on
|
||||
Grok-only machines): emit "MCP registration skipped — Claude Code not present.
|
||||
Grok/other hosts succeed via CLI + AGENTS.md; register `gbrain serve` in host
|
||||
MCP config only if desired." Continue to step 6 — **do not fail setup**.
|
||||
|
||||
If `claude` is present, ask: "Give Claude Code a typed tool surface
|
||||
for gbrain? (recommended yes)"
|
||||
|
||||
The registration form depends on the path picked in Step 2:
|
||||
|
|
|
|||
|
|
@ -34,8 +34,22 @@ Claude Code) can call it as both a CLI and an MCP tool.
|
|||
|
||||
**Scope honesty:** This skill's MCP registration step (5a) uses
|
||||
`claude mcp add` and targets Claude Code specifically. Other local hosts
|
||||
(Cursor, Codex CLI, etc.) will still get the gbrain CLI on PATH — they can
|
||||
register `gbrain serve` in their own MCP config manually after setup.
|
||||
(Cursor, Codex CLI, **Grok Build**, etc.) will still get the gbrain CLI on
|
||||
PATH — they can register `gbrain serve` in their own MCP config manually
|
||||
after setup.
|
||||
|
||||
**Grok Build success path (no Claude required):** setup is **COMPATIBLE** when
|
||||
(1) `gbrain` is on PATH, (2) the engine is healthy (`gbrain doctor`), and
|
||||
(3) AGENTS.md (or project docs) carry gbrain guidance via `/sync-gbrain`.
|
||||
Claude MCP registration is optional. Never mark setup FAILED only because
|
||||
`claude` is missing. Optional Grok MCP into `~/.grok/config.toml` is a stretch
|
||||
(absolute path, structured TOML RMW, gbrain stanza only, atomic write +
|
||||
backup; non-interactive refuses non-equivalent overwrite unless `--force` or
|
||||
a real TTY confirm).
|
||||
|
||||
**Bridge policy:** install prose alone does not green the primary path when
|
||||
the CLI is missing — detect/install guidance may be READY; overall bridge is
|
||||
DEPENDENT until `gbrain` is present.
|
||||
|
||||
**Audience:** local-Mac users. openclaw/hermes agents typically run in cloud
|
||||
docker containers with their own gbrain; "sharing" a brain between them and
|
||||
|
|
@ -504,9 +518,14 @@ doctor output and STOP.
|
|||
|
||||
---
|
||||
|
||||
## Step 5a: Register gbrain as Claude Code MCP (D18)
|
||||
## Step 5a: Register gbrain as Claude Code MCP (D18) — optional
|
||||
|
||||
Only if `which claude` resolves. Ask: "Give Claude Code a typed tool surface
|
||||
Only if `which claude` resolves. If `claude` is **not** on PATH (typical on
|
||||
Grok-only machines): emit "MCP registration skipped — Claude Code not present.
|
||||
Grok/other hosts succeed via CLI + AGENTS.md; register `gbrain serve` in host
|
||||
MCP config only if desired." Continue to step 6 — **do not fail setup**.
|
||||
|
||||
If `claude` is present, ask: "Give Claude Code a typed tool surface
|
||||
for gbrain? (recommended yes)"
|
||||
|
||||
The registration form depends on the path picked in Step 2:
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ separated tokens starting with `--`. Last flag wins on conflict.
|
|||
| `--no-dedupe` | — | Skip the dedupe check. |
|
||||
| `--no-gate` | OFF (gate is ON) | Skip the codex quality-score gate between Phase 4 and Phase 5. **Redaction (Phase 4.5a semantic + 4.5b regex) still runs — there is no flag that disables it.** |
|
||||
| `--audit` | OFF | Route Phase 5 to the Audit/Cleanup template (instead of Standard). |
|
||||
| `--execute` | conditional default (see Phase 5) | Spawn `claude -p` in a fresh worktree after filing the issue. |
|
||||
{{SPEC_EXECUTE_FLAG}}
|
||||
| `--no-execute` | — | File issue only; do NOT spawn agent (alias: `--file-only`). |
|
||||
| `--file-only` | — | Same as `--no-execute`. |
|
||||
| `--plan-file <path>` | inferred from harness | Load the spec into the specified plan file instead of inferring. |
|
||||
|
|
@ -447,14 +447,7 @@ git worktree add "$SPAWN_PATH" -b "$SPAWN_BRANCH" "$PIN_SHA" 2>&1
|
|||
in-progress changes will be visible to the agent. Cancel with Ctrl+C if not
|
||||
desired." Then fall back to current dir (still spawn).
|
||||
|
||||
If A and worktree created: spawn `claude -p` with the spec piped via stdin:
|
||||
|
||||
```bash
|
||||
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
|
||||
SPAWN_PID=$!
|
||||
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
|
||||
echo "Follow with: cd $SPAWN_PATH && claude --resume"
|
||||
```
|
||||
{{SPEC_SPAWN}}
|
||||
|
||||
Update archive frontmatter with `spec_worktree_path: $SPAWN_PATH` and
|
||||
`spec_executed: true` (atomic re-write).
|
||||
|
|
|
|||
|
|
@ -785,8 +785,12 @@ Skills that run plan reviews (`/plan-*-review`, `/codex review`) include the EXI
|
|||
You are running the canonical "keep this brain up to date" verb. /setup-gbrain
|
||||
installs gbrain once; /sync-gbrain runs every time the user wants the brain
|
||||
refreshed against this repo's current state, and refreshes the agent-side
|
||||
guidance in CLAUDE.md so the coding agent knows when to prefer `gbrain`
|
||||
search over Grep.
|
||||
guidance in CLAUDE.md / AGENTS.md so the coding agent knows when to prefer
|
||||
`gbrain` search over Grep.
|
||||
|
||||
**Grok Build:** success does not require Claude MCP. Require `gbrain` on PATH +
|
||||
engine healthy + AGENTS.md guidance refresh. If gbrain is absent, give install
|
||||
steps (DEPENDENT bridge) — do not silent-fail or demand `which claude`.
|
||||
|
||||
**Architecture (post-codex review):** This skill uses gbrain v0.20.0+'s
|
||||
**native code surfaces** (`gbrain sources add`, `gbrain sync --strategy code`,
|
||||
|
|
|
|||
|
|
@ -31,8 +31,12 @@ allowed-tools:
|
|||
You are running the canonical "keep this brain up to date" verb. /setup-gbrain
|
||||
installs gbrain once; /sync-gbrain runs every time the user wants the brain
|
||||
refreshed against this repo's current state, and refreshes the agent-side
|
||||
guidance in CLAUDE.md so the coding agent knows when to prefer `gbrain`
|
||||
search over Grep.
|
||||
guidance in CLAUDE.md / AGENTS.md so the coding agent knows when to prefer
|
||||
`gbrain` search over Grep.
|
||||
|
||||
**Grok Build:** success does not require Claude MCP. Require `gbrain` on PATH +
|
||||
engine healthy + AGENTS.md guidance refresh. If gbrain is absent, give install
|
||||
steps (DEPENDENT bridge) — do not silent-fail or demand `which claude`.
|
||||
|
||||
**Architecture (post-codex review):** This skill uses gbrain v0.20.0+'s
|
||||
**native code surfaces** (`gbrain sources add`, `gbrain sync --strategy code`,
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ GSTACK_ROOT="$HOME/.codex/skills/gstack"
|
|||
GSTACK_BIN="$GSTACK_ROOT/bin"
|
||||
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
|
||||
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
|
||||
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
|
||||
_UPD=$($GSTACK_BIN/gstack-update-check 2>/dev/null || .agents/skills/gstack/bin/gstack-update-check 2>/dev/null || true)
|
||||
[ -n "$_UPD" ] && echo "$_UPD" || true
|
||||
mkdir -p ~/.gstack/sessions
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ GSTACK_ROOT="$HOME/.factory/skills/gstack"
|
|||
GSTACK_BIN="$GSTACK_ROOT/bin"
|
||||
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
|
||||
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
|
||||
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
|
||||
_UPD=$($GSTACK_BIN/gstack-update-check 2>/dev/null || .factory/skills/gstack/bin/gstack-update-check 2>/dev/null || true)
|
||||
[ -n "$_UPD" ] && echo "$_UPD" || true
|
||||
mkdir -p ~/.gstack/sessions
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,600 @@
|
|||
/**
|
||||
* Behavioral unit tests for Grok Build packaging surfaces (code-review #8):
|
||||
* - generateSpecSpawn fail-closed allowlist + no default --always-approve
|
||||
* - GrokAdapter available timeout / --prompt-file path / auth redaction
|
||||
* - gstack-grok-compat-audit exit codes against fixture skills dirs
|
||||
*
|
||||
* Free (no API spend). Does not invoke real grok CLI for successful runs.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { spawnSync } from 'child_process';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { generateSpecSpawn, generateSpecExecuteFlag } from '../scripts/resolvers/spec-spawn';
|
||||
import type { TemplateContext } from '../scripts/resolvers/types';
|
||||
import { HOST_PATHS } from '../scripts/resolvers/types';
|
||||
import { GrokAdapter, parseGrokOutput, isStructurallyValidGrokAuthFile } from './helpers/providers/grok';
|
||||
import { estimateCostUsd, PRICING } from './helpers/pricing';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
const AUDIT_BIN = path.join(ROOT, 'bin', 'gstack-grok-compat-audit');
|
||||
|
||||
function makeCtx(host: 'grok-build' | 'claude'): TemplateContext {
|
||||
return {
|
||||
skillName: 'spec',
|
||||
tmplPath: path.join(ROOT, 'spec', 'SKILL.md.tmpl'),
|
||||
host,
|
||||
paths: HOST_PATHS[host],
|
||||
};
|
||||
}
|
||||
|
||||
// ─── generateSpecSpawn ───────────────────────────────────────
|
||||
|
||||
describe('generateSpecSpawn (Grok fail-closed)', () => {
|
||||
test('grok-build emits auth gate, ARCHIVE allowlist, and no default --always-approve', () => {
|
||||
const out = generateSpecSpawn(makeCtx('grok-build'));
|
||||
expect(out).toContain('command -v grok');
|
||||
expect(out).toContain('ARCHIVE_PATH');
|
||||
expect(out).toContain('SPAWN_PATH');
|
||||
expect(out).toContain('GSTACK_STATE_ROOT');
|
||||
expect(out).toContain('realpath');
|
||||
expect(out).toContain('fail closed');
|
||||
expect(out).toContain('grok --prompt-file');
|
||||
// Default spawn must NOT enable elevated auto-approve
|
||||
expect(out).not.toMatch(/grok --prompt-file[^\n]*--always-approve/);
|
||||
expect(out).toMatch(/opt-in only|does \*\*not\*\* pass `--always-approve`|--always-approve` is opt-in/i);
|
||||
});
|
||||
|
||||
test('grok-build --execute-claude path reuses ARCHIVE allowlist before cat|claude', () => {
|
||||
const out = generateSpecSpawn(makeCtx('grok-build'));
|
||||
expect(out).toContain('--execute-claude');
|
||||
// Claude branch must re-check allowlist, not bare cat only
|
||||
const claudeIdx = out.indexOf('--execute-claude');
|
||||
const after = out.slice(claudeIdx);
|
||||
expect(after).toContain('ARCHIVE_REAL');
|
||||
expect(after).toContain('STATE_PROJECTS');
|
||||
expect(after).toMatch(/cat "\$ARCHIVE_PATH" \| \(cd "\$SPAWN_PATH" && claude -p/);
|
||||
});
|
||||
|
||||
test('claude host still uses classic stdin pipe (no Grok-only gates)', () => {
|
||||
const out = generateSpecSpawn(makeCtx('claude'));
|
||||
expect(out).toContain('claude -p');
|
||||
expect(out).not.toContain('grok --prompt-file');
|
||||
});
|
||||
|
||||
test('generateSpecExecuteFlag is host-aware', () => {
|
||||
expect(generateSpecExecuteFlag(makeCtx('grok-build'))).toContain('grok --prompt-file');
|
||||
expect(generateSpecExecuteFlag(makeCtx('claude'))).toContain('claude -p');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── GrokAdapter ─────────────────────────────────────────────
|
||||
|
||||
describe('GrokAdapter behavioral unit', () => {
|
||||
const adapter = new GrokAdapter();
|
||||
let tmpHome: string;
|
||||
let prevHome: string | undefined;
|
||||
let prevXai: string | undefined;
|
||||
let prevGrok: string | undefined;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-adapter-'));
|
||||
prevHome = process.env.HOME;
|
||||
prevXai = process.env.XAI_API_KEY;
|
||||
prevGrok = process.env.GROK_API_KEY;
|
||||
process.env.HOME = tmpHome;
|
||||
delete process.env.XAI_API_KEY;
|
||||
delete process.env.GROK_API_KEY;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (prevHome === undefined) delete process.env.HOME;
|
||||
else process.env.HOME = prevHome;
|
||||
if (prevXai === undefined) delete process.env.XAI_API_KEY;
|
||||
else process.env.XAI_API_KEY = prevXai;
|
||||
if (prevGrok === undefined) delete process.env.GROK_API_KEY;
|
||||
else process.env.GROK_API_KEY = prevGrok;
|
||||
fs.rmSync(tmpHome, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('available() returns not-ok when CLI and auth both missing', async () => {
|
||||
// Isolate PATH so grok cannot be found even if installed on the machine.
|
||||
// Some environments still resolve `grok` via absolute exec caches — accept
|
||||
// either binary-missing OR auth-missing as a valid not-ok outcome.
|
||||
const prevPath = process.env.PATH;
|
||||
process.env.PATH = '/nonexistent-bin-dir-for-grok-avail';
|
||||
try {
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.reason).toMatch(/not found|Install Grok|PATH|No Grok auth|XAI_API_KEY|GROK_API_KEY/i);
|
||||
} finally {
|
||||
process.env.PATH = prevPath;
|
||||
}
|
||||
});
|
||||
|
||||
test('available() fails when binary present but no auth file and no env key names', async () => {
|
||||
// If grok is not on PATH in this environment, skip — unit still covers the
|
||||
// auth-file/env-name branch when binary is present.
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) {
|
||||
// No binary: available should report CLI missing under empty HOME
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(false);
|
||||
return;
|
||||
}
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.reason).toMatch(/No Grok auth|XAI_API_KEY|GROK_API_KEY/i);
|
||||
});
|
||||
|
||||
test('available() accepts non-empty auth.json object under HOME', async () => {
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) return; // cannot exercise auth-file path without binary
|
||||
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
|
||||
// OAuth-shaped key (URL) without logging real secrets — structure only
|
||||
fs.writeFileSync(
|
||||
path.join(tmpHome, '.grok', 'auth.json'),
|
||||
JSON.stringify({ 'https://auth.x.ai::fixture-user': { access: 'x' } }),
|
||||
);
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(true);
|
||||
});
|
||||
|
||||
test('available() fails closed on empty object auth.json {}', async () => {
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) return;
|
||||
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
|
||||
fs.writeFileSync(path.join(tmpHome, '.grok', 'auth.json'), '{}');
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.reason).toMatch(/No Grok auth|auth\.json|XAI_API_KEY|GROK_API_KEY/i);
|
||||
// Reason must never echo file contents / secrets
|
||||
expect(r.reason).not.toMatch(/access_token|sk-|Bearer/i);
|
||||
});
|
||||
|
||||
test('available() fails closed on zero-byte and invalid auth.json', async () => {
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) return;
|
||||
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
|
||||
const authPath = path.join(tmpHome, '.grok', 'auth.json');
|
||||
|
||||
fs.writeFileSync(authPath, '');
|
||||
expect((await adapter.available()).ok).toBe(false);
|
||||
|
||||
fs.writeFileSync(authPath, ' \n');
|
||||
expect((await adapter.available()).ok).toBe(false);
|
||||
|
||||
fs.writeFileSync(authPath, 'not-json');
|
||||
expect((await adapter.available()).ok).toBe(false);
|
||||
|
||||
fs.writeFileSync(authPath, 'null');
|
||||
expect((await adapter.available()).ok).toBe(false);
|
||||
|
||||
fs.writeFileSync(authPath, '[]');
|
||||
expect((await adapter.available()).ok).toBe(false);
|
||||
});
|
||||
|
||||
test('available() fails closed on whitespace-only env keys', async () => {
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) return;
|
||||
process.env.XAI_API_KEY = ' ';
|
||||
process.env.GROK_API_KEY = '\t';
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(false);
|
||||
expect(r.reason).not.toContain(' ');
|
||||
});
|
||||
|
||||
test('available() accepts non-blank env key without auth file', async () => {
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
if (which.status !== 0) return;
|
||||
process.env.XAI_API_KEY = 'test-not-a-real-key';
|
||||
const r = await adapter.available();
|
||||
expect(r.ok).toBe(true);
|
||||
});
|
||||
|
||||
test('isStructurallyValidGrokAuthFile pure checks', () => {
|
||||
const p = path.join(tmpHome, 'auth.json');
|
||||
fs.writeFileSync(p, '');
|
||||
expect(isStructurallyValidGrokAuthFile(p)).toBe(false);
|
||||
fs.writeFileSync(p, '{}');
|
||||
expect(isStructurallyValidGrokAuthFile(p)).toBe(false);
|
||||
fs.writeFileSync(p, '{"k":1}');
|
||||
expect(isStructurallyValidGrokAuthFile(p)).toBe(true);
|
||||
expect(isStructurallyValidGrokAuthFile(path.join(tmpHome, 'missing.json'))).toBe(false);
|
||||
});
|
||||
|
||||
test('run() uses --prompt-file for multi-line / large prompts (ARG_MAX safety)', async () => {
|
||||
// Force binary_missing path so we never hit a real CLI; still inspects argv construction
|
||||
// via the ENOENT error path. Large prompt must not throw ARG_MAX.
|
||||
const big = 'line\n'.repeat(500) + 'x'.repeat(3000);
|
||||
const prevPath = process.env.PATH;
|
||||
process.env.PATH = '/nonexistent-bin-dir-for-grok-test';
|
||||
try {
|
||||
const r = await adapter.run({
|
||||
prompt: big,
|
||||
workdir: tmpHome,
|
||||
timeoutMs: 2000,
|
||||
});
|
||||
expect(r.error?.code).toBe('binary_missing');
|
||||
expect(r.tokens).toEqual({ input: 0, output: 0 });
|
||||
} finally {
|
||||
process.env.PATH = prevPath;
|
||||
}
|
||||
});
|
||||
|
||||
test('run() redacts long token-shaped stderr on unknown errors', async () => {
|
||||
// Drive the unknown error path with a fake binary that exits non-zero
|
||||
const fakeBin = path.join(tmpHome, 'bin');
|
||||
fs.mkdirSync(fakeBin, { recursive: true });
|
||||
const grokSh = path.join(fakeBin, 'grok');
|
||||
fs.writeFileSync(
|
||||
grokSh,
|
||||
`#!/bin/sh\necho "auth token SECRET_TOKEN_ABCDEFGHIJKLMNOPQRSTUVWXYZ123456" >&2\nexit 1\n`,
|
||||
{ mode: 0o755 },
|
||||
);
|
||||
const prevPath = process.env.PATH;
|
||||
process.env.PATH = `${fakeBin}:${prevPath ?? ''}`;
|
||||
try {
|
||||
const r = await adapter.run({
|
||||
prompt: 'hi',
|
||||
workdir: tmpHome,
|
||||
timeoutMs: 3000,
|
||||
});
|
||||
// Auth keyword in stderr maps to auth code with redacted reason
|
||||
expect(r.error).toBeDefined();
|
||||
if (r.error?.code === 'auth') {
|
||||
expect(r.error.reason).not.toContain('SECRET_TOKEN');
|
||||
} else {
|
||||
expect(r.error!.reason).not.toContain('SECRET_TOKEN_ABCDEFGHIJKLMNOPQRSTUVWXYZ123456');
|
||||
expect(r.error!.reason).toMatch(/\[redacted\]|unknown|authentication/i);
|
||||
}
|
||||
} finally {
|
||||
process.env.PATH = prevPath;
|
||||
}
|
||||
});
|
||||
|
||||
test('run() passes --output-format json and parses usage from fixture CLI', async () => {
|
||||
const fakeBin = path.join(tmpHome, 'bin');
|
||||
fs.mkdirSync(fakeBin, { recursive: true });
|
||||
const grokSh = path.join(fakeBin, 'grok');
|
||||
// Echo argv so we can assert --output-format json; emit usage-bearing JSON on stdout
|
||||
const payload = JSON.stringify({
|
||||
text: 'hello',
|
||||
usage: { input_tokens: 1000, output_tokens: 500 },
|
||||
model: 'grok',
|
||||
});
|
||||
fs.writeFileSync(
|
||||
grokSh,
|
||||
`#!/bin/sh
|
||||
# record argv for assertions
|
||||
printf '%s\\n' "$*" > "${tmpHome}/grok-argv.txt"
|
||||
echo '${payload}'
|
||||
`,
|
||||
{ mode: 0o755 },
|
||||
);
|
||||
const prevPath = process.env.PATH;
|
||||
process.env.PATH = `${fakeBin}:${prevPath ?? ''}`;
|
||||
try {
|
||||
const r = await adapter.run({
|
||||
prompt: 'hi',
|
||||
workdir: tmpHome,
|
||||
timeoutMs: 3000,
|
||||
});
|
||||
expect(r.error).toBeUndefined();
|
||||
expect(r.output).toBe('hello');
|
||||
expect(r.tokens).toEqual({ input: 1000, output: 500 });
|
||||
expect(r.modelUsed).toBe('grok');
|
||||
// Cost honesty: non-zero tokens × official rates → non-zero USD
|
||||
const cost = adapter.estimateCost(r.tokens, r.modelUsed);
|
||||
expect(cost).toBeGreaterThan(0);
|
||||
const argv = fs.readFileSync(path.join(tmpHome, 'grok-argv.txt'), 'utf-8');
|
||||
expect(argv).toMatch(/--output-format\s+json|--output-format json/);
|
||||
} finally {
|
||||
process.env.PATH = prevPath;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── parseGrokOutput + pricing honesty ───────────────────────
|
||||
|
||||
describe('parseGrokOutput + Grok pricing', () => {
|
||||
test('parses characterized headless json shape (no usage → zero tokens)', () => {
|
||||
const raw = JSON.stringify({
|
||||
text: 'pong',
|
||||
stopReason: 'EndTurn',
|
||||
sessionId: '019f57be-82a4-7990-89ed-47a030fbaeb7',
|
||||
requestId: 'fe579ce3-777b-42e5-aea5-f2e25c24398d',
|
||||
thought: 'simple reply',
|
||||
});
|
||||
const p = parseGrokOutput(raw);
|
||||
expect(p.output).toBe('pong');
|
||||
expect(p.tokens).toEqual({ input: 0, output: 0 });
|
||||
});
|
||||
|
||||
test('parses usage when present (input_tokens / output_tokens)', () => {
|
||||
const p = parseGrokOutput(
|
||||
JSON.stringify({
|
||||
text: 'ok',
|
||||
usage: { input_tokens: 1_000_000, output_tokens: 1_000_000 },
|
||||
}),
|
||||
);
|
||||
expect(p.tokens).toEqual({ input: 1_000_000, output: 1_000_000 });
|
||||
// With U1 rates for `grok` ($1/$2 per MTok): 1 + 2 = $3
|
||||
expect(estimateCostUsd(p.tokens, 'grok')).toBe(3);
|
||||
});
|
||||
|
||||
test('parses prompt_tokens / completion_tokens aliases', () => {
|
||||
const p = parseGrokOutput(
|
||||
JSON.stringify({
|
||||
text: 'x',
|
||||
usage: { prompt_tokens: 100, completion_tokens: 50 },
|
||||
}),
|
||||
);
|
||||
expect(p.tokens).toEqual({ input: 100, output: 50 });
|
||||
});
|
||||
|
||||
test('plain text without usage → zero tokens, no throw', () => {
|
||||
const p = parseGrokOutput('just plain assistant text');
|
||||
expect(p.output).toBe('just plain assistant text');
|
||||
expect(p.tokens).toEqual({ input: 0, output: 0 });
|
||||
});
|
||||
|
||||
test('streaming-json NDJSON accumulates text; zero tokens without usage', () => {
|
||||
const raw = [
|
||||
'{"type":"thought","data":"hmm"}',
|
||||
'{"type":"text","data":"hel"}',
|
||||
'{"type":"text","data":"lo"}',
|
||||
'{"type":"end","stopReason":"EndTurn","sessionId":"x"}',
|
||||
].join('\n');
|
||||
const p = parseGrokOutput(raw);
|
||||
expect(p.output).toBe('hello');
|
||||
expect(p.tokens).toEqual({ input: 0, output: 0 });
|
||||
});
|
||||
|
||||
test('official Grok rates are non-zero and match rate math', () => {
|
||||
expect(PRICING['grok']?.input_per_mtok).toBe(1);
|
||||
expect(PRICING['grok']?.output_per_mtok).toBe(2);
|
||||
expect(PRICING['grok-build-0.1']?.input_per_mtok).toBe(1);
|
||||
expect(PRICING['grok-4.5']?.input_per_mtok).toBe(2);
|
||||
expect(PRICING['grok-4.5']?.output_per_mtok).toBe(6);
|
||||
// 1M in + 1M out at $1/$2 → $3.00
|
||||
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'grok')).toBe(3);
|
||||
// 1M in + 1M out at $2/$6 → $8.00
|
||||
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'grok-4.5')).toBe(8);
|
||||
});
|
||||
|
||||
test('unknown model returns 0 and does not throw; peer rows unchanged', () => {
|
||||
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'not-a-real-model-xyz')).toBe(0);
|
||||
expect(PRICING['claude-opus-4-7']?.input_per_mtok).toBe(15);
|
||||
expect(PRICING['gpt-5.4']?.input_per_mtok).toBe(2.5);
|
||||
expect(PRICING['gemini-2.5-pro']?.input_per_mtok).toBe(1.25);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
// ─── gstack-grok-compat-audit fixtures ───────────────────────
|
||||
|
||||
function writeMinimalRuntime(skillsDir: string, opts: { withReview?: boolean; withSpec?: boolean } = {}) {
|
||||
const { withReview = true, withSpec = true } = opts;
|
||||
const root = path.join(skillsDir, 'gstack');
|
||||
for (const rel of [
|
||||
'bin',
|
||||
'browse/dist',
|
||||
'browse/src',
|
||||
'scripts',
|
||||
'review/specialists',
|
||||
]) {
|
||||
fs.mkdirSync(path.join(root, rel), { recursive: true });
|
||||
}
|
||||
// When monorepo has design/dist + extension, audit requires them in the runtime root
|
||||
if (fs.existsSync(path.join(ROOT, 'design', 'dist'))) {
|
||||
fs.mkdirSync(path.join(root, 'design', 'dist'), { recursive: true });
|
||||
}
|
||||
if (fs.existsSync(path.join(ROOT, 'extension'))) {
|
||||
fs.mkdirSync(path.join(root, 'extension'), { recursive: true });
|
||||
}
|
||||
// browse-client required by skillify audit
|
||||
fs.mkdirSync(path.join(root, 'browse', 'src'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'browse', 'src', 'browse-client.ts'), '// fixture\n');
|
||||
if (withReview) {
|
||||
fs.writeFileSync(path.join(root, 'review', 'checklist.md'), '# checklist\n');
|
||||
fs.writeFileSync(path.join(root, 'review', 'TODOS-format.md'), '# todos\n');
|
||||
}
|
||||
// Audit compares fixture runtime against monorepo presence of design/extension.
|
||||
// Mirror those when the monorepo has them so the fixture is "complete" for phase a.
|
||||
if (fs.existsSync(path.join(ROOT, 'design', 'dist'))) {
|
||||
fs.mkdirSync(path.join(root, 'design', 'dist'), { recursive: true });
|
||||
}
|
||||
if (fs.existsSync(path.join(ROOT, 'extension'))) {
|
||||
fs.mkdirSync(path.join(root, 'extension'), { recursive: true });
|
||||
fs.writeFileSync(path.join(root, 'extension', 'manifest.json'), '{}\n');
|
||||
}
|
||||
// connect-chrome → open-gstack-browser shape
|
||||
const ogb = path.join(skillsDir, 'gstack-open-gstack-browser');
|
||||
fs.mkdirSync(ogb, { recursive: true });
|
||||
fs.writeFileSync(path.join(ogb, 'SKILL.md'), 'name: open-gstack-browser\n');
|
||||
fs.symlinkSync(ogb, path.join(skillsDir, 'connect-chrome'));
|
||||
|
||||
if (withSpec) {
|
||||
const spec = path.join(skillsDir, 'gstack-spec');
|
||||
fs.mkdirSync(spec, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(spec, 'SKILL.md'),
|
||||
[
|
||||
'---',
|
||||
'name: spec',
|
||||
'---',
|
||||
'Spawn **Grok** headless with --prompt-file.',
|
||||
'```bash',
|
||||
'grok --prompt-file "$ARCHIVE_PATH" --cwd "$SPAWN_PATH"',
|
||||
'```',
|
||||
'$GSTACK_ROOT',
|
||||
].join('\n'),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function runAudit(skillsDir: string, phase = 'a'): { status: number | null; stdout: string; stderr: string } {
|
||||
const r = spawnSync('bun', ['run', AUDIT_BIN, '--skills-dir', skillsDir, '--phase', phase], {
|
||||
cwd: ROOT,
|
||||
encoding: 'utf-8',
|
||||
timeout: 30000,
|
||||
});
|
||||
return {
|
||||
status: r.status,
|
||||
stdout: r.stdout?.toString() ?? '',
|
||||
stderr: r.stderr?.toString() ?? '',
|
||||
};
|
||||
}
|
||||
|
||||
describe('gstack-grok-compat-audit fixtures', () => {
|
||||
let skillsDir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
skillsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-audit-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(skillsDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('missing skills dir exits 1 INCOMPATIBLE', () => {
|
||||
const missing = path.join(skillsDir, 'does-not-exist');
|
||||
const r = runAudit(missing, 'a');
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stdout).toMatch(/INCOMPATIBLE|skills dir missing/i);
|
||||
});
|
||||
|
||||
test('minimal complete runtime exits 0 COMPATIBLE for phase a', () => {
|
||||
writeMinimalRuntime(skillsDir);
|
||||
const r = runAudit(skillsDir, 'a');
|
||||
expect(r.status).toBe(0);
|
||||
expect(r.stdout).toContain('COMPATIBLE');
|
||||
});
|
||||
|
||||
test('missing review/checklist.md fails phase a', () => {
|
||||
writeMinimalRuntime(skillsDir, { withReview: false });
|
||||
const r = runAudit(skillsDir, 'a');
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stdout).toMatch(/missing review\/checklist\.md|INCOMPATIBLE/);
|
||||
});
|
||||
|
||||
test('missing gstack runtime root fails', () => {
|
||||
fs.mkdirSync(skillsDir, { recursive: true });
|
||||
// empty skills dir — no gstack/
|
||||
const r = runAudit(skillsDir, 'a');
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stdout).toMatch(/missing runtime root|INCOMPATIBLE/);
|
||||
});
|
||||
|
||||
test('phase ab fails when neither gstack-spec nor spec package installed', () => {
|
||||
writeMinimalRuntime(skillsDir, { withSpec: false });
|
||||
const r = runAudit(skillsDir, 'ab');
|
||||
expect(r.status).toBe(1);
|
||||
expect(r.stdout).toMatch(/neither gstack-spec nor spec|INCOMPATIBLE/);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── setup create_grok_runtime_root preflight smoke ──────────
|
||||
|
||||
describe('create_grok_runtime_root preflight (setup smoke)', () => {
|
||||
test('refuses wipe when required monorepo assets missing', () => {
|
||||
const emptyRepo = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-setup-empty-'));
|
||||
const dest = path.join(emptyRepo, 'install-target', 'gstack');
|
||||
// Seed a live install that must NOT be wiped when preflight fails
|
||||
fs.mkdirSync(dest, { recursive: true });
|
||||
fs.writeFileSync(path.join(dest, 'KEEPME'), 'live');
|
||||
|
||||
const setupPath = path.join(ROOT, 'setup');
|
||||
// Source extracted functions from a temp file — do NOT eval via $(...) because
|
||||
// create_grok_runtime_root uses $$ for the staging suffix and command
|
||||
// substitution would expand it early.
|
||||
const extractScript = path.join(emptyRepo, 'extract-and-run.sh');
|
||||
fs.writeFileSync(
|
||||
extractScript,
|
||||
`#!/usr/bin/env bash
|
||||
set +e
|
||||
SETUP=${JSON.stringify(setupPath)}
|
||||
GSTACK_DIR=${JSON.stringify(emptyRepo)}
|
||||
DEST=${JSON.stringify(dest)}
|
||||
_link_or_copy() { ln -sfn "$1" "$2" 2>/dev/null || cp -R "$1" "$2"; }
|
||||
extract_fn() {
|
||||
local name="$1"
|
||||
local out="$2"
|
||||
awk -v name="$name" '
|
||||
$0 ~ "^" name "\\\\(\\\\)" {grab=1}
|
||||
grab {
|
||||
print
|
||||
for (i=1;i<=length($0);i++) {
|
||||
c=substr($0,i,1)
|
||||
if (c=="{") depth++
|
||||
if (c=="}") {
|
||||
depth--
|
||||
if (depth==0) { exit }
|
||||
}
|
||||
}
|
||||
}
|
||||
' "$SETUP" > "$out"
|
||||
}
|
||||
FN_DIR=$(mktemp -d)
|
||||
extract_fn _grok_link_under_monorepo "$FN_DIR/link.sh"
|
||||
extract_fn create_grok_runtime_root "$FN_DIR/create.sh"
|
||||
# shellcheck source=/dev/null
|
||||
. "$FN_DIR/link.sh"
|
||||
# shellcheck source=/dev/null
|
||||
. "$FN_DIR/create.sh"
|
||||
create_grok_runtime_root "$GSTACK_DIR" "$DEST"
|
||||
rc=$?
|
||||
rm -rf "$FN_DIR"
|
||||
if [ $rc -eq 0 ]; then
|
||||
echo "UNEXPECTED_SUCCESS"
|
||||
exit 2
|
||||
fi
|
||||
echo "PREFLIGHT_FAILED_AS_EXPECTED"
|
||||
if [ -f "$DEST/KEEPME" ]; then
|
||||
echo "LIVE_PRESERVED"
|
||||
exit 0
|
||||
fi
|
||||
echo "LIVE_WIPED"
|
||||
exit 3
|
||||
`,
|
||||
{ mode: 0o755 },
|
||||
);
|
||||
const r = spawnSync('bash', [extractScript], {
|
||||
encoding: 'utf-8',
|
||||
timeout: 15000,
|
||||
});
|
||||
fs.rmSync(emptyRepo, { recursive: true, force: true });
|
||||
|
||||
expect(r.stdout + r.stderr).toMatch(/preflight|required monorepo asset missing/i);
|
||||
expect(r.stdout).toContain('PREFLIGHT_FAILED_AS_EXPECTED');
|
||||
expect(r.stdout).toContain('LIVE_PRESERVED');
|
||||
expect(r.status).toBe(0);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
/**
|
||||
* Grok Build section-pointer mode (Option A token-ceiling fix).
|
||||
*
|
||||
* Carved skills must emit STOP-Read pointers + separate section files under
|
||||
* .grok/skills/gstack-<skill>/sections/, not a monolith that exceeds the
|
||||
* gen-skill-docs soft ceiling (~160KB / ~40k tokens).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import {
|
||||
hostUsesSectionPointers,
|
||||
externalSkillPackageName,
|
||||
sectionPointerPath,
|
||||
SECTION,
|
||||
SECTION_INDEX,
|
||||
} from '../scripts/resolvers/sections';
|
||||
import type { TemplateContext } from '../scripts/resolvers/types';
|
||||
import { HOST_PATHS } from '../scripts/resolvers/types';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
|
||||
function shipCtx(host: 'claude' | 'grok-build' | 'codex'): TemplateContext {
|
||||
return {
|
||||
skillName: 'ship',
|
||||
tmplPath: path.join(ROOT, 'ship', 'SKILL.md.tmpl'),
|
||||
host,
|
||||
paths: HOST_PATHS[host],
|
||||
};
|
||||
}
|
||||
|
||||
describe('hostUsesSectionPointers', () => {
|
||||
test('claude and grok-build use pointers; codex inlines', () => {
|
||||
expect(hostUsesSectionPointers('claude')).toBe(true);
|
||||
expect(hostUsesSectionPointers('grok-build')).toBe(true);
|
||||
expect(hostUsesSectionPointers('codex')).toBe(false);
|
||||
expect(hostUsesSectionPointers('factory')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('sectionPointerPath', () => {
|
||||
test('grok uses flat package under ~/.grok/skills', () => {
|
||||
expect(sectionPointerPath('grok-build', 'ship', 'tests.md', '$GSTACK_ROOT')).toBe(
|
||||
'~/.grok/skills/gstack-ship/sections/tests.md',
|
||||
);
|
||||
expect(externalSkillPackageName('ship')).toBe('gstack-ship');
|
||||
expect(externalSkillPackageName('gstack-upgrade')).toBe('gstack-upgrade');
|
||||
});
|
||||
|
||||
test('claude uses nested skillRoot layout', () => {
|
||||
expect(sectionPointerPath('claude', 'ship', 'tests.md', '~/.claude/skills/gstack')).toBe(
|
||||
'~/.claude/skills/gstack/ship/sections/tests.md',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('SECTION / SECTION_INDEX resolvers', () => {
|
||||
test('grok ship SECTION emits STOP-Read to gstack-ship package path', () => {
|
||||
const out = SECTION(shipCtx('grok-build'), ['tests']);
|
||||
expect(out).toContain('**STOP.**');
|
||||
expect(out).toContain('~/.grok/skills/gstack-ship/sections/tests.md');
|
||||
expect(out).not.toContain('## Test Framework Bootstrap'); // not inlined
|
||||
});
|
||||
|
||||
test('codex ship SECTION still inlines section body', () => {
|
||||
const out = SECTION(shipCtx('codex'), ['tests']);
|
||||
expect(out).not.toContain('**STOP.**');
|
||||
// tests.md.tmpl has substantive body (not a pointer)
|
||||
expect(out.length).toBeGreaterThan(200);
|
||||
});
|
||||
|
||||
test('grok SECTION_INDEX lists full paths; codex SECTION_INDEX is empty', () => {
|
||||
const grok = SECTION_INDEX(shipCtx('grok-build'), ['ship']);
|
||||
expect(grok).toContain('## Section index');
|
||||
expect(grok).toContain('~/.grok/skills/gstack-ship/sections/review-army.md');
|
||||
expect(SECTION_INDEX(shipCtx('codex'), ['ship'])).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('generated grok ship package (when present)', () => {
|
||||
const shipPkg = path.join(ROOT, '.grok', 'skills', 'gstack-ship');
|
||||
const skillMd = path.join(shipPkg, 'SKILL.md');
|
||||
const sectionsDir = path.join(shipPkg, 'sections');
|
||||
|
||||
test('gstack-ship SKILL.md is under soft token ceiling when generated', () => {
|
||||
if (!fs.existsSync(skillMd)) return; // gen not run in this env
|
||||
const bytes = fs.statSync(skillMd).size;
|
||||
// Soft ceiling in gen-skill-docs: 160_000 bytes (~40k tokens)
|
||||
expect(bytes).toBeLessThan(160_000);
|
||||
});
|
||||
|
||||
test('gstack-ship has carved section files + STOP pointers when generated', () => {
|
||||
if (!fs.existsSync(skillMd)) return;
|
||||
const body = fs.readFileSync(skillMd, 'utf-8');
|
||||
expect(body).toContain('**STOP.**');
|
||||
expect(body).toContain('~/.grok/skills/gstack-ship/sections/');
|
||||
// Heavy steps must not be inlined into the skeleton
|
||||
expect(body).not.toMatch(/## Step 9\.1: Review Army/);
|
||||
expect(fs.existsSync(path.join(sectionsDir, 'review-army.md'))).toBe(true);
|
||||
expect(fs.existsSync(path.join(sectionsDir, 'adversarial.md'))).toBe(true);
|
||||
expect(fs.existsSync(path.join(sectionsDir, 'tests.md'))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
|
@ -7,26 +7,29 @@
|
|||
* one. Per-provider auth/timeout/rate-limit errors don't abort the batch.
|
||||
*/
|
||||
|
||||
import type { ProviderAdapter, RunOpts, RunResult } from './providers/types';
|
||||
import type { ProviderAdapter, RunOpts, RunResult, Family } from './providers/types';
|
||||
import { ClaudeAdapter } from './providers/claude';
|
||||
import { GptAdapter } from './providers/gpt';
|
||||
import { GeminiAdapter } from './providers/gemini';
|
||||
import { GrokAdapter } from './providers/grok';
|
||||
|
||||
export type ProviderName = 'claude' | 'gpt' | 'gemini' | 'grok';
|
||||
|
||||
export interface BenchmarkInput {
|
||||
prompt: string;
|
||||
workdir: string;
|
||||
timeoutMs?: number;
|
||||
/** Adapter names to run (e.g., ['claude', 'gpt', 'gemini']). */
|
||||
providers: Array<'claude' | 'gpt' | 'gemini'>;
|
||||
/** Adapter names to run (e.g., ['claude', 'gpt', 'gemini', 'grok']). */
|
||||
providers: ProviderName[];
|
||||
/** Optional per-provider model overrides. */
|
||||
models?: Partial<Record<'claude' | 'gpt' | 'gemini', string>>;
|
||||
models?: Partial<Record<ProviderName, string>>;
|
||||
/** If true, skip providers whose available() returns !ok. If false, include them with error. */
|
||||
skipUnavailable?: boolean;
|
||||
}
|
||||
|
||||
export interface BenchmarkEntry {
|
||||
provider: string;
|
||||
family: 'claude' | 'gpt' | 'gemini';
|
||||
family: Family;
|
||||
available: boolean;
|
||||
unavailable_reason?: string;
|
||||
result?: RunResult;
|
||||
|
|
@ -44,10 +47,11 @@ export interface BenchmarkReport {
|
|||
entries: BenchmarkEntry[];
|
||||
}
|
||||
|
||||
const ADAPTERS: Record<'claude' | 'gpt' | 'gemini', () => ProviderAdapter> = {
|
||||
const ADAPTERS: Record<ProviderName, () => ProviderAdapter> = {
|
||||
claude: () => new ClaudeAdapter(),
|
||||
gpt: () => new GptAdapter(),
|
||||
gemini: () => new GeminiAdapter(),
|
||||
grok: () => new GrokAdapter(),
|
||||
};
|
||||
|
||||
export async function runBenchmark(input: BenchmarkInput): Promise<BenchmarkReport> {
|
||||
|
|
|
|||
|
|
@ -6,9 +6,17 @@
|
|||
* - Anthropic: https://www.anthropic.com/pricing#api
|
||||
* - OpenAI: https://openai.com/api/pricing/
|
||||
* - Google AI: https://ai.google.dev/pricing
|
||||
* - xAI: https://docs.x.ai/developers/pricing
|
||||
*
|
||||
* When a model isn't in the table, estimateCost returns 0 with a console warning.
|
||||
* Prefer adding a new row to the table over guessing.
|
||||
*
|
||||
* xAI rates (official docs, as of 2026-07): Code API grok-build-0.1 $1/$2;
|
||||
* Chat API grok-4.5 $2/$6; grok-4.3 and grok-4.20-* $1.25/$2.50.
|
||||
* Benchmark adapter default id `grok` maps to Code API grok-build-0.1
|
||||
* (Grok Build headless default). Alias `grok-4` maps to Chat mid-tier
|
||||
* grok-4.3 rates. Run cost still needs real token counts from the adapter —
|
||||
* rates alone cannot invent usage.
|
||||
*/
|
||||
|
||||
export interface ModelPricing {
|
||||
|
|
@ -32,6 +40,16 @@ export const PRICING: Record<string, ModelPricing> = {
|
|||
// Google
|
||||
'gemini-2.5-pro': { input_per_mtok: 1.25, output_per_mtok: 5.00, as_of: '2026-04' },
|
||||
'gemini-2.5-flash': { input_per_mtok: 0.30, output_per_mtok: 1.20, as_of: '2026-04' },
|
||||
|
||||
// xAI Grok — https://docs.x.ai/developers/pricing (as of 2026-07)
|
||||
'grok-build-0.1': { input_per_mtok: 1.00, output_per_mtok: 2.00, as_of: '2026-07' },
|
||||
'grok': { input_per_mtok: 1.00, output_per_mtok: 2.00, as_of: '2026-07' }, // alias → Code API
|
||||
'grok-4.5': { input_per_mtok: 2.00, output_per_mtok: 6.00, as_of: '2026-07' },
|
||||
'grok-4.3': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
|
||||
'grok-4': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' }, // alias → Chat mid-tier
|
||||
'grok-4.20-0309-reasoning': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
|
||||
'grok-4.20-0309-non-reasoning': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
|
||||
'grok-4.20-multi-agent-0309': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
|
||||
};
|
||||
|
||||
const WARNED = new Set<string>();
|
||||
|
|
|
|||
|
|
@ -0,0 +1,338 @@
|
|||
import type { ProviderAdapter, RunOpts, RunResult, AvailabilityCheck } from './types';
|
||||
import { estimateCostUsd } from '../pricing';
|
||||
import { execFileSync, spawnSync } from 'child_process';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
|
||||
export type GrokParsedOutput = {
|
||||
output: string;
|
||||
tokens: { input: number; output: number; cached?: number };
|
||||
toolCalls: number;
|
||||
modelUsed?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* Parse Grok headless stdout.
|
||||
*
|
||||
* Characterized 2026-07 against grok 0.2.93:
|
||||
* --output-format json →
|
||||
* { text, stopReason, sessionId, requestId, thought? }
|
||||
* --output-format streaming-json →
|
||||
* NDJSON {type:"thought"|"text"|"end", ...} — end has stopReason/sessionId
|
||||
* Neither shape exposes usage today. When a future CLI adds usage, accept
|
||||
* common field names (input_tokens/output_tokens, prompt_tokens/completion_tokens,
|
||||
* nested usage object). Never invent token counts from prompt/output length.
|
||||
*/
|
||||
export function parseGrokOutput(raw: string): GrokParsedOutput {
|
||||
const trimmed = raw.trim();
|
||||
if (!trimmed) {
|
||||
return { output: '', tokens: { input: 0, output: 0 }, toolCalls: 0 };
|
||||
}
|
||||
|
||||
// Single JSON object (default headless json format)
|
||||
try {
|
||||
const obj = JSON.parse(trimmed);
|
||||
if (obj && typeof obj === 'object' && !Array.isArray(obj)) {
|
||||
return parseGrokJsonObject(obj as Record<string, unknown>);
|
||||
}
|
||||
} catch {
|
||||
// fall through to NDJSON / plain text
|
||||
}
|
||||
|
||||
// streaming-json NDJSON lines
|
||||
if (trimmed.includes('\n') || trimmed.startsWith('{')) {
|
||||
const lines = trimmed.split('\n').map((l) => l.trim()).filter(Boolean);
|
||||
let looksNdjson = 0;
|
||||
let textParts: string[] = [];
|
||||
let tokens = { input: 0, output: 0 } as { input: number; output: number; cached?: number };
|
||||
let toolCalls = 0;
|
||||
let modelUsed: string | undefined;
|
||||
let sawUsage = false;
|
||||
|
||||
for (const line of lines) {
|
||||
try {
|
||||
const ev = JSON.parse(line) as Record<string, unknown>;
|
||||
looksNdjson++;
|
||||
const t = ev.type;
|
||||
if (t === 'text' && typeof ev.data === 'string') {
|
||||
textParts.push(ev.data);
|
||||
} else if (t === 'message' && typeof ev.data === 'string') {
|
||||
textParts.push(ev.data);
|
||||
} else if (typeof ev.text === 'string') {
|
||||
textParts.push(ev.text);
|
||||
}
|
||||
const fromEv = extractUsage(ev);
|
||||
if (fromEv) {
|
||||
tokens = fromEv.tokens;
|
||||
sawUsage = true;
|
||||
if (fromEv.cached !== undefined) tokens = { ...tokens, cached: fromEv.cached };
|
||||
}
|
||||
if (typeof ev.model === 'string') modelUsed = ev.model;
|
||||
if (typeof ev.modelUsed === 'string') modelUsed = ev.modelUsed;
|
||||
if (typeof ev.num_turns === 'number') toolCalls = ev.num_turns;
|
||||
if (typeof ev.tool_calls === 'number') toolCalls = ev.tool_calls;
|
||||
if (typeof ev.toolCallCount === 'number') toolCalls = ev.toolCallCount;
|
||||
} catch {
|
||||
// non-JSON line — ignore for NDJSON path
|
||||
}
|
||||
}
|
||||
|
||||
if (looksNdjson > 0 && (textParts.length > 0 || sawUsage || lines.length === looksNdjson)) {
|
||||
return {
|
||||
output: textParts.join(''),
|
||||
tokens: sawUsage ? tokens : { input: 0, output: 0 },
|
||||
toolCalls,
|
||||
modelUsed,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Plain text fallback — zero tokens (do not estimate from length)
|
||||
return { output: raw, tokens: { input: 0, output: 0 }, toolCalls: 0 };
|
||||
}
|
||||
|
||||
function parseGrokJsonObject(obj: Record<string, unknown>): GrokParsedOutput {
|
||||
// Preferred text fields for Grok Build; also accept Claude-like `result`
|
||||
let output = '';
|
||||
if (typeof obj.text === 'string') output = obj.text;
|
||||
else if (typeof obj.result === 'string') output = obj.result;
|
||||
else if (typeof obj.message === 'string') output = obj.message;
|
||||
else if (obj.result != null) output = String(obj.result);
|
||||
|
||||
const usage = extractUsage(obj);
|
||||
const tokens = usage
|
||||
? { input: usage.tokens.input, output: usage.tokens.output, ...(usage.cached !== undefined ? { cached: usage.cached } : {}) }
|
||||
: { input: 0, output: 0 };
|
||||
|
||||
const toolCalls =
|
||||
(typeof obj.num_turns === 'number' ? obj.num_turns : undefined) ??
|
||||
(typeof obj.tool_calls === 'number' ? obj.tool_calls : undefined) ??
|
||||
(typeof obj.toolCallCount === 'number' ? obj.toolCallCount : undefined) ??
|
||||
0;
|
||||
|
||||
const modelUsed =
|
||||
(typeof obj.model === 'string' ? obj.model : undefined) ??
|
||||
(typeof obj.modelUsed === 'string' ? obj.modelUsed : undefined) ??
|
||||
(typeof obj.model_id === 'string' ? obj.model_id : undefined);
|
||||
|
||||
return { output, tokens, toolCalls, modelUsed };
|
||||
}
|
||||
|
||||
function numField(obj: Record<string, unknown>, ...keys: string[]): number | undefined {
|
||||
for (const k of keys) {
|
||||
const v = obj[k];
|
||||
if (typeof v === 'number' && Number.isFinite(v) && v >= 0) return v;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** Extract token usage from a JSON object if present; otherwise undefined. */
|
||||
function extractUsage(
|
||||
obj: Record<string, unknown>,
|
||||
): { tokens: { input: number; output: number }; cached?: number } | undefined {
|
||||
const nested =
|
||||
obj.usage && typeof obj.usage === 'object' && !Array.isArray(obj.usage)
|
||||
? (obj.usage as Record<string, unknown>)
|
||||
: null;
|
||||
|
||||
const sources = nested ? [nested, obj] : [obj];
|
||||
for (const src of sources) {
|
||||
const input = numField(
|
||||
src,
|
||||
'input_tokens',
|
||||
'prompt_tokens',
|
||||
'inputTokens',
|
||||
'promptTokens',
|
||||
'input_token_count',
|
||||
);
|
||||
const output = numField(
|
||||
src,
|
||||
'output_tokens',
|
||||
'completion_tokens',
|
||||
'outputTokens',
|
||||
'completionTokens',
|
||||
'output_token_count',
|
||||
);
|
||||
if (input !== undefined || output !== undefined) {
|
||||
const cached = numField(
|
||||
src,
|
||||
'cache_read_input_tokens',
|
||||
'cached_prompt_tokens',
|
||||
'cached_tokens',
|
||||
'cached',
|
||||
);
|
||||
return {
|
||||
tokens: { input: input ?? 0, output: output ?? 0 },
|
||||
...(cached !== undefined ? { cached } : {}),
|
||||
};
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Structural auth check only — never log file or env values.
|
||||
* Valid auth.json: non-empty JSON object with ≥1 key (OAuth shapes use URL keys).
|
||||
* Reject: missing, empty/whitespace, invalid JSON, null, arrays, {}.
|
||||
*/
|
||||
export function isStructurallyValidGrokAuthFile(authPath: string): boolean {
|
||||
try {
|
||||
if (!fs.existsSync(authPath)) return false;
|
||||
const raw = fs.readFileSync(authPath, 'utf-8');
|
||||
if (!raw.trim()) return false;
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) return false;
|
||||
return Object.keys(parsed as object).length > 0;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function hasNonBlankEnvKey(): boolean {
|
||||
const xai = process.env.XAI_API_KEY?.trim();
|
||||
const grok = process.env.GROK_API_KEY?.trim();
|
||||
return !!(xai || grok);
|
||||
}
|
||||
|
||||
/**
|
||||
* Grok adapter — wraps the `grok` CLI via -p / --single / --prompt-file.
|
||||
*
|
||||
* Auth readiness: CLI present + (structurally valid ~/.grok/auth.json OR
|
||||
* non-blank XAI_API_KEY / GROK_API_KEY after trim). Never log secret values.
|
||||
* No network probe in available().
|
||||
*/
|
||||
export class GrokAdapter implements ProviderAdapter {
|
||||
readonly name = 'grok';
|
||||
readonly family = 'grok' as const;
|
||||
|
||||
async available(): Promise<AvailabilityCheck> {
|
||||
// Boolean PATH presence only — never log secrets. Bound to ≤2s like peer adapters.
|
||||
const which = spawnSync('sh', ['-c', 'command -v grok'], {
|
||||
timeout: 2000,
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
});
|
||||
let hasBinary = which.status === 0;
|
||||
if (!hasBinary) {
|
||||
try {
|
||||
execFileSync('grok', ['--version'], {
|
||||
encoding: 'utf-8',
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
timeout: 2000,
|
||||
});
|
||||
hasBinary = true;
|
||||
} catch {
|
||||
hasBinary = false;
|
||||
}
|
||||
}
|
||||
if (!hasBinary) {
|
||||
return {
|
||||
ok: false,
|
||||
reason: 'grok CLI not found on PATH. Install Grok Build from xAI, or ensure `grok` is on PATH.',
|
||||
};
|
||||
}
|
||||
|
||||
// Prefer HOME when set so hermetic tests / agent envs can isolate auth
|
||||
// discovery. Bun's os.homedir() ignores process.env.HOME (unlike Node).
|
||||
const home = process.env.HOME || os.homedir();
|
||||
const authPath = path.join(home, '.grok', 'auth.json');
|
||||
const hasValidAuthFile = isStructurallyValidGrokAuthFile(authPath);
|
||||
const hasKey = hasNonBlankEnvKey();
|
||||
if (!hasValidAuthFile && !hasKey) {
|
||||
return {
|
||||
ok: false,
|
||||
reason:
|
||||
'No Grok auth found. Log in via `grok` interactive session (non-empty auth.json), or export a non-blank XAI_API_KEY / GROK_API_KEY.',
|
||||
};
|
||||
}
|
||||
return { ok: true };
|
||||
}
|
||||
|
||||
async run(opts: RunOpts): Promise<RunResult> {
|
||||
const start = Date.now();
|
||||
// Prefer --prompt-file for multi-line / large prompts (ARG_MAX + quoting).
|
||||
// Short single-line prompts use --single to avoid temp files.
|
||||
const useFile =
|
||||
opts.prompt.includes('\n') || opts.prompt.length > 2000 || Buffer.byteLength(opts.prompt, 'utf8') > 2000;
|
||||
let promptFile: string | null = null;
|
||||
let args: string[];
|
||||
if (useFile) {
|
||||
promptFile = path.join(
|
||||
os.tmpdir(),
|
||||
`gstack-grok-bench-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.txt`,
|
||||
);
|
||||
fs.writeFileSync(promptFile, opts.prompt, 'utf-8');
|
||||
args = ['--prompt-file', promptFile, '--cwd', opts.workdir];
|
||||
} else {
|
||||
args = ['--single', opts.prompt];
|
||||
}
|
||||
// Request JSON so we can parse usage when the CLI exposes it (currently often omitted).
|
||||
args.push('--output-format', 'json');
|
||||
if (opts.model) args.push('--model', opts.model);
|
||||
if (opts.extraArgs) args.push(...opts.extraArgs);
|
||||
|
||||
try {
|
||||
const out = execFileSync('grok', args, {
|
||||
cwd: opts.workdir,
|
||||
timeout: opts.timeoutMs,
|
||||
encoding: 'utf-8',
|
||||
maxBuffer: 32 * 1024 * 1024,
|
||||
// Pipe stderr so auth-shaped tokens never inherit onto the parent console
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { ...process.env, GSTACK_HEADLESS: '1' },
|
||||
});
|
||||
const raw = typeof out === 'string' ? out : String(out);
|
||||
const parsed = parseGrokOutput(raw);
|
||||
return {
|
||||
output: parsed.output,
|
||||
tokens: parsed.tokens,
|
||||
durationMs: Date.now() - start,
|
||||
toolCalls: parsed.toolCalls,
|
||||
modelUsed: parsed.modelUsed || opts.model || 'grok',
|
||||
};
|
||||
} catch (err: unknown) {
|
||||
const durationMs = Date.now() - start;
|
||||
const e = err as { code?: string; stderr?: Buffer; signal?: string; message?: string };
|
||||
const stderr = e.stderr?.toString() ?? '';
|
||||
// Never surface raw stderr for auth paths (may contain token-shaped text).
|
||||
const safeSlice = (s: string) => s.replace(/[A-Za-z0-9_\-]{20,}/g, '[redacted]').slice(0, 200);
|
||||
if (e.signal === 'SIGTERM' || e.code === 'ETIMEDOUT') {
|
||||
return this.emptyResult(durationMs, { code: 'timeout', reason: `exceeded ${opts.timeoutMs}ms` }, opts.model);
|
||||
}
|
||||
if (/unauthorized|auth|login|api.?key/i.test(stderr)) {
|
||||
return this.emptyResult(durationMs, { code: 'auth', reason: 'authentication failed (details redacted)' }, opts.model);
|
||||
}
|
||||
if (/rate[- ]?limit|429/i.test(stderr)) {
|
||||
return this.emptyResult(durationMs, { code: 'rate_limit', reason: safeSlice(stderr) }, opts.model);
|
||||
}
|
||||
if (/ENOENT|not found/i.test(e.message ?? '') || e.code === 'ENOENT') {
|
||||
return this.emptyResult(durationMs, { code: 'binary_missing', reason: 'grok CLI not found' }, opts.model);
|
||||
}
|
||||
return this.emptyResult(durationMs, { code: 'unknown', reason: safeSlice(e.message ?? stderr ?? 'unknown') }, opts.model);
|
||||
} finally {
|
||||
if (promptFile) {
|
||||
try {
|
||||
fs.unlinkSync(promptFile);
|
||||
} catch {
|
||||
// best-effort cleanup of temp prompt file
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
estimateCost(tokens: { input: number; output: number; cached?: number }, model?: string): number {
|
||||
return estimateCostUsd(tokens, model ?? 'grok');
|
||||
}
|
||||
|
||||
private emptyResult(durationMs: number, error: RunResult['error'], model?: string): RunResult {
|
||||
return {
|
||||
output: '',
|
||||
tokens: { input: 0, output: 0 },
|
||||
durationMs,
|
||||
toolCalls: 0,
|
||||
modelUsed: model || 'grok',
|
||||
error,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -55,7 +55,7 @@ export interface AvailabilityCheck {
|
|||
reason?: string;
|
||||
}
|
||||
|
||||
export type Family = 'claude' | 'gpt' | 'gemini';
|
||||
export type Family = 'claude' | 'gpt' | 'gemini' | 'grok';
|
||||
|
||||
export interface ProviderAdapter {
|
||||
/** Stable name used in output tables and config (e.g., 'claude', 'gpt', 'gemini'). */
|
||||
|
|
|
|||
|
|
@ -22,16 +22,18 @@ import {
|
|||
slate,
|
||||
cursor,
|
||||
openclaw,
|
||||
grokBuild,
|
||||
} from '../hosts/index';
|
||||
import { HOST_PATHS } from '../scripts/resolvers/types';
|
||||
import { resolveDistBinary } from '../scripts/resolvers/browse';
|
||||
|
||||
const ROOT = path.resolve(import.meta.dir, '..');
|
||||
|
||||
// ─── hosts/index.ts ─────────────────────────────────────────
|
||||
|
||||
describe('hosts/index.ts', () => {
|
||||
test('ALL_HOST_CONFIGS has 10 hosts', () => {
|
||||
expect(ALL_HOST_CONFIGS.length).toBe(10);
|
||||
test('ALL_HOST_CONFIGS has 11 hosts', () => {
|
||||
expect(ALL_HOST_CONFIGS.length).toBe(11);
|
||||
});
|
||||
|
||||
test('ALL_HOST_NAMES matches config names', () => {
|
||||
|
|
@ -409,6 +411,12 @@ describe('golden-file regression', () => {
|
|||
const current = fs.readFileSync(path.join(ROOT, '.factory', 'skills', 'gstack-ship', 'SKILL.md'), 'utf-8');
|
||||
expect(current).toBe(golden);
|
||||
});
|
||||
|
||||
test('Grok Build ship skill matches golden baseline', () => {
|
||||
const golden = fs.readFileSync(path.join(GOLDEN_DIR, 'grok-build-ship-SKILL.md'), 'utf-8');
|
||||
const current = fs.readFileSync(path.join(ROOT, '.grok', 'skills', 'gstack-ship', 'SKILL.md'), 'utf-8');
|
||||
expect(current).toBe(golden);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Individual host config correctness ─────────────────────
|
||||
|
|
@ -535,4 +543,71 @@ describe('host config correctness', () => {
|
|||
expect(config.runtimeRoot.globalSymlinks).toContain('ETHOS.md');
|
||||
}
|
||||
});
|
||||
|
||||
test('grok-build runtime root lists complete packaging assets (U1)', () => {
|
||||
expect(grokBuild.name).toBe('grok-build');
|
||||
expect(grokBuild.usesEnvVars).toBe(true);
|
||||
const links = grokBuild.runtimeRoot.globalSymlinks;
|
||||
for (const asset of [
|
||||
'browse/src',
|
||||
'design/dist',
|
||||
'make-pdf/dist',
|
||||
'extension',
|
||||
'scripts',
|
||||
'review/specialists',
|
||||
]) {
|
||||
expect(links).toContain(asset);
|
||||
}
|
||||
expect(grokBuild.runtimeRoot.globalFiles?.review).toContain('checklist.md');
|
||||
expect(grokBuild.runtimeRoot.globalFiles?.review).toContain('design-checklist.md');
|
||||
expect(grokBuild.generation.skipSkills).toContain('codex');
|
||||
});
|
||||
|
||||
test('grok-build path rewrites suppress Claude host bleed (U3)', () => {
|
||||
expect(grokBuild.pathRewrites.some(r => r.from === 'CLAUDE.md' && r.to === 'AGENTS.md')).toBe(true);
|
||||
expect(grokBuild.pathRewrites.some(r => r.from === 'MODEL_OVERLAY: claude' && r.to === 'MODEL_OVERLAY: none')).toBe(true);
|
||||
expect(grokBuild.toolRewrites?.['AskUserQuestion']).toBe('ask_user_question');
|
||||
});
|
||||
|
||||
// Dual-write parity (#5): hosts/grok-build.ts runtimeRoot must appear in setup's
|
||||
// create_grok_runtime_root so packaging cannot skew silently.
|
||||
test('grok-build runtimeRoot dual-write parity with create_grok_runtime_root', () => {
|
||||
const setupPath = path.join(ROOT, 'setup');
|
||||
const setup = fs.readFileSync(setupPath, 'utf-8');
|
||||
const fnStart = setup.indexOf('create_grok_runtime_root()');
|
||||
expect(fnStart).toBeGreaterThanOrEqual(0);
|
||||
// Function ends at link_grok_skill_dirs (next sibling) — slice that range
|
||||
const fnEnd = setup.indexOf('link_grok_skill_dirs()', fnStart);
|
||||
expect(fnEnd).toBeGreaterThan(fnStart);
|
||||
const fnBody = setup.slice(fnStart, fnEnd);
|
||||
|
||||
for (const link of grokBuild.runtimeRoot.globalSymlinks) {
|
||||
// Each asset path must appear as a monorepo source under create_grok_runtime_root
|
||||
expect(fnBody).toContain(link);
|
||||
}
|
||||
for (const f of grokBuild.runtimeRoot.globalFiles?.review ?? []) {
|
||||
expect(fnBody).toContain(f);
|
||||
}
|
||||
// Preflight + atomic stage (review #6) stay wired
|
||||
expect(fnBody).toContain('preflight');
|
||||
expect(fnBody).toMatch(/\.next\.\$\$|staging/);
|
||||
// Core review files fail-closed (review #7): required=1 on link
|
||||
expect(fnBody).toMatch(
|
||||
/for f in checklist\.md TODOS-format\.md; do[\s\S]*?_grok_link_under_monorepo[^\n]* 1 \|\| return 1/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveDistBinary (U2 double-home fix)', () => {
|
||||
test('env-var browseDir never prefixes $HOME', () => {
|
||||
expect(resolveDistBinary('$GSTACK_BROWSE', 'browse')).toBe('$GSTACK_BROWSE/browse');
|
||||
expect(resolveDistBinary('$GSTACK_DESIGN', 'design')).toBe('$GSTACK_DESIGN/design');
|
||||
expect(resolveDistBinary('$GSTACK_MAKE_PDF', 'pdf')).toBe('$GSTACK_MAKE_PDF/pdf');
|
||||
});
|
||||
|
||||
test('tilde paths keep $HOME prefix', () => {
|
||||
expect(resolveDistBinary('~/.claude/skills/gstack/browse/dist', 'browse')).toBe(
|
||||
'$HOME/.claude/skills/gstack/browse/dist/browse',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -60,6 +60,16 @@ describe('setup: plan-tune hooks are non-interactive-safe', () => {
|
|||
expect(setupSrc).toMatch(/tr '\[:upper:\]' '\[:lower:\]'/);
|
||||
expect(setupSrc).toMatch(/PT_DECISION=\$\(printf .* tr/);
|
||||
});
|
||||
|
||||
test('plan-tune hooks are Claude-host-scoped (skip for --host grok-build)', () => {
|
||||
// Non-Claude host installs must not prompt to mutate ~/.claude/settings.json.
|
||||
// Explicit --plan-tune-hooks still allowed via _PT_EXPLICIT_YES.
|
||||
expect(setupSrc).toContain('_PT_EXPLICIT_YES');
|
||||
expect(setupSrc).toContain('[ "$INSTALL_CLAUDE" -eq 1 ] || [ "$_PT_EXPLICIT_YES" -eq 1 ]');
|
||||
// Codex .agents/ regen is also host-scoped — not always-on during grok-only setup.
|
||||
expect(setupSrc).toContain('[ "$INSTALL_CODEX" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]');
|
||||
expect(setupSrc).toContain('Generating .agents/ skill docs for Codex...');
|
||||
});
|
||||
});
|
||||
|
||||
describe('dev-setup: never silently mutates global settings.json', () => {
|
||||
|
|
|
|||
Loading…
Reference in New Issue