This commit is contained in:
Nehr 2026-07-14 18:26:06 -07:00 committed by GitHub
commit bbfc0969e2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
37 changed files with 3762 additions and 118 deletions

1
.gitignore vendored
View File

@ -19,6 +19,7 @@ bin/gstack-global-discover*
.opencode/
.slate/
.cursor/
.grok/
.openclaw/
.hermes/
.gbrain/

View File

@ -2,6 +2,55 @@
## NEXT PRIORITY
### P2: Grok Build full-compat follow-ups (after `feat/grok-build-host`)
**What:** Finish / harden the Grok-native packaging work landed on
`feat/grok-build-host` (runtime root, SETUP path fix, host-bleed cleanup,
bridges, `bin/gstack-grok-compat-audit`, section pointers for token ceiling).
These are the remaining product/ship items — not blockers for local Grok use.
**Context (landed on branch):**
- U1U3 packaging + U4U7 bridges + U9 audit gate
- Setup host-scoping (no always-on Codex `.agents/` regen / Claude plan-tune
prompt on `--host grok-build`)
- Option A: Grok carved skills use STOP-Read section pointers
(`~/.grok/skills/gstack-*/sections/`) — `/ship` always-loaded ~20k tokens
(was ~45k)
**Follow-ups to track:**
1. **Upstream PR** — open/update PR against `garrytan/gstack` for
`feat/grok-build-host` (VERSION/CHANGELOG/title deferred until local DoD
green on main merge path).
2. **U4 behavioral smoke** — if claiming `/spec --execute` COMPATIBLE (not
file-only), record multi-turn tool-use / cwd isolation / fail-closed auth
smoke or fixture; flag-string greps alone are not enough.
3. **Section pointers for other external hosts (optional)** — Codex ship is
still monolith ~39k (near soft ceiling). Reuse
`hostUsesSectionPointers` for Codex/Cursor/Factory when package paths are
portable (Codex: `.agents/skills/gstack-*` or global
`~/.codex/skills/gstack-*`).
4. **R10 / U8 optional `gstack-codex`** — only if product wants suite
completeness; keep bare `/codex` = OpenAI plugin; deny bare `codex` alias
on Grok install (partial deny already in `link_grok_skill_dirs`).
5. **Gbrain Grok MCP stretch** — optional `~/.grok/config.toml` gbrain stanza
(absolute path, TOML RMW, atomic write + backup; non-interactive refuse
overwrite without `--force`). Success path remains CLI + AGENTS.md.
6. **Audit harness productization** — keep `bin/gstack-grok-compat-audit` as
long-lived multi-host CI later, or document as branch/DoD gate only
(open question from plan ce-doc-review).
7. **Name collision note** — document `/review` vs compound-engineering
`code-review` routing in `~/.grok/AGENTS.md` (do not delete CE skill).
**Why:** Branch is implementation-ready for local Grok; these items close
honesty gaps, upstream ship, and optional suite completeness without
reopening Phase A packaging.
**Depends on / blocked by:** Nothing for local use. Upstream PR waits on
clean validation + VERSION/CHANGELOG if that is the release process.
---
### P1: #1882 — portable skill-install prefix (non-`gstack` install dirs break silently)
**What:** Every generated SKILL.md hardcodes the literal `~/.claude/skills/gstack/...`

View File

@ -20,11 +20,12 @@ allowed-tools:
## When to invoke this skill
Runs the same prompt through Claude,
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
and optionally quality via LLM judge. Answers "which model is actually best
for this skill?" with data instead of vibes. Separate from /benchmark, which
measures web page performance. Use when: "benchmark models", "compare models",
"which model is best for X", "cross-model comparison", "model shootout".
GPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,
cost, and optionally quality via LLM judge. Answers "which model is actually
best for this skill?" with data instead of vibes. Separate from /benchmark,
which measures web page performance. Use when: "benchmark models", "compare
models", "which model is best for X", "cross-model comparison", "model
shootout".
Voice triggers (speech-to-text aliases): "compare models", "model shootout", "which model is best".
@ -578,12 +579,12 @@ If C: ask for the path. Verify it exists. Use as positional argument.
## Step 2: Choose providers
```bash
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini --dry-run
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini,grok --dry-run
```
Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).
If ALL three show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, or `gemini login` / `export GOOGLE_API_KEY`.
If ALL show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, `gemini login` / `export GOOGLE_API_KEY`, or `grok` login / `export XAI_API_KEY`.
If at least one is OK: AskUserQuestion:
- **Simplify:** "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
@ -591,7 +592,8 @@ If at least one is OK: AskUserQuestion:
- **Options:**
- A) All authed providers. Completeness: 10/10.
- B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
- C) Pick two — specify on next turn. Completeness: 8/10.
- C) Only Grok (Grok-only machine). Completeness: 7/10.
- D) Pick two — specify on next turn. Completeness: 8/10.
---
@ -608,7 +610,8 @@ If judge is available, AskUserQuestion:
- A) Enable judge (adds ~$0.05). Completeness: 10/10.
- B) Skip judge — speed/cost/tokens only. Completeness: 7/10.
If judge is NOT available, skip this question and omit the `--judge` flag.
If judge is NOT available (Grok-only / no ANTHROPIC_API_KEY): skip this question
and omit `--judge` (Grok-or-skip). Speed/cost/tokens still work.
---

View File

@ -4,11 +4,12 @@ preamble-tier: 1
version: 1.0.0
description: |
Cross-model benchmark for gstack skills. Runs the same prompt through Claude,
GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,
and optionally quality via LLM judge. Answers "which model is actually best
for this skill?" with data instead of vibes. Separate from /benchmark, which
measures web page performance. Use when: "benchmark models", "compare models",
"which model is best for X", "cross-model comparison", "model shootout". (gstack)
GPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,
cost, and optionally quality via LLM judge. Answers "which model is actually
best for this skill?" with data instead of vibes. Separate from /benchmark,
which measures web page performance. Use when: "benchmark models", "compare
models", "which model is best for X", "cross-model comparison", "model
shootout". (gstack)
voice-triggers:
- "compare models"
- "model shootout"
@ -69,12 +70,12 @@ If C: ask for the path. Verify it exists. Use as positional argument.
## Step 2: Choose providers
```bash
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini --dry-run
"$BIN" --prompt "unused, dry-run" --models claude,gpt,gemini,grok --dry-run
```
Show the dry-run output. The "Adapter availability" section tells the user which providers will actually run (OK) vs skip (NOT READY — remediation hint included).
If ALL three show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, or `gemini login` / `export GOOGLE_API_KEY`.
If ALL show NOT READY: stop with a clear message — benchmark can't run without at least one authed provider. Suggest `claude login`, `codex login`, `gemini login` / `export GOOGLE_API_KEY`, or `grok` login / `export XAI_API_KEY`.
If at least one is OK: AskUserQuestion:
- **Simplify:** "Which models should we include? The dry-run above showed which are authed. Unauthed ones will be skipped cleanly — they won't abort the batch."
@ -82,7 +83,8 @@ If at least one is OK: AskUserQuestion:
- **Options:**
- A) All authed providers. Completeness: 10/10.
- B) Only Claude. Completeness: 6/10 (no cross-model signal — use /ship's review for solo claude benchmarks instead).
- C) Pick two — specify on next turn. Completeness: 8/10.
- C) Only Grok (Grok-only machine). Completeness: 7/10.
- D) Pick two — specify on next turn. Completeness: 8/10.
---
@ -99,7 +101,8 @@ If judge is available, AskUserQuestion:
- A) Enable judge (adds ~$0.05). Completeness: 10/10.
- B) Skip judge — speed/cost/tokens only. Completeness: 7/10.
If judge is NOT available, skip this question and omit the `--judge` flag.
If judge is NOT available (Grok-only / no ANTHROPIC_API_KEY): skip this question
and omit `--judge` (Grok-or-skip). Speed/cost/tokens still work.
---

362
bin/gstack-grok-compat-audit Executable file
View File

@ -0,0 +1,362 @@
#!/usr/bin/env bun
/**
* gstack-grok-compat-audit — Definition of Done gate for Grok Build packaging
* + bridge honesty (plan U9 / R11).
*
* Exit 0 only when packaging checks pass for installed gstack membership
* packages and bridge gates are not theater-greened.
*
* Usage:
* bin/gstack-grok-compat-audit
* bin/gstack-grok-compat-audit --skills-dir ~/.grok/skills
* bin/gstack-grok-compat-audit --phase a # packaging only
* bin/gstack-grok-compat-audit --phase ab # packaging + bridge gates (default)
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { execFileSync } from 'child_process';
const ROOT = path.resolve(import.meta.dir, '..');
const args = process.argv.slice(2);
function arg(name: string, def?: string): string | undefined {
const idx = args.findIndex(a => a === name || a.startsWith(name + '='));
if (idx < 0) return def;
const eq = args[idx].indexOf('=');
if (eq >= 0) return args[idx].slice(eq + 1);
return args[idx + 1] ?? def;
}
const SKILLS_DIR = path.resolve(
(arg('--skills-dir') ?? path.join(os.homedir(), '.grok', 'skills')).replace(/^~/, os.homedir()),
);
const PHASE = (arg('--phase') ?? 'ab').toLowerCase();
const GSTACK_ROOT = path.join(SKILLS_DIR, 'gstack');
type Severity = 'fail' | 'warn' | 'ok';
interface Finding {
severity: Severity;
area: string;
message: string;
}
const findings: Finding[] = [];
function fail(area: string, message: string) {
findings.push({ severity: 'fail', area, message });
}
function warn(area: string, message: string) {
findings.push({ severity: 'warn', area, message });
}
function ok(area: string, message: string) {
findings.push({ severity: 'ok', area, message });
}
function exists(p: string): boolean {
try {
fs.accessSync(p);
return true;
} catch {
return false;
}
}
function readText(p: string): string {
try {
return fs.readFileSync(p, 'utf-8');
} catch {
return '';
}
}
function isGstackMembership(dirName: string): boolean {
if (dirName === 'gstack' || dirName === 'connect-chrome' || dirName === 'browse') return true;
if (dirName.startsWith('gstack-')) return true;
// bare aliases that point at gstack packages (symlink or real SKILL with gstack markers)
const skill = path.join(SKILLS_DIR, dirName, 'SKILL.md');
if (!exists(skill)) return false;
const body = readText(skill);
return body.includes('$GSTACK_ROOT') || body.includes('gstack') || body.includes('AUTO-GENERATED');
}
function listMembershipPackages(): string[] {
if (!exists(SKILLS_DIR)) return [];
return fs
.readdirSync(SKILLS_DIR, { withFileTypes: true })
.filter(d => d.isDirectory() || d.isSymbolicLink())
.map(d => d.name)
.filter(isGstackMembership)
.sort();
}
// ─── Packaging suite (Phase A) ────────────────────────────────
function auditRuntimeRoot() {
const area = 'runtime-root';
if (!exists(GSTACK_ROOT)) {
fail(area, `missing runtime root: ${GSTACK_ROOT}`);
return;
}
const requiredDirs = [
'bin',
'browse/dist',
'browse/src',
'review/specialists',
'scripts',
];
for (const rel of requiredDirs) {
const p = path.join(GSTACK_ROOT, rel);
if (!exists(p)) fail(area, `missing required asset: ${rel}`);
else ok(area, `present: ${rel}`);
}
// Conditional when monorepo had them at install — soft-check if src monorepo has them
const monorepoHas = (rel: string) => exists(path.join(ROOT, rel));
if (monorepoHas('design/dist') && !exists(path.join(GSTACK_ROOT, 'design/dist'))) {
fail(area, 'monorepo has design/dist but runtime root does not');
}
if (monorepoHas('extension') && !exists(path.join(GSTACK_ROOT, 'extension'))) {
fail(area, 'monorepo has extension/ but runtime root does not');
}
if (monorepoHas('make-pdf/dist') && !exists(path.join(GSTACK_ROOT, 'make-pdf/dist'))) {
warn(area, 'monorepo has make-pdf/dist but runtime root does not (membership optional)');
}
for (const f of ['checklist.md', 'TODOS-format.md']) {
const p = path.join(GSTACK_ROOT, 'review', f);
if (!exists(p)) fail(area, `missing review/${f}`);
else ok(area, `present: review/${f}`);
}
}
function auditNoDoubleHome() {
const area = 'no-double-home';
const pkgs = listMembershipPackages();
let hits = 0;
for (const name of pkgs) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
const body = readText(skill);
if (/\$HOME\$GSTACK/.test(body)) {
fail(area, `${name}: contains $HOME$GSTACK (double-home path bug)`);
hits++;
}
}
if (hits === 0) ok(area, `no $HOME$GSTACK in ${pkgs.length} packages`);
}
function auditNoUngeneratedMonorepoPointers() {
const area = 'no-monorepo-skill-pointer';
// Skills under ~/.grok/skills must not be bare monorepo dirs with unrewritten Claude SKILL.md
// Heuristic: ungenerated monorepo package often has MODEL_OVERLAY: claude section + ~/.claude paths without rewrite
const pkgs = listMembershipPackages();
for (const name of pkgs) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
const body = readText(skill);
// Skip runtime root
if (name === 'gstack' && !body.includes('name:')) continue;
if (body.includes('~/.claude/skills/gstack') && !body.includes('$GSTACK_ROOT')) {
// Still has unrewritten Claude global path without env var form
fail(area, `${name}: unrewritten ~/.claude/skills/gstack pointer`);
}
}
ok(area, 'membership packages scanned for unrewritten Claude monorepo pointers');
}
function auditConnectChrome() {
const area = 'connect-chrome';
const link = path.join(SKILLS_DIR, 'connect-chrome');
if (!exists(link)) {
fail(area, 'connect-chrome missing under skills dir');
return;
}
try {
const target = fs.realpathSync(link);
if (!/open-gstack-browser/.test(target) && !/gstack-open-gstack-browser/.test(target)) {
fail(area, `connect-chrome does not resolve to open-gstack-browser (got ${target})`);
} else {
ok(area, `connect-chrome → ${target}`);
}
} catch (e) {
fail(area, `connect-chrome realpath failed: ${(e as Error).message}`);
}
}
function auditHostBleedSample() {
const area = 'host-bleed';
const samples = ['gstack-ship', 'ship', 'gstack-browse', 'browse', 'gstack-claude', 'claude'];
for (const name of samples) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
const body = readText(skill);
if (/MODEL_OVERLAY:\s*claude/.test(body)) {
fail(area, `${name}: MODEL_OVERLAY: claude still present`);
}
if (/## Model-Specific Behavioral Patch \(claude\)/.test(body)) {
fail(area, `${name}: Claude model overlay section body present`);
}
// Claude skill is allowed to mention claude CLI extensively
if (name !== 'claude' && name !== 'gstack-claude') {
if (/\bAskUserQuestion\b/.test(body) && !/ask_user_question/.test(body)) {
// May still appear in examples — soft
warn(area, `${name}: raw AskUserQuestion token (prefer ask_user_question rewrite)`);
}
}
if (name === 'claude' || name === 'gstack-claude') {
if (!/MULTI_CLI_BRIDGE/.test(body) && !/Step 0: Check Claude CLI/.test(body)) {
fail(area, `${name}: missing MULTI_CLI_BRIDGE / Step 0 detect`);
} else {
ok(area, `${name}: Step 0 / multi-CLI packaging present`);
}
}
}
ok(area, 'host-bleed sample scan complete');
}
function auditSkillifySdkPath() {
const area = 'skillify-sdk';
const src = path.join(GSTACK_ROOT, 'browse', 'src', 'browse-client.ts');
if (exists(src)) ok(area, 'browse/src/browse-client.ts resolvable under $GSTACK_ROOT');
else fail(area, 'browse/src/browse-client.ts missing — skillify SDK path broken');
}
// ─── Bridge gates (Phase B) ───────────────────────────────────
function auditBridges() {
// Install prose alone must not claim primary READY when CLI missing
const area = 'bridge-policy';
// Prefer installed runtime root binary (what users run); fall back to monorepo.
const installedBench = path.join(GSTACK_ROOT, 'bin', 'gstack-model-benchmark');
const monorepoBench = path.join(ROOT, 'bin', 'gstack-model-benchmark');
const benchBin = exists(installedBench) ? installedBench : monorepoBench;
if (!exists(installedBench) && exists(monorepoBench)) {
warn(area, 'benchmark: using monorepo binary (installed runtime bin missing) — packaging incomplete?');
}
try {
// No shell: argv list avoids monorepo-path injection.
const help = execFileSync(benchBin, ['--prompt', 'x', '--models', 'grok', '--dry-run'], {
encoding: 'utf-8',
timeout: 15000,
maxBuffer: 2 * 1024 * 1024,
});
if (/unknown provider.*grok/i.test(help)) {
fail(area, 'benchmark: grok still unknown provider');
} else if (/grok:\s*(OK|NOT READY)/i.test(help)) {
ok(area, `benchmark: grok provider wired via ${benchBin} (dry-run READY/NOT READY is boolean auth)`);
} else {
warn(area, `benchmark dry-run unexpected output: ${help.slice(0, 200)}`);
}
} catch (e) {
const msg = (e as { stdout?: string; message?: string }).stdout
?? (e as Error).message
?? String(e);
if (/unknown provider.*grok/i.test(msg)) fail(area, 'benchmark: grok unknown provider');
else warn(area, `benchmark dry-run error: ${String(msg).slice(0, 200)}`);
}
// spec: at least one installed package must exist and use Grok-native spawn
let specSeen = 0;
for (const name of ['gstack-spec', 'spec']) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
specSeen++;
const body = readText(skill);
if (/\$\(cat\s+["']?\$ARCHIVE/.test(body) || /grok[^\n]*\$\(cat/.test(body)) {
fail(area, `${name}: banned $(cat …) into grok argv`);
}
if (/grok --prompt-file/.test(body) || /Spawn \*\*Grok\*\*/.test(body)) {
ok(area, `${name}: Grok-native spawn present`);
} else if (/Spawned:.*claude -p/.test(body) && !/--execute-claude/.test(body)) {
fail(area, `${name}: default execute still only claude -p (no Grok spawn)`);
} else {
fail(area, `${name}: missing researched grok --prompt-file spawn`);
}
}
if (specSeen === 0) {
fail(area, 'spec: neither gstack-spec nor spec package installed under skills dir');
}
// setup-gbrain: must not hard-require which claude for success
let gbrainSeen = 0;
for (const name of ['gstack-setup-gbrain', 'setup-gbrain']) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
gbrainSeen++;
const body = readText(skill);
if (/Grok Build success path/.test(body) || /no Claude required/i.test(body)) {
ok(area, `${name}: Grok success path documented`);
} else {
warn(area, `${name}: missing explicit Grok success path (CLI+AGENTS)`);
}
}
if (gbrainSeen === 0) {
warn(area, 'setup-gbrain: package not installed (bridge not audited)');
}
// pair-agent: local must not hard-require ngrok; remote has install+security+teardown
let pairSeen = 0;
for (const name of ['gstack-pair-agent', 'pair-agent']) {
const skill = path.join(SKILLS_DIR, name, 'SKILL.md');
if (!exists(skill)) continue;
pairSeen++;
const body = readText(skill);
if (/teardown/i.test(body) && /ngrok/i.test(body) && /127\.0\.0\.1|security/i.test(body)) {
ok(area, `${name}: remote ngrok install+security+teardown present`);
} else {
warn(area, `${name}: remote security/teardown checklist incomplete`);
}
if (/DEPENDENT/.test(body)) ok(area, `${name}: honest DEPENDENT labeling`);
}
if (pairSeen === 0) {
warn(area, 'pair-agent: package not installed (bridge not audited)');
}
}
// ─── Main ─────────────────────────────────────────────────────
function main() {
console.log('== gstack-grok-compat-audit ==');
console.log(` skills-dir: ${SKILLS_DIR}`);
console.log(` phase: ${PHASE}`);
console.log(` monorepo: ${ROOT}`);
console.log('');
if (!exists(SKILLS_DIR)) {
fail('install', `skills dir missing: ${SKILLS_DIR} — run: cd ${ROOT} && ./setup --host grok-build`);
} else {
const pkgs = listMembershipPackages();
console.log(` membership packages: ${pkgs.length}`);
auditRuntimeRoot();
auditNoDoubleHome();
auditNoUngeneratedMonorepoPointers();
auditConnectChrome();
auditHostBleedSample();
auditSkillifySdkPath();
if (PHASE === 'ab' || PHASE === 'b' || PHASE === 'full') {
auditBridges();
}
}
const fails = findings.filter(f => f.severity === 'fail');
const warns = findings.filter(f => f.severity === 'warn');
const oks = findings.filter(f => f.severity === 'ok');
for (const f of findings) {
const tag = f.severity === 'fail' ? 'FAIL' : f.severity === 'warn' ? 'WARN' : 'OK ';
console.log(`[${tag}] ${f.area}: ${f.message}`);
}
console.log('');
console.log(`Summary: ${oks.length} ok, ${warns.length} warn, ${fails.length} fail`);
if (fails.length > 0) {
console.log('VERDICT: INCOMPATIBLE (packaging/bridge gate failed)');
process.exit(1);
}
console.log('VERDICT: packaging COMPATIBLE' + (PHASE.includes('b') || PHASE === 'full' ? ' (+ bridge gates honest)' : ''));
process.exit(0);
}
main();

View File

@ -7,7 +7,7 @@
* gstack-model-benchmark <skill-or-prompt-file> [options]
*
* Options:
* --models claude,gpt,gemini Comma-separated provider list (default: claude)
* --models claude,gpt,gemini,grok Comma-separated provider list (default: claude)
* --prompt "<text>" Inline prompt instead of a file
* --workdir <path> Working dir passed to each CLI (default: cwd)
* --timeout-ms <n> Per-provider timeout (default: 300000)
@ -15,27 +15,29 @@
* --skip-unavailable Skip providers that fail available() check
* (default: include them with unavailable marker)
* --judge Run Anthropic SDK judge on outputs for quality score
* (requires ANTHROPIC_API_KEY; adds ~$0.05 per call)
* (requires ANTHROPIC_API_KEY; Grok-only skips judge)
* --dry-run Validate flags + resolve auth, don't invoke providers
*
* Examples:
* gstack-model-benchmark --prompt "Write a haiku about databases" --models claude,gpt
* gstack-model-benchmark ./test-prompt.txt --models claude,gpt,gemini --judge
* gstack-model-benchmark --prompt "hi" --models claude,gpt,gemini --dry-run
* gstack-model-benchmark --prompt "hi" --models grok --dry-run
*/
import '../lib/conductor-env-shim';
import * as fs from 'fs';
import * as path from 'path';
import { runBenchmark, formatTable, formatJson, formatMarkdown, type BenchmarkInput } from '../test/helpers/benchmark-runner';
import { runBenchmark, formatTable, formatJson, formatMarkdown, type BenchmarkInput, type ProviderName } from '../test/helpers/benchmark-runner';
import { ClaudeAdapter } from '../test/helpers/providers/claude';
import { GptAdapter } from '../test/helpers/providers/gpt';
import { GeminiAdapter } from '../test/helpers/providers/gemini';
import { GrokAdapter } from '../test/helpers/providers/grok';
const ADAPTER_FACTORIES = {
const ADAPTER_FACTORIES: Record<ProviderName, () => { name: string; available: () => Promise<{ ok: boolean; reason?: string }> }> = {
claude: () => new ClaudeAdapter(),
gpt: () => new GptAdapter(),
gemini: () => new GeminiAdapter(),
grok: () => new GrokAdapter(),
};
type OutputFormat = 'table' | 'json' | 'markdown';
@ -76,13 +78,13 @@ function positionalArgs(args: string[]): string[] {
return positional;
}
function parseProviders(s: string | undefined): Array<'claude' | 'gpt' | 'gemini'> {
function parseProviders(s: string | undefined): ProviderName[] {
if (!s) return ['claude'];
const seen = new Set<'claude' | 'gpt' | 'gemini'>();
const seen = new Set<ProviderName>();
for (const p of s.split(',').map(x => x.trim()).filter(Boolean)) {
if (p === 'claude' || p === 'gpt' || p === 'gemini') seen.add(p);
if (p === 'claude' || p === 'gpt' || p === 'gemini' || p === 'grok') seen.add(p);
else {
console.error(`WARN: unknown provider '${p}' — skipping. Valid: claude, gpt, gemini.`);
console.error(`WARN: unknown provider '${p}' — skipping. Valid: claude, gpt, gemini, grok.`);
}
}
return seen.size ? Array.from(seen) : ['claude'];
@ -129,11 +131,16 @@ async function main(): Promise<void> {
const report = await runBenchmark(input);
if (doJudge) {
try {
const { judgeEntries } = await import('../test/helpers/benchmark-judge');
await judgeEntries(report);
} catch (err) {
console.error(`WARN: judge unavailable: ${(err as Error).message}`);
// Grok-or-skip: judge requires Anthropic; Grok-only machines skip cleanly
if (!process.env.ANTHROPIC_API_KEY) {
console.error('WARN: judge skipped — ANTHROPIC_API_KEY not set (Grok-or-skip judge).');
} else {
try {
const { judgeEntries } = await import('../test/helpers/benchmark-judge');
await judgeEntries(report);
} catch (err) {
console.error(`WARN: judge unavailable: ${(err as Error).message}`);
}
}
}
@ -149,7 +156,7 @@ async function main(): Promise<void> {
async function dryRunReport(opts: {
prompt: string;
providers: Array<'claude' | 'gpt' | 'gemini'>;
providers: ProviderName[];
workdir: string;
timeoutMs: number;
output: OutputFormat;

View File

@ -30,6 +30,11 @@ modify files.
The generated external invocation name is `gstack-claude`.
**MULTI_CLI_BRIDGE:** Primary modes always call the Claude Code CLI. Packaging and
Step 0 detect/install are first-class on this host; this skill does **not** work
without Claude installed and authenticated. Residual foreign-CLI dependency is
expected and labeled — never claim "works without Claude."
---
## Step 0: Check Claude CLI

113
hosts/grok-build.ts Normal file
View File

@ -0,0 +1,113 @@
import type { HostConfig } from '../scripts/host-config';
/**
* Grok Build (xAI) host based on community PR #2028 (adamouabakar),
* aligned with cursor/factory host patterns and Grok skill discovery.
*
* Skills generate into <repo>/.grok/skills/gstack-* and install under
* ~/.grok/skills/ (flat skill packages + runtime root at ~/.grok/skills/gstack).
*
* Grok discovers user skills from ~/.grok/skills/<name>/SKILL.md.
* Frontmatter `name:` stays unprefixed (browse, ship, ) so slash commands
* remain /browse, /ship, etc. Directory names use gstack- prefix (external hosts).
*/
const grokBuild: HostConfig = {
name: 'grok-build',
displayName: 'Grok Build',
cliCommand: 'grok',
cliAliases: ['grok-build'],
// Relative to $HOME for global install path docs / preamble
globalRoot: '.grok/skills/gstack',
// Project-local runtime root (optional team mode)
localSkillRoot: '.grok/skills/gstack',
// Gitignored generated skill docs live here
hostSubdir: '.grok',
usesEnvVars: true,
frontmatter: {
mode: 'allowlist',
// Keep triggers so Grok model-invocation routing still works
keepFields: ['name', 'description', 'triggers', 'allowed-tools'],
descriptionLimit: null,
},
generation: {
generateMetadata: false,
// Upstream convention: /codex skill is a Claude↔Codex bridge; all external
// hosts skip it (host-config.test.ts). Grok users already have the openai-codex
// plugin; keep /claude as an optional outside-voice skill.
skipSkills: ['codex'],
},
pathRewrites: [
{ from: '~/.claude/skills/gstack', to: '$GSTACK_ROOT' },
{ from: '.claude/skills/gstack', to: '.grok/skills/gstack' },
{ from: '.claude/skills/review', to: '.grok/skills/gstack/review' },
{ from: '.claude/skills', to: '.grok/skills' },
{ from: '~/.claude/skills', to: '~/.grok/skills' },
{ from: 'CLAUDE.md', to: 'AGENTS.md' },
// Defense-in-depth if a Claude model overlay ever slips through (U3 / KTD6)
{ from: 'MODEL_OVERLAY: claude', to: 'MODEL_OVERLAY: none' },
{ from: 'use the Skill tool', to: 'invoke the skill via slash command or skill load' },
],
toolRewrites: {
'use the Bash tool': 'run this command in the shell',
'use the Write tool': 'create this file',
'use the Read tool': 'read the file',
'use the Edit tool': 'edit the file',
'use the Agent tool': 'dispatch a subagent',
'use the Grep tool': 'search for',
'use the Glob tool': 'find files matching',
'use the Skill tool': 'invoke the skill via slash command or skill load',
AskUserQuestion: 'ask_user_question',
ExitPlanMode: 'exit_plan_mode',
},
// Suppress Claude-only outside-voice orchestration that assumes Claude can
// spawn Codex as itself. Keep plan/review skills; they still run on Grok.
suppressedResolvers: [
'GBRAIN_CONTEXT_LOAD',
'GBRAIN_SAVE_RESULTS',
],
// Thin runtime root: every asset skills resolve via $GSTACK_ROOT (R1 / U1).
// Dual-write with setup create_grok_runtime_root — keep lists in sync.
runtimeRoot: {
globalSymlinks: [
'bin',
'browse/dist',
'browse/bin',
'browse/src',
'design/dist',
'make-pdf/dist',
'extension',
'scripts',
'review/specialists',
'gstack-upgrade',
'ETHOS.md',
],
globalFiles: {
review: [
'checklist.md',
'TODOS-format.md',
'design-checklist.md',
'greptile-triage.md',
],
},
},
install: {
prefixable: false,
linkingStrategy: 'symlink-generated',
},
learningsMode: 'basic',
boundaryInstruction:
'IMPORTANT: Prefer ~/.grok/skills/gstack and $GSTACK_ROOT over ~/.claude/skills/gstack. ' +
'Do not assume Claude Code tools (TodoWrite, Skill tool, Claude-in-Chrome MCP). ' +
'Use Grok shell/read/edit/web tools and ask_user_question. Prefer /browse over browser MCPs.',
};
export default grokBuild;

View File

@ -16,9 +16,10 @@ import cursor from './cursor';
import openclaw from './openclaw';
import hermes from './hermes';
import gbrain from './gbrain';
import grokBuild from './grok-build';
/** All registered host configs. Add new hosts here. */
export const ALL_HOST_CONFIGS: HostConfig[] = [claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain];
export const ALL_HOST_CONFIGS: HostConfig[] = [claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain, grokBuild];
/** Map from host name to config. */
export const HOST_CONFIG_MAP: Record<string, HostConfig> = Object.fromEntries(
@ -65,4 +66,4 @@ export function getExternalHosts(): HostConfig[] {
}
// Re-export individual configs for direct import
export { claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain };
export { claude, codex, factory, kiro, opencode, slate, cursor, openclaw, hermes, gbrain, grokBuild };

View File

@ -994,6 +994,27 @@ browser to the internet securely).
STOP here. Wait for the user to install ngrok and re-invoke.
**Bridge status:** Remote pair is **DEPENDENT** on ngrok until the tunnel binary
is installed and authenticated. Local same-machine pair remains zero-extra-dep
and must never hard-require ngrok.
### Remote path security + teardown checklist
Remote pairing expands attack surface vs local pair. Always:
1. **Authenticated tunnel only** — never run an unauthenticated public expose.
2. **Bind local pair endpoint to 127.0.0.1** — do not advertise LAN bind.
3. **Authtoken out-of-band** — user pastes token into `ngrok config add-authtoken`;
never echo the token or tunnel URL into session transcripts beyond what the
CLI already printed for the user to copy.
4. **Teardown after pair** — when pairing ends, stop the ngrok tunnel process
(or tell the user: `pkill -f 'ngrok http'` / dashboard stop). Do not leave
a long-lived public tunnel running.
5. **Verify:** `ngrok config check` after auth; `which ngrok` for install.
If ngrok is missing, report bridge DEPENDENT (install path present) — do **not**
mark the remote primary path READY/COMPATIBLE from install prose alone.
## Step 5: Verify connection
After the user pastes the instructions into the other agent, wait a moment then check:

View File

@ -189,6 +189,27 @@ browser to the internet securely).
STOP here. Wait for the user to install ngrok and re-invoke.
**Bridge status:** Remote pair is **DEPENDENT** on ngrok until the tunnel binary
is installed and authenticated. Local same-machine pair remains zero-extra-dep
and must never hard-require ngrok.
### Remote path security + teardown checklist
Remote pairing expands attack surface vs local pair. Always:
1. **Authenticated tunnel only** — never run an unauthenticated public expose.
2. **Bind local pair endpoint to 127.0.0.1** — do not advertise LAN bind.
3. **Authtoken out-of-band** — user pastes token into `ngrok config add-authtoken`;
never echo the token or tunnel URL into session transcripts beyond what the
CLI already printed for the user to copy.
4. **Teardown after pair** — when pairing ends, stop the ngrok tunnel process
(or tell the user: `pkill -f 'ngrok http'` / dashboard stop). Do not leave
a long-lived public tunnel running.
5. **Verify:** `ngrok config check` after auth; `which ngrok` for install.
If ngrok is missing, report bridge DEPENDENT (install path present) — do **not**
mark the remote primary path READY/COMPATIBLE from install prose alone.
## Step 5: Verify connection
After the user pastes the instructions into the other agent, wait a moment then check:

View File

@ -18,6 +18,7 @@ import * as path from 'path';
import type { Host, TemplateContext } from './resolvers/types';
import { HOST_PATHS, unwrapResolver } from './resolvers/types';
import { RESOLVERS } from './resolvers/index';
import { hostUsesSectionPointers } from './resolvers/sections';
import { externalSkillName, extractHookSafetyProse as _extractHookSafetyProse, extractNameAndDescription as _extractNameAndDescription, condenseOpenAIShortDescription as _condenseOpenAIShortDescription, generateOpenAIYaml as _generateOpenAIYaml } from './resolvers/codex-helpers';
import { generatePlanCompletionAuditShip, generatePlanCompletionAuditReview, generatePlanVerificationExec } from './resolvers/review';
import { ALL_HOST_CONFIGS, ALL_HOST_NAMES, resolveHostArg, getHostConfig } from '../hosts/index';
@ -91,9 +92,11 @@ let HOST: Host = HOST_ARG_VAL === 'all' ? 'claude' : HOST_ARG_VAL;
// ─── Model Overlay Selection ────────────────────────────────
// --model is explicit. We do NOT auto-detect from host (host ≠ model).
// Default is 'claude'. Missing overlay file → empty string (graceful).
// Default is 'claude' for most hosts. Grok Build suppresses the Claude
// model overlay body unless --model is passed explicitly (U3 / KTD6).
import { ALL_MODEL_NAMES, resolveModel, type Model } from './models';
const MODEL_ARG = process.argv.find(a => a.startsWith('--model'));
const MODEL_EXPLICIT = !!MODEL_ARG;
const MODEL_ARG_VAL: Model = (() => {
if (!MODEL_ARG) return 'claude';
const val = MODEL_ARG.includes('=') ? MODEL_ARG.split('=')[1] : process.argv[process.argv.indexOf(MODEL_ARG) + 1];
@ -729,9 +732,12 @@ function buildContext(
const preambleTier = tierMatch ? parseInt(tierMatch[1], 10) : undefined;
const interactiveMatch = tmplContent.match(/^interactive:\s*(true|false)\s*$/m);
const interactive = interactiveMatch ? interactiveMatch[1] === 'true' : undefined;
// Grok-native packaging: no Claude MODEL_OVERLAY section unless user forced --model
const modelForHost: Model | undefined =
host === 'grok-build' && !MODEL_EXPLICIT ? undefined : MODEL_ARG_VAL;
return {
skillName, tmplPath, benefitsFrom, host, paths: HOST_PATHS[host],
preambleTier, model: MODEL_ARG_VAL, interactive, explainLevel: EXPLAIN_LEVEL,
preambleTier, model: modelForHost, interactive, explainLevel: EXPLAIN_LEVEL,
};
}
@ -1019,14 +1025,15 @@ for (const currentHost of hostsToRun) {
}
}
// ─── Section generation (v2 plan T9, Claude-first carve) ───
// On-demand sections/*.md for carved skills. Generated for CLAUDE ONLY:
// every other host inlines section content via the {{SECTION:id}} resolver
// (keeping the full monolith skill), so they need no section files and we
// sidestep host-portable section paths until that plumbing lands. No-op for
// any skill without a sections/ dir. Mirrors the SKILL.md DRY_RUN handling so
// sections participate in the freshness gate.
for (const sec of currentHost === 'claude' ? discoverSectionTemplates(ROOT) : []) {
// ─── Section generation (v2 plan T9 carve) ───
// On-demand sections/*.md for carved skills.
// Pointer hosts (Claude, Grok Build): emit section files next to the skill
// package; {{SECTION:id}} is a STOP-Read pointer (keeps the skeleton small).
// Inline hosts (Codex, Factory, …): {{SECTION:id}} inlines content into the
// monolith SKILL.md — no separate section files.
// No-op for any skill without a sections/ dir. Mirrors the SKILL.md DRY_RUN
// handling so sections participate in the freshness gate.
for (const sec of hostUsesSectionPointers(currentHost) ? discoverSectionTemplates(ROOT) : []) {
if (currentHostConfig.generation.includeSkills?.length &&
!currentHostConfig.generation.includeSkills.includes(sec.skillDir)) continue;
if (currentHostConfig.generation.skipSkills?.length &&

View File

@ -15,7 +15,7 @@
},
"benchmark-models": {
"lead": "Cross-model benchmark for gstack skills.",
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost,\nand optionally quality via LLM judge. Answers \"which model is actually best\nfor this skill?\" with data instead of vibes. Separate from /benchmark, which\nmeasures web page performance. Use when: \"benchmark models\", \"compare models\",\n\"which model is best for X\", \"cross-model comparison\", \"model shootout\".",
"routing": "Runs the same prompt through Claude,\nGPT (via Codex CLI), Gemini, and Grok side-by-side — compares latency, tokens,\ncost, and optionally quality via LLM judge. Answers \"which model is actually\nbest for this skill?\" with data instead of vibes. Separate from /benchmark,\nwhich measures web page performance. Use when: \"benchmark models\", \"compare\nmodels\", \"which model is best for X\", \"cross-model comparison\", \"model\nshootout\".",
"voice_line": "Voice triggers (speech-to-text aliases): \"compare models\", \"model shootout\", \"which model is best\"."
},
"browse": {

View File

@ -99,14 +99,24 @@ export function generateSnapshotFlags(_ctx: TemplateContext): string {
return lines.join('\n');
}
/** Resolve dist binary path: env-var hosts use $GSTACK_* (never $HOME+$GSTACK_*). */
export function resolveDistBinary(dir: string, binary: string): string {
if (dir.startsWith('$')) {
// e.g. $GSTACK_BROWSE already points at .../browse/dist
return `${dir}/${binary}`;
}
return `$HOME${dir.replace(/^~/, '')}/${binary}`;
}
export function generateBrowseSetup(ctx: TemplateContext): string {
const globalBrowse = resolveDistBinary(ctx.paths.browseDir, 'browse');
return `## SETUP (run this check BEFORE any browse command)
\`\`\`bash
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
B=""
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse" ] && B="$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse"
[ -z "$B" ] && B="$HOME${ctx.paths.browseDir.replace(/^~/, '')}/browse"
[ -z "$B" ] && B="${globalBrowse}"
if [ -x "$B" ]; then
echo "READY: $B"
else

View File

@ -1,5 +1,6 @@
import type { TemplateContext } from './types';
import { AI_SLOP_BLACKLIST, OPENAI_HARD_REJECTIONS, OPENAI_LITMUS_CHECKS } from './constants';
import { resolveDistBinary } from './browse';
export function generateDesignReviewLite(ctx: TemplateContext): string {
const litmusList = OPENAI_LITMUS_CHECKS.map((item, i) => `${i + 1}. ${item}`).join(' ');
@ -786,13 +787,15 @@ Source: [OpenAI "Designing Delightful Frontends with GPT-5.4"](https://developer
}
export function generateDesignSetup(ctx: TemplateContext): string {
const globalDesign = resolveDistBinary(ctx.paths.designDir, 'design');
const globalBrowse = resolveDistBinary(ctx.paths.browseDir, 'browse');
return `## DESIGN SETUP (run this check BEFORE any design mockup command)
\`\`\`bash
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
D=""
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design" ] && D="$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design"
[ -z "$D" ] && D="$HOME${ctx.paths.designDir.replace(/^~/, '')}/design"
[ -z "$D" ] && D="${globalDesign}"
if [ -x "$D" ]; then
echo "DESIGN_READY: $D"
else
@ -800,7 +803,7 @@ else
fi
B=""
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse" ] && B="$_ROOT/${ctx.paths.localSkillRoot}/browse/dist/browse"
[ -z "$B" ] && B="$HOME${ctx.paths.browseDir.replace(/^~/, '')}/browse"
[ -z "$B" ] && B="${globalBrowse}"
if [ -x "$B" ]; then
echo "BROWSE_READY: $B"
else
@ -831,13 +834,14 @@ data, not project files. They persist across branches, conversations, and worksp
}
export function generateDesignMockup(ctx: TemplateContext): string {
const globalDesign = resolveDistBinary(ctx.paths.designDir, 'design');
return `## Visual Design Exploration
\`\`\`bash
_ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
D=""
[ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design" ] && D="$_ROOT/${ctx.paths.localSkillRoot}/design/dist/design"
[ -z "$D" ] && D="$HOME${ctx.paths.designDir.replace(/^~/, '')}/design"
[ -z "$D" ] && D="${globalDesign}"
[ -x "$D" ] && echo "DESIGN_READY" || echo "DESIGN_NOT_AVAILABLE"
\`\`\`

View File

@ -36,6 +36,7 @@ import { generateMakePdfSetup } from './make-pdf';
import { generateTasksSectionEmit, generateTasksSectionAggregate } from './tasks-section';
import { SECTION, SECTION_INDEX } from './sections';
import { generateRedactTaxonomyTable, generateRedactInvocationBlock } from './redact-doc';
import { generateSpecSpawn, generateSpecExecuteFlag } from './spec-spawn';
export const RESOLVERS: Record<string, ResolverValue> = {
SLUG_EVAL: generateSlugEval,
@ -100,6 +101,8 @@ export const RESOLVERS: Record<string, ResolverValue> = {
MAKE_PDF_SETUP: generateMakePdfSetup,
TASKS_SECTION_EMIT: generateTasksSectionEmit,
TASKS_SECTION_AGGREGATE: generateTasksSectionAggregate,
SPEC_SPAWN: generateSpecSpawn,
SPEC_EXECUTE_FLAG: generateSpecExecuteFlag,
SECTION,
SECTION_INDEX,
};

View File

@ -1,4 +1,5 @@
import type { TemplateContext } from './types';
import { resolveDistBinary } from './browse';
/**
* {{MAKE_PDF_SETUP}} emits the shell preamble that resolves $P to the
@ -8,10 +9,11 @@ import type { TemplateContext } from './types';
*
* Resolution order (matches src/browseClient.ts::resolveBrowseBin):
* 1. Local skill root: $_ROOT/{localSkillRoot}/make-pdf/dist/pdf
* 2. Global: ~/{globalRoot}/make-pdf/dist/pdf
* 2. Global: $GSTACK_MAKE_PDF/pdf or ~/{globalRoot}/make-pdf/dist/pdf
* 3. Env override (MAKE_PDF_BIN) for contributor dev builds
*/
export function generateMakePdfSetup(ctx: TemplateContext): string {
const globalPdf = resolveDistBinary(ctx.paths.makePdfDir, 'pdf');
return `## MAKE-PDF SETUP (run this check BEFORE any make-pdf command)
\`\`\`bash
@ -19,7 +21,7 @@ _ROOT=$(git rev-parse --show-toplevel 2>/dev/null)
P=""
[ -n "$MAKE_PDF_BIN" ] && [ -x "$MAKE_PDF_BIN" ] && P="$MAKE_PDF_BIN"
[ -z "$P" ] && [ -n "$_ROOT" ] && [ -x "$_ROOT/${ctx.paths.localSkillRoot}/make-pdf/dist/pdf" ] && P="$_ROOT/${ctx.paths.localSkillRoot}/make-pdf/dist/pdf"
[ -z "$P" ] && P="$HOME${ctx.paths.makePdfDir.replace(/^~/, '')}/pdf"
[ -z "$P" ] && P="${globalPdf}"
if [ -x "$P" ]; then
echo "MAKE_PDF_READY: $P"
alias _p_="$P" # shellcheck alias helper (not exported)

View File

@ -10,6 +10,7 @@ GSTACK_ROOT="$HOME/${hostConfig.globalRoot}"
GSTACK_BIN="$GSTACK_ROOT/bin"
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
`
: '';

View File

@ -1,25 +1,28 @@
/**
* Section resolvers (v2 plan T9, Claude-first carve).
* Section resolvers (v2 plan T9 carve).
*
* A carved skill keeps its prose-heavy steps in `<skill>/sections/<id>.md`, read
* on demand. The SAME template ships to every host, so these resolvers make the
* carve host-aware:
*
* - On CLAUDE: {{SECTION:id}} emits a STOP-Read pointer to the generated section
* file (the skeleton), and the section .md is generated + installed separately.
* - On every OTHER host: {{SECTION:id}} INLINES the section template's content,
* so external hosts keep the full monolith ship skill (no section files, no
* host-portable-path problem). Inlined content keeps its own {{RESOLVER}}
* tokens, which the generator's multi-pass resolve expands.
* - On CLAUDE: SECTION:id emits a STOP-Read pointer to the generated section
* file under the nested monorepo install ({skillRoot}/{skill}/sections/).
* - On GROK-BUILD: same pointer mode, but paths use the flat Grok package layout
* (~/.grok/skills/gstack-{skill}/sections/). Section files are generated into
* each package's sections/ dir and ride along with package install.
* - On every OTHER host: SECTION placeholders INLINE the section template content,
* so those hosts keep the full monolith skill (no section files, no
* host-portable-path problem). Inlined content keeps its own resolver tokens,
* which the generator's multi-pass resolve expands.
*
* {{SECTION_INDEX:skill}} renders the situationsection table from the PASSIVE
* manifest on Claude (empty on other hosts they have no sections). The manifest
* SECTION_INDEX renders the situation-to-section table from the PASSIVE
* manifest on pointer hosts (empty when sections are inlined). The manifest
* is the single source of id/file/title/trigger text (CM2; v2_PLAN.md:663).
*/
import * as fs from 'fs';
import * as path from 'path';
import type { ResolverFn, TemplateContext } from './types';
import type { Host, ResolverFn, TemplateContext } from './types';
const ROOT = path.resolve(import.meta.dir, '..', '..');
@ -34,6 +37,21 @@ interface SectionManifest {
sections: SectionEntry[];
}
/** Hosts that load carved sections on demand (not monolith-inline). */
export function hostUsesSectionPointers(host: Host | string): boolean {
return host === 'claude' || host === 'grok-build';
}
/**
* External package dir name for flat hosts (gstack-ship, gstack-upgrade, ).
* Mirrors gen-skill-docs externalSkillName for skill dirs.
*/
export function externalSkillPackageName(skillName: string): string {
if (skillName === '.' || skillName === '' || skillName === 'gstack') return 'gstack';
if (skillName.startsWith('gstack-')) return skillName;
return `gstack-${skillName}`;
}
function loadManifest(skill: string): SectionManifest {
const p = path.join(ROOT, skill, 'sections', 'manifest.json');
const raw = fs.readFileSync(p, 'utf-8');
@ -49,35 +67,56 @@ function findSection(skill: string, id: string): SectionEntry {
}
/**
* {{SECTION:id}} pointer on Claude, inline on other hosts.
* Claude path uses the stable gstack-root install (`{skillRoot}/{skill}/sections/`),
* which always exists, instead of a naked relative path (Codex outside-voice #7).
* Absolute-style path the agent should Read for a section file.
* Claude: nested monorepo install under skillRoot.
* Grok: flat package next to the thin runtime root.
*/
export function sectionPointerPath(
host: Host | string,
skillName: string,
sectionFile: string,
skillRoot: string,
): string {
if (host === 'grok-build') {
const pkg = externalSkillPackageName(skillName);
return `~/.grok/skills/${pkg}/sections/${sectionFile}`;
}
// Claude (and any future nested-install pointer host)
return `${skillRoot}/${skillName}/sections/${sectionFile}`;
}
function stopReadDirective(sectionPath: string, trigger: string): string {
return [
`> **STOP.** Before ${trigger}, Read \`${sectionPath}\` and execute it`,
`> in full. Do not work from memory — that section is the source of truth for this step.`,
].join('\n');
}
/**
* SECTION:id pointer on Claude/Grok, inline on other hosts.
*/
export const SECTION: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
const id = args?.[0];
if (!id) throw new Error('{{SECTION:id}} requires a section id');
const entry = findSection(ctx.skillName, id);
if (ctx.host === 'claude') {
const sectionPath = `${ctx.paths.skillRoot}/${ctx.skillName}/sections/${entry.file}`;
return [
`> **STOP.** Before ${entry.trigger}, Read \`${sectionPath}\` and execute it`,
`> in full. Do not work from memory — that section is the source of truth for this step.`,
].join('\n');
if (hostUsesSectionPointers(ctx.host)) {
const sectionPath = sectionPointerPath(ctx.host, ctx.skillName, entry.file, ctx.paths.skillRoot);
return stopReadDirective(sectionPath, entry.trigger);
}
// Non-Claude hosts inline the section template content (monolith preserved).
// Non-pointer hosts inline the section template content (monolith preserved).
// Inner {{RESOLVER}} tokens are expanded by the generator's multi-pass resolve.
const tmplPath = path.join(ROOT, ctx.skillName, 'sections', `${entry.file}.tmpl`);
return fs.readFileSync(tmplPath, 'utf-8').trimEnd();
};
/**
* {{SECTION_INDEX:skill}} situationsection table from the passive manifest.
* Claude only; other hosts inline everything so an index would be noise.
* SECTION_INDEX situation-to-section table from the passive manifest.
* Pointer hosts only; inline hosts have no separate section files.
*/
export const SECTION_INDEX: ResolverFn = (ctx: TemplateContext, args?: string[]): string => {
if (ctx.host !== 'claude') return '';
if (!hostUsesSectionPointers(ctx.host)) return '';
const skill = args?.[0] ?? ctx.skillName;
const manifest = loadManifest(skill);
const lines: string[] = [
@ -90,7 +129,11 @@ export const SECTION_INDEX: ResolverFn = (ctx: TemplateContext, args?: string[])
'|------|-------------------|',
];
for (const s of manifest.sections) {
lines.push(`| ${s.trigger} | \`sections/${s.file}\` |`);
const sectionPath = sectionPointerPath(ctx.host, skill, s.file, ctx.paths.skillRoot);
// Table shows the resolvable path (Grok: full ~/.grok/...; Claude: short sections/)
const display =
ctx.host === 'grok-build' ? `\`${sectionPath}\`` : `\`sections/${s.file}\``;
lines.push(`| ${s.trigger} | ${display} |`);
}
return lines.join('\n');
};

View File

@ -0,0 +1,118 @@
import type { TemplateContext } from './types';
/**
* Host-native agent spawn for /spec --execute (U4 / R3).
*
* Grok research matrix (local CLI grok 0.2.x):
* -p / --single <PROMPT> short single-turn
* --prompt-file <PATH> single-turn from file (preferred for archives)
* --cwd <CWD> working directory
* --always-approve elevated auto-approve (opt-in / documented)
* Never: $(cat ) into argv (ARG_MAX); never invent --permission-mode acceptEdits.
*
* Claude host generation unchanged: stdin pipe into claude -p.
*/
export function generateSpecSpawn(ctx: TemplateContext): string {
if (ctx.host === 'grok-build') {
return `If A and worktree created: spawn **Grok** headless with the archived
spec as a prompt file (never \`$(cat …)\` into argv — ARG_MAX / quoting risk).
**Auth gate (fail closed):** before spawn, verify Grok is available and configured:
\`\`\`bash
command -v grok >/dev/null 2>&1 || { echo "STOP: grok CLI not on PATH. Install Grok Build, or re-run with --no-execute / --file-only."; exit 1; }
if [ ! -f "$HOME/.grok/auth.json" ] && [ -z "\${XAI_API_KEY:-}\${GROK_API_KEY:-}" ]; then
echo "STOP: Grok not authenticated (no ~/.grok/auth.json and no XAI_API_KEY/GROK_API_KEY). Log in via \`grok\`, or use --no-execute."
exit 1
fi
# ARCHIVE_PATH must stay under SPAWN_PATH or the gstack projects allowlist (fail closed).
if [ ! -f "$ARCHIVE_PATH" ]; then
echo "STOP: ARCHIVE_PATH missing: $ARCHIVE_PATH"; exit 1
fi
if command -v realpath >/dev/null 2>&1; then
ARCHIVE_REAL=$(realpath "$ARCHIVE_PATH")
else
ARCHIVE_REAL=$(cd "$(dirname "$ARCHIVE_PATH")" && pwd -P)/$(basename "$ARCHIVE_PATH")
fi
SPAWN_REAL=$(cd "$SPAWN_PATH" 2>/dev/null && pwd -P || echo "")
if [ -z "$SPAWN_REAL" ]; then
echo "STOP: SPAWN_PATH is not a real directory: $SPAWN_PATH"; exit 1
fi
STATE_PROJECTS="\${GSTACK_STATE_ROOT:-\$HOME/.gstack}/projects"
case "$ARCHIVE_REAL" in
"$STATE_PROJECTS"/*|"$SPAWN_REAL"/*) ;; # allowlisted
*)
echo "STOP: ARCHIVE_PATH realpath not under SPAWN_PATH or allowlisted archive dir ($STATE_PROJECTS)."; exit 1
;;
esac
\`\`\`
**Security:** default spawn does **not** pass \`--always-approve\` (elevated
auto-approve). Only spawn after the user confirmed the D16 gate. If the user
explicitly opts into unattended tool use, append \`--always-approve\` to the
command below never enable it by default. Spec archives must not contain
secrets. Third-party note: archive body is sent to xAI for processing.
\`\`\`bash
# Prefer --prompt-file (researched). Elevated --always-approve is opt-in only.
(cd "$SPAWN_PATH" && grok --prompt-file "$ARCHIVE_PATH" --cwd "$SPAWN_PATH" 2>&1) &
SPAWN_PID=$!
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
echo "Follow with: cd $SPAWN_PATH && grok --continue"
\`\`\`
**Optional Claude execute:** if the user asked for \`--execute-claude\` instead of
default Grok execute, and \`claude\` is on PATH, you may spawn — only after the
same ARCHIVE_PATH allowlist gate as the Grok path (reuse the block above; never
pipe an unallowlisted archive):
\`\`\`bash
# Re-run allowlist (same fail-closed rules as Grok path) before cat|claude.
if [ ! -f "$ARCHIVE_PATH" ]; then
echo "STOP: ARCHIVE_PATH missing: $ARCHIVE_PATH"; exit 1
fi
if command -v realpath >/dev/null 2>&1; then
ARCHIVE_REAL=$(realpath "$ARCHIVE_PATH")
else
ARCHIVE_REAL=$(cd "$(dirname "$ARCHIVE_PATH")" && pwd -P)/$(basename "$ARCHIVE_PATH")
fi
SPAWN_REAL=$(cd "$SPAWN_PATH" 2>/dev/null && pwd -P || echo "")
if [ -z "$SPAWN_REAL" ]; then
echo "STOP: SPAWN_PATH is not a real directory: $SPAWN_PATH"; exit 1
fi
STATE_PROJECTS="\${GSTACK_STATE_ROOT:-\$HOME/.gstack}/projects"
case "$ARCHIVE_REAL" in
"$STATE_PROJECTS"/*|"$SPAWN_REAL"/*) ;; # allowlisted
*)
echo "STOP: ARCHIVE_PATH realpath not under SPAWN_PATH or allowlisted archive dir ($STATE_PROJECTS)."; exit 1
;;
esac
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
\`\`\`
Do **not** silently fall through to Claude when Grok is missing STOP instead.
If no safe multi-line file ingest is available on an older Grok CLI (no
\`--prompt-file\`), demote to \`--no-execute\` / file-only and tell the user:
"This Grok CLI lacks --prompt-file; filed the issue only. Upgrade Grok Build or
use --execute-claude if Claude is installed."`;
}
// Claude + all other hosts: classic claude -p stdin pipe
return `If A and worktree created: spawn \`claude -p\` with the spec piped via stdin:
\`\`\`bash
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
SPAWN_PID=$!
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
echo "Follow with: cd $SPAWN_PATH && claude --resume"
\`\`\``;
}
/** Flag-table row for --execute (host-aware description). */
export function generateSpecExecuteFlag(ctx: TemplateContext): string {
if (ctx.host === 'grok-build') {
return '| `--execute` | conditional default (see Phase 5) | Spawn `grok --prompt-file` headless in a fresh worktree after filing the issue. |';
}
return '| `--execute` | conditional default (see Phase 5) | Spawn `claude -p` in a fresh worktree after filing the issue. |';
}

303
setup
View File

@ -24,6 +24,8 @@ FACTORY_SKILLS="$HOME/.factory/skills"
FACTORY_GSTACK="$FACTORY_SKILLS/gstack"
OPENCODE_SKILLS="$HOME/.config/opencode/skills"
OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"
GROK_SKILLS="$HOME/.grok/skills"
GROK_GSTACK="$GROK_SKILLS/gstack"
IS_WINDOWS=0
case "$(uname -s)" in
@ -85,7 +87,7 @@ NO_TEAM_MODE=0
PLAN_TUNE_HOOKS_MODE="" # "" = resolve from env/config/prompt; "yes"/"no" = explicit
while [ $# -gt 0 ]; do
case "$1" in
--host) [ -z "$2" ] && echo "Missing value for --host (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, or auto)" >&2 && exit 1; HOST="$2"; shift 2 ;;
--host) [ -z "$2" ] && echo "Missing value for --host (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, grok-build, grok, or auto)" >&2 && exit 1; HOST="$2"; shift 2 ;;
--host=*) HOST="${1#--host=}"; shift ;;
--local) LOCAL_INSTALL=1; shift ;;
--prefix) SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1; shift ;;
@ -100,8 +102,11 @@ while [ $# -gt 0 ]; do
esac
done
# Alias: --host grok → grok-build
if [ "$HOST" = "grok" ]; then HOST="grok-build"; fi
case "$HOST" in
claude|codex|kiro|factory|opencode|auto) ;;
claude|codex|kiro|factory|opencode|grok-build|auto) ;;
openclaw)
echo ""
echo "OpenClaw integration uses a different model — OpenClaw spawns Claude Code"
@ -136,7 +141,7 @@ case "$HOST" in
echo "GBrain setup and brain skills ship from the GBrain repo."
echo ""
exit 0 ;;
*) echo "Unknown --host value: $HOST (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, or auto)" >&2; exit 1 ;;
*) echo "Unknown --host value: $HOST (expected claude, codex, kiro, factory, opencode, openclaw, hermes, gbrain, grok-build, grok, or auto)" >&2; exit 1 ;;
esac
# ─── Resolve skill prefix preference ─────────────────────────
@ -200,14 +205,16 @@ INSTALL_CODEX=0
INSTALL_KIRO=0
INSTALL_FACTORY=0
INSTALL_OPENCODE=0
INSTALL_GROK=0
if [ "$HOST" = "auto" ]; then
command -v claude >/dev/null 2>&1 && INSTALL_CLAUDE=1
command -v codex >/dev/null 2>&1 && INSTALL_CODEX=1
command -v kiro-cli >/dev/null 2>&1 && INSTALL_KIRO=1
command -v droid >/dev/null 2>&1 && INSTALL_FACTORY=1
command -v opencode >/dev/null 2>&1 && INSTALL_OPENCODE=1
command -v grok >/dev/null 2>&1 && INSTALL_GROK=1
# If none found, default to claude
if [ "$INSTALL_CLAUDE" -eq 0 ] && [ "$INSTALL_CODEX" -eq 0 ] && [ "$INSTALL_KIRO" -eq 0 ] && [ "$INSTALL_FACTORY" -eq 0 ] && [ "$INSTALL_OPENCODE" -eq 0 ]; then
if [ "$INSTALL_CLAUDE" -eq 0 ] && [ "$INSTALL_CODEX" -eq 0 ] && [ "$INSTALL_KIRO" -eq 0 ] && [ "$INSTALL_FACTORY" -eq 0 ] && [ "$INSTALL_OPENCODE" -eq 0 ] && [ "$INSTALL_GROK" -eq 0 ]; then
INSTALL_CLAUDE=1
fi
elif [ "$HOST" = "claude" ]; then
@ -220,6 +227,8 @@ elif [ "$HOST" = "factory" ]; then
INSTALL_FACTORY=1
elif [ "$HOST" = "opencode" ]; then
INSTALL_OPENCODE=1
elif [ "$HOST" = "grok-build" ]; then
INSTALL_GROK=1
fi
migrate_direct_codex_install() {
@ -438,16 +447,13 @@ if [ ! -x "$BROWSE_BIN" ]; then
exit 1
fi
# 1b. Generate .agents/ Codex skill docs — always regenerate to prevent stale descriptions.
# 1b. Generate .agents/ Codex skill docs — only when installing for Codex.
# .agents/ is no longer committed — generated at setup time from .tmpl templates.
# bun run build already does this, but we need it when NEEDS_BUILD=0 (binary is fresh).
# Always regenerate: generation is fast (<2s) and mtime-based staleness checks are fragile
# (miss stale files when timestamps match after clone/checkout/upgrade).
AGENTS_DIR="$SOURCE_GSTACK_DIR/.agents/skills"
NEEDS_AGENTS_GEN=1
if [ "$NEEDS_AGENTS_GEN" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
log "Generating .agents/ skill docs..."
# Host-scoped: --host grok-build / claude / factory / etc. must not pay Codex regen cost
# or imply a Codex install is in progress.
if [ "$INSTALL_CODEX" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
log "Generating .agents/ skill docs for Codex..."
(
cd "$SOURCE_GSTACK_DIR"
bun_cmd install --frozen-lockfile 2>/dev/null || bun_cmd install
@ -475,6 +481,16 @@ if [ "$INSTALL_OPENCODE" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
)
fi
# 1e. Generate .grok/ Grok Build skill docs
if [ "$INSTALL_GROK" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]; then
log "Generating .grok/ skill docs for Grok Build..."
(
cd "$SOURCE_GSTACK_DIR"
bun_cmd install --frozen-lockfile 2>/dev/null || bun_cmd install
bun_cmd run gen:skill-docs --host grok-build
)
fi
# 2. Ensure Playwright's Chromium is available
if ! ensure_playwright_browser; then
echo "Installing Playwright Chromium..."
@ -972,6 +988,247 @@ link_opencode_skill_dirs() {
fi
}
# ─── Grok Build (xAI) runtime root + skill links ───────────────────────────
# Grok discovers skills from ~/.grok/skills/<pkg>/SKILL.md.
# Generated packages live in <repo>/.grok/skills/gstack-* (path-rewritten).
# Runtime assets (bin, browse, design, specialists, …) live at ~/.grok/skills/gstack.
#
# Security: every required link target must realpath under monorepo install root.
# Dual-write with hosts/grok-build.ts runtimeRoot — keep lists in sync (R1 / U1).
_grok_link_under_monorepo() {
local monorepo_root="$1"
local src="$2"
local dst="$3"
local required="${4:-0}" # 1 = fail closed if missing when expected
if [ ! -e "$src" ]; then
if [ "$required" = "1" ]; then
echo "error: required Grok runtime asset missing: $src" >&2
echo " monorepo root: $monorepo_root" >&2
echo " Re-run from a full gstack checkout after builds complete." >&2
return 1
fi
return 0
fi
local src_real monorepo_real
monorepo_real=$(cd "$monorepo_root" 2>/dev/null && pwd -P)
# Fail closed: empty monorepo_real makes "$monorepo_real"/* expand to /* and
# match every absolute path — never allow that.
if [ -z "$monorepo_real" ] || [ ! -d "$monorepo_real" ]; then
echo "error: refusing Grok runtime link — monorepo root unresolved: $monorepo_root" >&2
return 1
fi
# Prefer full realpath when available so a final-component symlink cannot escape.
if command -v realpath >/dev/null 2>&1; then
src_real=$(realpath "$src" 2>/dev/null) || src_real=""
else
src_real=$(cd "$(dirname "$src")" 2>/dev/null && pwd -P)/$(basename "$src")
fi
if [ -z "$src_real" ]; then
echo "error: refusing Grok runtime link — cannot realpath src: $src" >&2
return 1
fi
case "$src_real" in
"$monorepo_real"|"$monorepo_real"/*) ;;
*)
echo "error: refusing Grok runtime link outside monorepo:" >&2
echo " src=$src_real" >&2
echo " monorepo=$monorepo_real" >&2
return 1
;;
esac
_link_or_copy "$src" "$dst"
}
create_grok_runtime_root() {
local gstack_dir="$1"
local grok_gstack="$2"
local generated_root="$gstack_dir/.grok/skills/gstack"
local staging=""
local req
# Preflight required monorepo assets BEFORE wiping the live install.
# Without this, a mid-install failure under set -e leaves an empty/half tree
# after rm -rf (review finding #6).
for req in bin browse/dist browse/src scripts review/specialists; do
if [ ! -e "$gstack_dir/$req" ]; then
echo "error: preflight — required monorepo asset missing: $gstack_dir/$req" >&2
echo " Run a full monorepo build (./setup or bun run build) before installing Grok runtime." >&2
return 1
fi
done
# Core review files audited by gstack-grok-compat-audit — fail closed when monorepo has review/
if [ -d "$gstack_dir/review" ]; then
for req in checklist.md TODOS-format.md; do
if [ ! -f "$gstack_dir/review/$req" ]; then
echo "error: preflight — core review file missing: $gstack_dir/review/$req" >&2
return 1
fi
done
fi
# design/dist required when monorepo ships the design package
if [ -d "$gstack_dir/design" ] && [ ! -d "$gstack_dir/design/dist" ]; then
echo "error: preflight — monorepo has design/ but design/dist is missing — run design build" >&2
return 1
fi
# Stage into a sibling .next tree, then atomic rename into place so a partial
# link failure never leaves the live ~/.grok/skills/gstack empty.
staging="${grok_gstack}.next.$$"
rm -rf "$staging"
mkdir -p "$staging" \
"$staging/browse" \
"$staging/design" \
"$staging/make-pdf" \
"$staging/gstack-upgrade" \
"$staging/review" || return 1
# Cleanup staging on any failure from here until atomic swap
# shellcheck disable=SC2064
trap 'rm -rf "$staging"' RETURN
if [ -f "$generated_root/SKILL.md" ]; then
_link_or_copy "$generated_root/SKILL.md" "$staging/SKILL.md"
elif [ -f "$gstack_dir/SKILL.md" ]; then
_link_or_copy "$gstack_dir/SKILL.md" "$staging/SKILL.md"
fi
# Required core assets (fail closed) — dual-write with hosts/grok-build.ts runtimeRoot
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/bin" "$staging/bin" 1 || return 1
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/dist" "$staging/browse/dist" 1 || return 1
if [ -d "$gstack_dir/browse/bin" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/bin" "$staging/browse/bin" 0 || return 1
fi
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/browse/src" "$staging/browse/src" 1 || return 1
if [ -d "$gstack_dir/design/dist" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/design/dist" "$staging/design/dist" 1 || return 1
fi
# make-pdf/dist — link when present (membership skill)
if [ -d "$gstack_dir/make-pdf/dist" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/make-pdf/dist" "$staging/make-pdf/dist" 0 || return 1
fi
# extension/ — required when monorepo has it
if [ -d "$gstack_dir/extension" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/extension" "$staging/extension" 1 || return 1
fi
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/scripts" "$staging/scripts" 1 || return 1
# Core review files — required=1 so install policy matches gstack-grok-compat-audit (#7)
for f in checklist.md TODOS-format.md; do
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/$f" "$staging/review/$f" 1 || return 1
done
# Optional review assets (soft when monorepo lacks them)
for f in design-checklist.md greptile-triage.md; do
if [ -f "$gstack_dir/review/$f" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/$f" "$staging/review/$f" 0 || return 1
fi
done
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/review/specialists" "$staging/review/specialists" 1 || return 1
if [ -f "$gstack_dir/gstack-upgrade/SKILL.md" ] || [ -f "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" ]; then
if [ -f "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" ]; then
_link_or_copy "$gstack_dir/.grok/skills/gstack-upgrade/SKILL.md" "$staging/gstack-upgrade/SKILL.md"
else
_link_or_copy "$gstack_dir/gstack-upgrade/SKILL.md" "$staging/gstack-upgrade/SKILL.md"
fi
fi
if [ -f "$gstack_dir/ETHOS.md" ]; then
_grok_link_under_monorepo "$gstack_dir" "$gstack_dir/ETHOS.md" "$staging/ETHOS.md" 0 || return 1
fi
# Atomic swap: only wipe live install after staging is fully linked
trap - RETURN
if [ -L "$grok_gstack" ]; then
rm -f "$grok_gstack"
elif [ -d "$grok_gstack" ] && [ "$grok_gstack" != "$gstack_dir" ]; then
rm -rf "$grok_gstack"
fi
# Prefer rename; fall back to mv -T when available for non-empty dest edge cases
if ! mv "$staging" "$grok_gstack" 2>/dev/null; then
echo "error: failed to promote staged Grok runtime root to $grok_gstack" >&2
rm -rf "$staging"
return 1
fi
}
link_grok_skill_dirs() {
local gstack_dir="$1"
local skills_dir="$2"
local grok_dir="$gstack_dir/.grok/skills"
local linked=()
if [ ! -d "$grok_dir" ]; then
echo " Generating .grok/ skill docs..."
( cd "$gstack_dir" && bun run gen:skill-docs --host grok-build )
fi
if [ ! -d "$grok_dir" ]; then
echo " warning: .grok/skills/ generation failed — run 'bun run gen:skill-docs --host grok-build' manually" >&2
return 1
fi
mkdir -p "$skills_dir"
for skill_dir in "$grok_dir"/gstack*/; do
if [ -f "$skill_dir/SKILL.md" ]; then
skill_name="$(basename "$skill_dir")"
# Runtime root is installed separately as ~/.grok/skills/gstack
[ "$skill_name" = "gstack" ] && continue
target="$skills_dir/$skill_name"
# Replace stale monorepo flat symlinks (browse → ~/gstack/browse) with generated packages
if [ -L "$target" ] || [ ! -e "$target" ]; then
_link_or_copy "$skill_dir" "$target"
linked+=("$skill_name")
elif [ -d "$target" ] && [ ! -f "$target/SKILL.md" ]; then
rm -rf "$target"
_link_or_copy "$skill_dir" "$target"
linked+=("$skill_name")
else
# Force refresh: generated content is authoritative for this host
rm -rf "$target"
_link_or_copy "$skill_dir" "$target"
linked+=("$skill_name")
fi
# Also expose unprefixed alias (browse) so slash names match frontmatter
# without depending on Grok's name: field alone.
# Deny bare `codex` alias (OpenAI plugin owns /codex); optional gstack-codex only.
bare_name="${skill_name#gstack-}"
if [ -n "$bare_name" ] && [ "$bare_name" != "$skill_name" ] && [ "$bare_name" != "codex" ]; then
bare_target="$skills_dir/$bare_name"
# Only alias if missing or already a symlink (don't clobber real non-gstack skills)
if [ -L "$bare_target" ] || [ ! -e "$bare_target" ]; then
_link_or_copy "$skill_dir" "$bare_target"
fi
fi
fi
done
# Backwards-compat alias: /connect-chrome → generated open-gstack-browser (U2 / KTD7)
# Force-refresh stale monorepo targets that pointed at source tree packages.
local _ogb_src=""
if [ -d "$skills_dir/gstack-open-gstack-browser" ]; then
_ogb_src="$skills_dir/gstack-open-gstack-browser"
elif [ -d "$grok_dir/gstack-open-gstack-browser" ]; then
_ogb_src="$grok_dir/gstack-open-gstack-browser"
fi
if [ -n "$_ogb_src" ]; then
rm -rf "$skills_dir/connect-chrome"
_link_or_copy "$_ogb_src" "$skills_dir/connect-chrome"
linked+=("connect-chrome→open-gstack-browser")
fi
if [ ${#linked[@]} -gt 0 ]; then
echo " linked skills: ${linked[*]}"
fi
}
# 4. Install for Claude (default)
SKILLS_BASENAME="$(basename "$INSTALL_SKILLS_DIR")"
SKILLS_PARENT_BASENAME="$(basename "$(dirname "$INSTALL_SKILLS_DIR")")"
@ -1193,6 +1450,17 @@ if [ "$INSTALL_OPENCODE" -eq 1 ]; then
echo " opencode skills: $OPENCODE_SKILLS"
fi
# 6d. Install for Grok Build (xAI)
if [ "$INSTALL_GROK" -eq 1 ]; then
mkdir -p "$GROK_SKILLS"
create_grok_runtime_root "$SOURCE_GSTACK_DIR" "$GROK_GSTACK"
link_grok_skill_dirs "$SOURCE_GSTACK_DIR" "$GROK_SKILLS"
echo "gstack ready (grok-build)."
echo " browse: $BROWSE_BIN"
echo " grok skills: $GROK_SKILLS"
echo " runtime root: $GROK_GSTACK"
fi
# 7. Create .agents/ sidecar symlinks for the real Codex skill target.
# The root Codex skill ends up pointing at $SOURCE_GSTACK_DIR/.agents/skills/gstack,
# so the runtime assets must live there for both global and repo-local installs.
@ -1337,6 +1605,10 @@ fi
# something at runtime instead of being agent-convention. Explicit consent UX
# per D4 + Codex: never mutate settings.json silently.
#
# Host-scoped: these hooks only affect Claude Code. Skip the prompt entirely for
# non-Claude host installs (e.g. --host grok-build) unless the user explicitly
# passed --plan-tune-hooks. Grok /plan-tune still works via skill-text path.
#
# Idempotent via _gstack_source tag = 'plan-tune-cathedral'. If both hooks
# already registered under that tag, the install is a no-op (no prompt).
PLAN_TUNE_LOG_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/question-log-hook"
@ -1344,7 +1616,14 @@ PLAN_TUNE_PREF_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/question-preference-h
AUQ_ERROR_FALLBACK_HOOK="$SOURCE_GSTACK_DIR/hosts/claude/hooks/auq-error-fallback-hook"
PLAN_TUNE_INSTALL_MARKER="$HOME/.gstack/.plan-tune-hooks-prompted"
# Explicit --plan-tune-hooks still allowed from any host install (user intent).
_PT_EXPLICIT_YES=0
case "$(printf '%s' "${PLAN_TUNE_HOOKS_MODE:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')" in
y|yes|true|install|on|1) _PT_EXPLICIT_YES=1 ;;
esac
if [ "$NO_TEAM_MODE" -ne 1 ] \
&& { [ "$INSTALL_CLAUDE" -eq 1 ] || [ "$_PT_EXPLICIT_YES" -eq 1 ]; } \
&& [ -x "$SETTINGS_HOOK" ] \
&& [ -x "$PLAN_TUNE_LOG_HOOK" ] \
&& [ -x "$PLAN_TUNE_PREF_HOOK" ]; then

View File

@ -788,8 +788,22 @@ Claude Code) can call it as both a CLI and an MCP tool.
**Scope honesty:** This skill's MCP registration step (5a) uses
`claude mcp add` and targets Claude Code specifically. Other local hosts
(Cursor, Codex CLI, etc.) will still get the gbrain CLI on PATH — they can
register `gbrain serve` in their own MCP config manually after setup.
(Cursor, Codex CLI, **Grok Build**, etc.) will still get the gbrain CLI on
PATH — they can register `gbrain serve` in their own MCP config manually
after setup.
**Grok Build success path (no Claude required):** setup is **COMPATIBLE** when
(1) `gbrain` is on PATH, (2) the engine is healthy (`gbrain doctor`), and
(3) AGENTS.md (or project docs) carry gbrain guidance via `/sync-gbrain`.
Claude MCP registration is optional. Never mark setup FAILED only because
`claude` is missing. Optional Grok MCP into `~/.grok/config.toml` is a stretch
(absolute path, structured TOML RMW, gbrain stanza only, atomic write +
backup; non-interactive refuses non-equivalent overwrite unless `--force` or
a real TTY confirm).
**Bridge policy:** install prose alone does not green the primary path when
the CLI is missing — detect/install guidance may be READY; overall bridge is
DEPENDENT until `gbrain` is present.
**Audience:** local-Mac users. openclaw/hermes agents typically run in cloud
docker containers with their own gbrain; "sharing" a brain between them and
@ -1258,9 +1272,14 @@ doctor output and STOP.
---
## Step 5a: Register gbrain as Claude Code MCP (D18)
## Step 5a: Register gbrain as Claude Code MCP (D18) — optional
Only if `which claude` resolves. Ask: "Give Claude Code a typed tool surface
Only if `which claude` resolves. If `claude` is **not** on PATH (typical on
Grok-only machines): emit "MCP registration skipped — Claude Code not present.
Grok/other hosts succeed via CLI + AGENTS.md; register `gbrain serve` in host
MCP config only if desired." Continue to step 6 — **do not fail setup**.
If `claude` is present, ask: "Give Claude Code a typed tool surface
for gbrain? (recommended yes)"
The registration form depends on the path picked in Step 2:

View File

@ -34,8 +34,22 @@ Claude Code) can call it as both a CLI and an MCP tool.
**Scope honesty:** This skill's MCP registration step (5a) uses
`claude mcp add` and targets Claude Code specifically. Other local hosts
(Cursor, Codex CLI, etc.) will still get the gbrain CLI on PATH — they can
register `gbrain serve` in their own MCP config manually after setup.
(Cursor, Codex CLI, **Grok Build**, etc.) will still get the gbrain CLI on
PATH — they can register `gbrain serve` in their own MCP config manually
after setup.
**Grok Build success path (no Claude required):** setup is **COMPATIBLE** when
(1) `gbrain` is on PATH, (2) the engine is healthy (`gbrain doctor`), and
(3) AGENTS.md (or project docs) carry gbrain guidance via `/sync-gbrain`.
Claude MCP registration is optional. Never mark setup FAILED only because
`claude` is missing. Optional Grok MCP into `~/.grok/config.toml` is a stretch
(absolute path, structured TOML RMW, gbrain stanza only, atomic write +
backup; non-interactive refuses non-equivalent overwrite unless `--force` or
a real TTY confirm).
**Bridge policy:** install prose alone does not green the primary path when
the CLI is missing — detect/install guidance may be READY; overall bridge is
DEPENDENT until `gbrain` is present.
**Audience:** local-Mac users. openclaw/hermes agents typically run in cloud
docker containers with their own gbrain; "sharing" a brain between them and
@ -504,9 +518,14 @@ doctor output and STOP.
---
## Step 5a: Register gbrain as Claude Code MCP (D18)
## Step 5a: Register gbrain as Claude Code MCP (D18) — optional
Only if `which claude` resolves. Ask: "Give Claude Code a typed tool surface
Only if `which claude` resolves. If `claude` is **not** on PATH (typical on
Grok-only machines): emit "MCP registration skipped — Claude Code not present.
Grok/other hosts succeed via CLI + AGENTS.md; register `gbrain serve` in host
MCP config only if desired." Continue to step 6 — **do not fail setup**.
If `claude` is present, ask: "Give Claude Code a typed tool surface
for gbrain? (recommended yes)"
The registration form depends on the path picked in Step 2:

View File

@ -60,7 +60,7 @@ separated tokens starting with `--`. Last flag wins on conflict.
| `--no-dedupe` | — | Skip the dedupe check. |
| `--no-gate` | OFF (gate is ON) | Skip the codex quality-score gate between Phase 4 and Phase 5. **Redaction (Phase 4.5a semantic + 4.5b regex) still runs — there is no flag that disables it.** |
| `--audit` | OFF | Route Phase 5 to the Audit/Cleanup template (instead of Standard). |
| `--execute` | conditional default (see Phase 5) | Spawn `claude -p` in a fresh worktree after filing the issue. |
{{SPEC_EXECUTE_FLAG}}
| `--no-execute` | — | File issue only; do NOT spawn agent (alias: `--file-only`). |
| `--file-only` | — | Same as `--no-execute`. |
| `--plan-file <path>` | inferred from harness | Load the spec into the specified plan file instead of inferring. |
@ -447,14 +447,7 @@ git worktree add "$SPAWN_PATH" -b "$SPAWN_BRANCH" "$PIN_SHA" 2>&1
in-progress changes will be visible to the agent. Cancel with Ctrl+C if not
desired." Then fall back to current dir (still spawn).
If A and worktree created: spawn `claude -p` with the spec piped via stdin:
```bash
cat "$ARCHIVE_PATH" | (cd "$SPAWN_PATH" && claude -p 2>&1) &
SPAWN_PID=$!
echo "Spawned: PID $SPAWN_PID in $SPAWN_PATH (branch $SPAWN_BRANCH)"
echo "Follow with: cd $SPAWN_PATH && claude --resume"
```
{{SPEC_SPAWN}}
Update archive frontmatter with `spec_worktree_path: $SPAWN_PATH` and
`spec_executed: true` (atomic re-write).

View File

@ -785,8 +785,12 @@ Skills that run plan reviews (`/plan-*-review`, `/codex review`) include the EXI
You are running the canonical "keep this brain up to date" verb. /setup-gbrain
installs gbrain once; /sync-gbrain runs every time the user wants the brain
refreshed against this repo's current state, and refreshes the agent-side
guidance in CLAUDE.md so the coding agent knows when to prefer `gbrain`
search over Grep.
guidance in CLAUDE.md / AGENTS.md so the coding agent knows when to prefer
`gbrain` search over Grep.
**Grok Build:** success does not require Claude MCP. Require `gbrain` on PATH +
engine healthy + AGENTS.md guidance refresh. If gbrain is absent, give install
steps (DEPENDENT bridge) — do not silent-fail or demand `which claude`.
**Architecture (post-codex review):** This skill uses gbrain v0.20.0+'s
**native code surfaces** (`gbrain sources add`, `gbrain sync --strategy code`,

View File

@ -31,8 +31,12 @@ allowed-tools:
You are running the canonical "keep this brain up to date" verb. /setup-gbrain
installs gbrain once; /sync-gbrain runs every time the user wants the brain
refreshed against this repo's current state, and refreshes the agent-side
guidance in CLAUDE.md so the coding agent knows when to prefer `gbrain`
search over Grep.
guidance in CLAUDE.md / AGENTS.md so the coding agent knows when to prefer
`gbrain` search over Grep.
**Grok Build:** success does not require Claude MCP. Require `gbrain` on PATH +
engine healthy + AGENTS.md guidance refresh. If gbrain is absent, give install
steps (DEPENDENT bridge) — do not silent-fail or demand `which claude`.
**Architecture (post-codex review):** This skill uses gbrain v0.20.0+'s
**native code surfaces** (`gbrain sources add`, `gbrain sync --strategy code`,

View File

@ -19,6 +19,7 @@ GSTACK_ROOT="$HOME/.codex/skills/gstack"
GSTACK_BIN="$GSTACK_ROOT/bin"
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
_UPD=$($GSTACK_BIN/gstack-update-check 2>/dev/null || .agents/skills/gstack/bin/gstack-update-check 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD" || true
mkdir -p ~/.gstack/sessions

View File

@ -21,6 +21,7 @@ GSTACK_ROOT="$HOME/.factory/skills/gstack"
GSTACK_BIN="$GSTACK_ROOT/bin"
GSTACK_BROWSE="$GSTACK_ROOT/browse/dist"
GSTACK_DESIGN="$GSTACK_ROOT/design/dist"
GSTACK_MAKE_PDF="$GSTACK_ROOT/make-pdf/dist"
_UPD=$($GSTACK_BIN/gstack-update-check 2>/dev/null || .factory/skills/gstack/bin/gstack-update-check 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD" || true
mkdir -p ~/.gstack/sessions

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,600 @@
/**
* Behavioral unit tests for Grok Build packaging surfaces (code-review #8):
* - generateSpecSpawn fail-closed allowlist + no default --always-approve
* - GrokAdapter available timeout / --prompt-file path / auth redaction
* - gstack-grok-compat-audit exit codes against fixture skills dirs
*
* Free (no API spend). Does not invoke real grok CLI for successful runs.
*/
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { generateSpecSpawn, generateSpecExecuteFlag } from '../scripts/resolvers/spec-spawn';
import type { TemplateContext } from '../scripts/resolvers/types';
import { HOST_PATHS } from '../scripts/resolvers/types';
import { GrokAdapter, parseGrokOutput, isStructurallyValidGrokAuthFile } from './helpers/providers/grok';
import { estimateCostUsd, PRICING } from './helpers/pricing';
const ROOT = path.resolve(import.meta.dir, '..');
const AUDIT_BIN = path.join(ROOT, 'bin', 'gstack-grok-compat-audit');
function makeCtx(host: 'grok-build' | 'claude'): TemplateContext {
return {
skillName: 'spec',
tmplPath: path.join(ROOT, 'spec', 'SKILL.md.tmpl'),
host,
paths: HOST_PATHS[host],
};
}
// ─── generateSpecSpawn ───────────────────────────────────────
describe('generateSpecSpawn (Grok fail-closed)', () => {
test('grok-build emits auth gate, ARCHIVE allowlist, and no default --always-approve', () => {
const out = generateSpecSpawn(makeCtx('grok-build'));
expect(out).toContain('command -v grok');
expect(out).toContain('ARCHIVE_PATH');
expect(out).toContain('SPAWN_PATH');
expect(out).toContain('GSTACK_STATE_ROOT');
expect(out).toContain('realpath');
expect(out).toContain('fail closed');
expect(out).toContain('grok --prompt-file');
// Default spawn must NOT enable elevated auto-approve
expect(out).not.toMatch(/grok --prompt-file[^\n]*--always-approve/);
expect(out).toMatch(/opt-in only|does \*\*not\*\* pass `--always-approve`|--always-approve` is opt-in/i);
});
test('grok-build --execute-claude path reuses ARCHIVE allowlist before cat|claude', () => {
const out = generateSpecSpawn(makeCtx('grok-build'));
expect(out).toContain('--execute-claude');
// Claude branch must re-check allowlist, not bare cat only
const claudeIdx = out.indexOf('--execute-claude');
const after = out.slice(claudeIdx);
expect(after).toContain('ARCHIVE_REAL');
expect(after).toContain('STATE_PROJECTS');
expect(after).toMatch(/cat "\$ARCHIVE_PATH" \| \(cd "\$SPAWN_PATH" && claude -p/);
});
test('claude host still uses classic stdin pipe (no Grok-only gates)', () => {
const out = generateSpecSpawn(makeCtx('claude'));
expect(out).toContain('claude -p');
expect(out).not.toContain('grok --prompt-file');
});
test('generateSpecExecuteFlag is host-aware', () => {
expect(generateSpecExecuteFlag(makeCtx('grok-build'))).toContain('grok --prompt-file');
expect(generateSpecExecuteFlag(makeCtx('claude'))).toContain('claude -p');
});
});
// ─── GrokAdapter ─────────────────────────────────────────────
describe('GrokAdapter behavioral unit', () => {
const adapter = new GrokAdapter();
let tmpHome: string;
let prevHome: string | undefined;
let prevXai: string | undefined;
let prevGrok: string | undefined;
beforeEach(() => {
tmpHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-adapter-'));
prevHome = process.env.HOME;
prevXai = process.env.XAI_API_KEY;
prevGrok = process.env.GROK_API_KEY;
process.env.HOME = tmpHome;
delete process.env.XAI_API_KEY;
delete process.env.GROK_API_KEY;
});
afterEach(() => {
if (prevHome === undefined) delete process.env.HOME;
else process.env.HOME = prevHome;
if (prevXai === undefined) delete process.env.XAI_API_KEY;
else process.env.XAI_API_KEY = prevXai;
if (prevGrok === undefined) delete process.env.GROK_API_KEY;
else process.env.GROK_API_KEY = prevGrok;
fs.rmSync(tmpHome, { recursive: true, force: true });
});
test('available() returns not-ok when CLI and auth both missing', async () => {
// Isolate PATH so grok cannot be found even if installed on the machine.
// Some environments still resolve `grok` via absolute exec caches — accept
// either binary-missing OR auth-missing as a valid not-ok outcome.
const prevPath = process.env.PATH;
process.env.PATH = '/nonexistent-bin-dir-for-grok-avail';
try {
const r = await adapter.available();
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/not found|Install Grok|PATH|No Grok auth|XAI_API_KEY|GROK_API_KEY/i);
} finally {
process.env.PATH = prevPath;
}
});
test('available() fails when binary present but no auth file and no env key names', async () => {
// If grok is not on PATH in this environment, skip — unit still covers the
// auth-file/env-name branch when binary is present.
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) {
// No binary: available should report CLI missing under empty HOME
const r = await adapter.available();
expect(r.ok).toBe(false);
return;
}
const r = await adapter.available();
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/No Grok auth|XAI_API_KEY|GROK_API_KEY/i);
});
test('available() accepts non-empty auth.json object under HOME', async () => {
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) return; // cannot exercise auth-file path without binary
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
// OAuth-shaped key (URL) without logging real secrets — structure only
fs.writeFileSync(
path.join(tmpHome, '.grok', 'auth.json'),
JSON.stringify({ 'https://auth.x.ai::fixture-user': { access: 'x' } }),
);
const r = await adapter.available();
expect(r.ok).toBe(true);
});
test('available() fails closed on empty object auth.json {}', async () => {
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) return;
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
fs.writeFileSync(path.join(tmpHome, '.grok', 'auth.json'), '{}');
const r = await adapter.available();
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/No Grok auth|auth\.json|XAI_API_KEY|GROK_API_KEY/i);
// Reason must never echo file contents / secrets
expect(r.reason).not.toMatch(/access_token|sk-|Bearer/i);
});
test('available() fails closed on zero-byte and invalid auth.json', async () => {
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) return;
fs.mkdirSync(path.join(tmpHome, '.grok'), { recursive: true });
const authPath = path.join(tmpHome, '.grok', 'auth.json');
fs.writeFileSync(authPath, '');
expect((await adapter.available()).ok).toBe(false);
fs.writeFileSync(authPath, ' \n');
expect((await adapter.available()).ok).toBe(false);
fs.writeFileSync(authPath, 'not-json');
expect((await adapter.available()).ok).toBe(false);
fs.writeFileSync(authPath, 'null');
expect((await adapter.available()).ok).toBe(false);
fs.writeFileSync(authPath, '[]');
expect((await adapter.available()).ok).toBe(false);
});
test('available() fails closed on whitespace-only env keys', async () => {
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) return;
process.env.XAI_API_KEY = ' ';
process.env.GROK_API_KEY = '\t';
const r = await adapter.available();
expect(r.ok).toBe(false);
expect(r.reason).not.toContain(' ');
});
test('available() accepts non-blank env key without auth file', async () => {
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
if (which.status !== 0) return;
process.env.XAI_API_KEY = 'test-not-a-real-key';
const r = await adapter.available();
expect(r.ok).toBe(true);
});
test('isStructurallyValidGrokAuthFile pure checks', () => {
const p = path.join(tmpHome, 'auth.json');
fs.writeFileSync(p, '');
expect(isStructurallyValidGrokAuthFile(p)).toBe(false);
fs.writeFileSync(p, '{}');
expect(isStructurallyValidGrokAuthFile(p)).toBe(false);
fs.writeFileSync(p, '{"k":1}');
expect(isStructurallyValidGrokAuthFile(p)).toBe(true);
expect(isStructurallyValidGrokAuthFile(path.join(tmpHome, 'missing.json'))).toBe(false);
});
test('run() uses --prompt-file for multi-line / large prompts (ARG_MAX safety)', async () => {
// Force binary_missing path so we never hit a real CLI; still inspects argv construction
// via the ENOENT error path. Large prompt must not throw ARG_MAX.
const big = 'line\n'.repeat(500) + 'x'.repeat(3000);
const prevPath = process.env.PATH;
process.env.PATH = '/nonexistent-bin-dir-for-grok-test';
try {
const r = await adapter.run({
prompt: big,
workdir: tmpHome,
timeoutMs: 2000,
});
expect(r.error?.code).toBe('binary_missing');
expect(r.tokens).toEqual({ input: 0, output: 0 });
} finally {
process.env.PATH = prevPath;
}
});
test('run() redacts long token-shaped stderr on unknown errors', async () => {
// Drive the unknown error path with a fake binary that exits non-zero
const fakeBin = path.join(tmpHome, 'bin');
fs.mkdirSync(fakeBin, { recursive: true });
const grokSh = path.join(fakeBin, 'grok');
fs.writeFileSync(
grokSh,
`#!/bin/sh\necho "auth token SECRET_TOKEN_ABCDEFGHIJKLMNOPQRSTUVWXYZ123456" >&2\nexit 1\n`,
{ mode: 0o755 },
);
const prevPath = process.env.PATH;
process.env.PATH = `${fakeBin}:${prevPath ?? ''}`;
try {
const r = await adapter.run({
prompt: 'hi',
workdir: tmpHome,
timeoutMs: 3000,
});
// Auth keyword in stderr maps to auth code with redacted reason
expect(r.error).toBeDefined();
if (r.error?.code === 'auth') {
expect(r.error.reason).not.toContain('SECRET_TOKEN');
} else {
expect(r.error!.reason).not.toContain('SECRET_TOKEN_ABCDEFGHIJKLMNOPQRSTUVWXYZ123456');
expect(r.error!.reason).toMatch(/\[redacted\]|unknown|authentication/i);
}
} finally {
process.env.PATH = prevPath;
}
});
test('run() passes --output-format json and parses usage from fixture CLI', async () => {
const fakeBin = path.join(tmpHome, 'bin');
fs.mkdirSync(fakeBin, { recursive: true });
const grokSh = path.join(fakeBin, 'grok');
// Echo argv so we can assert --output-format json; emit usage-bearing JSON on stdout
const payload = JSON.stringify({
text: 'hello',
usage: { input_tokens: 1000, output_tokens: 500 },
model: 'grok',
});
fs.writeFileSync(
grokSh,
`#!/bin/sh
# record argv for assertions
printf '%s\\n' "$*" > "${tmpHome}/grok-argv.txt"
echo '${payload}'
`,
{ mode: 0o755 },
);
const prevPath = process.env.PATH;
process.env.PATH = `${fakeBin}:${prevPath ?? ''}`;
try {
const r = await adapter.run({
prompt: 'hi',
workdir: tmpHome,
timeoutMs: 3000,
});
expect(r.error).toBeUndefined();
expect(r.output).toBe('hello');
expect(r.tokens).toEqual({ input: 1000, output: 500 });
expect(r.modelUsed).toBe('grok');
// Cost honesty: non-zero tokens × official rates → non-zero USD
const cost = adapter.estimateCost(r.tokens, r.modelUsed);
expect(cost).toBeGreaterThan(0);
const argv = fs.readFileSync(path.join(tmpHome, 'grok-argv.txt'), 'utf-8');
expect(argv).toMatch(/--output-format\s+json|--output-format json/);
} finally {
process.env.PATH = prevPath;
}
});
});
// ─── parseGrokOutput + pricing honesty ───────────────────────
describe('parseGrokOutput + Grok pricing', () => {
test('parses characterized headless json shape (no usage → zero tokens)', () => {
const raw = JSON.stringify({
text: 'pong',
stopReason: 'EndTurn',
sessionId: '019f57be-82a4-7990-89ed-47a030fbaeb7',
requestId: 'fe579ce3-777b-42e5-aea5-f2e25c24398d',
thought: 'simple reply',
});
const p = parseGrokOutput(raw);
expect(p.output).toBe('pong');
expect(p.tokens).toEqual({ input: 0, output: 0 });
});
test('parses usage when present (input_tokens / output_tokens)', () => {
const p = parseGrokOutput(
JSON.stringify({
text: 'ok',
usage: { input_tokens: 1_000_000, output_tokens: 1_000_000 },
}),
);
expect(p.tokens).toEqual({ input: 1_000_000, output: 1_000_000 });
// With U1 rates for `grok` ($1/$2 per MTok): 1 + 2 = $3
expect(estimateCostUsd(p.tokens, 'grok')).toBe(3);
});
test('parses prompt_tokens / completion_tokens aliases', () => {
const p = parseGrokOutput(
JSON.stringify({
text: 'x',
usage: { prompt_tokens: 100, completion_tokens: 50 },
}),
);
expect(p.tokens).toEqual({ input: 100, output: 50 });
});
test('plain text without usage → zero tokens, no throw', () => {
const p = parseGrokOutput('just plain assistant text');
expect(p.output).toBe('just plain assistant text');
expect(p.tokens).toEqual({ input: 0, output: 0 });
});
test('streaming-json NDJSON accumulates text; zero tokens without usage', () => {
const raw = [
'{"type":"thought","data":"hmm"}',
'{"type":"text","data":"hel"}',
'{"type":"text","data":"lo"}',
'{"type":"end","stopReason":"EndTurn","sessionId":"x"}',
].join('\n');
const p = parseGrokOutput(raw);
expect(p.output).toBe('hello');
expect(p.tokens).toEqual({ input: 0, output: 0 });
});
test('official Grok rates are non-zero and match rate math', () => {
expect(PRICING['grok']?.input_per_mtok).toBe(1);
expect(PRICING['grok']?.output_per_mtok).toBe(2);
expect(PRICING['grok-build-0.1']?.input_per_mtok).toBe(1);
expect(PRICING['grok-4.5']?.input_per_mtok).toBe(2);
expect(PRICING['grok-4.5']?.output_per_mtok).toBe(6);
// 1M in + 1M out at $1/$2 → $3.00
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'grok')).toBe(3);
// 1M in + 1M out at $2/$6 → $8.00
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'grok-4.5')).toBe(8);
});
test('unknown model returns 0 and does not throw; peer rows unchanged', () => {
expect(estimateCostUsd({ input: 1_000_000, output: 1_000_000 }, 'not-a-real-model-xyz')).toBe(0);
expect(PRICING['claude-opus-4-7']?.input_per_mtok).toBe(15);
expect(PRICING['gpt-5.4']?.input_per_mtok).toBe(2.5);
expect(PRICING['gemini-2.5-pro']?.input_per_mtok).toBe(1.25);
});
});
// ─── gstack-grok-compat-audit fixtures ───────────────────────
function writeMinimalRuntime(skillsDir: string, opts: { withReview?: boolean; withSpec?: boolean } = {}) {
const { withReview = true, withSpec = true } = opts;
const root = path.join(skillsDir, 'gstack');
for (const rel of [
'bin',
'browse/dist',
'browse/src',
'scripts',
'review/specialists',
]) {
fs.mkdirSync(path.join(root, rel), { recursive: true });
}
// When monorepo has design/dist + extension, audit requires them in the runtime root
if (fs.existsSync(path.join(ROOT, 'design', 'dist'))) {
fs.mkdirSync(path.join(root, 'design', 'dist'), { recursive: true });
}
if (fs.existsSync(path.join(ROOT, 'extension'))) {
fs.mkdirSync(path.join(root, 'extension'), { recursive: true });
}
// browse-client required by skillify audit
fs.mkdirSync(path.join(root, 'browse', 'src'), { recursive: true });
fs.writeFileSync(path.join(root, 'browse', 'src', 'browse-client.ts'), '// fixture\n');
if (withReview) {
fs.writeFileSync(path.join(root, 'review', 'checklist.md'), '# checklist\n');
fs.writeFileSync(path.join(root, 'review', 'TODOS-format.md'), '# todos\n');
}
// Audit compares fixture runtime against monorepo presence of design/extension.
// Mirror those when the monorepo has them so the fixture is "complete" for phase a.
if (fs.existsSync(path.join(ROOT, 'design', 'dist'))) {
fs.mkdirSync(path.join(root, 'design', 'dist'), { recursive: true });
}
if (fs.existsSync(path.join(ROOT, 'extension'))) {
fs.mkdirSync(path.join(root, 'extension'), { recursive: true });
fs.writeFileSync(path.join(root, 'extension', 'manifest.json'), '{}\n');
}
// connect-chrome → open-gstack-browser shape
const ogb = path.join(skillsDir, 'gstack-open-gstack-browser');
fs.mkdirSync(ogb, { recursive: true });
fs.writeFileSync(path.join(ogb, 'SKILL.md'), 'name: open-gstack-browser\n');
fs.symlinkSync(ogb, path.join(skillsDir, 'connect-chrome'));
if (withSpec) {
const spec = path.join(skillsDir, 'gstack-spec');
fs.mkdirSync(spec, { recursive: true });
fs.writeFileSync(
path.join(spec, 'SKILL.md'),
[
'---',
'name: spec',
'---',
'Spawn **Grok** headless with --prompt-file.',
'```bash',
'grok --prompt-file "$ARCHIVE_PATH" --cwd "$SPAWN_PATH"',
'```',
'$GSTACK_ROOT',
].join('\n'),
);
}
}
function runAudit(skillsDir: string, phase = 'a'): { status: number | null; stdout: string; stderr: string } {
const r = spawnSync('bun', ['run', AUDIT_BIN, '--skills-dir', skillsDir, '--phase', phase], {
cwd: ROOT,
encoding: 'utf-8',
timeout: 30000,
});
return {
status: r.status,
stdout: r.stdout?.toString() ?? '',
stderr: r.stderr?.toString() ?? '',
};
}
describe('gstack-grok-compat-audit fixtures', () => {
let skillsDir: string;
beforeEach(() => {
skillsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-audit-'));
});
afterEach(() => {
fs.rmSync(skillsDir, { recursive: true, force: true });
});
test('missing skills dir exits 1 INCOMPATIBLE', () => {
const missing = path.join(skillsDir, 'does-not-exist');
const r = runAudit(missing, 'a');
expect(r.status).toBe(1);
expect(r.stdout).toMatch(/INCOMPATIBLE|skills dir missing/i);
});
test('minimal complete runtime exits 0 COMPATIBLE for phase a', () => {
writeMinimalRuntime(skillsDir);
const r = runAudit(skillsDir, 'a');
expect(r.status).toBe(0);
expect(r.stdout).toContain('COMPATIBLE');
});
test('missing review/checklist.md fails phase a', () => {
writeMinimalRuntime(skillsDir, { withReview: false });
const r = runAudit(skillsDir, 'a');
expect(r.status).toBe(1);
expect(r.stdout).toMatch(/missing review\/checklist\.md|INCOMPATIBLE/);
});
test('missing gstack runtime root fails', () => {
fs.mkdirSync(skillsDir, { recursive: true });
// empty skills dir — no gstack/
const r = runAudit(skillsDir, 'a');
expect(r.status).toBe(1);
expect(r.stdout).toMatch(/missing runtime root|INCOMPATIBLE/);
});
test('phase ab fails when neither gstack-spec nor spec package installed', () => {
writeMinimalRuntime(skillsDir, { withSpec: false });
const r = runAudit(skillsDir, 'ab');
expect(r.status).toBe(1);
expect(r.stdout).toMatch(/neither gstack-spec nor spec|INCOMPATIBLE/);
});
});
// ─── setup create_grok_runtime_root preflight smoke ──────────
describe('create_grok_runtime_root preflight (setup smoke)', () => {
test('refuses wipe when required monorepo assets missing', () => {
const emptyRepo = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-grok-setup-empty-'));
const dest = path.join(emptyRepo, 'install-target', 'gstack');
// Seed a live install that must NOT be wiped when preflight fails
fs.mkdirSync(dest, { recursive: true });
fs.writeFileSync(path.join(dest, 'KEEPME'), 'live');
const setupPath = path.join(ROOT, 'setup');
// Source extracted functions from a temp file — do NOT eval via $(...) because
// create_grok_runtime_root uses $$ for the staging suffix and command
// substitution would expand it early.
const extractScript = path.join(emptyRepo, 'extract-and-run.sh');
fs.writeFileSync(
extractScript,
`#!/usr/bin/env bash
set +e
SETUP=${JSON.stringify(setupPath)}
GSTACK_DIR=${JSON.stringify(emptyRepo)}
DEST=${JSON.stringify(dest)}
_link_or_copy() { ln -sfn "$1" "$2" 2>/dev/null || cp -R "$1" "$2"; }
extract_fn() {
local name="$1"
local out="$2"
awk -v name="$name" '
$0 ~ "^" name "\\\\(\\\\)" {grab=1}
grab {
print
for (i=1;i<=length($0);i++) {
c=substr($0,i,1)
if (c=="{") depth++
if (c=="}") {
depth--
if (depth==0) { exit }
}
}
}
' "$SETUP" > "$out"
}
FN_DIR=$(mktemp -d)
extract_fn _grok_link_under_monorepo "$FN_DIR/link.sh"
extract_fn create_grok_runtime_root "$FN_DIR/create.sh"
# shellcheck source=/dev/null
. "$FN_DIR/link.sh"
# shellcheck source=/dev/null
. "$FN_DIR/create.sh"
create_grok_runtime_root "$GSTACK_DIR" "$DEST"
rc=$?
rm -rf "$FN_DIR"
if [ $rc -eq 0 ]; then
echo "UNEXPECTED_SUCCESS"
exit 2
fi
echo "PREFLIGHT_FAILED_AS_EXPECTED"
if [ -f "$DEST/KEEPME" ]; then
echo "LIVE_PRESERVED"
exit 0
fi
echo "LIVE_WIPED"
exit 3
`,
{ mode: 0o755 },
);
const r = spawnSync('bash', [extractScript], {
encoding: 'utf-8',
timeout: 15000,
});
fs.rmSync(emptyRepo, { recursive: true, force: true });
expect(r.stdout + r.stderr).toMatch(/preflight|required monorepo asset missing/i);
expect(r.stdout).toContain('PREFLIGHT_FAILED_AS_EXPECTED');
expect(r.stdout).toContain('LIVE_PRESERVED');
expect(r.status).toBe(0);
});
});

View File

@ -0,0 +1,104 @@
/**
* Grok Build section-pointer mode (Option A token-ceiling fix).
*
* Carved skills must emit STOP-Read pointers + separate section files under
* .grok/skills/gstack-<skill>/sections/, not a monolith that exceeds the
* gen-skill-docs soft ceiling (~160KB / ~40k tokens).
*/
import { describe, test, expect } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import {
hostUsesSectionPointers,
externalSkillPackageName,
sectionPointerPath,
SECTION,
SECTION_INDEX,
} from '../scripts/resolvers/sections';
import type { TemplateContext } from '../scripts/resolvers/types';
import { HOST_PATHS } from '../scripts/resolvers/types';
const ROOT = path.resolve(import.meta.dir, '..');
function shipCtx(host: 'claude' | 'grok-build' | 'codex'): TemplateContext {
return {
skillName: 'ship',
tmplPath: path.join(ROOT, 'ship', 'SKILL.md.tmpl'),
host,
paths: HOST_PATHS[host],
};
}
describe('hostUsesSectionPointers', () => {
test('claude and grok-build use pointers; codex inlines', () => {
expect(hostUsesSectionPointers('claude')).toBe(true);
expect(hostUsesSectionPointers('grok-build')).toBe(true);
expect(hostUsesSectionPointers('codex')).toBe(false);
expect(hostUsesSectionPointers('factory')).toBe(false);
});
});
describe('sectionPointerPath', () => {
test('grok uses flat package under ~/.grok/skills', () => {
expect(sectionPointerPath('grok-build', 'ship', 'tests.md', '$GSTACK_ROOT')).toBe(
'~/.grok/skills/gstack-ship/sections/tests.md',
);
expect(externalSkillPackageName('ship')).toBe('gstack-ship');
expect(externalSkillPackageName('gstack-upgrade')).toBe('gstack-upgrade');
});
test('claude uses nested skillRoot layout', () => {
expect(sectionPointerPath('claude', 'ship', 'tests.md', '~/.claude/skills/gstack')).toBe(
'~/.claude/skills/gstack/ship/sections/tests.md',
);
});
});
describe('SECTION / SECTION_INDEX resolvers', () => {
test('grok ship SECTION emits STOP-Read to gstack-ship package path', () => {
const out = SECTION(shipCtx('grok-build'), ['tests']);
expect(out).toContain('**STOP.**');
expect(out).toContain('~/.grok/skills/gstack-ship/sections/tests.md');
expect(out).not.toContain('## Test Framework Bootstrap'); // not inlined
});
test('codex ship SECTION still inlines section body', () => {
const out = SECTION(shipCtx('codex'), ['tests']);
expect(out).not.toContain('**STOP.**');
// tests.md.tmpl has substantive body (not a pointer)
expect(out.length).toBeGreaterThan(200);
});
test('grok SECTION_INDEX lists full paths; codex SECTION_INDEX is empty', () => {
const grok = SECTION_INDEX(shipCtx('grok-build'), ['ship']);
expect(grok).toContain('## Section index');
expect(grok).toContain('~/.grok/skills/gstack-ship/sections/review-army.md');
expect(SECTION_INDEX(shipCtx('codex'), ['ship'])).toBe('');
});
});
describe('generated grok ship package (when present)', () => {
const shipPkg = path.join(ROOT, '.grok', 'skills', 'gstack-ship');
const skillMd = path.join(shipPkg, 'SKILL.md');
const sectionsDir = path.join(shipPkg, 'sections');
test('gstack-ship SKILL.md is under soft token ceiling when generated', () => {
if (!fs.existsSync(skillMd)) return; // gen not run in this env
const bytes = fs.statSync(skillMd).size;
// Soft ceiling in gen-skill-docs: 160_000 bytes (~40k tokens)
expect(bytes).toBeLessThan(160_000);
});
test('gstack-ship has carved section files + STOP pointers when generated', () => {
if (!fs.existsSync(skillMd)) return;
const body = fs.readFileSync(skillMd, 'utf-8');
expect(body).toContain('**STOP.**');
expect(body).toContain('~/.grok/skills/gstack-ship/sections/');
// Heavy steps must not be inlined into the skeleton
expect(body).not.toMatch(/## Step 9\.1: Review Army/);
expect(fs.existsSync(path.join(sectionsDir, 'review-army.md'))).toBe(true);
expect(fs.existsSync(path.join(sectionsDir, 'adversarial.md'))).toBe(true);
expect(fs.existsSync(path.join(sectionsDir, 'tests.md'))).toBe(true);
});
});

View File

@ -7,26 +7,29 @@
* one. Per-provider auth/timeout/rate-limit errors don't abort the batch.
*/
import type { ProviderAdapter, RunOpts, RunResult } from './providers/types';
import type { ProviderAdapter, RunOpts, RunResult, Family } from './providers/types';
import { ClaudeAdapter } from './providers/claude';
import { GptAdapter } from './providers/gpt';
import { GeminiAdapter } from './providers/gemini';
import { GrokAdapter } from './providers/grok';
export type ProviderName = 'claude' | 'gpt' | 'gemini' | 'grok';
export interface BenchmarkInput {
prompt: string;
workdir: string;
timeoutMs?: number;
/** Adapter names to run (e.g., ['claude', 'gpt', 'gemini']). */
providers: Array<'claude' | 'gpt' | 'gemini'>;
/** Adapter names to run (e.g., ['claude', 'gpt', 'gemini', 'grok']). */
providers: ProviderName[];
/** Optional per-provider model overrides. */
models?: Partial<Record<'claude' | 'gpt' | 'gemini', string>>;
models?: Partial<Record<ProviderName, string>>;
/** If true, skip providers whose available() returns !ok. If false, include them with error. */
skipUnavailable?: boolean;
}
export interface BenchmarkEntry {
provider: string;
family: 'claude' | 'gpt' | 'gemini';
family: Family;
available: boolean;
unavailable_reason?: string;
result?: RunResult;
@ -44,10 +47,11 @@ export interface BenchmarkReport {
entries: BenchmarkEntry[];
}
const ADAPTERS: Record<'claude' | 'gpt' | 'gemini', () => ProviderAdapter> = {
const ADAPTERS: Record<ProviderName, () => ProviderAdapter> = {
claude: () => new ClaudeAdapter(),
gpt: () => new GptAdapter(),
gemini: () => new GeminiAdapter(),
grok: () => new GrokAdapter(),
};
export async function runBenchmark(input: BenchmarkInput): Promise<BenchmarkReport> {

View File

@ -6,9 +6,17 @@
* - Anthropic: https://www.anthropic.com/pricing#api
* - OpenAI: https://openai.com/api/pricing/
* - Google AI: https://ai.google.dev/pricing
* - xAI: https://docs.x.ai/developers/pricing
*
* When a model isn't in the table, estimateCost returns 0 with a console warning.
* Prefer adding a new row to the table over guessing.
*
* xAI rates (official docs, as of 2026-07): Code API grok-build-0.1 $1/$2;
* Chat API grok-4.5 $2/$6; grok-4.3 and grok-4.20-* $1.25/$2.50.
* Benchmark adapter default id `grok` maps to Code API grok-build-0.1
* (Grok Build headless default). Alias `grok-4` maps to Chat mid-tier
* grok-4.3 rates. Run cost still needs real token counts from the adapter
* rates alone cannot invent usage.
*/
export interface ModelPricing {
@ -32,6 +40,16 @@ export const PRICING: Record<string, ModelPricing> = {
// Google
'gemini-2.5-pro': { input_per_mtok: 1.25, output_per_mtok: 5.00, as_of: '2026-04' },
'gemini-2.5-flash': { input_per_mtok: 0.30, output_per_mtok: 1.20, as_of: '2026-04' },
// xAI Grok — https://docs.x.ai/developers/pricing (as of 2026-07)
'grok-build-0.1': { input_per_mtok: 1.00, output_per_mtok: 2.00, as_of: '2026-07' },
'grok': { input_per_mtok: 1.00, output_per_mtok: 2.00, as_of: '2026-07' }, // alias → Code API
'grok-4.5': { input_per_mtok: 2.00, output_per_mtok: 6.00, as_of: '2026-07' },
'grok-4.3': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
'grok-4': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' }, // alias → Chat mid-tier
'grok-4.20-0309-reasoning': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
'grok-4.20-0309-non-reasoning': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
'grok-4.20-multi-agent-0309': { input_per_mtok: 1.25, output_per_mtok: 2.50, as_of: '2026-07' },
};
const WARNED = new Set<string>();

View File

@ -0,0 +1,338 @@
import type { ProviderAdapter, RunOpts, RunResult, AvailabilityCheck } from './types';
import { estimateCostUsd } from '../pricing';
import { execFileSync, spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
export type GrokParsedOutput = {
output: string;
tokens: { input: number; output: number; cached?: number };
toolCalls: number;
modelUsed?: string;
};
/**
* Parse Grok headless stdout.
*
* Characterized 2026-07 against grok 0.2.93:
* --output-format json
* { text, stopReason, sessionId, requestId, thought? }
* --output-format streaming-json
* NDJSON {type:"thought"|"text"|"end", ...} end has stopReason/sessionId
* Neither shape exposes usage today. When a future CLI adds usage, accept
* common field names (input_tokens/output_tokens, prompt_tokens/completion_tokens,
* nested usage object). Never invent token counts from prompt/output length.
*/
export function parseGrokOutput(raw: string): GrokParsedOutput {
const trimmed = raw.trim();
if (!trimmed) {
return { output: '', tokens: { input: 0, output: 0 }, toolCalls: 0 };
}
// Single JSON object (default headless json format)
try {
const obj = JSON.parse(trimmed);
if (obj && typeof obj === 'object' && !Array.isArray(obj)) {
return parseGrokJsonObject(obj as Record<string, unknown>);
}
} catch {
// fall through to NDJSON / plain text
}
// streaming-json NDJSON lines
if (trimmed.includes('\n') || trimmed.startsWith('{')) {
const lines = trimmed.split('\n').map((l) => l.trim()).filter(Boolean);
let looksNdjson = 0;
let textParts: string[] = [];
let tokens = { input: 0, output: 0 } as { input: number; output: number; cached?: number };
let toolCalls = 0;
let modelUsed: string | undefined;
let sawUsage = false;
for (const line of lines) {
try {
const ev = JSON.parse(line) as Record<string, unknown>;
looksNdjson++;
const t = ev.type;
if (t === 'text' && typeof ev.data === 'string') {
textParts.push(ev.data);
} else if (t === 'message' && typeof ev.data === 'string') {
textParts.push(ev.data);
} else if (typeof ev.text === 'string') {
textParts.push(ev.text);
}
const fromEv = extractUsage(ev);
if (fromEv) {
tokens = fromEv.tokens;
sawUsage = true;
if (fromEv.cached !== undefined) tokens = { ...tokens, cached: fromEv.cached };
}
if (typeof ev.model === 'string') modelUsed = ev.model;
if (typeof ev.modelUsed === 'string') modelUsed = ev.modelUsed;
if (typeof ev.num_turns === 'number') toolCalls = ev.num_turns;
if (typeof ev.tool_calls === 'number') toolCalls = ev.tool_calls;
if (typeof ev.toolCallCount === 'number') toolCalls = ev.toolCallCount;
} catch {
// non-JSON line — ignore for NDJSON path
}
}
if (looksNdjson > 0 && (textParts.length > 0 || sawUsage || lines.length === looksNdjson)) {
return {
output: textParts.join(''),
tokens: sawUsage ? tokens : { input: 0, output: 0 },
toolCalls,
modelUsed,
};
}
}
// Plain text fallback — zero tokens (do not estimate from length)
return { output: raw, tokens: { input: 0, output: 0 }, toolCalls: 0 };
}
function parseGrokJsonObject(obj: Record<string, unknown>): GrokParsedOutput {
// Preferred text fields for Grok Build; also accept Claude-like `result`
let output = '';
if (typeof obj.text === 'string') output = obj.text;
else if (typeof obj.result === 'string') output = obj.result;
else if (typeof obj.message === 'string') output = obj.message;
else if (obj.result != null) output = String(obj.result);
const usage = extractUsage(obj);
const tokens = usage
? { input: usage.tokens.input, output: usage.tokens.output, ...(usage.cached !== undefined ? { cached: usage.cached } : {}) }
: { input: 0, output: 0 };
const toolCalls =
(typeof obj.num_turns === 'number' ? obj.num_turns : undefined) ??
(typeof obj.tool_calls === 'number' ? obj.tool_calls : undefined) ??
(typeof obj.toolCallCount === 'number' ? obj.toolCallCount : undefined) ??
0;
const modelUsed =
(typeof obj.model === 'string' ? obj.model : undefined) ??
(typeof obj.modelUsed === 'string' ? obj.modelUsed : undefined) ??
(typeof obj.model_id === 'string' ? obj.model_id : undefined);
return { output, tokens, toolCalls, modelUsed };
}
function numField(obj: Record<string, unknown>, ...keys: string[]): number | undefined {
for (const k of keys) {
const v = obj[k];
if (typeof v === 'number' && Number.isFinite(v) && v >= 0) return v;
}
return undefined;
}
/** Extract token usage from a JSON object if present; otherwise undefined. */
function extractUsage(
obj: Record<string, unknown>,
): { tokens: { input: number; output: number }; cached?: number } | undefined {
const nested =
obj.usage && typeof obj.usage === 'object' && !Array.isArray(obj.usage)
? (obj.usage as Record<string, unknown>)
: null;
const sources = nested ? [nested, obj] : [obj];
for (const src of sources) {
const input = numField(
src,
'input_tokens',
'prompt_tokens',
'inputTokens',
'promptTokens',
'input_token_count',
);
const output = numField(
src,
'output_tokens',
'completion_tokens',
'outputTokens',
'completionTokens',
'output_token_count',
);
if (input !== undefined || output !== undefined) {
const cached = numField(
src,
'cache_read_input_tokens',
'cached_prompt_tokens',
'cached_tokens',
'cached',
);
return {
tokens: { input: input ?? 0, output: output ?? 0 },
...(cached !== undefined ? { cached } : {}),
};
}
}
return undefined;
}
/**
* Structural auth check only never log file or env values.
* Valid auth.json: non-empty JSON object with 1 key (OAuth shapes use URL keys).
* Reject: missing, empty/whitespace, invalid JSON, null, arrays, {}.
*/
export function isStructurallyValidGrokAuthFile(authPath: string): boolean {
try {
if (!fs.existsSync(authPath)) return false;
const raw = fs.readFileSync(authPath, 'utf-8');
if (!raw.trim()) return false;
const parsed = JSON.parse(raw) as unknown;
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) return false;
return Object.keys(parsed as object).length > 0;
} catch {
return false;
}
}
function hasNonBlankEnvKey(): boolean {
const xai = process.env.XAI_API_KEY?.trim();
const grok = process.env.GROK_API_KEY?.trim();
return !!(xai || grok);
}
/**
* Grok adapter wraps the `grok` CLI via -p / --single / --prompt-file.
*
* Auth readiness: CLI present + (structurally valid ~/.grok/auth.json OR
* non-blank XAI_API_KEY / GROK_API_KEY after trim). Never log secret values.
* No network probe in available().
*/
export class GrokAdapter implements ProviderAdapter {
readonly name = 'grok';
readonly family = 'grok' as const;
async available(): Promise<AvailabilityCheck> {
// Boolean PATH presence only — never log secrets. Bound to ≤2s like peer adapters.
const which = spawnSync('sh', ['-c', 'command -v grok'], {
timeout: 2000,
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
});
let hasBinary = which.status === 0;
if (!hasBinary) {
try {
execFileSync('grok', ['--version'], {
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'ignore'],
timeout: 2000,
});
hasBinary = true;
} catch {
hasBinary = false;
}
}
if (!hasBinary) {
return {
ok: false,
reason: 'grok CLI not found on PATH. Install Grok Build from xAI, or ensure `grok` is on PATH.',
};
}
// Prefer HOME when set so hermetic tests / agent envs can isolate auth
// discovery. Bun's os.homedir() ignores process.env.HOME (unlike Node).
const home = process.env.HOME || os.homedir();
const authPath = path.join(home, '.grok', 'auth.json');
const hasValidAuthFile = isStructurallyValidGrokAuthFile(authPath);
const hasKey = hasNonBlankEnvKey();
if (!hasValidAuthFile && !hasKey) {
return {
ok: false,
reason:
'No Grok auth found. Log in via `grok` interactive session (non-empty auth.json), or export a non-blank XAI_API_KEY / GROK_API_KEY.',
};
}
return { ok: true };
}
async run(opts: RunOpts): Promise<RunResult> {
const start = Date.now();
// Prefer --prompt-file for multi-line / large prompts (ARG_MAX + quoting).
// Short single-line prompts use --single to avoid temp files.
const useFile =
opts.prompt.includes('\n') || opts.prompt.length > 2000 || Buffer.byteLength(opts.prompt, 'utf8') > 2000;
let promptFile: string | null = null;
let args: string[];
if (useFile) {
promptFile = path.join(
os.tmpdir(),
`gstack-grok-bench-${process.pid}-${Date.now()}-${Math.random().toString(36).slice(2)}.txt`,
);
fs.writeFileSync(promptFile, opts.prompt, 'utf-8');
args = ['--prompt-file', promptFile, '--cwd', opts.workdir];
} else {
args = ['--single', opts.prompt];
}
// Request JSON so we can parse usage when the CLI exposes it (currently often omitted).
args.push('--output-format', 'json');
if (opts.model) args.push('--model', opts.model);
if (opts.extraArgs) args.push(...opts.extraArgs);
try {
const out = execFileSync('grok', args, {
cwd: opts.workdir,
timeout: opts.timeoutMs,
encoding: 'utf-8',
maxBuffer: 32 * 1024 * 1024,
// Pipe stderr so auth-shaped tokens never inherit onto the parent console
stdio: ['ignore', 'pipe', 'pipe'],
env: { ...process.env, GSTACK_HEADLESS: '1' },
});
const raw = typeof out === 'string' ? out : String(out);
const parsed = parseGrokOutput(raw);
return {
output: parsed.output,
tokens: parsed.tokens,
durationMs: Date.now() - start,
toolCalls: parsed.toolCalls,
modelUsed: parsed.modelUsed || opts.model || 'grok',
};
} catch (err: unknown) {
const durationMs = Date.now() - start;
const e = err as { code?: string; stderr?: Buffer; signal?: string; message?: string };
const stderr = e.stderr?.toString() ?? '';
// Never surface raw stderr for auth paths (may contain token-shaped text).
const safeSlice = (s: string) => s.replace(/[A-Za-z0-9_\-]{20,}/g, '[redacted]').slice(0, 200);
if (e.signal === 'SIGTERM' || e.code === 'ETIMEDOUT') {
return this.emptyResult(durationMs, { code: 'timeout', reason: `exceeded ${opts.timeoutMs}ms` }, opts.model);
}
if (/unauthorized|auth|login|api.?key/i.test(stderr)) {
return this.emptyResult(durationMs, { code: 'auth', reason: 'authentication failed (details redacted)' }, opts.model);
}
if (/rate[- ]?limit|429/i.test(stderr)) {
return this.emptyResult(durationMs, { code: 'rate_limit', reason: safeSlice(stderr) }, opts.model);
}
if (/ENOENT|not found/i.test(e.message ?? '') || e.code === 'ENOENT') {
return this.emptyResult(durationMs, { code: 'binary_missing', reason: 'grok CLI not found' }, opts.model);
}
return this.emptyResult(durationMs, { code: 'unknown', reason: safeSlice(e.message ?? stderr ?? 'unknown') }, opts.model);
} finally {
if (promptFile) {
try {
fs.unlinkSync(promptFile);
} catch {
// best-effort cleanup of temp prompt file
}
}
}
}
estimateCost(tokens: { input: number; output: number; cached?: number }, model?: string): number {
return estimateCostUsd(tokens, model ?? 'grok');
}
private emptyResult(durationMs: number, error: RunResult['error'], model?: string): RunResult {
return {
output: '',
tokens: { input: 0, output: 0 },
durationMs,
toolCalls: 0,
modelUsed: model || 'grok',
error,
};
}
}

View File

@ -55,7 +55,7 @@ export interface AvailabilityCheck {
reason?: string;
}
export type Family = 'claude' | 'gpt' | 'gemini';
export type Family = 'claude' | 'gpt' | 'gemini' | 'grok';
export interface ProviderAdapter {
/** Stable name used in output tables and config (e.g., 'claude', 'gpt', 'gemini'). */

View File

@ -22,16 +22,18 @@ import {
slate,
cursor,
openclaw,
grokBuild,
} from '../hosts/index';
import { HOST_PATHS } from '../scripts/resolvers/types';
import { resolveDistBinary } from '../scripts/resolvers/browse';
const ROOT = path.resolve(import.meta.dir, '..');
// ─── hosts/index.ts ─────────────────────────────────────────
describe('hosts/index.ts', () => {
test('ALL_HOST_CONFIGS has 10 hosts', () => {
expect(ALL_HOST_CONFIGS.length).toBe(10);
test('ALL_HOST_CONFIGS has 11 hosts', () => {
expect(ALL_HOST_CONFIGS.length).toBe(11);
});
test('ALL_HOST_NAMES matches config names', () => {
@ -409,6 +411,12 @@ describe('golden-file regression', () => {
const current = fs.readFileSync(path.join(ROOT, '.factory', 'skills', 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(current).toBe(golden);
});
test('Grok Build ship skill matches golden baseline', () => {
const golden = fs.readFileSync(path.join(GOLDEN_DIR, 'grok-build-ship-SKILL.md'), 'utf-8');
const current = fs.readFileSync(path.join(ROOT, '.grok', 'skills', 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(current).toBe(golden);
});
});
// ─── Individual host config correctness ─────────────────────
@ -535,4 +543,71 @@ describe('host config correctness', () => {
expect(config.runtimeRoot.globalSymlinks).toContain('ETHOS.md');
}
});
test('grok-build runtime root lists complete packaging assets (U1)', () => {
expect(grokBuild.name).toBe('grok-build');
expect(grokBuild.usesEnvVars).toBe(true);
const links = grokBuild.runtimeRoot.globalSymlinks;
for (const asset of [
'browse/src',
'design/dist',
'make-pdf/dist',
'extension',
'scripts',
'review/specialists',
]) {
expect(links).toContain(asset);
}
expect(grokBuild.runtimeRoot.globalFiles?.review).toContain('checklist.md');
expect(grokBuild.runtimeRoot.globalFiles?.review).toContain('design-checklist.md');
expect(grokBuild.generation.skipSkills).toContain('codex');
});
test('grok-build path rewrites suppress Claude host bleed (U3)', () => {
expect(grokBuild.pathRewrites.some(r => r.from === 'CLAUDE.md' && r.to === 'AGENTS.md')).toBe(true);
expect(grokBuild.pathRewrites.some(r => r.from === 'MODEL_OVERLAY: claude' && r.to === 'MODEL_OVERLAY: none')).toBe(true);
expect(grokBuild.toolRewrites?.['AskUserQuestion']).toBe('ask_user_question');
});
// Dual-write parity (#5): hosts/grok-build.ts runtimeRoot must appear in setup's
// create_grok_runtime_root so packaging cannot skew silently.
test('grok-build runtimeRoot dual-write parity with create_grok_runtime_root', () => {
const setupPath = path.join(ROOT, 'setup');
const setup = fs.readFileSync(setupPath, 'utf-8');
const fnStart = setup.indexOf('create_grok_runtime_root()');
expect(fnStart).toBeGreaterThanOrEqual(0);
// Function ends at link_grok_skill_dirs (next sibling) — slice that range
const fnEnd = setup.indexOf('link_grok_skill_dirs()', fnStart);
expect(fnEnd).toBeGreaterThan(fnStart);
const fnBody = setup.slice(fnStart, fnEnd);
for (const link of grokBuild.runtimeRoot.globalSymlinks) {
// Each asset path must appear as a monorepo source under create_grok_runtime_root
expect(fnBody).toContain(link);
}
for (const f of grokBuild.runtimeRoot.globalFiles?.review ?? []) {
expect(fnBody).toContain(f);
}
// Preflight + atomic stage (review #6) stay wired
expect(fnBody).toContain('preflight');
expect(fnBody).toMatch(/\.next\.\$\$|staging/);
// Core review files fail-closed (review #7): required=1 on link
expect(fnBody).toMatch(
/for f in checklist\.md TODOS-format\.md; do[\s\S]*?_grok_link_under_monorepo[^\n]* 1 \|\| return 1/,
);
});
});
describe('resolveDistBinary (U2 double-home fix)', () => {
test('env-var browseDir never prefixes $HOME', () => {
expect(resolveDistBinary('$GSTACK_BROWSE', 'browse')).toBe('$GSTACK_BROWSE/browse');
expect(resolveDistBinary('$GSTACK_DESIGN', 'design')).toBe('$GSTACK_DESIGN/design');
expect(resolveDistBinary('$GSTACK_MAKE_PDF', 'pdf')).toBe('$GSTACK_MAKE_PDF/pdf');
});
test('tilde paths keep $HOME prefix', () => {
expect(resolveDistBinary('~/.claude/skills/gstack/browse/dist', 'browse')).toBe(
'$HOME/.claude/skills/gstack/browse/dist/browse',
);
});
});

View File

@ -60,6 +60,16 @@ describe('setup: plan-tune hooks are non-interactive-safe', () => {
expect(setupSrc).toMatch(/tr '\[:upper:\]' '\[:lower:\]'/);
expect(setupSrc).toMatch(/PT_DECISION=\$\(printf .* tr/);
});
test('plan-tune hooks are Claude-host-scoped (skip for --host grok-build)', () => {
// Non-Claude host installs must not prompt to mutate ~/.claude/settings.json.
// Explicit --plan-tune-hooks still allowed via _PT_EXPLICIT_YES.
expect(setupSrc).toContain('_PT_EXPLICIT_YES');
expect(setupSrc).toContain('[ "$INSTALL_CLAUDE" -eq 1 ] || [ "$_PT_EXPLICIT_YES" -eq 1 ]');
// Codex .agents/ regen is also host-scoped — not always-on during grok-only setup.
expect(setupSrc).toContain('[ "$INSTALL_CODEX" -eq 1 ] && [ "$NEEDS_BUILD" -eq 0 ]');
expect(setupSrc).toContain('Generating .agents/ skill docs for Codex...');
});
});
describe('dev-setup: never silently mutates global settings.json', () => {