gstack/test
Garry Tan 5a11abef6f
test(benchmark-providers): drop literal 'ok' assertion on gemini smoke
The gemini live-smoke test was failing intermittently when the Gemini CLI
returned empty output for the trivial "say ok" prompt — likely a CLI
parser miss on a successful run rather than the model failing the task.
The whole point of this smoke is "did the adapter wire up and the run
terminate without error?", not "did the model say the literal word ok",
so we drop the toLowerCase().toContain('ok') assertion in favor of an
adapter-shape check.

This brings the gemini smoke in line with what we actually care about at
the gate tier: cross-provider adapter wiring stays unbroken.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:46:21 -07:00
..
fixtures test: refresh ship goldens + ratchet preamble budget for #1205 2026-05-10 11:18:57 -07:00
helpers test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
agent-sdk-runner.test.ts v1.11.1.0 fix: plan-mode handshake + canUseTool test harness (#1182) 2026-04-24 00:04:53 -07:00
analytics.test.ts feat: safety hook skills + skill usage telemetry (v0.7.1) (#189) 2026-03-18 23:57:59 -05:00
audit-compliance.test.ts feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040) 2026-04-19 17:50:31 +08:00
benchmark-cli.test.ts feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040) 2026-04-19 17:50:31 +08:00
benchmark-runner.test.ts feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040) 2026-04-19 17:50:31 +08:00
brain-sync.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
builder-profile.test.ts feat: relationship closing — office-hours adapts to repeat users (v0.16.2.0) (#937) 2026-04-08 22:21:28 -10:00
codex-e2e-plan-format.test.ts fix(plan-reviews): restore RECOMMENDATION + Completeness split + Codex ELI10 (v1.6.3.0) (#1149) 2026-04-23 07:25:20 -07:00
codex-e2e.test.ts feat: worktree isolation for E2E tests + infrastructure elegance (v0.11.12.0) (#425) 2026-03-23 23:05:22 -07:00
codex-hardening.test.ts codex + Apple Silicon hardening wave (v0.18.4.0) (#1056) 2026-04-18 12:30:54 +08:00
codex-resume-flag-semantics.test.ts v1.30.0.0 fix wave: 21 community PRs + Windows CI extension + codex flag-semantics smoke (#1391) 2026-05-09 08:06:47 -07:00
context-save-hardening.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
diff-scope.test.ts feat: Review Army — parallel specialist reviewers for /review (v0.14.3.0) (#692) 2026-03-30 22:07:50 -06:00
e2e-harness-audit.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
explain-level-config.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
gbrain-detect-install.test.ts v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183) 2026-04-24 01:38:21 -07:00
gbrain-lib-verify.test.ts v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183) 2026-04-24 01:38:21 -07:00
gbrain-repo-policy.test.ts v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183) 2026-04-24 01:38:21 -07:00
gbrain-sources.test.ts v1.26.3.0 feat: /sync-gbrain skill + native code-surface orchestrator (#1314) 2026-05-04 09:29:48 -07:00
gbrain-supabase-provision.test.ts v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183) 2026-04-24 01:38:21 -07:00
gemini-e2e.test.ts feat: Confusion Protocol, Hermes + GBrain hosts, brain-first resolver (v0.18.0.0) (#1005) 2026-04-16 10:41:38 -07:00
gen-skill-docs.test.ts test: refresh ship goldens + ratchet preamble budget for #1205 2026-05-10 11:18:57 -07:00
global-discover.test.ts fix: close redundant PRs + friendly error on all design commands (v0.15.8.1) (#817) 2026-04-05 02:02:06 -07:00
gstack-artifacts-init.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
gstack-artifacts-url.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
gstack-brain-context-load.test.ts v1.26.0.0 feat: V1 transcript ingest + per-skill gbrain manifests + retrieval surface (#1298) 2026-05-02 08:40:30 -07:00
gstack-developer-profile.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
gstack-gbrain-detect-mcp-mode.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
gstack-gbrain-mcp-verify.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
gstack-gbrain-source-wireup.test.ts v1.17.0.0: setup-gbrain wireup ships the gbrain federation surface (#1234) 2026-04-28 01:17:54 -07:00
gstack-gbrain-sync.test.ts v1.29.0.0 feat: worktree-aware gbrain code sources via path-hash IDs and CWD pin (#1382) 2026-05-08 12:46:15 -07:00
gstack-memory-helpers.test.ts v1.26.0.0 feat: V1 transcript ingest + per-skill gbrain manifests + retrieval surface (#1298) 2026-05-02 08:40:30 -07:00
gstack-memory-ingest.test.ts test(memory-ingest): regression for NUL-byte strip on gbrain put body 2026-05-10 11:06:00 -07:00
gstack-next-version.test.ts v1.11.0.0 feat(ship): workspace-aware version allocation (#1168) 2026-04-23 23:03:27 -07:00
gstack-paths.test.ts v1.24.0.0 feat: cross-platform hardening — curated Windows lane + Bun.which resolver + path-portability helper (#1252) 2026-05-01 07:21:28 -07:00
gstack-question-log.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
gstack-question-preference.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
gstack-upgrade-migration-v1_17_0_0.test.ts v1.17.0.0: setup-gbrain wireup ships the gbrain federation surface (#1234) 2026-04-28 01:17:54 -07:00
helpers-unit.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
hook-scripts.test.ts v1.31.1.0 fix wave: 3 community PRs (careful BSD sed, codex Step 0 rename, make-pdf setup ordering) (#1413) 2026-05-10 06:57:24 -07:00
host-config.test.ts community wave: 6 PRs + hardening (v0.18.1.0) (#1028) 2026-04-17 00:45:13 -07:00
jargon-list.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
learnings-injection.test.ts fix: community security wave — 8 PRs, 4 contributors (v0.15.13.0) (#847) 2026-04-06 00:47:04 -07:00
learnings.test.ts feat: GStack Learns — per-project self-learning infrastructure (v0.13.4.0) (#622) 2026-03-29 17:02:01 -06:00
llm-judge-recommendation.test.ts v1.25.1.0 fix: office-hours Phase 4 STOP gate + AskUserQuestion recommendation judge (#1296) 2026-05-01 19:51:51 -07:00
llms-txt-shape.test.ts v1.28.0.0 feat: browse --headed/--proxy/--navigate + gstack/llms.txt + webdriver-only stealth (#1363) 2026-05-07 20:14:59 -07:00
migration-checkpoint-ownership.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
migrations-v1.27.0.0.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
model-overlay-opus-4-7.test.ts v1.13.0.0 feat: add Claude outside-voice skill (#1212) 2026-04-25 11:52:48 -07:00
no-stale-gstack-brain-refs.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
openclaw-native-skills.test.ts community wave: 6 PRs + hardening (v0.18.1.0) (#1028) 2026-04-17 00:45:13 -07:00
plan-tune.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
post-rename-doc-regen.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
pr-title-rewrite.test.ts v1.23.0.0 feat: always prefix PR titles with v<VERSION> (#1284) 2026-05-01 07:06:37 -07:00
preamble-compose.test.ts v1.10.0.0: fix AskUserQuestion cadence + Pros/Cons format upgrade (#1178) 2026-04-23 18:25:34 -07:00
readme-throughput.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
relink.test.ts fix: headed browser auto-shutdown + disconnect cleanup (v0.18.1.0) (#1025) 2026-04-16 15:39:44 -07:00
resolver-ask-user-format.test.ts v1.10.0.0: fix AskUserQuestion cadence + Pros/Cons format upgrade (#1178) 2026-04-23 18:25:34 -07:00
review-log.test.ts fix: community PRs + security hardening + E2E stability (v0.12.7.0) (#552) 2026-03-26 23:21:27 -06:00
secret-sink-harness.test.ts v1.12.0.0 feat: /setup-gbrain — coding-agent onboarding for gbrain (#1183) 2026-04-24 01:38:21 -07:00
setup-codesign.test.ts codex + Apple Silicon hardening wave (v0.18.4.0) (#1056) 2026-04-18 12:30:54 +08:00
setup-gbrain-path4-structure.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
ship-plan-completion-invariants.test.ts v1.30.0.0 fix wave: 21 community PRs + Windows CI extension + codex flag-semantics smoke (#1391) 2026-05-09 08:06:47 -07:00
ship-version-sync.test.ts fix(ship): detect + repair VERSION/package.json drift in Step 12 (v1.1.1.0) (#1063) 2026-04-18 23:58:59 +08:00
skill-budget-regression.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-collision-sentinel.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
skill-cross-model-recommendation-emit.test.ts v1.25.1.0 fix: office-hours Phase 4 STOP gate + AskUserQuestion recommendation judge (#1296) 2026-05-01 19:51:51 -07:00
skill-e2e-ask-user-question-format-compliance.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-e2e-auto-decide-preserved.test.ts v1.25.0.0 fix: AskUserQuestion resolves to host MCP variant when native is disallowed (#1287) 2026-05-01 08:45:36 -07:00
skill-e2e-autoplan-chain.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-e2e-autoplan-dual-voice.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
skill-e2e-benchmark-providers.test.ts test(benchmark-providers): drop literal 'ok' assertion on gemini smoke 2026-05-11 09:46:21 -07:00
skill-e2e-brain-privacy-gate.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
skill-e2e-bws.test.ts feat(v1.9.0.0): gbrain-sync — cross-machine gstack memory (#1151) 2026-04-23 17:54:54 -07:00
skill-e2e-context-skills.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
skill-e2e-cso.test.ts feat: /cso v2 — infrastructure-first security audit (v0.11.6.0) (#384) 2026-03-23 06:57:22 -07:00
skill-e2e-deploy.test.ts feat: /land-and-deploy first-run dry run + staging-first + trust ladder (v0.12.2.0) (#518) 2026-03-26 11:08:31 -07:00
skill-e2e-design.test.ts test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
skill-e2e-learnings.test.ts feat: recursive self-improvement — operational learning + full skill wiring (v0.13.8.0) (#647) 2026-03-31 23:08:22 -06:00
skill-e2e-memory-pipeline.test.ts v1.26.3.0 feat: /sync-gbrain skill + native code-surface orchestrator (#1314) 2026-05-04 09:29:48 -07:00
skill-e2e-office-hours-auto-mode.test.ts v1.25.0.0 fix: AskUserQuestion resolves to host MCP variant when native is disallowed (#1287) 2026-05-01 08:45:36 -07:00
skill-e2e-office-hours-phase4.test.ts v1.25.1.0 fix: office-hours Phase 4 STOP gate + AskUserQuestion recommendation judge (#1296) 2026-05-01 19:51:51 -07:00
skill-e2e-office-hours.test.ts feat: mode-posture energy fix for /plan-ceo-review and /office-hours (v1.1.2.0) (#1065) 2026-04-19 05:44:39 +08:00
skill-e2e-opus-47.test.ts feat(v1.5.2.0): Opus 4.7 migration — model overlay, voice, routing (#1117) 2026-04-22 01:06:22 -07:00
skill-e2e-overlay-harness.test.ts feat(v1.10.1.0): overlay efficacy harness + Opus 4.7 fanout nudge removal (#1166) 2026-04-23 18:42:58 -07:00
skill-e2e-plan-ceo-finding-count.test.ts v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire) (#1255) 2026-04-30 02:50:09 -07:00
skill-e2e-plan-ceo-finding-floor.test.ts v1.27.1.0 fix: anti-shortcut clause + gate-tier AskUserQuestion floor tests for all plan-* skills (#1354) 2026-05-06 20:27:20 -07:00
skill-e2e-plan-ceo-mode-routing.test.ts v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire) (#1255) 2026-04-30 02:50:09 -07:00
skill-e2e-plan-ceo-plan-mode.test.ts v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390) 2026-05-09 17:01:13 -07:00
skill-e2e-plan-design-finding-count.test.ts v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire) (#1255) 2026-04-30 02:50:09 -07:00
skill-e2e-plan-design-finding-floor.test.ts v1.27.1.0 fix: anti-shortcut clause + gate-tier AskUserQuestion floor tests for all plan-* skills (#1354) 2026-05-06 20:27:20 -07:00
skill-e2e-plan-design-plan-mode.test.ts v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390) 2026-05-09 17:01:13 -07:00
skill-e2e-plan-design-with-ui.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-e2e-plan-devex-finding-count.test.ts v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire) (#1255) 2026-04-30 02:50:09 -07:00
skill-e2e-plan-devex-finding-floor.test.ts v1.27.1.0 fix: anti-shortcut clause + gate-tier AskUserQuestion floor tests for all plan-* skills (#1354) 2026-05-06 20:27:20 -07:00
skill-e2e-plan-devex-plan-mode.test.ts v1.26.2.0 fix: plan-eng-review STOP gates always fire AskUserQuestion + report-at-bottom contract enforcement (#1313) 2026-05-03 20:26:59 -07:00
skill-e2e-plan-eng-finding-count.test.ts v1.21.1.0 test: tighten plan-ceo-review smoke (Step 0 must fire) (#1255) 2026-04-30 02:50:09 -07:00
skill-e2e-plan-eng-finding-floor.test.ts v1.27.1.0 fix: anti-shortcut clause + gate-tier AskUserQuestion floor tests for all plan-* skills (#1354) 2026-05-06 20:27:20 -07:00
skill-e2e-plan-eng-multi-finding-batching.test.ts v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390) 2026-05-09 17:01:13 -07:00
skill-e2e-plan-eng-plan-mode.test.ts v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390) 2026-05-09 17:01:13 -07:00
skill-e2e-plan-format.test.ts v1.25.1.0 fix: office-hours Phase 4 STOP gate + AskUserQuestion recommendation judge (#1296) 2026-05-01 19:51:51 -07:00
skill-e2e-plan-mode-no-op.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-e2e-plan-prosons.test.ts v1.10.0.0: fix AskUserQuestion cadence + Pros/Cons format upgrade (#1178) 2026-04-23 18:25:34 -07:00
skill-e2e-plan-tune.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
skill-e2e-plan.test.ts test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
skill-e2e-qa-bugs.test.ts test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
skill-e2e-qa-workflow.test.ts feat: CI evals on Ubicloud — 12 parallel runners + Docker image (v0.11.10.0) (#360) 2026-03-23 10:17:33 -07:00
skill-e2e-review-army.test.ts feat: Review Army — parallel specialist reviewers for /review (v0.14.3.0) (#692) 2026-03-30 22:07:50 -06:00
skill-e2e-review.test.ts test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
skill-e2e-session-intelligence.test.ts fix(checkpoint): rename /checkpoint → /context-save + /context-restore (v1.0.1.0) (#1064) 2026-04-19 08:38:19 +08:00
skill-e2e-setup-gbrain-bad-token.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
skill-e2e-setup-gbrain-remote.test.ts v1.27.0.0 feat: /setup-gbrain Path 4 (remote MCP) + brain → artifacts rename (#1351) 2026-05-06 19:37:53 -07:00
skill-e2e-ship-idempotency.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00
skill-e2e-sidebar.test.ts feat: declarative multi-host platform + OpenCode, Slate, Cursor, OpenClaw (v0.15.5.0) (#793) 2026-04-04 15:32:20 -07:00
skill-e2e-skillify.test.ts v1.20.0.0 feat: browser-skills runtime + gbrain-support carryover (#1233) 2026-04-28 20:08:04 -07:00
skill-e2e-workflow.test.ts test: bump remaining claude-opus-4-6 → 4-7 references 2026-05-10 11:08:26 -07:00
skill-e2e.test.ts feat: recursive self-improvement — operational learning + full skill wiring (v0.13.8.0) (#647) 2026-03-31 23:08:22 -06:00
skill-llm-eval.test.ts feat: voice directive for all skills (v0.12.3.0) (#520) 2026-03-26 17:31:53 -06:00
skill-parser.test.ts feat: SKILL.md template system, 3-tier testing, DX tools (v0.3.3) (#41) 2026-03-13 21:08:12 -07:00
skill-routing-e2e.test.ts feat: Confusion Protocol, Hermes + GBrain hosts, brain-first resolver (v0.18.0.0) (#1005) 2026-04-16 10:41:38 -07:00
skill-validation.test.ts v1.30.0.0 fix wave: 21 community PRs + Windows CI extension + codex flag-semantics smoke (#1391) 2026-05-09 08:06:47 -07:00
taste-engine.test.ts feat(v1.3.0.0): open agents learnings + cross-model benchmark skill (#1040) 2026-04-19 17:50:31 +08:00
team-mode.test.ts feat(v1.5.2.0): Opus 4.7 migration — model overlay, voice, routing (#1117) 2026-04-22 01:06:22 -07:00
telemetry.test.ts feat: community wave — 7 fixes, relink, sidebar Write, discoverability (v0.13.5.0) (#641) 2026-03-29 21:43:36 -06:00
test-free-shards.test.ts v1.24.0.0 feat: cross-platform hardening — curated Windows lane + Bun.which resolver + path-portability helper (#1252) 2026-05-01 07:21:28 -07:00
timeline.test.ts feat: Session Intelligence Layer — /checkpoint + /health + context recovery (v0.15.0.0) (#733) 2026-04-01 00:50:42 -06:00
touchfiles.test.ts v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390) 2026-05-09 17:01:13 -07:00
uninstall.test.ts feat: community PRs — faster install, skill namespacing, uninstall, Codex fallback, Windows fix, Python patterns (v0.12.9.0) (#561) 2026-03-27 00:44:37 -06:00
upgrade-migration-v1.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
v0-dormancy.test.ts feat: gstack v1 — simpler prompts + real LOC receipts (v1.0.0.0) (#1039) 2026-04-18 15:05:42 +08:00
worktree.test.ts feat: content security — 4-layer prompt injection defense for pair-agent (#815) 2026-04-06 14:41:06 -07:00
writing-style-resolver.test.ts v1.15.0.0 feat: slim preamble + real-PTY plan-mode E2E harness (#1215) 2026-04-26 13:55:13 -07:00