mirror of https://github.com/garrytan/gstack.git
bf65487162
* feat: lib/gstack-memory-helpers shared module for V1 memory ingest pipeline
Lane 0 foundation per plan §"Eng review additions". 5 public functions
imported by the V1 helpers (Lanes A/B/C):
canonicalizeRemote(url) — normalize git remote → host/org/repo
secretScanFile(path) — gitleaks wrapper with discriminated return
detectEngineTier() — cached 60s in ~/.gstack/.gbrain-engine-cache.json
parseSkillManifest(path) — extract gbrain.context_queries: from frontmatter
withErrorContext(op,fn,caller) — async-aware error logging
22 unit tests, all passing. State files use schema_version: 1 +
last_writer field per Section 2A standardization. Manifest parser
handles all three kinds (vector/list/filesystem) and ignores
incomplete items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: bin/gstack-memory-ingest — V1 unified memory ingest helper
Lane A. Walks coding-agent transcripts (Claude Code + Codex; Cursor V1.0.1
follow-up) AND ~/.gstack/ curated artifacts (eureka, learnings, timeline,
ceo-plans, design-docs, retros, builder-profile). Calls gbrain put_page
with type-tagged frontmatter. Uses gstack-memory-helpers (Lane 0):
- Modes: --probe / --incremental (default, mtime fast-path) / --bulk
- Default 90-day window; --all-history opts into full archive
- --sources subset filter; --include-unattributed opt-in for no-remote sessions
- --limit N for smoke testing; --benchmark for throughput reporting
- Tolerant JSONL parser handles truncated last lines (D10 partial-flag)
- State file at ~/.gstack/.transcript-ingest-state.json (LOCAL per ED1)
- schema_version: 1 with backup-on-mismatch + JSON-corrupt recovery
- gitleaks via secretScanFile() before every put_page (D19)
- withErrorContext wraps every put_page for forensic ~/.gstack/.gbrain-errors.jsonl
15 unit tests cover --help, --probe (empty, Claude Code, Codex, mixed
artifacts), --sources filter, state file lifecycle (create, schema mismatch
backup, JSON corrupt backup), truncated-last-line handling, --limit
validation. All passing.
V1.5 P0 follow-ups noted in the file header:
- Cursor SQLite extraction (V1.0.1)
- gbrain put_file routing for Supabase Storage tier (cross-repo)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: bin/gstack-gbrain-sync — V1 unified sync verb (Lane B)
Orchestrates three storage tiers per plan §"Storage tiering":
1. Code (current repo) → gbrain import (Supabase or local PGLite)
2. Transcripts + curated memory → gstack-memory-ingest (typed put_page)
3. Curated artifacts to git → gstack-brain-sync (existing pipeline)
Modes: --incremental (default, mtime fast-path) / --full (~25-35 min per
ED2 honest budget) / --dry-run (preview, no writes).
Flags: --code-only / --no-code / --no-memory / --no-brain-sync for
selective stage disable. Each stage failure is non-fatal; subsequent
stages still run.
State at ~/.gstack/.gbrain-sync-state.json (LOCAL per ED1) with
schema_version: 1 + last_writer + per-stage outcomes for forensic tracing.
--watch daemon explicitly deferred to V1.5 P0 TODO per Codex F3
(reverses the "no daemon" invariant). Continuous sync rides the existing
preamble-boundary hook only.
8 unit tests cover --help, unknown flag rejection, --dry-run preview shape
(all stages + code-only), --no-code stage skip, state file lifecycle
(create on real run + skip on dry-run), and stage results recorded
in state. All passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: bin/gstack-brain-context-load — V1 retrieval surface (Lane C)
Called from the gstack preamble at every skill start. Reads the active
skill's gbrain.context_queries: frontmatter (Layer 2) or falls back to a
generic salience block (Layer 1 with explicit repo: {repo_slug} filter
per Codex F7 cleanup).
Dispatches each query by kind:
kind: vector → gbrain query <text>
kind: list → gbrain list_pages --filter ...
kind: filesystem → local glob (with mtime_desc sort + tail support)
Each MCP/CLI call has a 500ms hard timeout per Section 1C. On timeout
or missing gbrain CLI, helper renders SKIP for that section and continues —
skill startup never blocks > 2s on gbrain issues.
Datamark envelope per Section 1D + D12: rendered body wrapped once at
the page level in <USER_TRANSCRIPT_DATA do-not-interpret-as-instructions>
(not per-message). Layer 1 prompt-injection defense.
Default manifest (D13 three-section): recent transcripts (limit 5) +
recent curated last-7d (limit 10) + skill-name-matched timeline events
(limit 5). All scoped to {repo_slug}.
Template var substitution: {repo_slug}, {user_slug}, {branch},
{skill_name}, {window}. Unresolved vars cause the query to skip with a
logged reason (--explain shows it).
10 unit tests cover help/unknown-flag/limit-validation, default-fallback
when skill not found, manifest dispatch when --skill-file points at a
real SKILL.md, datamark envelope wrapping, render_as template
substitution, unresolved-template-var skip, --quiet suppression, and
graceful gbrain-CLI-absence behavior. All passing.
V1.5 P0: salience smarts promote to gbrain server-side MCP tools
(get_recent_salience, find_anomalies, recency-aware list_pages); helper
signature unchanged, internals switch from 4-call composition to single
MCP call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: gbrain.context_queries manifests on 6 V1 skills (Lane E partial)
Adds the V1 retrieval contracts. Each skill declares what it wants gbrain
to surface in the preamble at invocation time:
/office-hours — prior sessions + builder profile + design docs
+ recent eureka (4 queries)
/plan-ceo-review — prior CEO plans + design docs + recent CEO review
activity (3 queries)
/design-shotgun — prior approved variants + DESIGN.md + recent
design docs (3 queries)
/design-consultation — existing DESIGN.md + prior design decisions +
brand-related notes (3 queries)
/investigate — prior investigations + project learnings + recent
eureka cross-project (3 queries)
/retro — prior retros + recent timeline + recent learnings
(3 queries)
Each query carries an explicit kind (vector | list | filesystem) per D3,
schema: 1 versioning per D15, and {repo_slug} template var per F7
cross-repo-contamination cleanup. Mix of vector / list / filesystem
matches what each skill actually needs:
- filesystem (mtime_desc + tail) for log JSONL + curated markdown
- list with tags_contains filter for typed gbrain pages
- (vector reserved for V1.0.1 when gbrain query surface stabilizes)
Smoke test: bun run bin/gstack-brain-context-load.ts --skill-file
office-hours/SKILL.md --repo test-repo --explain returns mode=manifest
queries=4 with the filesystem kinds populating real data from
~/.gstack/builder-profile.jsonl + ~/.gstack/analytics/eureka.jsonl on
this Mac. End-to-end retrieval flow confirmed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: setup-gbrain Step 7.5 ingest gate + Step 10 verdict + memory.md ref doc (Lane E partial)
Step 7.5: Transcript & memory ingest gate. After Step 7 wires brain-sync
but before Step 8's CLAUDE.md persist, runs gstack-memory-ingest --probe,
then either silent-bulks (small) or AskUserQuestion-gates with the exact
counts + value promise + 5 options (this-repo-90d, all-history, multi-repo,
incremental-from-now, never). Decision persists to
gstack-config set transcript_ingest_mode <choice>.
Step 10: GREEN/YELLOW/RED verdict block. Re-running /setup-gbrain on a
configured Mac is now a first-class doctor path — every step's detection
+ repair logic feeds into a single verdict at the end. Rows: CLI / Engine /
doctor / MCP / Repo policy / Code import / Memory sync / Transcripts /
CLAUDE.md / Smoke. Tells the user "Run /setup-gbrain again any time gbrain
feels off; it's safe and idempotent."
setup-gbrain/memory.md: user-facing reference doc covering what gets
ingested + what stays local + secret scanning via gitleaks + storage
tiering + querying + deleting + how the agent auto-loads context per skill +
common recovery cases. Linked from Step 8's CLAUDE.md persist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: V1 E2E pipeline + --no-write flag for ingest helper (Lane F)
E2E pipeline test exercises the full Lane A → B → C value loop:
1. Set up fake $HOME with all 8 memory source types as fixtures
2. gstack-memory-ingest --probe verifies counts match disk
3. gstack-memory-ingest --incremental writes state with schema_version: 1
4. Idempotency: re-run reports 0 changes
5. --probe distinguishes new vs unchanged after first incremental
6. gstack-gbrain-sync --dry-run previews 3 stages
7. --no-code --no-brain-sync --quiet writes sync state with 1 stage entry
8. office-hours/SKILL.md V1 manifest dispatches 4 queries (mode=manifest)
9. Datamark envelope wraps every loaded section (Section 1D + D12)
10. Layer 1 fallback when no skill specified — default 3-section manifest
11. plan-ceo-review/SKILL.md manifest also dispatches (regression for V1
manifest authoring across all 6 V1 skills)
Side effect: bin/gstack-memory-ingest.ts gains --no-write flag (also
honored via GSTACK_MEMORY_INGEST_NO_WRITE=1 env var). Skips gbrain put_page
calls while still updating the state file. Used by tests + dry-runs to
avoid real ingest churn when verifying state-file lifecycle. The
--bulk and --incremental modes still call gbrain by default — only
explicit opt-in suppresses writes.
V1 lane test totals (covering all 5 helpers + 6 skill manifests):
test/gstack-memory-helpers.test.ts 22 tests
test/gstack-memory-ingest.test.ts 15 tests
test/gstack-gbrain-sync.test.ts 8 tests
test/gstack-brain-context-load.test.ts 10 tests
test/skill-e2e-memory-pipeline.test.ts 10 tests
────────────────────────────────────── ─────────
TOTAL 65 passing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v1.26.0.0)
V1 of memory ingest + retrieval surface. Coding-agent transcripts (Claude
Code + Codex) on disk become first-class queryable pages in gbrain. Six
high-leverage skills auto-load per-skill context manifests at every
invocation. Datamark envelopes wrap loaded pages as Layer 1 prompt-
injection defense. Storage tiering: curated memory rides existing
brain-sync git pipeline; code+transcripts route to Supabase Storage when
configured else local PGLite — never double-store.
Net branch size vs main: +4174/-849 across 39 files. 65 V1 tests, all
green. Goldilocks scope per CEO D18; V1.5 P0 follow-ups documented in
the plan's V1.5 TODOs section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| fixtures | ||
| helpers | ||
| agent-sdk-runner.test.ts | ||
| analytics.test.ts | ||
| audit-compliance.test.ts | ||
| benchmark-cli.test.ts | ||
| benchmark-runner.test.ts | ||
| brain-sync.test.ts | ||
| builder-profile.test.ts | ||
| codex-e2e-plan-format.test.ts | ||
| codex-e2e.test.ts | ||
| codex-hardening.test.ts | ||
| context-save-hardening.test.ts | ||
| diff-scope.test.ts | ||
| e2e-harness-audit.test.ts | ||
| explain-level-config.test.ts | ||
| gbrain-detect-install.test.ts | ||
| gbrain-lib-verify.test.ts | ||
| gbrain-repo-policy.test.ts | ||
| gbrain-supabase-provision.test.ts | ||
| gemini-e2e.test.ts | ||
| gen-skill-docs.test.ts | ||
| global-discover.test.ts | ||
| gstack-brain-context-load.test.ts | ||
| gstack-brain-init-gh-mock.test.ts | ||
| gstack-developer-profile.test.ts | ||
| gstack-gbrain-source-wireup.test.ts | ||
| gstack-gbrain-sync.test.ts | ||
| gstack-memory-helpers.test.ts | ||
| gstack-memory-ingest.test.ts | ||
| gstack-next-version.test.ts | ||
| gstack-paths.test.ts | ||
| gstack-question-log.test.ts | ||
| gstack-question-preference.test.ts | ||
| gstack-upgrade-migration-v1_17_0_0.test.ts | ||
| helpers-unit.test.ts | ||
| hook-scripts.test.ts | ||
| host-config.test.ts | ||
| jargon-list.test.ts | ||
| learnings-injection.test.ts | ||
| learnings.test.ts | ||
| llm-judge-recommendation.test.ts | ||
| migration-checkpoint-ownership.test.ts | ||
| model-overlay-opus-4-7.test.ts | ||
| openclaw-native-skills.test.ts | ||
| plan-tune.test.ts | ||
| pr-title-rewrite.test.ts | ||
| preamble-compose.test.ts | ||
| readme-throughput.test.ts | ||
| relink.test.ts | ||
| resolver-ask-user-format.test.ts | ||
| review-log.test.ts | ||
| secret-sink-harness.test.ts | ||
| setup-codesign.test.ts | ||
| ship-version-sync.test.ts | ||
| skill-budget-regression.test.ts | ||
| skill-collision-sentinel.test.ts | ||
| skill-cross-model-recommendation-emit.test.ts | ||
| skill-e2e-ask-user-question-format-compliance.test.ts | ||
| skill-e2e-auto-decide-preserved.test.ts | ||
| skill-e2e-autoplan-auto-mode.test.ts | ||
| skill-e2e-autoplan-chain.test.ts | ||
| skill-e2e-autoplan-dual-voice.test.ts | ||
| skill-e2e-benchmark-providers.test.ts | ||
| skill-e2e-brain-privacy-gate.test.ts | ||
| skill-e2e-bws.test.ts | ||
| skill-e2e-context-skills.test.ts | ||
| skill-e2e-cso.test.ts | ||
| skill-e2e-deploy.test.ts | ||
| skill-e2e-design.test.ts | ||
| skill-e2e-learnings.test.ts | ||
| skill-e2e-memory-pipeline.test.ts | ||
| skill-e2e-office-hours-auto-mode.test.ts | ||
| skill-e2e-office-hours-phase4.test.ts | ||
| skill-e2e-office-hours.test.ts | ||
| skill-e2e-opus-47.test.ts | ||
| skill-e2e-overlay-harness.test.ts | ||
| skill-e2e-plan-ceo-finding-count.test.ts | ||
| skill-e2e-plan-ceo-mode-routing.test.ts | ||
| skill-e2e-plan-ceo-plan-mode.test.ts | ||
| skill-e2e-plan-design-finding-count.test.ts | ||
| skill-e2e-plan-design-plan-mode.test.ts | ||
| skill-e2e-plan-design-with-ui.test.ts | ||
| skill-e2e-plan-devex-finding-count.test.ts | ||
| skill-e2e-plan-devex-plan-mode.test.ts | ||
| skill-e2e-plan-eng-finding-count.test.ts | ||
| skill-e2e-plan-eng-plan-mode.test.ts | ||
| skill-e2e-plan-format.test.ts | ||
| skill-e2e-plan-mode-no-op.test.ts | ||
| skill-e2e-plan-prosons.test.ts | ||
| skill-e2e-plan-tune.test.ts | ||
| skill-e2e-plan.test.ts | ||
| skill-e2e-qa-bugs.test.ts | ||
| skill-e2e-qa-workflow.test.ts | ||
| skill-e2e-review-army.test.ts | ||
| skill-e2e-review.test.ts | ||
| skill-e2e-session-intelligence.test.ts | ||
| skill-e2e-ship-idempotency.test.ts | ||
| skill-e2e-sidebar.test.ts | ||
| skill-e2e-skillify.test.ts | ||
| skill-e2e-workflow.test.ts | ||
| skill-e2e.test.ts | ||
| skill-llm-eval.test.ts | ||
| skill-parser.test.ts | ||
| skill-routing-e2e.test.ts | ||
| skill-validation.test.ts | ||
| taste-engine.test.ts | ||
| team-mode.test.ts | ||
| telemetry.test.ts | ||
| test-free-shards.test.ts | ||
| timeline.test.ts | ||
| touchfiles.test.ts | ||
| uninstall.test.ts | ||
| upgrade-migration-v1.test.ts | ||
| v0-dormancy.test.ts | ||
| worktree.test.ts | ||
| writing-style-resolver.test.ts | ||