mirror of https://github.com/garrytan/gstack.git
2 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Garry Tan
|
675717e320
|
v1.17.0.0: setup-gbrain wireup ships the gbrain federation surface (#1234)
* feat: gstack-gbrain-source-wireup helper + 13 unit tests
The new bin/gstack-gbrain-source-wireup is the single helper that registers
the gstack brain repo as a gbrain federated source via `git worktree`, runs
incremental sync, and supports --uninstall + --probe + --strict modes.
Replaces the dead `consumers.json + ingest_url + /ingest-repo` HTTP wireup
introduced in v1.12.0.0 — that endpoint never shipped on the gbrain side.
The federation surface (`gbrain sources` / `gbrain sync`) shipped in gbrain
v0.18.0; this helper adapts to its actual semantics (no `sources update`, so
path drift recovery is `remove + re-add`; no `--install-cron` either, so
freshness rides on the existing skill-end push hook).
Source-id derivation is multi-fallback: ~/.gstack/.git origin URL →
~/.gstack-brain-remote.txt → --source-id flag. This makes `--uninstall`
work even after `~/.gstack/.git` is destroyed by the parent uninstall script.
Worktree is `--detach`ed at $GSTACK_HOME's HEAD because main is already
checked out there; advance is a re-checkout of the parent's current HEAD,
not a `git pull`. Divergence recovery removes + re-adds the worktree.
Test suite covers 13 cases: fresh-state registration, idempotent re-runs,
drift recovery, --strict failure modes, source-id fallback chain, --probe
non-mutation, sync errors, and --uninstall. Fake gbrain on $PATH, real git
ops at GSTACK_HOME tmp dir.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: wire setup-gbrain + brain-restore + brain-uninstall to use the helper
setup-gbrain Step 7 now invokes gstack-gbrain-source-wireup --strict after
gstack-brain-init + gbrain_sync_mode is set. Strict mode means the user sees
the failure rather than silently ending up with an unwired brain.
bin/gstack-brain-init drops 60 lines of dead code: the HTTP POST to
${GBRAIN_URL}/ingest-repo, the GBRAIN_URL_VAL/GBRAIN_TOKEN_VAL probes, the
consumers.json writer, and the chore commit step. CONSUMERS_FILE variable
declaration removed. The closing message no longer points at the dead
gstack-brain-consumer add path.
bin/gstack-brain-restore drops the 18-line consumers.json token-rehydration
block (was a no-op for the only consumer that ever existed). Adds a
best-effort wireup invocation after the brain-repo clone so 2nd-Mac restore
gets gbrain federation automatically. Failure prints a stderr WARNING but
does not abort the restore — restore's primary job is the git clone.
bin/gstack-brain-uninstall calls the helper's --uninstall mode (which
removes the gbrain source registration, the git worktree, and the
future-launchd-plist stub) before the existing legacy consumers.json
removal. Ordering is fragile-by-design: helper derives source-id via
multi-fallback so it works even after .git is destroyed.
bin/gstack-brain-consumer gets a DEPRECATED header note. Stays in the tree
for one cycle of grace; removal in v1.13.0.0.
setup-gbrain/SKILL.md is regenerated from the .tmpl via gen:skill-docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: v1.12.3.0 migration — wire existing brain-sync repos into gbrain
Idempotent migration script. For users who already opted into brain-sync
before this release (gbrain_sync_mode != off, ~/.gstack/.git exists), runs
the new gstack-gbrain-source-wireup helper so their existing brain repo
becomes searchable via gbrain immediately on /gstack-upgrade.
Skip conditions (each ends with exit 0):
- HOME unset or empty (defensive)
- gbrain_sync_mode = off or empty (user opted out)
- no ~/.gstack/.git (brain-init never ran)
- helper missing on disk (broken install)
No --strict on the helper invocation: missing or old gbrain is a benign
skip during a batch upgrade rather than a blocker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v1.12.3.0: setup-gbrain wireup ships the gbrain federation surface
Bumps VERSION 1.12.2.0 → 1.12.3.0 with a release-notes-format entry in
CHANGELOG.md. After upgrade, the placeholder consumers.json wireup is gone,
gbrain sources + sync + skill-end hook is the new path, your gstack memory
is actually searchable in gbrain.
The CHANGELOG entry follows the release-summary format from CLAUDE.md:
two-line bold headline, lead paragraph naming what shipped, "verify after
upgrade" command block readers can run on their own brain to see the
delta, then the standard Itemized changes / What this means / For
contributors sections.
Three pre-existing test failures on this branch are flagged in the
contributor section: the GSTACK_HOME isolation test (reads Garry's actual
~/.gstack/config.yaml), the 2MB tracked-binary test (security-bench
fixtures > 2MB), and the Opus 4.7 pacing-directive test (overlay text
drifted). All three were verified to fail on the base branch too — out
of scope for this PR, follow-up needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: helper locks GBRAIN_DATABASE_URL at startup, defends against config rewrites
The wireup helper previously read ~/.gbrain/config.json on every gbrain
subprocess invocation. On Garry's Mac, multiple concurrent test runs and
agent integrations were rewriting that file mid-sync, redirecting the
wireup at the wrong brain partway through a 4-min initial import.
This commit adds a `--database-url <url>` flag to the helper and locks
the URL at startup. Precedence:
1. --database-url flag (explicit caller intent)
2. GBRAIN_DATABASE_URL / DATABASE_URL env (CI / manual override)
3. read once from ~/.gbrain/config.json (default)
Whichever wins gets exported as GBRAIN_DATABASE_URL for every child
`gbrain` invocation. Per gbrain's loadConfig at src/core/config.ts:53,
env-var URLs override the file URL — so a process that flips config.json
between two of our gbrain calls can't redirect us. Defense-in-depth:
once the URL is locked, the wireup completes against the original brain
even under hostile filesystem conditions.
setup-gbrain/SKILL.md.tmpl Step 7 now reads the URL out of config.json
once (via python3 inline) and passes it explicitly with --database-url,
so even the very first wireup call is decoupled from config.json mutability.
Three new test cases cover the lock behavior:
- --database-url flag is exported to child gbrain calls
- falls back to ~/.gbrain/config.json when no flag and no env
- flag overrides env GBRAIN_DATABASE_URL and config.json values
The fake gbrain in the test suite now records GBRAIN_DATABASE_URL alongside
each call so tests can assert the helper exported the locked URL.
Total test count: 13 → 16 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump v1.12.3.0 references to v1.15.1.0 to match merged-with-main release
Internal-only renames after merging origin/main bumped this branch's release
target from v1.12.3.0 → v1.15.1.0:
- gstack-upgrade/migrations/v1.12.3.0.sh → v1.15.1.0.sh (rename + log-prefix
bump from "[v1.12.3.0]" to "[v1.15.1.0]")
- bin/gstack-brain-consumer header: "DEPRECATED in v1.12.3.0" → "DEPRECATED in
v1.15.1.0"; removal target bumped from v1.13.0.0 → v1.16.0.0 (next minor
after v1.15.1.0).
- bin/gstack-brain-uninstall: "no longer written ... since v1.12.3.0" →
"since v1.15.1.0".
No behavior change. Test suite still 16/16 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: 10 new cases close coverage gaps (helper defensive paths + migration)
/ship Step 7 coverage audit reported 48% (22/46 branches). Added 10 cases
covering the highest-impact gaps:
Helper (test/gstack-gbrain-source-wireup.test.ts, +3 cases → 19 total):
- --uninstall when gbrain is missing: best-effort exit 0, worktree still cleaned
- --no-pull skips HEAD advance on existing worktree (was untested)
- Stray non-git directory at worktree path is cleaned up + worktree created
Migration (test/gstack-upgrade-migration-v1_15_1_0.test.ts, NEW, 7 cases):
- HOME unset → defensive exit 0
- gbrain_sync_mode=off → exit 0 silently
- gbrain_sync_mode unset → exit 0 silently
- no ~/.gstack/.git → exit 0 silently
- helper missing on PATH → warning + exit 0
- happy path → invokes helper without --strict
- helper exits non-zero → migration prints retry hint, still exits 0 (non-blocking)
Also syncs package.json version from 1.15.0.0 → 1.15.1.0 to match VERSION
file (DRIFT_STALE_PKG repair from /ship Step 12 idempotency check; was a
manual-edit-bypass artifact from the merge step).
Coverage estimate: 48% → ~75%. Mainline + migration script + key defensive
paths all exercised. 26 tests total covering the new code surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: pre-landing review auto-fixes (5 correctness + observability)
/ship Step 9 review surfaced 9 INFORMATIONAL findings on the new helper +
migration. Five auto-fixed with no behavior regression (26/26 tests pass):
bin/gstack-gbrain-source-wireup:
- Version compare: put floor "0.18.0" first in `sort -V` stdin so equal-or-
greater $v always sorts to position 2. Stable across sort implementations.
- _worktree_add_detached: drop `2>/dev/null` on the `worktree add`, surface
git's stderr through `prefix` so users see WHY adds fail (disk, perms).
- ensure_worktree: same observability fix on the `git checkout --detach` path
during HEAD-advance, so users see the actual git error before recovery.
- do_probe: replace `[ -d X ] || [ -f X ] && set=present` (precedence trap —
the `&&` short-circuits when the dir branch fails) with explicit if-block.
- do_probe: capture `check_source_state`'s return code explicitly via
`set +e; ...; rc=$?; set -e`. `$?` after an `if`/`elif` chain is fragile
under set -e and may not reach the elif under some shell versions.
- do_wireup: same explicit return-code capture for `ensure_worktree`. The
prior `ensure_worktree || { if [ $? = 2 ]; ...` pattern relied on `$?`
reflecting the function's return after `||`, which is implementation-defined.
gstack-upgrade/migrations/v1.15.1.0.sh:
- Trim whitespace from `gstack-config get gbrain_sync_mode` output via
`tr -d '[:space:]'`. Trailing newlines would mis-classify "off\n" as a
non-empty non-off mode and incorrectly invoke the helper.
Skipped findings (cosmetic / out of scope):
- `python3 -c` reads `~/.gbrain/config.json` via `expanduser` instead of
the helper's `$GBRAIN_CONFIG` variable (cosmetic; HONORS HOME override).
- Long sync-failure error message could truncate to last N lines (cosmetic
log readability).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: adversarial review hardening (rm safety, jq probe, secret redaction, multi-Mac)
/ship Step 11 adversarial review surfaced 7 CRITICAL issues. Five fixed
inline (no behavior regression, 26/26 tests still pass):
bin/gstack-gbrain-source-wireup:
1. **rm -rf path validation** (was: F-c-CRITICAL 9/10).
Added `safe_rm_worktree` helper that refuses any path not strictly under
$HOME/, plus dangerous-path allowlist for /, /Users, $HOME root. Replaces
raw `rm -rf "$WORKTREE"` calls (lines 161, 169 originally). If user sets
GSTACK_BRAIN_WORKTREE="" or "/", the helper now dies cleanly instead of
nuking the home dir or root.
2. **jq dependency probe** (was: F-c-CRITICAL 9/10).
`check_source_state` now hard-fails with a clear message if jq is missing,
instead of silently returning "absent" → re-add → die-on-duplicate. Plus
trims whitespace from jq output (`tr -d '[:space:]'`) to defend against
gbrain emitting `\n` for missing fields. Header comment claimed jq was a
transitive dep; now we enforce it.
3. **Python heredoc warns on JSON parse failure** (was: F-c-CRITICAL 8/10).
Previously `except Exception: pass` silently swallowed malformed JSON,
leaving _locked_url empty and defeating the URL-lock defense. Now writes
the parse error to a temp file and warns the user that the URL was not
locked. Also passes the config path via env var (GBRAIN_CONFIG_PATH)
instead of hardcoded `~/.gbrain/config.json`, respecting any HOME override.
4. **Multi-Mac source-id collision fix** (was: F-c-CRITICAL 9/10).
When `check_source_state` returns 1 (source exists at different path), the
helper used to remove + re-add. Two Macs sharing one Supabase brain would
ping-pong the local_path metadata on every sync. Now: if the existing
path's basename matches the local worktree's basename (likely another
machine's local copy of the SAME brain repo), skip re-registration and
sync against the local worktree. gbrain stores pages by content; metadata
is informational. No more ping-pong.
5. **Redact DB URL from sync-failure error message** (was: F-c-CRITICAL 7/10).
`gbrain sync` failures used to echo the full stderr (which can contain
the postgres connection string with password) into the user's terminal
and any log redirect. Now we sed-replace any `postgres://...` with
`postgres://***REDACTED***` before the die() call, and only show the
last 10 lines.
Bonus minor fix: `die()` now uses `$1` instead of `$*` for the warn
message, so the exit-code arg ($2) doesn't get appended to the warning text.
Acknowledged-but-deferred:
- GBRAIN_DATABASE_URL env exposure on Linux via /proc/$PID/environ. This is
a Linux-only concern; gstack is Mac-targeted today and macOS restricts
process env reads. Document as a follow-up if Linux support lands.
- gbrain version parser brittleness if gbrain switches to "v0.18.0" prefix.
Defensive only; current gbrain output matches `gbrain X.Y.Z` exactly.
- bash 3.2 PIPESTATUS reliability. Tests pass on the host bash version (3.2+
via macOS); modern bash 5.x is widely available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: sync gbrain-source-wireup helper into USING_GBRAIN + gbrain-sync
USING_GBRAIN_WITH_GSTACK.md: add gstack-gbrain-source-wireup row to the bin
helpers table — describes federation registration via `gbrain sources add` +
worktree, lists flags, calls out it replaces the dead consumers.json/ingest-repo
HTTP wireup.
docs/gbrain-sync.md: replace the `gstack-brain-reader add --ingest-url` step
in gstack-brain-init's flow (which targeted the never-shipped /ingest-repo
endpoint) with the real flow — federate via gbrain sources + worktree, point
to bin/gstack-gbrain-source-wireup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* v1.16.1.0: rebump after queue-collision (PR #1233 took v1.16.0.0)
CI's "Check VERSION is not stale vs queue" job (job 73105686380) failed
with: "VERSION drift: PR #1234 claims v1.15.1.0 but the queue has moved —
next free slot is v1.16.1.0." PR #1233 (garrytan/browserharness) entered
the queue claiming v1.16.0.0 between when this branch's prior /ship ran
and when CI evaluated, so v1.15.1.0 is stale. Rebumping on top.
Files updated:
- VERSION 1.15.1.0 → 1.16.1.0
- package.json 1.15.1.0 → 1.16.1.0
- CHANGELOG.md heading + Before/After columns 1.15.1.0 → 1.16.1.0
- CHANGELOG removal target (consumers.json + config keys) 1.16.0.0 → 1.17.0.0
- gstack-upgrade/migrations/v1.15.1.0.sh → renamed v1.16.1.0.sh + log prefix
- bin/gstack-brain-consumer "DEPRECATED in" + "removal in" 1.15.1.0/1.16.0.0 → 1.16.1.0/1.17.0.0
- bin/gstack-brain-uninstall "since vX.Y.Z.W" 1.15.1.0 → 1.16.1.0
- test/gstack-upgrade-migration-v1_15_1_0.test.ts → renamed v1_16_1_0.test.ts
No behavior change. 26/26 wireup + migration tests still pass on the rename.
Full bun test suite: exit 0, 0 failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v1.17.0.0: rebump again — bump-detection now classifies branch as MINOR
CI's version-stale check (job 73106360896) failed: PR #1234 claims v1.16.1.0
but the queue moved to v1.17.0.0. Root cause: bumping 1.15.1.0 → 1.16.1.0
to dodge the prior collision turned the branch's diff classification from
PATCH (1.15.0 → 1.15.1) into MINOR (1.15.0 → 1.16.x). detect-bump.ts now
sees MINOR, gstack-next-version walks the MINOR lane past #1233's
v1.16.0.0 claim, and the next free slot is v1.17.0.0.
Honestly accurate per CLAUDE.md scale-aware bumps: this branch IS a
MINOR ("substantial new capability shipped — skill, harness, command,
big refactor"). The new helper + migration + integration totals ~1200
lines added across 11 files with 26 new tests. PATCH was always the
wrong honest classification; the queue collision forced the right
answer.
Files updated:
- VERSION 1.16.1.0 → 1.17.0.0
- package.json 1.16.1.0 → 1.17.0.0
- CHANGELOG.md heading + After column 1.16.1.0 → 1.17.0.0
- CHANGELOG removal targets 1.17.0.0 → 1.18.0.0
- gstack-upgrade/migrations/v1.16.1.0.sh → renamed v1.17.0.0.sh + log prefix
- bin/gstack-brain-consumer "DEPRECATED in" + "removal in" 1.16.1.0/1.17.0.0 → 1.17.0.0/1.18.0.0
- bin/gstack-brain-uninstall "since vX.Y.Z.W" 1.16.1.0 → 1.17.0.0
- test/gstack-upgrade-migration-v1_16_1_0.test.ts → renamed v1_17_0_0.test.ts
26/26 tests still pass. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
Garry Tan
|
9dbaf906cf
|
feat(v1.9.0.0): gbrain-sync — cross-machine gstack memory (#1151)
* feat(gbrain-sync): queue primitives + writer shims
Adds bin/gstack-brain-enqueue (atomic append to sync queue) and
bin/gstack-jsonl-merge (git merge driver, ts-sort with SHA-256 fallback).
Wires one backgrounded enqueue call into learnings-log, timeline-log,
review-log, and developer-profile --migrate. question-log and
question-preferences stay local per Codex v2 decision.
gstack-config gains gbrain_sync_mode (off/artifacts-only/full) and
gbrain_sync_mode_prompted keys, plus GSTACK_HOME env alignment so
tests don't leak into real ~/.gstack/config.yaml.
* feat(gbrain-sync): --once drain + secret scan + push
bin/gstack-brain-sync is the core sync binary. Subcommands: --once
(drain queue, allowlist-filter, privacy-class-filter, secret-scan
staged diff, commit with template, push with fetch+merge retry),
--status, --skip-file <path>, --drop-queue --yes, --discover-new
(cursor-based detection of artifact writes that skip the shim).
Secret regex families: AWS keys, GitHub tokens (ghp_/gho_/ghu_/ghs_/
ghr_/github_pat_), OpenAI sk-, PEM blocks, JWTs, bearer-token-in-JSON.
On hit: unstage, preserve queue, print remediation hint (--skip-file
or edit), exit clean. No daemon — invoked by preamble at skill
boundaries.
* feat(gbrain-sync): init, restore, uninstall, consumer registry
bin/gstack-brain-init: idempotent first-run. git init ~/.gstack/,
.gitignore=*, canonical .brain-allowlist + .brain-privacy-map.json,
pre-commit secret-scan hook (defense-in-depth), merge driver registration
via git config, gh repo create --private OR arbitrary --remote <url>,
initial push, ~/.gstack-brain-remote.txt for new-machine discovery,
GBrain consumer registration via HTTP POST.
bin/gstack-brain-restore: safe new-machine bootstrap. Refuses clobber
of existing allowlisted files, clones to staging, rsync-copies tracked
files, re-registers merge drivers (required — not cloned from remote),
rehydrates consumers.json, prompts for per-consumer tokens.
bin/gstack-brain-uninstall: clean off-ramp. Removes .git + .brain-*
files + consumers.json + config keys. Preserves user data (learnings,
plans, retros, profile). Optional --delete-remote for GitHub repos.
bin/gstack-brain-consumer + bin/gstack-brain-reader (symlink alias):
registry management. Internal 'consumer' term; user-facing 'reader'
per DX review decision.
* feat(gbrain-sync): preamble block — privacy gate + boundary sync
scripts/resolvers/preamble/generate-brain-sync-block.ts emits bash that
runs at every skill invocation:
- Detects ~/.gstack-brain-remote.txt on machines without local .git
and surfaces a restore-available hint (does NOT auto-run restore).
- Runs gstack-brain-sync --once at skill start to drain any pending
writes (and at skill end via prose instruction).
- Once-per-day auto-pull (cached via .brain-last-pull) for append-only
JSONL files.
- Emits BRAIN_SYNC: status line every skill run.
Also emits prose for the host LLM to fire the one-time privacy
stop-gate (full / artifacts-only / off) when gbrain is detected and
gbrain_sync_mode_prompted is false. Wired into preamble.ts composition.
* test(gbrain-sync): 27-test consolidated suite
test/brain-sync.test.ts covers:
- Config: validation, defaults, GSTACK_HOME env isolation
- Enqueue: no-op gates, skip list, concurrent atomicity, JSON escape
- JSONL merge driver: 3-way + ts-sort + SHA-256 fallback
- Init + sync: canonical file creation, merge driver registration,
push-reject + fetch+merge retry path
- Init refuses different remote (idempotency)
- Cross-machine restore round-trip (machine A write → machine B sees)
- Secret scan across all 6 regex families (AWS, GH, OpenAI, PEM, JWT,
bearer-JSON). --skip-file unblock remediation
- Uninstall removes sync config, preserves user data
- --discover-new idempotence via mtime+size cursor
Behaviors verified via integration smokes during implementation. Known
follow-up: bun-test 5s default timeout needs 30s wrapper for
spawnSync-heavy tests.
* docs(gbrain-sync): user guide + error lookup + README section
docs/gbrain-sync.md: setup walkthrough, privacy modes, cross-machine
workflow, secret protection, two-machine conflict handling, uninstall,
troubleshooting reference.
docs/gbrain-sync-errors.md: problem/cause/fix index for every
user-visible error. Patterned on Rust's error docs + Stripe's API
error reference.
README.md: 'Cross-machine memory with GBrain sync' section near the
top (discovery moment), plus docs-table entry.
* chore: bump version and changelog (v1.7.0.0)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: regenerate SKILL.md files for gbrain-sync preamble block
Re-runs bun run gen:skill-docs after adding generateBrainSyncBlock
to scripts/resolvers/preamble.ts in
|