mirror of https://github.com/garrytan/gstack.git
2 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Garry Tan
|
a861c00cfa
|
v1.58.3.0 feat: gbrowser anti-detection Layer C stealth (#2047)
* feat: Layer C stealth — chrome.*, Notification, per-install hardware, toString Proxy (gbrowser T1+T3+D6)
Three additions stacked into the existing applyStealth() init script
to close the visible automation tells that today push GBrowser users
into Google's /sorry/index captcha and similar:
T1 — Strip Playwright's automation default args:
--enable-automation (kills "Chrome is being
controlled" infobar)
--disable-popup-blocking, --disable-component-update,
--disable-default-apps (Patchright's list — each
is a documented tell)
Now centralized in STEALTH_IGNORE_DEFAULT_ARGS export, used by BOTH
launchHeaded() and handoff() (the headless → headed re-launch path).
D6 — Drop "GStackBrowser" UA branding suffix:
Real Chrome's UA ends `Safari/537.36`, not `Safari/537.36 GStackBrowser`.
The branded suffix was a high-entropy classifier for any vendor that
grep'd UA for known automation/test-browser strings. Branding still
lives in the wrapper .app name + Dock icon + tray — does not need
to leak via the UA string for the product to be "GBrowser." Resolves
the "looks like Chrome but identifies as GStackBrowser" contradiction
codex review #18 flagged.
T3 — Layer C init-script additions in stealth.ts:
1. Function.prototype.toString Proxy (must run first). Wraps every
patched getter / function in a WeakSet so they report
`function NAME() { [native code] }` at every recursion depth,
defeating the depth-3+ integrity check
(fn.toString.toString.toString().includes('[native code]')).
2. window.chrome.runtime / chrome.app / chrome.csi / chrome.loadTimes
restoration with full enum shape (OnInstalledReason, PlatformArch,
PlatformOs, etc.) + method bodies. Real Chrome ships these; their
absence is universally checked. Vendor research (gbrowser plan
deep-dive on Cloudflare + DataDome) confirmed both vendors probe
this shape directly.
3. Notification.permission aligned to 'default'. The existing inline
addInitScript already spoofs permissions.query({name:'notifications'})
to return 'prompt' — Notification.permission being 'denied' while
Permissions returns 'prompt' is a cross-source inconsistency that
detectors flag specifically.
4. Per-install hardware values via GSTACK_HW_CONCURRENCY /
GSTACK_DEVICE_MEMORY env vars (set by gbd's host_profile.go from
system_profiler + sysctl). Reporting real host values within the
Chrome shape avoids the cross-user GBrowser fingerprint cluster
that hardcoded defaults would create. Codex review #10 flagged
hardcoding as creating contradictions across Apple Silicon / Intel
/ UA-CH architecture.
5. Selenium 25-global cleanup + PhantomJS + NightmareJS + Watir +
Playwright (__pwInitScripts, __playwright__binding__) static-name
deletion. The inline block continues to handle the dynamic
cdc_/__webdriver/__selenium/__driver prefixes.
D7 (codex correction) kept: still do NOT fake navigator.plugins or
navigator.languages. Synthesizing those triggers MORE consistency
flags from modern fingerprinters than letting Chromium surface them
natively.
Test coverage:
- 15 new tests in stealth-layer-c.test.ts covering: launch-flag
exports, script structure, toString-Proxy installs first, every
spoof present, hardware values interpolated from input (not
hardcoded), Selenium global cleanup spot-check, no GStackBrowser
leak in stealth payload, backwards-compat exports preserved.
- All 8 existing stealth-webdriver tests still pass.
- All 2 existing browser-manager-unit tests still pass.
For GBrowser specifically: this is the gstack-side half of Phase 1 / T1
+ T3 + D6 in the anti-detection plan. The gbrowser repo's submodule
pointer bump will land alongside this.
* feat: buildGStackLaunchArgs — Pack 1 cmdline-switch construction for gbrowser
New stealth.ts export that turns the GSTACK_* env vars (already populated
by gbrowser's gbd from host_profile.go) into the --gstack-* cmdline
switches the Pack 1 Chromium patches read at WebGL getParameter,
NavigatorUA::userAgentData, NavigatorConcurrentHardware::hardwareConcurrency,
and NavigatorDeviceMemory::deviceMemory time.
Wired into all three launchArgs sites: launch() (headless), launchHeaded()
(real product path), and handoff() (headless → headed re-launch).
Mapping:
GSTACK_GPU_VENDOR → --gstack-gpu-vendor
GSTACK_GPU_RENDERER → --gstack-gpu-renderer
GSTACK_PLATFORM → --gstack-ua-platform (with mapping:
MacARM/MacIntel → macOS, Win32 → Windows,
Linux x86_64 → Linux)
GSTACK_GPU_CHIPSET → --gstack-ua-model
GSTACK_HW_CONCURRENCY → --gstack-hw-concurrency
GSTACK_DEVICE_MEMORY → --gstack-device-memory
Each switch is emitted only when its env var is non-empty — empty
values fall through to the patch's "no override" path, which returns
the real Chromium native value. Safe to ship on Chromium builds
without the Pack 1 patches applied (zero behavior change).
The patches themselves live in the gbrowser repo at chromium/patches/
{webgl-vendor-spoof,ua-client-hints-stealth,worker-navigator-stealth}.patch.
Both halves (gstack arg construction + gbrowser C++ patches) must
land + Chromium rebuild before the spoof reaches the WebGL/UA-CH/
hardware accessors. Currently dormant until then.
Tests (browse/test/stealth-layer-c.test.ts):
7 new buildGStackLaunchArgs cases — empty env, all-populated, partial,
platform mapping (MacARM/MacIntel/Win32/Linux), unrecognized platform
fallthrough, vendor-with-spaces escape-safety.
All 32 stealth/browser-manager tests pass.
For GBrowser specifically: gstack-side half of the Pack 1 flag plumbing.
gbrowser repo will bump the submodule pointer to this commit, then re-run
bun run test/anti-bot/evidence-run.ts to verify creepjs's "33% headless"
score drops after Pack 1 + Chromium rebuild.
* feat: buildGStackLaunchArgs adds --gstack-suppress-prepare-stack-trace
Pack 2 / B11 flag plumbing for the new
error-preparestacktrace-stealth.patch in gbrowser/chromium/patches/.
Always emit --gstack-suppress-prepare-stack-trace unless the caller
explicitly sets GSTACK_CDP_STEALTH=off in the environment. Off by
default in patch behavior (no-op without the C++ patch), so this is
safe on stock Playwright Chromium too.
Closes the Cloudflare canary trick where a page sets
Error.prepareStackTrace and watches for it to fire during CDP
serialization of a logged Error object.
Tests:
All 33 stealth/browser-manager tests pass. New cases:
- GSTACK_CDP_STEALTH=off disables suppression
- empty env still emits the always-on flag (count=1)
- all-populated env now emits 7 flags (was 6)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(browse): enable Chromium sandbox on headed launchPersistentContext
Mirrors v1.40.0.1 from main lineage (PR #1617). Cherry-picked onto
gbrowser-anti-detection so the GBrowser submodule can consume the fix
without waiting for main to merge.
Playwright auto-adds --no-sandbox whenever chromiumSandbox !== true
(playwright-core/lib/server/chromium/chromium.js:291-292). The headless
chromium.launch() site set the option; the two headed sites
(launchHeaded() and handoff()) did not. Every headed launch on macOS
and Linux showed Chromium's yellow "unsupported command-line flag:
--no-sandbox" infobar.
shouldEnableChromiumSandbox() centralizes the Win32 / CI / CONTAINER /
root heuristic that previously lived only in the headless path's
explicit --no-sandbox push at :225. All three launch sites now use the
helper, and six unit tests pin the policy across darwin, linux, win32,
CI, CONTAINER, and root.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v1.40.0.2 fix(browse): Cmd+Q on managed Chromium stops triggering supervisor respawn
Three browser.on('disconnected') handlers in browse/src/browser-manager.ts
(launch, launchHeaded, handoff) each exited with a non-zero code on every
disconnect, regardless of cause. Process supervisors that consume our exit
code (gbrowser's gbd HealthMonitor in cmd/gbd/health.go) treated user
Cmd+Q identical to a Chromium crash and respawned with exponential
backoff, so the visible browser kept reappearing after the user closed it.
Add resolveDisconnectCause(browser) that reads the underlying ChildProcess
exitCode + signalCode (waiting up to 1s for the exit event if the
disconnected event fired first). Exit code 0 + no signal = clean user
quit; anything else = crash, signal-kill, or OOM.
Wire the resolver into all three disconnect handlers:
- launch() (headless): clean → exit 0, crash → exit 1 (was always 1)
- launchHeaded() (headed): clean → exit 0, crash → exit 2 (was always 2)
onDisconnect() cleanup callback still runs in both cases.
- handoff() (re-launch): same as launch() via the helper.
Preserve the per-path crash codes (1 vs 2) so any supervisor that
differentiated headed vs headless crashes keeps working.
Seven new unit tests in browse-manager-unit.test.ts cover the resolver
across already-exited, signal-killed (SIGSEGV / SIGKILL), async exits,
and null-browser inputs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(browse): apply stealth on every launch path + share automation-artifact cleanup
handoff() built cmdline args but never called applyStealth, so a handed-off
browser had no JS stealth (no webdriver mask, no chrome.* shape, no toString
proxy). And the cdc_/Permissions cleanup shim lived inline in launchHeaded()
only, so headless launch() reported Notification.permission='default' without
the matching permissions.query='prompt' answer — the exact cross-source
inconsistency the shim exists to prevent.
Move the cleanup into AUTOMATION_ARTIFACT_CLEANUP_SCRIPT inside applyStealth so
all three launch paths (launch, launchHeaded, handoff) get identical stealth,
and call applyStealth(newContext) in handoff() before restoreState() navigates.
A static tripwire in browser-manager-unit.test.ts fails CI if any launch path
drops the applyStealth call again.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(browse): make --gstack-suppress-prepare-stack-trace opt-in, not default-on
buildGStackLaunchArgs() pushed the flag unless GSTACK_CDP_STEALTH=off, i.e.
on-by-default — contradicting its own comment ("off by default, only for
gbrowser builds"). The switch is read by a C++ patch that only exists in
gbrowser; on stock Playwright Chromium it is an unknown switch.
Flip to opt-in: emit only when GSTACK_CDP_STEALTH is on/1/true. gbd opts in by
exporting GSTACK_CDP_STEALTH=on; stock installs leave it unset so the flag
never reaches a Chromium that wouldn't understand it. Comment now matches code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(browse): correct stale stealth comments
The file-level stealth.ts docstring claimed "we DON'T fake navigator.plugins"
while the same file now ships EXTENDED_STEALTH_SCRIPT, which does fake plugins
when GSTACK_STEALTH=extended. Clarify that Layer C (the always-on default)
doesn't fake plugins and the opt-in extended mode does, as the documented
"actively lies, may break sites" escape hatch.
Also fix the launch()/launchHeaded() comments that said "mask navigator.webdriver
only" — applyStealth (Layer C) also restores window.chrome.*, aligns
Notification.permission, and sets per-install hardware.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(browse): runtime + extended-mode coverage for the stealth blend
The stealth tests were all static string-shape assertions; nothing executed
the script in a real page. Add real-Chromium runtime checks via applyStealth +
page.evaluate:
- Layer C runtime: window.chrome.* rich shape, Notification.permission='default'
paired with permissions.query notifications='prompt' (guards the shim now
running on every path), and patched getters reporting [native code].
- Per-install hardware: navigator.hardwareConcurrency/deviceMemory reflect the
GSTACK_* env profile.
- Extended-mode blend: navigator.plugins is faked when GSTACK_STEALTH=extended,
Layer C still wins window.chrome.runtime, and navigator.webdriver stays false
(own-prop getter survives extended's prototype delete).
- Persistent-context (launchHeaded/handoff) parity now uses a page created
AFTER applyStealth — the old test checked pages()[0], which predates the init
script, so webdriver was false only via the launch arg, not Layer C.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(browse): handoff() + launchHeaded() spread the shared STEALTH_LAUNCH_ARGS
handoff() built its launch args from only ['--hide-crash-restore-bubble',
...buildGStackLaunchArgs()], omitting STEALTH_LAUNCH_ARGS — so a handed-off
browser kept the --disable-blink-features=AutomationControlled tell that
launch() and launchHeaded() strip. launchHeaded() also hardcoded the flag as a
literal. Both now spread the shared constant, so the AutomationControlled flag
lives in one place across all three launch paths.
Tripwires: STEALTH_LAUNCH_ARGS spread into >= 3 sites (no inline literal) and
STEALTH_IGNORE_DEFAULT_ARGS wired into both persistent-context paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(browse): drop dead HostProfile.platform, export test internals
HostProfile.platform was set by readHostProfile but never read by
buildStealthScript — the platform spoof is owned by the UA-CH cmdline switch in
buildGStackLaunchArgs (which reads GSTACK_PLATFORM directly). Remove the dead
field. Export readHostProfile and AUTOMATION_ARTIFACT_CLEANUP_SCRIPT so their
clamp/shape invariants can be unit-tested. Correct the stale "25 Selenium
globals" count comment and note the extended cdc_ scan is redundant-but-retained
for standalone use.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(browse): cover readHostProfile clamp, toString depth-3, chrome.* calls
Pre-landing review coverage gaps:
- readHostProfile clamps 0/negative/NaN/missing env to 8 (a deviceMemory=0 or
NaN would be a glaring bot tell) — now asserted.
- toString proxy survives the depth-3 recursion trick
(fn.toString.toString.toString().includes('[native code]')), the headline
claim that was only tested at depth-1.
- chrome.csi() and chrome.loadTimes() are invoked (not just typeof-checked) and
runtime.connect() throws the native-shaped "No matching signature" error.
- AUTOMATION_ARTIFACT_CLEANUP_SCRIPT static shape (cdc_/__webdriver strip +
notifications->prompt) as a hermetic backup for the live-Chromium pairing test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(browse): recreateContext() re-applies stealth (closes 4th un-stealth path)
useragent and viewport --scale route through recreateContext(), which rebuilds
the BrowserContext via newContext() — a fresh context with no init scripts. It
never called applyStealth, so a routine useragent/viewport-scale command
silently dropped webdriver masking, window.chrome.* shape, hardware spoof, and
the cdc/Permissions cleanup on every restored page. Caught by the cross-model
adversarial review (Codex) after the Claude pass and eng review missed it.
Both the main and fallback paths now call applyStealth before any page is
created. The launch-path tripwire is raised to >= 4 sites and now asserts the
recreateContext() body specifically, so the regression class can't recur.
Also documents the load-bearing trust assumption on buildGStackLaunchArgs /
readHostProfile (GSTACK_* must be gbd-sourced, never page/remote data — the
injection-safety argument depends on it) and the notifications-permission
spoof tradeoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: bump version and changelog (v1.58.3.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync browser stealth docs to Layer C (v1.58.3.0)
BROWSER.md "Stealth scope" still described the default as navigator.webdriver
masking only; Layer C is now the always-on default across all four
context-creation paths. Update the stealth-scope prose, the "What GStack
Browser means" blurb (stock-Chrome UA, no GStackBrowser suffix, captchas can
still get through at the CDP layer), the stealth.ts source-map line, and the
env-vars table (GSTACK_STEALTH, GSTACK_CDP_STEALTH, GSTACK_GPU_*, GSTACK_PLATFORM,
GSTACK_HW_CONCURRENCY/GSTACK_DEVICE_MEMORY + the explicit --gstack-* switches and
ignoreDefaultArgs stripping). Correct the stale "narrows to navigator.webdriver
masking only" premise on the open CDP-patch TODO (the TODO itself stays open —
the CDP-protocol layer is still unaddressed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
Garry Tan
|
443bde054c
|
v1.28.0.0 feat: browse --headed/--proxy/--navigate + gstack/llms.txt + webdriver-only stealth (#1363)
* feat(browse): SOCKS5 bridge with auth + cred redaction helper
Adds browse/src/socks-bridge.ts: a 127.0.0.1-only SOCKS5 listener that
accepts unauthenticated connections from Chromium and relays them through
an authenticated upstream proxy. Chromium does not prompt for SOCKS5 auth
at launch, so this bridge is the workaround for using auth-required
residential SOCKS5 upstreams.
- startSocksBridge({ upstream, port: 0 }) → ephemeral 127.0.0.1 listener
- testUpstream({ upstream, retries: 3, backoffMs: 500, budgetMs: 5000 })
pre-flight that connects to a known endpoint (default 1.1.1.1:443)
- Stream-error policy: kill affected client + upstream sockets on any
error mid-stream; no transport retries (a transport-layer retry can
corrupt browser traffic)
Adds browse/src/proxy-redact.ts: single source of truth for redacting
credentials in any logged proxy URL or upstream config. Every code path
that prints proxy config goes through this helper.
Adds the socks npm dep (~30KB) and 16 tests covering: 127.0.0.1-only
bind, byte-for-byte round trip through the bridge, auth rejection,
mid-stream upstream drop kills client conn, listener teardown,
testUpstream success + retry-exhaust paths, redaction of every
credential shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(browse): --proxy and --headed flags wire bridge into daemon
Adds the global --proxy <url> and --headed flags to the browse CLI.
Resolves cred policy and routes the daemon launch through the SOCKS5
bridge (or pass-through for HTTP/HTTPS) before chromium.launch().
CLI (cli.ts):
- extractGlobalFlags() strips --proxy/--headed from argv, parses URL via
Node URL class, validates D9 cred-mixing (env BROWSE_PROXY_USER/PASS
+ URL creds → exit 1 with hint), composes canonical proxy URL with
resolved creds, computes a stable configHash for daemon-mismatch
- ensureServer() now reads existing daemon's configHash from state file
and refuses (exit 1 with disconnect hint) if --proxy/--headed mismatch
the existing daemon. No silent restart that would drop tab state.
- All proxy-related stderr lines go through redactProxyUrl
proxy-config.ts (new):
- parseProxyConfig() — URL parser + D9 cred-mixing detector + scheme allowlist
- computeConfigHash() — stable hash of (proxy URL minus creds + headed flag)
- toUpstreamConfig() — map ParsedProxyConfig → socks-bridge.UpstreamConfig
Server (server.ts):
- Reads BROWSE_PROXY_URL at startup; for SOCKS5+auth, runs testUpstream
pre-flight (5s budget, 3 retries, 500ms backoff) and exits 1 on failure
with redacted error
- Spawns startSocksBridge() on 127.0.0.1:<ephemeral> and points
Chromium at it via socks5://127.0.0.1:<port>
- HTTP/HTTPS or unauth SOCKS5 → pass-through to chromium.launch
proxy.server (with username/password if present)
- State file gains optional configHash for daemon-mismatch check
- Bridge tears down via process.on('exit')
Browser manager (browser-manager.ts):
- New setProxyConfig({ server, username, password }) called by server.ts
before launch
- chromium.launch() and both launchPersistentContext sites pass the
proxy config through when set
Tests: 22 new across proxy-config (parse + cred-mixing + hash stability)
and extractGlobalFlags (flag stripping + cred-mixing rejection + cred
rotation hash stability + redaction).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(browse): Xvfb auto-spawn with PID + start-time validation
Adds browse/src/xvfb.ts: a Linux-only Xvfb auto-spawn module for
running headed Chromium in containers without DISPLAY. The module
walks a display range to pick a free one (never hardcodes :99) and
validates orphan PIDs by BOTH /proc/<pid>/cmdline matching 'Xvfb' AND
start-time matching the recorded value before sending any signal.
Defends against PID reuse — refuses to kill anything that doesn't
match both checks.
- shouldSpawnXvfb(env, platform) — pure decision: skip on macOS/Windows,
on Linux skip when DISPLAY or WAYLAND_DISPLAY is set (codex F2)
- pickFreeDisplay(99..120) — probes via xdpyinfo
- spawnXvfb(display) — returns { pid, startTime, display } handle
- isOurXvfb(pid, startTime) — both-checks validator
- cleanupXvfb(state) — best-effort, validates ownership before SIGTERM
Wired into server.ts startup: when shouldSpawnXvfb says yes, picks a
free display, spawns Xvfb, sets DISPLAY for chromium.launchHeaded, and
records xvfbPid/xvfbStartTime/xvfbDisplay in the state file. Cleanup
runs on process.on('exit'). The CLI's disconnect path also runs
cleanupXvfb() in the force-cleanup branch when the server is dead.
Disconnect now applies to any non-default daemon (headed mode OR
configHash-tagged daemon — i.e. one started with --proxy/--headed),
not just headed mode.
Adds xvfb + x11-utils to .github/docker/Dockerfile.ci so CI exercises
the Linux container --headed path on every run. Without it the most
common production path would go untested.
Tests: 17 new across decision logic, PID validation defenses
(cmdline mismatch, start-time mismatch), no-op safety on bad inputs,
and a Linux+Xvfb-installed gate for the spawn → validate → cleanup
round trip. Tests skip on macOS/Windows automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(browse): webdriver-mask stealth + Chromium-through-bridge e2e
D7 (codex narrowing): mask navigator.webdriver only via addInitScript.
The wintermute approach (fake plugins=[1..5], fake languages=['en-US',
'en'], stub window.chrome) is intentionally NOT applied — modern
fingerprinters check consistency between plugins.length, languages,
userAgent, and platform, and synthesizing fixed values can flag MORE
bot-like, not less. The honest minimum is webdriver, which Chromium
exposes as a known automation tell.
Adds browse/src/stealth.ts: single source of truth for the stealth
init script and launch args. Both browser-manager.launch() (headless)
and launchHeaded() (persistent context with extension) call
applyStealth(context) and pass STEALTH_LAUNCH_ARGS into chromium.launch.
The pre-existing launchHeaded stealth that did fake plugins/languages
is removed for the same reason. The cdc_/__webdriver runtime cleanup
and Permissions API patch are kept — they remove automation-injected
artifacts, not synthesize fake natural-browser values.
Adds bridge-chromium-e2e.test.ts (codex F3): the test that proves the
FEATURE works. Real Chromium with proxy.server = 'socks5://127.0.0.1:
<bridgePort>' navigates to a local HTTP fixture; the auth upstream's
connect counter and the HTTP fixture's hit counter both increment,
proving traffic actually traversed bridge → auth-upstream → destination.
Without this test, we could ship a working byte-relay and a broken
Chromium integration and never know.
Adds bridge-port-restart.test.ts (codex F1, reframed): old test
assumed two daemons coexist, which contradicts D2 single-daemon model.
Reframed as restart-then-restart, asserting fresh ephemeral ports
(never the hardcoded 1090) on each spin-up.
Adds stealth-webdriver.test.ts: navigator.webdriver=false in both
fresh contexts and persistent contexts; navigator.plugins/languages
are NOT replaced with the wintermute fake list (D7 verification).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(gstack): generate llms.txt — single-file capability index for AI agents
Adds scripts/gen-llms-txt.ts: produces gstack/llms.txt at repo root,
indexing every skill (47), every browse command (75), and design
commands when the design CLI is present. Per the llmstxt.org
convention, agents can read one file to learn what gstack offers
instead of crawling 47 SKILL.md files.
Sources:
- skill SKILL.md.tmpl frontmatter (name + description block scalar)
- browse/src/commands.ts COMMAND_DESCRIPTIONS (sorted by category)
- design/src/commands.ts COMMAND_DESCRIPTIONS if present (best-effort)
Wired into scripts/gen-skill-docs.ts as a post-step so it regenerates
on every `bun run gen:skill-docs` (the same script that re-emits all
SKILL.md files). Failures are non-fatal warnings, not build breaks —
the generator never blocks SKILL.md regen.
Strict mode (--strict, also used by tests) throws when a skill is
missing name or description in its frontmatter, catching missing
metadata before it ships.
Tests: shape (top-level sections, sort order, single-line summary
discipline), every-skill-and-command-appears, strict-mode rejection of
incomplete frontmatter, and freshness check that the committed
gstack/llms.txt matches what the generator produces now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(browse): --navigate flag on download for browser-triggered files
Adds the --navigate strategy from community PR #1355 (originally from
@garrytan-agents). When set, download navigates to the URL with
waitUntil:'commit' and captures the resulting browser download via
page.waitForEvent('download'), then saves via download.saveAs().
Handles URLs that trigger files via Content-Disposition headers,
multi-hop CDN redirects requiring browser cookies, or anti-bot CDN
chains where page.request.fetch() can't follow the auth/redirect
chain.
Defaults still use the existing direct-fetch strategy. --navigate is
opt-in.
Goes through the same validateNavigationUrl SSRF gate as goto, so
download --navigate cannot reach IPv4 metadata endpoints (AWS IMDSv1,
GCP/Azure equivalents) or arbitrary internal hosts.
Inferred content type from suggested filename for common extensions
(epub, pdf, zip, gz, mp3/mp4, jpg/jpeg/png, txt, html, json) — falls
back to application/octet-stream. Same 200MB cap as Strategy 1.
Frames the use case generically (anti-bot CDN, Content-Disposition,
redirect chains) rather than naming any specific site, per project
voice rules.
Co-Authored-By: @garrytan-agents
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: v1.28.0.0 — browse SKILL section + VERSION + CHANGELOG
VERSION 1.27.1.0 → 1.28.0.0 (MINOR — substantial new capability:
five new flags/features, ~600 LOC added, new socks dep, multiple
new modules).
browse/SKILL.md.tmpl: new "Headed Mode + Proxy + Anti-Bot Sites"
section between User Handoff and Snapshot Flags. Documents
--headed (auto-Xvfb on Linux), --proxy (with embedded SOCKS5
bridge for auth), download --navigate, the cred-mixing policy,
daemon-discipline (refuse-on-mismatch), the narrowed
webdriver-only stealth, container support caveats, and the
fail-fast/no-retry failure modes.
CHANGELOG entry follows the release-summary format from CLAUDE.md:
two-line headline, lead paragraph, "The numbers that matter"
table tied to specific test files that prove each capability,
"What this means for AI agents" closing tied to a real workflow
shift, then itemized Added/Changed/Fixed/For-contributors
sections.
Browse SKILL.md regenerated via bun run gen:skill-docs.
gstack/llms.txt regenerated automatically from the same pipeline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(browse): integration coverage for daemon mismatch + proxy fail-fast
Adds two integration tests that exercise the full process boundary,
not just the module-level wiring.
daemon-mismatch-refuse.test.ts (D2):
- Stubs a healthy state file with a fake configHash and a fake /health
HTTP server, runs the actual cli.ts binary with a mismatching
--proxy, asserts exit 1 + 'different config' / 'browse disconnect'
hint in stderr.
- Same shape with the plain-daemon-meets---headed case.
- Positive case: matching configHash → CLI does NOT emit the mismatch
hint (regardless of whether the actual command succeeds).
server-proxy-fail-fast.test.ts:
- Starts the rejecting SOCKS5 upstream, spawns server.ts with
BROWSE_PROXY_URL pointing at it, BROWSE_HEADLESS_SKIP=1 to skip
Chromium launch.
- Asserts exit 1, 'FAIL upstream' in stderr (testUpstream pre-flight
ran), no raw credential leakage in any output (redaction works on
the failure path), and exit within 30s upper bound.
Both tests use the existing spawn-bun-cli pattern from
commands.test.ts so they run on the same CI infrastructure as the
rest of the bun test suite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(gen-skill-docs): keep module sync so test require() still works
Two regressions caught by the full test suite after the v1.28.0.0
landing pass:
1) package.json version mismatch — VERSION was bumped to 1.28.0.0
but package.json still pinned to 1.27.1.0.
test/gen-skill-docs.test.ts asserts they match.
2) Top-level await in scripts/gen-llms-txt.ts (CLI entry block) and
scripts/gen-skill-docs.ts (post-step) made gen-skill-docs an
async module. test/gen-skill-docs.test.ts uses require() to pull
extractVoiceTriggers/processVoiceTriggers from gen-skill-docs,
which Bun rejects on async modules with:
"TypeError: require() async module ... unsupported.
use 'await import()' instead."
Fix: wrap the await blocks in void IIFEs so the modules remain sync
from a require() perspective.
After fix: all 379 gen-skill-docs tests pass, all 77 new feature
tests pass (3 skipped on macOS — Linux+Xvfb gates).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(browse): apply codex adversarial findings on the new lifecycle
Codex outside-voice review caught five real production-failure modes in
the v1.28.0.0 proxy/headed lifecycle. Fixed:
1) `browse disconnect` skip-graceful for proxy-only daemons
(browse/src/cli.ts). The graceful /command POST went out with stray
`domains,` shorthand and (even fixed) the server's disconnect handler
only tears down headed mode — proxy-only daemons returned 200 "Not
in headed mode" while leaving the bridge running. Now disconnect
short-circuits to force-cleanup for non-headed daemons, which kicks
process.on('exit') in server.ts to close the bridge + Xvfb.
2) sendCommand crash retry preserves --proxy / --headed
(browse/src/cli.ts). The ECONNRESET retry path called startServer()
with no extraEnv, silently dropping the proxied flags. A daemon that
died mid-command would silently restart in default direct/headless
mode and bypass the SOCKS bridge. Now reapplies BROWSE_PROXY_URL,
BROWSE_HEADED, and BROWSE_CONFIG_HASH from the resolved global flags.
3) `connect` honors --proxy (browse/src/cli.ts). The headed-mode
`connect` command built its own serverEnv that didn't include
BROWSE_PROXY_URL, so `browse --proxy <url> connect` launched headed
Chromium without the proxy. Now threads proxyUrl + configHash into
the connect serverEnv.
4) SOCKS5 bridge handles fragmented TCP frames
(browse/src/socks-bridge.ts). Previously used once('data') and
parsed each chunk as a complete SOCKS5 frame — TCP doesn't preserve
message boundaries and split greetings/CONNECT requests caused
intermittent handshake failures. Replaced with a single state
machine that buffers chunks and uses size predicates on the SOCKS5
header to know when a complete frame has arrived. Pauses the client
socket during upstream connect and replays any remainder bytes
into the upstream on success.
5) Xvfb cleanup-then-state-delete ordering
(browse/src/server.ts). emergencyCleanup() previously deleted the
state file BEFORE any Xvfb cleanup could read it, orphaning Xvfb
on uncaughtException / unhandledRejection. Now reads the state
file first, calls cleanupXvfb() (which validates cmdline +
start-time before kill), then deletes the state file.
Adds a regression test for #4: writes the SOCKS5 greeting + CONNECT
one byte at a time with 5ms ticks, asserts a clean round trip after
the fragmented handshake.
Codex's sixth finding (bridge advertises NO_AUTH on 127.0.0.1, so any
co-located process can use the authenticated upstream) is documented
as a known limitation — gstack's threat model assumes single-user
hosts. Adding bridge-side auth is a separate change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: update BROWSER.md + TODOS.md for v1.28.0.0
BROWSER.md picks up a "Headed mode + proxy + browser-native downloads
(v1.28.0.0)" subsection inside Real-browser mode plus the new source-map
entries (socks-bridge.ts, proxy-config.ts, proxy-redact.ts, xvfb.ts,
stealth.ts). TODOS.md anti-bot-stealth item updated to reflect the v1.28
narrowing — the "fake plugins" line is no longer accurate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(ci): include bun.lock in image build for deterministic install
CI evals all failed on PR #1363 with:
error: Could not resolve: "smart-buffer". Maybe you need to "bun install"?
error: Could not resolve: "ip-address". Maybe you need to "bun install"?
at /opt/node_modules_cache/socks/build/client/socksclient.js:15
The cached node_modules layer in the pre-baked Docker image had
`socks` (the new dep) but was missing its transitive deps (smart-buffer,
ip-address). The image build copied only package.json into the build
context — without bun.lock, `bun install` resolved a different tree
than local `bun install` did, dropping required transitive deps.
Reproduces locally as 229 packages (correct) when bun.lock is present
or absent. Why CI diverged isn't fully understood — possibly Docker
layer cache reuse across image rebuilds — but the deterministic fix is
to include the lockfile in the image build context and use
`--frozen-lockfile`, matching what every CI doc recommends.
Changes:
- .github/docker/Dockerfile.ci: COPY bun.lock alongside package.json,
switch `bun install` → `bun install --frozen-lockfile` so any future
lockfile drift fails loudly during image build instead of producing
a partially-installed cache that breaks downstream eval jobs.
- .github/workflows/evals.yml: include bun.lock in the image-tag hash
so adding/removing a dep invalidates the image, AND copy bun.lock
into the docker context alongside package.json.
- .github/workflows/evals-periodic.yml: same updates.
- .github/workflows/ci-image.yml: rebuild trigger now fires on bun.lock
changes too; build context includes bun.lock.
Image hash changes → fresh image gets built on next CI run → install
matches the lockfile exactly → no missing transitive deps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): use hardlink copy instead of symlink for node_modules cache
After the bun.lock fix landed, the eval matrix STILL failed identically:
Could not resolve: "smart-buffer" / "ip-address"
at /opt/node_modules_cache/socks/build/client/socksclient.js
But the hash-tagged image actually contains smart-buffer + ip-address +
socks all flat in /opt/node_modules_cache (verified by pulling and
inspecting the image). 207 packages, all present.
Root cause: the workflow used `ln -s /opt/node_modules_cache node_modules`
to restore deps. Bun build (and Node module resolution generally) walks
a file's realpath to find sibling deps. From the symlinked
/workspace/node_modules/socks/build/client/socksclient.js, realpath
resolves to /opt/node_modules_cache/socks/build/client/socksclient.js,
and walking up to find a node_modules/smart-buffer dir fails — there's
no `node_modules` segment in the realpath.
Switch `ln -s` → `cp -al` (hardlink-copy). Each file in the cache becomes
a hardlink at /workspace/node_modules/<pkg>, sharing inodes (no data
copy). Realpath of /workspace/node_modules/socks/.../socksclient.js
stays inside /workspace/node_modules, so sibling deps resolve correctly.
Speed is comparable to symlink — `cp -al` on ~200 packages on tmpfs is
sub-second. Same caching story preserved.
Both evals.yml and evals-periodic.yml updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): cp -r instead of cp -al — /opt and /workspace are different filesystems
The hardlink-copy fix landed and immediately broke with:
cp: cannot create hard link 'node_modules/<file>' to
'/opt/node_modules_cache/<file>': Invalid cross-device link
GitHub Actions runners mount the workspace volume at /workspace
(overlay-fs layered onto the runner image), and /opt is the runner
image's own filesystem. Cross-filesystem hardlinks aren't supported.
Switch `cp -al` → `cp -r`. Cost: ~5s for ~200 packages of small JS
files vs ~0s for the broken symlink. Still cheaper than the ~15s
`bun install` fallback. Realpath of /workspace/node_modules/<pkg>/...
stays inside /workspace, so bun build's sibling-dep resolution works.
Both evals.yml and evals-periodic.yml updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|