mirror of https://github.com/garrytan/gstack.git
Merge 1799f5a3de into 2be6c06ba8
This commit is contained in:
Topherhindman
GitHub
commit
4d513bae49
|
|
@ -280,6 +280,7 @@ from `snapshot`, or `@c` refs from `snapshot -C`. Full table:
|
|||
| `cookie-import-browser [browser] [--domain d]` | Import from installed Chromium browsers (interactive picker, or `--domain` for direct import) |
|
||||
| `header <name>:<value>` | Set custom request header (sensitive values auto-redacted) |
|
||||
| `useragent <string>` | Set user agent (triggers context recreation, invalidates refs) |
|
||||
| `record start\|stop` | Toggle video recording (triggers context recreation, invalidates refs) |
|
||||
|
||||
### Tabs + frames
|
||||
|
||||
|
|
@ -333,6 +334,9 @@ from `snapshot`, or `@c` refs from `snapshot -C`. Full table:
|
|||
| `chain` (JSON via stdin) | Run a sequence of commands. Pipe `[["cmd","arg1",...],...]` to `$B chain`. Stops at first error. |
|
||||
| `inbox [--clear]` | List messages from sidebar scout inbox |
|
||||
| `watch [stop]` | Passive observation — periodic snapshots while user browses; `stop` returns summary |
|
||||
| `record start [dir] [--size WxH]` | Record video of browser activity; one `.webm` per tab that rendered. Rebuilds the context, so refs are invalidated. Headless only, control scope |
|
||||
| `record stop` | Flush and list the video files this recording produced |
|
||||
| `record status` | Report the active recording directory, or that none is running |
|
||||
|
||||
### Browser-skills runtime
|
||||
|
||||
|
|
|
|||
|
|
@ -664,7 +664,35 @@ $B diff https://staging.app.com https://prod.app.com
|
|||
### 11. Show screenshots to the user
|
||||
After `$B screenshot`, `$B snapshot -a -o`, or `$B responsive`, always use the Read tool on the output PNG(s) so the user can see them. Without this, screenshots are invisible.
|
||||
|
||||
### 12. Render local HTML (no HTTP server needed)
|
||||
### 12. Record a video of an interactive bug
|
||||
A screenshot proves what a page looked like; it can't show what a page *did*. When
|
||||
the bug is in the timing — a double-submit, a loading flicker, focus jumping, a
|
||||
drag that drops in the wrong place — record the repro instead of describing it.
|
||||
|
||||
```bash
|
||||
$B record start # or: record start /tmp/repro --size 1280x720
|
||||
$B goto https://app.example.com/checkout
|
||||
$B click @e4
|
||||
$B record stop # flushes and lists the .webm files
|
||||
```
|
||||
|
||||
Recording is a browser-context setting, so `start` and `stop` each rebuild the
|
||||
context. Cookies, storage, and open tabs survive that, but `@e` refs do not —
|
||||
re-snapshot after `record stop` before you act on the page again. One `.webm` is
|
||||
written per tab that rendered while recording ran, including tabs you closed
|
||||
along the way; a tab opened and closed in the same instant never paints and
|
||||
produces nothing. Keep clips short: a few seconds around the moment it breaks
|
||||
beats a minute of navigation. Stick with screenshots for static bugs (a typo, a
|
||||
clipped element, a wrong color) — they are cheaper to produce and easier to read.
|
||||
|
||||
`record stop` is what hands you the file paths, so stop before you walk away: a
|
||||
recording still running when the daemon idles out leaves its `.webm` in the
|
||||
directory, but nothing prints the paths. Recording is headless-only (`handoff`
|
||||
and `connect` hand the browser to the user, and their window is theirs to
|
||||
capture), and it needs control scope — the video keeps whatever was on screen,
|
||||
including anything typed into a login form.
|
||||
|
||||
### 13. Render local HTML (no HTTP server needed)
|
||||
Two paths, pick the cleaner one:
|
||||
```bash
|
||||
# HTML file on disk → goto file:// (absolute, or cwd-relative)
|
||||
|
|
@ -679,7 +707,7 @@ $B load-html /tmp/tweet.html
|
|||
|
||||
`goto file://...` is usually cleaner (URL is saved in state, relative asset URLs resolve against the file's dir, scale changes replay naturally). `load-html` uses `page.setContent()` — URL stays `about:blank`, but the content survives `viewport --scale` via in-memory replay. Both are scoped to files under cwd or `$TMPDIR`.
|
||||
|
||||
### 13. Retina screenshots (deviceScaleFactor)
|
||||
### 14. Retina screenshots (deviceScaleFactor)
|
||||
```bash
|
||||
$B viewport 480x600 --scale 2 # 2x deviceScaleFactor
|
||||
$B load-html /tmp/tweet.html # or: $B goto file://./tweet.html
|
||||
|
|
@ -688,7 +716,7 @@ $B screenshot /tmp/out.png --selector .tweet-card
|
|||
```
|
||||
Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode.
|
||||
|
||||
### 14. Offline render mode (rasterize your own HTML/JSON, zero network)
|
||||
### 15. Offline render mode (rasterize your own HTML/JSON, zero network)
|
||||
|
||||
This is the blessed path for "I just want to turn my own local HTML or JSON into a
|
||||
PNG/PDF/bytes on disk" — Excalidraw diagrams, tweet/quote cards, og-images,
|
||||
|
|
@ -1025,6 +1053,7 @@ $B prettyscreenshot --cleanup --scroll-to ".pricing" --width 1440 ~/Desktop/hero
|
|||
| `domain-skill save|list|show|edit|promote-to-global|rollback|rm <host?>` | Per-site notes the agent writes for itself. Host is derived from the active tab. Lifecycle: `save` adds a quarantined note → after N=3 successful uses without the prompt-injection classifier flagging it, the note auto-promotes to "active" → `promote-to-global` lifts it to the global tier (machine-wide, all projects). The classifier flag is set automatically by the L4 prompt-injection scan; agents do not set it manually. Use `list` / `show` to inspect, `edit` to revise, `rollback` to demote, `rm` to tombstone. |
|
||||
| `frame <sel|@ref|--name n|--url pattern|main>` | Switch to iframe context (or main to return) |
|
||||
| `inbox [--clear]` | List messages from sidebar scout inbox |
|
||||
| `record start [dir] [--size WxH] | record stop | record status` | Record video of browser activity (interactive bug repros) |
|
||||
| `skill list|show|run|test|rm <name?> [--arg k=v]... [--timeout=Ns]` | Run a browser-skill: deterministic Playwright script that drives the daemon over loopback HTTP. 3-tier lookup (project > global > bundled). Spawned scripts get a per-spawn scoped token (read+write only) — never the daemon root token. |
|
||||
| `watch [stop]` | Passive observation — periodic snapshots while user browses |
|
||||
|
||||
|
|
|
|||
|
|
@ -111,7 +111,35 @@ $B diff https://staging.app.com https://prod.app.com
|
|||
### 11. Show screenshots to the user
|
||||
After `$B screenshot`, `$B snapshot -a -o`, or `$B responsive`, always use the Read tool on the output PNG(s) so the user can see them. Without this, screenshots are invisible.
|
||||
|
||||
### 12. Render local HTML (no HTTP server needed)
|
||||
### 12. Record a video of an interactive bug
|
||||
A screenshot proves what a page looked like; it can't show what a page *did*. When
|
||||
the bug is in the timing — a double-submit, a loading flicker, focus jumping, a
|
||||
drag that drops in the wrong place — record the repro instead of describing it.
|
||||
|
||||
```bash
|
||||
$B record start # or: record start /tmp/repro --size 1280x720
|
||||
$B goto https://app.example.com/checkout
|
||||
$B click @e4
|
||||
$B record stop # flushes and lists the .webm files
|
||||
```
|
||||
|
||||
Recording is a browser-context setting, so `start` and `stop` each rebuild the
|
||||
context. Cookies, storage, and open tabs survive that, but `@e` refs do not —
|
||||
re-snapshot after `record stop` before you act on the page again. One `.webm` is
|
||||
written per tab that rendered while recording ran, including tabs you closed
|
||||
along the way; a tab opened and closed in the same instant never paints and
|
||||
produces nothing. Keep clips short: a few seconds around the moment it breaks
|
||||
beats a minute of navigation. Stick with screenshots for static bugs (a typo, a
|
||||
clipped element, a wrong color) — they are cheaper to produce and easier to read.
|
||||
|
||||
`record stop` is what hands you the file paths, so stop before you walk away: a
|
||||
recording still running when the daemon idles out leaves its `.webm` in the
|
||||
directory, but nothing prints the paths. Recording is headless-only (`handoff`
|
||||
and `connect` hand the browser to the user, and their window is theirs to
|
||||
capture), and it needs control scope — the video keeps whatever was on screen,
|
||||
including anything typed into a login form.
|
||||
|
||||
### 13. Render local HTML (no HTTP server needed)
|
||||
Two paths, pick the cleaner one:
|
||||
```bash
|
||||
# HTML file on disk → goto file:// (absolute, or cwd-relative)
|
||||
|
|
@ -126,7 +154,7 @@ $B load-html /tmp/tweet.html
|
|||
|
||||
`goto file://...` is usually cleaner (URL is saved in state, relative asset URLs resolve against the file's dir, scale changes replay naturally). `load-html` uses `page.setContent()` — URL stays `about:blank`, but the content survives `viewport --scale` via in-memory replay. Both are scoped to files under cwd or `$TMPDIR`.
|
||||
|
||||
### 13. Retina screenshots (deviceScaleFactor)
|
||||
### 14. Retina screenshots (deviceScaleFactor)
|
||||
```bash
|
||||
$B viewport 480x600 --scale 2 # 2x deviceScaleFactor
|
||||
$B load-html /tmp/tweet.html # or: $B goto file://./tweet.html
|
||||
|
|
@ -135,7 +163,7 @@ $B screenshot /tmp/out.png --selector .tweet-card
|
|||
```
|
||||
Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode.
|
||||
|
||||
### 14. Offline render mode (rasterize your own HTML/JSON, zero network)
|
||||
### 15. Offline render mode (rasterize your own HTML/JSON, zero network)
|
||||
|
||||
This is the blessed path for "I just want to turn my own local HTML or JSON into a
|
||||
PNG/PDF/bytes on disk" — Excalidraw diagrams, tweet/quote cards, og-images,
|
||||
|
|
|
|||
|
|
@ -205,6 +205,16 @@ export type { RefEntry };
|
|||
// Re-export TabSession for consumers
|
||||
export { TabSession };
|
||||
|
||||
/** Outcome of a recording: the files it produced, and anything that went wrong. */
|
||||
export interface RecordingResult {
|
||||
/** Video files with bytes in them. */
|
||||
videos: string[];
|
||||
/** Files that were created but never received bytes — missing evidence, not saved video. */
|
||||
empty: string[];
|
||||
/** Degraded context rebuild or unreadable directory; null when the flush was clean. */
|
||||
warning: string | null;
|
||||
}
|
||||
|
||||
export interface BrowserState {
|
||||
cookies: Cookie[];
|
||||
pages: Array<{
|
||||
|
|
@ -249,6 +259,17 @@ export class BrowserManager {
|
|||
private deviceScaleFactor: number = 1;
|
||||
private currentViewport: { width: number; height: number } = { width: 1280, height: 720 };
|
||||
|
||||
// ─── Video recording (context option) ────────────────────────
|
||||
// Playwright records at the context level and flushes each page's .webm when
|
||||
// the context closes, so both start and stop go through recreateContext().
|
||||
// Null means not recording.
|
||||
private recording: {
|
||||
dir: string;
|
||||
/** Videos already in `dir` when this recording started — not ours to report. */
|
||||
preexisting: Set<string>;
|
||||
size?: { width: number; height: number };
|
||||
} | null = null;
|
||||
|
||||
/** Server port — set after server starts, used by cookie-import-browser command */
|
||||
public serverPort: number = 0;
|
||||
|
||||
|
|
@ -483,14 +504,7 @@ export class BrowserManager {
|
|||
void handleChromiumDisconnect(this.browser);
|
||||
});
|
||||
|
||||
const contextOptions: BrowserContextOptions = {
|
||||
viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
|
||||
deviceScaleFactor: this.deviceScaleFactor,
|
||||
};
|
||||
if (this.customUserAgent) {
|
||||
contextOptions.userAgent = this.customUserAgent;
|
||||
}
|
||||
this.context = await this.browser.newContext(contextOptions);
|
||||
this.context = await this.browser.newContext(this.buildContextOptions());
|
||||
|
||||
if (Object.keys(this.extraHeaders).length > 0) {
|
||||
await this.context.setExtraHTTPHeaders(this.extraHeaders);
|
||||
|
|
@ -1501,14 +1515,7 @@ export class BrowserManager {
|
|||
await this.context.close().catch(() => {});
|
||||
|
||||
// 3. Create new context with updated settings
|
||||
const contextOptions: BrowserContextOptions = {
|
||||
viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
|
||||
deviceScaleFactor: this.deviceScaleFactor,
|
||||
};
|
||||
if (this.customUserAgent) {
|
||||
contextOptions.userAgent = this.customUserAgent;
|
||||
}
|
||||
this.context = await this.browser.newContext(contextOptions);
|
||||
this.context = await this.browser.newContext(this.buildContextOptions());
|
||||
|
||||
// Re-apply stealth: newContext() is a fresh context with no init scripts,
|
||||
// so a useragent / viewport --scale rebuild would otherwise drop the
|
||||
|
|
@ -1533,14 +1540,7 @@ export class BrowserManager {
|
|||
this.tabSessions.clear();
|
||||
if (this.context) await this.context.close().catch(() => {});
|
||||
|
||||
const contextOptions: BrowserContextOptions = {
|
||||
viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
|
||||
deviceScaleFactor: this.deviceScaleFactor,
|
||||
};
|
||||
if (this.customUserAgent) {
|
||||
contextOptions.userAgent = this.customUserAgent;
|
||||
}
|
||||
this.context = await this.browser!.newContext(contextOptions);
|
||||
this.context = await this.browser!.newContext(this.buildContextOptions());
|
||||
// Stealth applies to the fallback blank context too.
|
||||
const { applyStealth } = await import('./stealth');
|
||||
await applyStealth(this.context);
|
||||
|
|
@ -1553,6 +1553,134 @@ export class BrowserManager {
|
|||
}
|
||||
}
|
||||
|
||||
// ─── Video Recording ─────────────────────────────────────────
|
||||
/**
|
||||
* Start recording video of browser activity into `dir`.
|
||||
*
|
||||
* Recording is a context option, so this rebuilds the context through
|
||||
* recreateContext(), which preserves cookies, storage, and open tabs.
|
||||
*
|
||||
* Returns the result of any recording this call superseded, plus the warning
|
||||
* recreateContext() uses to report a degraded rebuild. Both are the caller's
|
||||
* to report: a superseded take is already on disk, and a caller who is not
|
||||
* handed its paths has lost it.
|
||||
*/
|
||||
async startRecording(
|
||||
dir: string,
|
||||
size?: { width: number; height: number },
|
||||
): Promise<{ superseded: RecordingResult | null; warning: string | null }> {
|
||||
if (this.connectionMode === 'headed') {
|
||||
throw new Error('record is not supported in headed mode — the visible window is the real browser. Run `$B disconnect` first, or capture the screen with an OS recorder.');
|
||||
}
|
||||
if (!this.browser || !this.context) {
|
||||
throw new Error('Browser not launched');
|
||||
}
|
||||
|
||||
const superseded = this.recording ? await this.stopRecording() : null;
|
||||
|
||||
mkdirSecure(dir);
|
||||
this.recording = { dir, preexisting: this.videoNamesIn(dir), ...(size ? { size } : {}) };
|
||||
// A degraded rebuild still records: the fallback context is built from
|
||||
// buildContextOptions() too, so the flag stays set and stop finds the files.
|
||||
const warning = await this.recreateContext();
|
||||
|
||||
return { superseded, warning };
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop recording and return every video the recording produced.
|
||||
*
|
||||
* Playwright writes one .webm per page and only flushes on context close, so
|
||||
* this clears the flag and rebuilds the context to force the flush, then reads
|
||||
* the recording directory. Reading the directory rather than the live pages is
|
||||
* what catches a tab that was closed mid-recording — its video is on disk, and
|
||||
* enumerating open pages would miss it.
|
||||
*
|
||||
* `warning` carries a degraded rebuild (the flush is that rebuild, so a
|
||||
* failure there is a flush failure) and any error reading the directory back.
|
||||
* Files that never received bytes are reported as `empty` rather than counted
|
||||
* as saved.
|
||||
*/
|
||||
async stopRecording(): Promise<RecordingResult> {
|
||||
if (!this.recording) return { videos: [], empty: [], warning: null };
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { dir, preexisting } = this.recording;
|
||||
this.recording = null;
|
||||
|
||||
// Closing the old context is what flushes the .webm files to disk. In headed
|
||||
// mode that context is already gone (handoff replaced it, flushing as it
|
||||
// closed), and recreateContext() would throw, so only rebuild when we own a
|
||||
// headless context to rebuild.
|
||||
let warning: string | null = null;
|
||||
if (this.connectionMode !== 'headed') {
|
||||
warning = await this.recreateContext();
|
||||
}
|
||||
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = fs.readdirSync(dir).filter((name: string) => name.endsWith('.webm'));
|
||||
} catch (err: unknown) {
|
||||
const detail = err instanceof Error ? err.message : String(err);
|
||||
return { videos: [], empty: [], warning: warning ?? `Recording directory could not be read: ${detail}` };
|
||||
}
|
||||
|
||||
const videos: string[] = [];
|
||||
const empty: string[] = [];
|
||||
for (const name of entries) {
|
||||
// Skip videos that were already there: a directory reused across takes
|
||||
// would otherwise report the earlier one as part of this recording.
|
||||
if (preexisting.has(name)) continue;
|
||||
const full = path.join(dir, name);
|
||||
try {
|
||||
(fs.statSync(full).size > 0 ? videos : empty).push(full);
|
||||
} catch {
|
||||
empty.push(full);
|
||||
}
|
||||
}
|
||||
|
||||
return { videos, empty, warning };
|
||||
}
|
||||
|
||||
/** Names of the .webm files currently in `dir`; empty when it does not exist yet. */
|
||||
private videoNamesIn(dir: string): Set<string> {
|
||||
const fs = require('fs');
|
||||
try {
|
||||
return new Set<string>(fs.readdirSync(dir).filter((name: string) => name.endsWith('.webm')));
|
||||
} catch {
|
||||
return new Set<string>();
|
||||
}
|
||||
}
|
||||
|
||||
/** Directory the active recording is writing to, or null when not recording. */
|
||||
getRecordingDir(): string | null {
|
||||
return this.recording?.dir ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Context options every context in this manager is built from — the initial
|
||||
* launch, the recreateContext() rebuild, and its clean-slate fallback. These
|
||||
* settings only apply at context construction, so each one has to be repeated
|
||||
* on every rebuild to survive it.
|
||||
*/
|
||||
private buildContextOptions(): BrowserContextOptions {
|
||||
const options: BrowserContextOptions = {
|
||||
viewport: { width: this.currentViewport.width, height: this.currentViewport.height },
|
||||
deviceScaleFactor: this.deviceScaleFactor,
|
||||
};
|
||||
if (this.customUserAgent) {
|
||||
options.userAgent = this.customUserAgent;
|
||||
}
|
||||
if (this.recording) {
|
||||
options.recordVideo = {
|
||||
dir: this.recording.dir,
|
||||
...(this.recording.size ? { size: this.recording.size } : {}),
|
||||
};
|
||||
}
|
||||
return options;
|
||||
}
|
||||
|
||||
/**
|
||||
* Change deviceScaleFactor + viewport size atomically.
|
||||
*
|
||||
|
|
|
|||
|
|
@ -39,6 +39,7 @@ export const META_COMMANDS = new Set([
|
|||
'connect', 'disconnect', 'focus',
|
||||
'inbox',
|
||||
'watch',
|
||||
'record',
|
||||
'state',
|
||||
'frame',
|
||||
'ux-audit',
|
||||
|
|
@ -168,6 +169,8 @@ export const COMMAND_DESCRIPTIONS: Record<string, { category: string; descriptio
|
|||
'inbox': { category: 'Meta', description: 'List messages from sidebar scout inbox', usage: 'inbox [--clear]' },
|
||||
// Watch
|
||||
'watch': { category: 'Meta', description: 'Passive observation — periodic snapshots while user browses', usage: 'watch [stop]' },
|
||||
// Record
|
||||
'record': { category: 'Meta', description: 'Record video of browser activity (interactive bug repros)', usage: 'record start [dir] [--size WxH] | record stop | record status' },
|
||||
// State
|
||||
'state': { category: 'Server', description: 'Save/load browser state (cookies + URLs)', usage: 'state save|load <name>' },
|
||||
// Frame
|
||||
|
|
@ -225,6 +228,7 @@ export function canonicalizeCommand(cmd: string): string {
|
|||
*/
|
||||
export const NEW_IN_VERSION: Record<string, string> = {
|
||||
'load-html': '0.19.0.0',
|
||||
'record': '1.62.0.0',
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
* Meta commands — tabs, server control, screenshots, chain, diff, snapshot
|
||||
*/
|
||||
|
||||
import type { BrowserManager } from './browser-manager';
|
||||
import type { BrowserManager, RecordingResult } from './browser-manager';
|
||||
import { handleSnapshot } from './snapshot';
|
||||
import { getCleanText } from './read-commands';
|
||||
import { READ_COMMANDS, WRITE_COMMANDS, META_COMMANDS, PAGE_CONTENT_COMMANDS, wrapUntrustedContent, canonicalizeCommand } from './commands';
|
||||
|
|
@ -23,6 +23,29 @@ import { resolveConfig } from './config';
|
|||
import { filterSessionCookies } from './session-persist';
|
||||
import type { Frame } from 'playwright';
|
||||
|
||||
/**
|
||||
* Render a recording outcome. Shared by `record start` (for a take it
|
||||
* superseded) and `record stop` so a superseded recording is reported the same
|
||||
* way as one the caller stopped on purpose.
|
||||
*/
|
||||
function formatRecordingResult(result: RecordingResult, headline: string): string[] {
|
||||
const lines: string[] = [];
|
||||
if (result.videos.length > 0) {
|
||||
lines.push(`${headline} — ${result.videos.length} video${result.videos.length === 1 ? '' : 's'}:`);
|
||||
lines.push(...result.videos.map(v => ` ${v}`));
|
||||
} else {
|
||||
lines.push(`${headline} — no video was written. A recording only captures pages that rendered while it ran.`);
|
||||
}
|
||||
if (result.empty.length > 0) {
|
||||
lines.push('Empty (never flushed) — missing evidence, not saved video:');
|
||||
lines.push(...result.empty.map(v => ` ${v}`));
|
||||
}
|
||||
if (result.warning) {
|
||||
lines.push(`⚠️ ${result.warning}`);
|
||||
}
|
||||
return lines;
|
||||
}
|
||||
|
||||
/** Tokenize a pipe segment respecting double-quoted strings. */
|
||||
function tokenizePipeSegment(segment: string): string[] {
|
||||
const tokens: string[] = [];
|
||||
|
|
@ -836,6 +859,70 @@ export async function handleMetaCommand(
|
|||
return 'WATCHING — observing user browsing. Periodic snapshots every 5s.\nRun `$B watch stop` to stop and get summary.';
|
||||
}
|
||||
|
||||
// ─── Record ─────────────────────────────────────────
|
||||
case 'record': {
|
||||
const action = args[0];
|
||||
if (!action) throw new Error('Usage: record start [dir] [--size WxH] | record stop | record status');
|
||||
|
||||
if (action === 'status') {
|
||||
const dir = bm.getRecordingDir();
|
||||
return dir ? `Recording → ${dir}` : 'Not recording.';
|
||||
}
|
||||
|
||||
if (action === 'stop') {
|
||||
if (!bm.getRecordingDir()) return 'Not recording (nothing to stop).';
|
||||
return [
|
||||
...formatRecordingResult(await bm.stopRecording(), 'Recording stopped'),
|
||||
'Refs are stale after a recording stops — re-snapshot before acting on the page.',
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
if (action === 'start') {
|
||||
let dirArg: string | undefined;
|
||||
let size: { width: number; height: number } | undefined;
|
||||
|
||||
for (let i = 1; i < args.length; i++) {
|
||||
const token = args[i];
|
||||
if (token === '--size') {
|
||||
const value = args[++i];
|
||||
if (!value) throw new Error('record start --size: missing value (e.g. --size 1280x720)');
|
||||
const match = /^(\d+)x(\d+)$/.exec(value);
|
||||
if (!match) throw new Error(`record start --size: expected WxH, got '${value}'`);
|
||||
const width = parseInt(match[1], 10);
|
||||
const height = parseInt(match[2], 10);
|
||||
if (width < 1 || height < 1) throw new Error(`record start --size: dimensions must be positive, got '${value}'`);
|
||||
size = { width, height };
|
||||
} else if (token.startsWith('--')) {
|
||||
throw new Error(`record start: unknown flag '${token}'`);
|
||||
} else if (dirArg === undefined) {
|
||||
dirArg = token;
|
||||
} else {
|
||||
throw new Error(`record start: unexpected argument '${token}'. Usage: record start [dir] [--size WxH]`);
|
||||
}
|
||||
}
|
||||
|
||||
// Each recording gets its own directory: Playwright names videos by an
|
||||
// internal id, so a shared directory makes the videos of two recordings
|
||||
// indistinguishable after the fact.
|
||||
const targetDir = dirArg
|
||||
? path.resolve(dirArg)
|
||||
: path.join(TEMP_DIR, `browse-record-${new Date().toISOString().replace(/[:.]/g, '-')}`);
|
||||
validateOutputPath(targetDir);
|
||||
|
||||
const { superseded, warning } = await bm.startRecording(targetDir, size);
|
||||
const lines: string[] = [];
|
||||
if (superseded) {
|
||||
lines.push(...formatRecordingResult(superseded, 'Stopped the recording already running'));
|
||||
}
|
||||
lines.push(`Recording → ${targetDir}`);
|
||||
lines.push('Run `$B record stop` to flush and list the video files.');
|
||||
if (warning) lines.push(`⚠️ ${warning}`);
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
throw new Error(`record: unknown action '${action}'. Usage: record start|stop|status`);
|
||||
}
|
||||
|
||||
// ─── Inbox ──────────────────────────────────────────
|
||||
case 'inbox': {
|
||||
const { execSync } = await import('child_process');
|
||||
|
|
|
|||
|
|
@ -64,6 +64,11 @@ export const SCOPE_ADMIN = new Set([
|
|||
/** Browser-wide destructive commands — can kill the server, disconnect headed mode */
|
||||
export const SCOPE_CONTROL = new Set([
|
||||
'state', 'handoff', 'resume', 'stop', 'restart', 'connect', 'disconnect',
|
||||
// `record` sits here rather than in meta for two reasons: it rebuilds the
|
||||
// context out from under every open tab (same blast radius as `state`), and it
|
||||
// writes whatever is on screen — including anything typed into a login form —
|
||||
// to disk as video.
|
||||
'record',
|
||||
]);
|
||||
|
||||
/** Meta commands — generally safe but some need scope checking */
|
||||
|
|
|
|||
|
|
@ -0,0 +1,349 @@
|
|||
/**
|
||||
* Tests for the record command — video capture of browser activity.
|
||||
*
|
||||
* Integration tests drive a real Playwright browser against the local test
|
||||
* server and assert on the bytes that land on disk, because the failure this
|
||||
* command has to avoid is reporting a video that was never flushed.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { startTestServer } from './test-server';
|
||||
import { BrowserManager } from '../src/browser-manager';
|
||||
import { handleWriteCommand as _handleWriteCommand } from '../src/write-commands';
|
||||
import { handleMetaCommand } from '../src/meta-commands';
|
||||
import { META_COMMANDS, COMMAND_DESCRIPTIONS } from '../src/commands';
|
||||
import { SCOPE_CONTROL, SCOPE_READ, SCOPE_WRITE, checkScope } from '../src/token-registry';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { TEMP_DIR } from '../src/platform';
|
||||
|
||||
const handleWriteCommand = (cmd: string, args: string[], b: BrowserManager) =>
|
||||
_handleWriteCommand(cmd, args, b.getActiveSession(), b);
|
||||
|
||||
const shutdown = async () => {};
|
||||
|
||||
let testServer: ReturnType<typeof startTestServer>;
|
||||
let bm: BrowserManager;
|
||||
let baseUrl: string;
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
/** A recording directory inside the browse temp root, cleaned up in afterAll. */
|
||||
function recordingDir(label: string): string {
|
||||
const dir = fs.mkdtempSync(path.join(TEMP_DIR, `browse-record-test-${label}-`));
|
||||
tempDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function webmsIn(dir: string): string[] {
|
||||
return fs.readdirSync(dir).filter(name => name.endsWith('.webm'));
|
||||
}
|
||||
|
||||
/**
|
||||
* Read PixelWidth (EBML id 0xB0) and PixelHeight (0xBA) out of a WebM header.
|
||||
* Both are short unsigned ints, so the 1- and 2-byte length forms cover every
|
||||
* viewport we can ask for. Avoids depending on ffprobe being installed in CI.
|
||||
*/
|
||||
function videoDimensions(file: string): { width: number; height: number } {
|
||||
const head = fs.readFileSync(file).subarray(0, 4096);
|
||||
const read = (id: number): number => {
|
||||
for (let i = 0; i < head.length - 4; i++) {
|
||||
if (head[i] !== id) continue;
|
||||
if (head[i + 1] === 0x81) return head[i + 2];
|
||||
if (head[i + 1] === 0x82) return (head[i + 2] << 8) | head[i + 3];
|
||||
}
|
||||
return 0;
|
||||
};
|
||||
return { width: read(0xb0), height: read(0xba) };
|
||||
}
|
||||
|
||||
/** Give a page long enough to paint that Playwright is certain to emit a video. */
|
||||
async function settle(ms = 400): Promise<void> {
|
||||
await new Promise(resolve => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
testServer = startTestServer(0);
|
||||
baseUrl = testServer.url;
|
||||
bm = new BrowserManager();
|
||||
await bm.launch();
|
||||
});
|
||||
|
||||
// Close the browser rather than forcing the process down: a process.exit here
|
||||
// truncates whatever else the runner still had queued. close() races Chromium's
|
||||
// shutdown against its own 5s cap, so the hook needs more than bun's 5s default.
|
||||
afterAll(async () => {
|
||||
await bm.close().catch(() => {});
|
||||
try { testServer.server.stop(); } catch {}
|
||||
for (const dir of tempDirs) {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}, 20000);
|
||||
|
||||
// ─── Registration ───────────────────────────────────────────────
|
||||
|
||||
describe('record registration', () => {
|
||||
test('is a known meta command with usage text', () => {
|
||||
expect(META_COMMANDS.has('record')).toBe(true);
|
||||
expect(COMMAND_DESCRIPTIONS['record']?.usage).toContain('record start');
|
||||
});
|
||||
|
||||
test('is control-scoped — it rebuilds the context and writes screen content to disk', () => {
|
||||
expect(SCOPE_CONTROL.has('record')).toBe(true);
|
||||
expect(SCOPE_READ.has('record')).toBe(false);
|
||||
expect(SCOPE_WRITE.has('record')).toBe(false);
|
||||
});
|
||||
|
||||
test('a read-only token cannot record, a control token can', () => {
|
||||
const readOnly = { token: 't', clientId: 'agent', type: 'session' as const, scopes: ['read' as const] };
|
||||
const control = { token: 't', clientId: 'agent', type: 'session' as const, scopes: ['control' as const] };
|
||||
expect(checkScope(readOnly, 'record')).toBe(false);
|
||||
expect(checkScope(control, 'record')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Argument handling ──────────────────────────────────────────
|
||||
|
||||
describe('record argument handling', () => {
|
||||
test('status reports not-recording before any start', async () => {
|
||||
const result = await handleMetaCommand('record', ['status'], bm, shutdown);
|
||||
expect(result).toBe('Not recording.');
|
||||
});
|
||||
|
||||
test('stop without an active recording is a no-op, not an error', async () => {
|
||||
const result = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
expect(result).toContain('Not recording');
|
||||
});
|
||||
|
||||
test('missing action, unknown action, and unknown flag all explain the usage', async () => {
|
||||
expect(handleMetaCommand('record', [], bm, shutdown)).rejects.toThrow(/Usage: record start/);
|
||||
expect(handleMetaCommand('record', ['bogus'], bm, shutdown)).rejects.toThrow(/unknown action/);
|
||||
expect(handleMetaCommand('record', ['start', '--bogus'], bm, shutdown)).rejects.toThrow(/unknown flag/);
|
||||
});
|
||||
|
||||
test('malformed --size is rejected before a context is rebuilt', async () => {
|
||||
expect(handleMetaCommand('record', ['start', '--size'], bm, shutdown)).rejects.toThrow(/missing value/);
|
||||
expect(handleMetaCommand('record', ['start', '--size', '1280'], bm, shutdown)).rejects.toThrow(/expected WxH/);
|
||||
expect(handleMetaCommand('record', ['start', '--size', '0x720'], bm, shutdown)).rejects.toThrow(/must be positive/);
|
||||
expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toBe('Not recording.');
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Recording ──────────────────────────────────────────────────
|
||||
|
||||
describe('record capture', () => {
|
||||
test('start → activity → stop writes a playable webm and reports its path', async () => {
|
||||
const dir = recordingDir('basic');
|
||||
const started = await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
expect(started).toContain(dir);
|
||||
expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toContain(dir);
|
||||
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm);
|
||||
|
||||
const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
const videos = webmsIn(dir);
|
||||
expect(videos.length).toBeGreaterThan(0);
|
||||
expect(stopped).toContain(videos[0]);
|
||||
|
||||
// EBML magic bytes — a real Matroska/WebM container, not an empty file.
|
||||
const bytes = fs.readFileSync(path.join(dir, videos[0]));
|
||||
expect(bytes.length).toBeGreaterThan(0);
|
||||
expect([bytes[0], bytes[1], bytes[2], bytes[3]]).toEqual([0x1a, 0x45, 0xdf, 0xa3]);
|
||||
|
||||
expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toBe('Not recording.');
|
||||
}, 30000);
|
||||
|
||||
test('the browser still works after a recording stops', async () => {
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
const url = await handleMetaCommand('url', [], bm, shutdown);
|
||||
expect(url).toContain('/snapshot.html');
|
||||
}, 15000);
|
||||
|
||||
test('a tab closed mid-recording still has its video reported', async () => {
|
||||
const dir = recordingDir('closed-tab');
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
await handleMetaCommand('newtab', [baseUrl + '/forms.html'], bm, shutdown);
|
||||
// A tab has to actually paint before Playwright emits a video for it; a tab
|
||||
// opened and closed inside the same few milliseconds produces nothing.
|
||||
await settle();
|
||||
await handleMetaCommand('closetab', [], bm, shutdown);
|
||||
|
||||
const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
const videos = webmsIn(dir);
|
||||
|
||||
// Enumerating live pages at stop time would report one video here. Every
|
||||
// file the recording produced has to be listed, or the caller is told they
|
||||
// have less evidence than they do.
|
||||
expect(videos.length).toBeGreaterThanOrEqual(2);
|
||||
for (const name of videos) {
|
||||
expect(stopped).toContain(path.join(dir, name));
|
||||
}
|
||||
}, 45000);
|
||||
|
||||
test('cookies survive the context rebuild that start and stop perform', async () => {
|
||||
const dir = recordingDir('state');
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await bm.getActiveSession().page.context().addCookies([
|
||||
{ name: 'record_state_probe', value: 'kept', url: baseUrl },
|
||||
]);
|
||||
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
const during = await bm.getActiveSession().page.context().cookies();
|
||||
expect(during.some(c => c.name === 'record_state_probe')).toBe(true);
|
||||
|
||||
await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
const after = await bm.getActiveSession().page.context().cookies();
|
||||
expect(after.some(c => c.name === 'record_state_probe')).toBe(true);
|
||||
}, 30000);
|
||||
|
||||
test('starting while already recording reports the take it superseded', async () => {
|
||||
const first = recordingDir('first');
|
||||
const second = recordingDir('second');
|
||||
|
||||
await handleMetaCommand('record', ['start', first], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
const restarted = await handleMetaCommand('record', ['start', second], bm, shutdown);
|
||||
|
||||
// The first take is on disk. Telling the caller only about the new
|
||||
// recording would strand it somewhere they were never given the path to.
|
||||
const firstVideos = webmsIn(first);
|
||||
expect(firstVideos.length).toBeGreaterThan(0);
|
||||
for (const name of firstVideos) {
|
||||
expect(restarted).toContain(path.join(first, name));
|
||||
}
|
||||
expect(restarted).toContain(second);
|
||||
expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toContain(second);
|
||||
|
||||
await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm);
|
||||
await settle();
|
||||
await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
expect(webmsIn(second).length).toBeGreaterThan(0);
|
||||
}, 60000);
|
||||
|
||||
test('--size sets the recorded frame size', async () => {
|
||||
const dir = recordingDir('size');
|
||||
await handleMetaCommand('record', ['start', dir, '--size', '640x480'], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
|
||||
const videos = webmsIn(dir);
|
||||
expect(videos.length).toBeGreaterThan(0);
|
||||
expect(videoDimensions(path.join(dir, videos[0]))).toEqual({ width: 640, height: 480 });
|
||||
}, 30000);
|
||||
|
||||
test('a reused directory does not re-report the earlier take', async () => {
|
||||
const dir = recordingDir('reused');
|
||||
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
const firstStop = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
const afterFirst = webmsIn(dir);
|
||||
expect(afterFirst.length).toBeGreaterThan(0);
|
||||
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm);
|
||||
await settle();
|
||||
const secondStop = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
|
||||
// Both takes live in the same directory, so the second report must name
|
||||
// only what the second take produced.
|
||||
const listed = (report: string) =>
|
||||
report.split('\n').map(l => l.trim()).filter(l => l.endsWith('.webm'));
|
||||
expect(webmsIn(dir).length).toBeGreaterThan(afterFirst.length);
|
||||
expect(listed(firstStop).length).toBeGreaterThan(0);
|
||||
expect(listed(secondStop).length).toBeGreaterThan(0);
|
||||
for (const earlier of listed(firstStop)) {
|
||||
expect(listed(secondStop)).not.toContain(earlier);
|
||||
}
|
||||
}, 60000);
|
||||
});
|
||||
|
||||
// ─── Failure reporting ──────────────────────────────────────────
|
||||
|
||||
describe('record failure reporting', () => {
|
||||
test('a degraded flush is surfaced, not swallowed', async () => {
|
||||
const dir = recordingDir('degraded');
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
|
||||
// The rebuild IS the flush, so recreateContext reporting a degraded rebuild
|
||||
// means the flush degraded. Silently dropping it tells the caller their
|
||||
// evidence is fine when their tabs were just reset.
|
||||
const real = (bm as any).recreateContext.bind(bm);
|
||||
(bm as any).recreateContext = async () => 'Context recreation failed: simulated. Browser reset to blank tab.';
|
||||
try {
|
||||
const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
expect(stopped).toContain('Context recreation failed');
|
||||
} finally {
|
||||
(bm as any).recreateContext = real;
|
||||
}
|
||||
}, 30000);
|
||||
|
||||
test('an unreadable recording directory is reported as a failure', async () => {
|
||||
const dir = recordingDir('vanished');
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
|
||||
const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
expect(stopped).toMatch(/could not be read/i);
|
||||
}, 30000);
|
||||
|
||||
test('stop after a switch to headed mode still hands back the videos', async () => {
|
||||
const dir = recordingDir('headed');
|
||||
await handleMetaCommand('record', ['start', dir], bm, shutdown);
|
||||
await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm);
|
||||
await settle();
|
||||
|
||||
// handoff/connect close the recording context (flushing as they go) and
|
||||
// leave the manager headed, where recreateContext() throws. The videos are
|
||||
// already on disk, so stop has to hand them back rather than raise.
|
||||
await (bm as any).context.close().catch(() => {});
|
||||
(bm as any).connectionMode = 'headed';
|
||||
try {
|
||||
const result = await bm.stopRecording();
|
||||
expect(result.videos.length).toBeGreaterThan(0);
|
||||
expect(bm.getRecordingDir()).toBeNull();
|
||||
} finally {
|
||||
(bm as any).connectionMode = 'launched';
|
||||
await (bm as any).recreateContext();
|
||||
}
|
||||
}, 30000);
|
||||
});
|
||||
|
||||
// ─── Path policy ────────────────────────────────────────────────
|
||||
|
||||
describe('record path policy', () => {
|
||||
test('a directory outside the safe roots is refused', async () => {
|
||||
expect(
|
||||
handleMetaCommand('record', ['start', '/etc/browse-record-should-not-exist'], bm, shutdown),
|
||||
).rejects.toThrow(/must be within/i);
|
||||
expect(fs.existsSync('/etc/browse-record-should-not-exist')).toBe(false);
|
||||
});
|
||||
|
||||
test('a rejected start leaves no directory behind', async () => {
|
||||
const dir = path.join(TEMP_DIR, `browse-record-rejected-${Date.now()}`);
|
||||
expect(
|
||||
handleMetaCommand('record', ['start', dir, '--size', 'nonsense'], bm, shutdown),
|
||||
).rejects.toThrow(/expected WxH/);
|
||||
expect(fs.existsSync(dir)).toBe(false);
|
||||
});
|
||||
|
||||
test('no directory argument records under the temp dir', async () => {
|
||||
const started = await handleMetaCommand('record', ['start'], bm, shutdown);
|
||||
const match = /Recording → (\S+)/.exec(started);
|
||||
expect(match).not.toBeNull();
|
||||
const dir = match![1];
|
||||
tempDirs.push(dir);
|
||||
expect(dir).toContain('browse-record-');
|
||||
await handleMetaCommand('record', ['stop'], bm, shutdown);
|
||||
}, 30000);
|
||||
});
|
||||
|
|
@ -117,6 +117,7 @@ Run with `browse <command> [args]`. Full reference: `browse/SKILL.md`.
|
|||
- `domain-skill save|list|show|edit|promote-to-global|rollback|rm <host?>`: Per-site notes the agent writes for itself.
|
||||
- `frame <sel|@ref|--name n|--url pattern|main>`: Switch to iframe context (or main to return)
|
||||
- `inbox [--clear]`: List messages from sidebar scout inbox
|
||||
- `record start [dir] [--size WxH] | record stop | record status`: Record video of browser activity (interactive bug repros)
|
||||
- `skill list|show|run|test|rm <name?> [--arg k=v]... [--timeout=Ns]`: Run a browser-skill: deterministic Playwright script that drives the daemon over loopback HTTP.
|
||||
- `watch [stop]`: Passive observation — periodic snapshots while user browses
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue