diff --git a/BROWSER.md b/BROWSER.md index 1ab7a1e60..7a9c2bdb9 100644 --- a/BROWSER.md +++ b/BROWSER.md @@ -280,6 +280,7 @@ from `snapshot`, or `@c` refs from `snapshot -C`. Full table: | `cookie-import-browser [browser] [--domain d]` | Import from installed Chromium browsers (interactive picker, or `--domain` for direct import) | | `header :` | Set custom request header (sensitive values auto-redacted) | | `useragent ` | Set user agent (triggers context recreation, invalidates refs) | +| `record start\|stop` | Toggle video recording (triggers context recreation, invalidates refs) | ### Tabs + frames @@ -333,6 +334,9 @@ from `snapshot`, or `@c` refs from `snapshot -C`. Full table: | `chain` (JSON via stdin) | Run a sequence of commands. Pipe `[["cmd","arg1",...],...]` to `$B chain`. Stops at first error. | | `inbox [--clear]` | List messages from sidebar scout inbox | | `watch [stop]` | Passive observation — periodic snapshots while user browses; `stop` returns summary | +| `record start [dir] [--size WxH]` | Record video of browser activity; one `.webm` per tab that rendered. Rebuilds the context, so refs are invalidated. Headless only, control scope | +| `record stop` | Flush and list the video files this recording produced | +| `record status` | Report the active recording directory, or that none is running | ### Browser-skills runtime diff --git a/browse/SKILL.md b/browse/SKILL.md index d977f363f..9adda1b57 100644 --- a/browse/SKILL.md +++ b/browse/SKILL.md @@ -664,7 +664,35 @@ $B diff https://staging.app.com https://prod.app.com ### 11. Show screenshots to the user After `$B screenshot`, `$B snapshot -a -o`, or `$B responsive`, always use the Read tool on the output PNG(s) so the user can see them. Without this, screenshots are invisible. -### 12. Render local HTML (no HTTP server needed) +### 12. Record a video of an interactive bug +A screenshot proves what a page looked like; it can't show what a page *did*. When +the bug is in the timing — a double-submit, a loading flicker, focus jumping, a +drag that drops in the wrong place — record the repro instead of describing it. + +```bash +$B record start # or: record start /tmp/repro --size 1280x720 +$B goto https://app.example.com/checkout +$B click @e4 +$B record stop # flushes and lists the .webm files +``` + +Recording is a browser-context setting, so `start` and `stop` each rebuild the +context. Cookies, storage, and open tabs survive that, but `@e` refs do not — +re-snapshot after `record stop` before you act on the page again. One `.webm` is +written per tab that rendered while recording ran, including tabs you closed +along the way; a tab opened and closed in the same instant never paints and +produces nothing. Keep clips short: a few seconds around the moment it breaks +beats a minute of navigation. Stick with screenshots for static bugs (a typo, a +clipped element, a wrong color) — they are cheaper to produce and easier to read. + +`record stop` is what hands you the file paths, so stop before you walk away: a +recording still running when the daemon idles out leaves its `.webm` in the +directory, but nothing prints the paths. Recording is headless-only (`handoff` +and `connect` hand the browser to the user, and their window is theirs to +capture), and it needs control scope — the video keeps whatever was on screen, +including anything typed into a login form. + +### 13. Render local HTML (no HTTP server needed) Two paths, pick the cleaner one: ```bash # HTML file on disk → goto file:// (absolute, or cwd-relative) @@ -679,7 +707,7 @@ $B load-html /tmp/tweet.html `goto file://...` is usually cleaner (URL is saved in state, relative asset URLs resolve against the file's dir, scale changes replay naturally). `load-html` uses `page.setContent()` — URL stays `about:blank`, but the content survives `viewport --scale` via in-memory replay. Both are scoped to files under cwd or `$TMPDIR`. -### 13. Retina screenshots (deviceScaleFactor) +### 14. Retina screenshots (deviceScaleFactor) ```bash $B viewport 480x600 --scale 2 # 2x deviceScaleFactor $B load-html /tmp/tweet.html # or: $B goto file://./tweet.html @@ -688,7 +716,7 @@ $B screenshot /tmp/out.png --selector .tweet-card ``` Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode. -### 14. Offline render mode (rasterize your own HTML/JSON, zero network) +### 15. Offline render mode (rasterize your own HTML/JSON, zero network) This is the blessed path for "I just want to turn my own local HTML or JSON into a PNG/PDF/bytes on disk" — Excalidraw diagrams, tweet/quote cards, og-images, @@ -1025,6 +1053,7 @@ $B prettyscreenshot --cleanup --scroll-to ".pricing" --width 1440 ~/Desktop/hero | `domain-skill save|list|show|edit|promote-to-global|rollback|rm ` | Per-site notes the agent writes for itself. Host is derived from the active tab. Lifecycle: `save` adds a quarantined note → after N=3 successful uses without the prompt-injection classifier flagging it, the note auto-promotes to "active" → `promote-to-global` lifts it to the global tier (machine-wide, all projects). The classifier flag is set automatically by the L4 prompt-injection scan; agents do not set it manually. Use `list` / `show` to inspect, `edit` to revise, `rollback` to demote, `rm` to tombstone. | | `frame ` | Switch to iframe context (or main to return) | | `inbox [--clear]` | List messages from sidebar scout inbox | +| `record start [dir] [--size WxH] | record stop | record status` | Record video of browser activity (interactive bug repros) | | `skill list|show|run|test|rm [--arg k=v]... [--timeout=Ns]` | Run a browser-skill: deterministic Playwright script that drives the daemon over loopback HTTP. 3-tier lookup (project > global > bundled). Spawned scripts get a per-spawn scoped token (read+write only) — never the daemon root token. | | `watch [stop]` | Passive observation — periodic snapshots while user browses | diff --git a/browse/SKILL.md.tmpl b/browse/SKILL.md.tmpl index 1da7698b2..65c2bbd17 100644 --- a/browse/SKILL.md.tmpl +++ b/browse/SKILL.md.tmpl @@ -111,7 +111,35 @@ $B diff https://staging.app.com https://prod.app.com ### 11. Show screenshots to the user After `$B screenshot`, `$B snapshot -a -o`, or `$B responsive`, always use the Read tool on the output PNG(s) so the user can see them. Without this, screenshots are invisible. -### 12. Render local HTML (no HTTP server needed) +### 12. Record a video of an interactive bug +A screenshot proves what a page looked like; it can't show what a page *did*. When +the bug is in the timing — a double-submit, a loading flicker, focus jumping, a +drag that drops in the wrong place — record the repro instead of describing it. + +```bash +$B record start # or: record start /tmp/repro --size 1280x720 +$B goto https://app.example.com/checkout +$B click @e4 +$B record stop # flushes and lists the .webm files +``` + +Recording is a browser-context setting, so `start` and `stop` each rebuild the +context. Cookies, storage, and open tabs survive that, but `@e` refs do not — +re-snapshot after `record stop` before you act on the page again. One `.webm` is +written per tab that rendered while recording ran, including tabs you closed +along the way; a tab opened and closed in the same instant never paints and +produces nothing. Keep clips short: a few seconds around the moment it breaks +beats a minute of navigation. Stick with screenshots for static bugs (a typo, a +clipped element, a wrong color) — they are cheaper to produce and easier to read. + +`record stop` is what hands you the file paths, so stop before you walk away: a +recording still running when the daemon idles out leaves its `.webm` in the +directory, but nothing prints the paths. Recording is headless-only (`handoff` +and `connect` hand the browser to the user, and their window is theirs to +capture), and it needs control scope — the video keeps whatever was on screen, +including anything typed into a login form. + +### 13. Render local HTML (no HTTP server needed) Two paths, pick the cleaner one: ```bash # HTML file on disk → goto file:// (absolute, or cwd-relative) @@ -126,7 +154,7 @@ $B load-html /tmp/tweet.html `goto file://...` is usually cleaner (URL is saved in state, relative asset URLs resolve against the file's dir, scale changes replay naturally). `load-html` uses `page.setContent()` — URL stays `about:blank`, but the content survives `viewport --scale` via in-memory replay. Both are scoped to files under cwd or `$TMPDIR`. -### 13. Retina screenshots (deviceScaleFactor) +### 14. Retina screenshots (deviceScaleFactor) ```bash $B viewport 480x600 --scale 2 # 2x deviceScaleFactor $B load-html /tmp/tweet.html # or: $B goto file://./tweet.html @@ -135,7 +163,7 @@ $B screenshot /tmp/out.png --selector .tweet-card ``` Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode. -### 14. Offline render mode (rasterize your own HTML/JSON, zero network) +### 15. Offline render mode (rasterize your own HTML/JSON, zero network) This is the blessed path for "I just want to turn my own local HTML or JSON into a PNG/PDF/bytes on disk" — Excalidraw diagrams, tweet/quote cards, og-images, diff --git a/browse/src/browser-manager.ts b/browse/src/browser-manager.ts index af601b5d1..7ebc2d594 100644 --- a/browse/src/browser-manager.ts +++ b/browse/src/browser-manager.ts @@ -205,6 +205,16 @@ export type { RefEntry }; // Re-export TabSession for consumers export { TabSession }; +/** Outcome of a recording: the files it produced, and anything that went wrong. */ +export interface RecordingResult { + /** Video files with bytes in them. */ + videos: string[]; + /** Files that were created but never received bytes — missing evidence, not saved video. */ + empty: string[]; + /** Degraded context rebuild or unreadable directory; null when the flush was clean. */ + warning: string | null; +} + export interface BrowserState { cookies: Cookie[]; pages: Array<{ @@ -249,6 +259,17 @@ export class BrowserManager { private deviceScaleFactor: number = 1; private currentViewport: { width: number; height: number } = { width: 1280, height: 720 }; + // ─── Video recording (context option) ──────────────────────── + // Playwright records at the context level and flushes each page's .webm when + // the context closes, so both start and stop go through recreateContext(). + // Null means not recording. + private recording: { + dir: string; + /** Videos already in `dir` when this recording started — not ours to report. */ + preexisting: Set; + size?: { width: number; height: number }; + } | null = null; + /** Server port — set after server starts, used by cookie-import-browser command */ public serverPort: number = 0; @@ -483,14 +504,7 @@ export class BrowserManager { void handleChromiumDisconnect(this.browser); }); - const contextOptions: BrowserContextOptions = { - viewport: { width: this.currentViewport.width, height: this.currentViewport.height }, - deviceScaleFactor: this.deviceScaleFactor, - }; - if (this.customUserAgent) { - contextOptions.userAgent = this.customUserAgent; - } - this.context = await this.browser.newContext(contextOptions); + this.context = await this.browser.newContext(this.buildContextOptions()); if (Object.keys(this.extraHeaders).length > 0) { await this.context.setExtraHTTPHeaders(this.extraHeaders); @@ -1501,14 +1515,7 @@ export class BrowserManager { await this.context.close().catch(() => {}); // 3. Create new context with updated settings - const contextOptions: BrowserContextOptions = { - viewport: { width: this.currentViewport.width, height: this.currentViewport.height }, - deviceScaleFactor: this.deviceScaleFactor, - }; - if (this.customUserAgent) { - contextOptions.userAgent = this.customUserAgent; - } - this.context = await this.browser.newContext(contextOptions); + this.context = await this.browser.newContext(this.buildContextOptions()); // Re-apply stealth: newContext() is a fresh context with no init scripts, // so a useragent / viewport --scale rebuild would otherwise drop the @@ -1533,14 +1540,7 @@ export class BrowserManager { this.tabSessions.clear(); if (this.context) await this.context.close().catch(() => {}); - const contextOptions: BrowserContextOptions = { - viewport: { width: this.currentViewport.width, height: this.currentViewport.height }, - deviceScaleFactor: this.deviceScaleFactor, - }; - if (this.customUserAgent) { - contextOptions.userAgent = this.customUserAgent; - } - this.context = await this.browser!.newContext(contextOptions); + this.context = await this.browser!.newContext(this.buildContextOptions()); // Stealth applies to the fallback blank context too. const { applyStealth } = await import('./stealth'); await applyStealth(this.context); @@ -1553,6 +1553,134 @@ export class BrowserManager { } } + // ─── Video Recording ───────────────────────────────────────── + /** + * Start recording video of browser activity into `dir`. + * + * Recording is a context option, so this rebuilds the context through + * recreateContext(), which preserves cookies, storage, and open tabs. + * + * Returns the result of any recording this call superseded, plus the warning + * recreateContext() uses to report a degraded rebuild. Both are the caller's + * to report: a superseded take is already on disk, and a caller who is not + * handed its paths has lost it. + */ + async startRecording( + dir: string, + size?: { width: number; height: number }, + ): Promise<{ superseded: RecordingResult | null; warning: string | null }> { + if (this.connectionMode === 'headed') { + throw new Error('record is not supported in headed mode — the visible window is the real browser. Run `$B disconnect` first, or capture the screen with an OS recorder.'); + } + if (!this.browser || !this.context) { + throw new Error('Browser not launched'); + } + + const superseded = this.recording ? await this.stopRecording() : null; + + mkdirSecure(dir); + this.recording = { dir, preexisting: this.videoNamesIn(dir), ...(size ? { size } : {}) }; + // A degraded rebuild still records: the fallback context is built from + // buildContextOptions() too, so the flag stays set and stop finds the files. + const warning = await this.recreateContext(); + + return { superseded, warning }; + } + + /** + * Stop recording and return every video the recording produced. + * + * Playwright writes one .webm per page and only flushes on context close, so + * this clears the flag and rebuilds the context to force the flush, then reads + * the recording directory. Reading the directory rather than the live pages is + * what catches a tab that was closed mid-recording — its video is on disk, and + * enumerating open pages would miss it. + * + * `warning` carries a degraded rebuild (the flush is that rebuild, so a + * failure there is a flush failure) and any error reading the directory back. + * Files that never received bytes are reported as `empty` rather than counted + * as saved. + */ + async stopRecording(): Promise { + if (!this.recording) return { videos: [], empty: [], warning: null }; + + const fs = require('fs'); + const path = require('path'); + const { dir, preexisting } = this.recording; + this.recording = null; + + // Closing the old context is what flushes the .webm files to disk. In headed + // mode that context is already gone (handoff replaced it, flushing as it + // closed), and recreateContext() would throw, so only rebuild when we own a + // headless context to rebuild. + let warning: string | null = null; + if (this.connectionMode !== 'headed') { + warning = await this.recreateContext(); + } + + let entries: string[]; + try { + entries = fs.readdirSync(dir).filter((name: string) => name.endsWith('.webm')); + } catch (err: unknown) { + const detail = err instanceof Error ? err.message : String(err); + return { videos: [], empty: [], warning: warning ?? `Recording directory could not be read: ${detail}` }; + } + + const videos: string[] = []; + const empty: string[] = []; + for (const name of entries) { + // Skip videos that were already there: a directory reused across takes + // would otherwise report the earlier one as part of this recording. + if (preexisting.has(name)) continue; + const full = path.join(dir, name); + try { + (fs.statSync(full).size > 0 ? videos : empty).push(full); + } catch { + empty.push(full); + } + } + + return { videos, empty, warning }; + } + + /** Names of the .webm files currently in `dir`; empty when it does not exist yet. */ + private videoNamesIn(dir: string): Set { + const fs = require('fs'); + try { + return new Set(fs.readdirSync(dir).filter((name: string) => name.endsWith('.webm'))); + } catch { + return new Set(); + } + } + + /** Directory the active recording is writing to, or null when not recording. */ + getRecordingDir(): string | null { + return this.recording?.dir ?? null; + } + + /** + * Context options every context in this manager is built from — the initial + * launch, the recreateContext() rebuild, and its clean-slate fallback. These + * settings only apply at context construction, so each one has to be repeated + * on every rebuild to survive it. + */ + private buildContextOptions(): BrowserContextOptions { + const options: BrowserContextOptions = { + viewport: { width: this.currentViewport.width, height: this.currentViewport.height }, + deviceScaleFactor: this.deviceScaleFactor, + }; + if (this.customUserAgent) { + options.userAgent = this.customUserAgent; + } + if (this.recording) { + options.recordVideo = { + dir: this.recording.dir, + ...(this.recording.size ? { size: this.recording.size } : {}), + }; + } + return options; + } + /** * Change deviceScaleFactor + viewport size atomically. * diff --git a/browse/src/commands.ts b/browse/src/commands.ts index 73bc9ab1b..de44266c8 100644 --- a/browse/src/commands.ts +++ b/browse/src/commands.ts @@ -39,6 +39,7 @@ export const META_COMMANDS = new Set([ 'connect', 'disconnect', 'focus', 'inbox', 'watch', + 'record', 'state', 'frame', 'ux-audit', @@ -168,6 +169,8 @@ export const COMMAND_DESCRIPTIONS: Record' }, // Frame @@ -225,6 +228,7 @@ export function canonicalizeCommand(cmd: string): string { */ export const NEW_IN_VERSION: Record = { 'load-html': '0.19.0.0', + 'record': '1.62.0.0', }; /** diff --git a/browse/src/meta-commands.ts b/browse/src/meta-commands.ts index 521c65a20..d8e5f399e 100644 --- a/browse/src/meta-commands.ts +++ b/browse/src/meta-commands.ts @@ -2,7 +2,7 @@ * Meta commands — tabs, server control, screenshots, chain, diff, snapshot */ -import type { BrowserManager } from './browser-manager'; +import type { BrowserManager, RecordingResult } from './browser-manager'; import { handleSnapshot } from './snapshot'; import { getCleanText } from './read-commands'; import { READ_COMMANDS, WRITE_COMMANDS, META_COMMANDS, PAGE_CONTENT_COMMANDS, wrapUntrustedContent, canonicalizeCommand } from './commands'; @@ -23,6 +23,29 @@ import { resolveConfig } from './config'; import { filterSessionCookies } from './session-persist'; import type { Frame } from 'playwright'; +/** + * Render a recording outcome. Shared by `record start` (for a take it + * superseded) and `record stop` so a superseded recording is reported the same + * way as one the caller stopped on purpose. + */ +function formatRecordingResult(result: RecordingResult, headline: string): string[] { + const lines: string[] = []; + if (result.videos.length > 0) { + lines.push(`${headline} — ${result.videos.length} video${result.videos.length === 1 ? '' : 's'}:`); + lines.push(...result.videos.map(v => ` ${v}`)); + } else { + lines.push(`${headline} — no video was written. A recording only captures pages that rendered while it ran.`); + } + if (result.empty.length > 0) { + lines.push('Empty (never flushed) — missing evidence, not saved video:'); + lines.push(...result.empty.map(v => ` ${v}`)); + } + if (result.warning) { + lines.push(`⚠️ ${result.warning}`); + } + return lines; +} + /** Tokenize a pipe segment respecting double-quoted strings. */ function tokenizePipeSegment(segment: string): string[] { const tokens: string[] = []; @@ -836,6 +859,70 @@ export async function handleMetaCommand( return 'WATCHING — observing user browsing. Periodic snapshots every 5s.\nRun `$B watch stop` to stop and get summary.'; } + // ─── Record ───────────────────────────────────────── + case 'record': { + const action = args[0]; + if (!action) throw new Error('Usage: record start [dir] [--size WxH] | record stop | record status'); + + if (action === 'status') { + const dir = bm.getRecordingDir(); + return dir ? `Recording → ${dir}` : 'Not recording.'; + } + + if (action === 'stop') { + if (!bm.getRecordingDir()) return 'Not recording (nothing to stop).'; + return [ + ...formatRecordingResult(await bm.stopRecording(), 'Recording stopped'), + 'Refs are stale after a recording stops — re-snapshot before acting on the page.', + ].join('\n'); + } + + if (action === 'start') { + let dirArg: string | undefined; + let size: { width: number; height: number } | undefined; + + for (let i = 1; i < args.length; i++) { + const token = args[i]; + if (token === '--size') { + const value = args[++i]; + if (!value) throw new Error('record start --size: missing value (e.g. --size 1280x720)'); + const match = /^(\d+)x(\d+)$/.exec(value); + if (!match) throw new Error(`record start --size: expected WxH, got '${value}'`); + const width = parseInt(match[1], 10); + const height = parseInt(match[2], 10); + if (width < 1 || height < 1) throw new Error(`record start --size: dimensions must be positive, got '${value}'`); + size = { width, height }; + } else if (token.startsWith('--')) { + throw new Error(`record start: unknown flag '${token}'`); + } else if (dirArg === undefined) { + dirArg = token; + } else { + throw new Error(`record start: unexpected argument '${token}'. Usage: record start [dir] [--size WxH]`); + } + } + + // Each recording gets its own directory: Playwright names videos by an + // internal id, so a shared directory makes the videos of two recordings + // indistinguishable after the fact. + const targetDir = dirArg + ? path.resolve(dirArg) + : path.join(TEMP_DIR, `browse-record-${new Date().toISOString().replace(/[:.]/g, '-')}`); + validateOutputPath(targetDir); + + const { superseded, warning } = await bm.startRecording(targetDir, size); + const lines: string[] = []; + if (superseded) { + lines.push(...formatRecordingResult(superseded, 'Stopped the recording already running')); + } + lines.push(`Recording → ${targetDir}`); + lines.push('Run `$B record stop` to flush and list the video files.'); + if (warning) lines.push(`⚠️ ${warning}`); + return lines.join('\n'); + } + + throw new Error(`record: unknown action '${action}'. Usage: record start|stop|status`); + } + // ─── Inbox ────────────────────────────────────────── case 'inbox': { const { execSync } = await import('child_process'); diff --git a/browse/src/token-registry.ts b/browse/src/token-registry.ts index 161b26b6d..256764b11 100644 --- a/browse/src/token-registry.ts +++ b/browse/src/token-registry.ts @@ -64,6 +64,11 @@ export const SCOPE_ADMIN = new Set([ /** Browser-wide destructive commands — can kill the server, disconnect headed mode */ export const SCOPE_CONTROL = new Set([ 'state', 'handoff', 'resume', 'stop', 'restart', 'connect', 'disconnect', + // `record` sits here rather than in meta for two reasons: it rebuilds the + // context out from under every open tab (same blast radius as `state`), and it + // writes whatever is on screen — including anything typed into a login form — + // to disk as video. + 'record', ]); /** Meta commands — generally safe but some need scope checking */ diff --git a/browse/test/record.test.ts b/browse/test/record.test.ts new file mode 100644 index 000000000..6c59b8059 --- /dev/null +++ b/browse/test/record.test.ts @@ -0,0 +1,349 @@ +/** + * Tests for the record command — video capture of browser activity. + * + * Integration tests drive a real Playwright browser against the local test + * server and assert on the bytes that land on disk, because the failure this + * command has to avoid is reporting a video that was never flushed. + */ + +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import { startTestServer } from './test-server'; +import { BrowserManager } from '../src/browser-manager'; +import { handleWriteCommand as _handleWriteCommand } from '../src/write-commands'; +import { handleMetaCommand } from '../src/meta-commands'; +import { META_COMMANDS, COMMAND_DESCRIPTIONS } from '../src/commands'; +import { SCOPE_CONTROL, SCOPE_READ, SCOPE_WRITE, checkScope } from '../src/token-registry'; +import * as fs from 'fs'; +import * as path from 'path'; +import { TEMP_DIR } from '../src/platform'; + +const handleWriteCommand = (cmd: string, args: string[], b: BrowserManager) => + _handleWriteCommand(cmd, args, b.getActiveSession(), b); + +const shutdown = async () => {}; + +let testServer: ReturnType; +let bm: BrowserManager; +let baseUrl: string; +const tempDirs: string[] = []; + +/** A recording directory inside the browse temp root, cleaned up in afterAll. */ +function recordingDir(label: string): string { + const dir = fs.mkdtempSync(path.join(TEMP_DIR, `browse-record-test-${label}-`)); + tempDirs.push(dir); + return dir; +} + +function webmsIn(dir: string): string[] { + return fs.readdirSync(dir).filter(name => name.endsWith('.webm')); +} + +/** + * Read PixelWidth (EBML id 0xB0) and PixelHeight (0xBA) out of a WebM header. + * Both are short unsigned ints, so the 1- and 2-byte length forms cover every + * viewport we can ask for. Avoids depending on ffprobe being installed in CI. + */ +function videoDimensions(file: string): { width: number; height: number } { + const head = fs.readFileSync(file).subarray(0, 4096); + const read = (id: number): number => { + for (let i = 0; i < head.length - 4; i++) { + if (head[i] !== id) continue; + if (head[i + 1] === 0x81) return head[i + 2]; + if (head[i + 1] === 0x82) return (head[i + 2] << 8) | head[i + 3]; + } + return 0; + }; + return { width: read(0xb0), height: read(0xba) }; +} + +/** Give a page long enough to paint that Playwright is certain to emit a video. */ +async function settle(ms = 400): Promise { + await new Promise(resolve => setTimeout(resolve, ms)); +} + +beforeAll(async () => { + testServer = startTestServer(0); + baseUrl = testServer.url; + bm = new BrowserManager(); + await bm.launch(); +}); + +// Close the browser rather than forcing the process down: a process.exit here +// truncates whatever else the runner still had queued. close() races Chromium's +// shutdown against its own 5s cap, so the hook needs more than bun's 5s default. +afterAll(async () => { + await bm.close().catch(() => {}); + try { testServer.server.stop(); } catch {} + for (const dir of tempDirs) { + fs.rmSync(dir, { recursive: true, force: true }); + } +}, 20000); + +// ─── Registration ─────────────────────────────────────────────── + +describe('record registration', () => { + test('is a known meta command with usage text', () => { + expect(META_COMMANDS.has('record')).toBe(true); + expect(COMMAND_DESCRIPTIONS['record']?.usage).toContain('record start'); + }); + + test('is control-scoped — it rebuilds the context and writes screen content to disk', () => { + expect(SCOPE_CONTROL.has('record')).toBe(true); + expect(SCOPE_READ.has('record')).toBe(false); + expect(SCOPE_WRITE.has('record')).toBe(false); + }); + + test('a read-only token cannot record, a control token can', () => { + const readOnly = { token: 't', clientId: 'agent', type: 'session' as const, scopes: ['read' as const] }; + const control = { token: 't', clientId: 'agent', type: 'session' as const, scopes: ['control' as const] }; + expect(checkScope(readOnly, 'record')).toBe(false); + expect(checkScope(control, 'record')).toBe(true); + }); +}); + +// ─── Argument handling ────────────────────────────────────────── + +describe('record argument handling', () => { + test('status reports not-recording before any start', async () => { + const result = await handleMetaCommand('record', ['status'], bm, shutdown); + expect(result).toBe('Not recording.'); + }); + + test('stop without an active recording is a no-op, not an error', async () => { + const result = await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(result).toContain('Not recording'); + }); + + test('missing action, unknown action, and unknown flag all explain the usage', async () => { + expect(handleMetaCommand('record', [], bm, shutdown)).rejects.toThrow(/Usage: record start/); + expect(handleMetaCommand('record', ['bogus'], bm, shutdown)).rejects.toThrow(/unknown action/); + expect(handleMetaCommand('record', ['start', '--bogus'], bm, shutdown)).rejects.toThrow(/unknown flag/); + }); + + test('malformed --size is rejected before a context is rebuilt', async () => { + expect(handleMetaCommand('record', ['start', '--size'], bm, shutdown)).rejects.toThrow(/missing value/); + expect(handleMetaCommand('record', ['start', '--size', '1280'], bm, shutdown)).rejects.toThrow(/expected WxH/); + expect(handleMetaCommand('record', ['start', '--size', '0x720'], bm, shutdown)).rejects.toThrow(/must be positive/); + expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toBe('Not recording.'); + }); +}); + +// ─── Recording ────────────────────────────────────────────────── + +describe('record capture', () => { + test('start → activity → stop writes a playable webm and reports its path', async () => { + const dir = recordingDir('basic'); + const started = await handleMetaCommand('record', ['start', dir], bm, shutdown); + expect(started).toContain(dir); + expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toContain(dir); + + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm); + + const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown); + const videos = webmsIn(dir); + expect(videos.length).toBeGreaterThan(0); + expect(stopped).toContain(videos[0]); + + // EBML magic bytes — a real Matroska/WebM container, not an empty file. + const bytes = fs.readFileSync(path.join(dir, videos[0])); + expect(bytes.length).toBeGreaterThan(0); + expect([bytes[0], bytes[1], bytes[2], bytes[3]]).toEqual([0x1a, 0x45, 0xdf, 0xa3]); + + expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toBe('Not recording.'); + }, 30000); + + test('the browser still works after a recording stops', async () => { + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + const url = await handleMetaCommand('url', [], bm, shutdown); + expect(url).toContain('/snapshot.html'); + }, 15000); + + test('a tab closed mid-recording still has its video reported', async () => { + const dir = recordingDir('closed-tab'); + await handleMetaCommand('record', ['start', dir], bm, shutdown); + + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + await handleMetaCommand('newtab', [baseUrl + '/forms.html'], bm, shutdown); + // A tab has to actually paint before Playwright emits a video for it; a tab + // opened and closed inside the same few milliseconds produces nothing. + await settle(); + await handleMetaCommand('closetab', [], bm, shutdown); + + const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown); + const videos = webmsIn(dir); + + // Enumerating live pages at stop time would report one video here. Every + // file the recording produced has to be listed, or the caller is told they + // have less evidence than they do. + expect(videos.length).toBeGreaterThanOrEqual(2); + for (const name of videos) { + expect(stopped).toContain(path.join(dir, name)); + } + }, 45000); + + test('cookies survive the context rebuild that start and stop perform', async () => { + const dir = recordingDir('state'); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await bm.getActiveSession().page.context().addCookies([ + { name: 'record_state_probe', value: 'kept', url: baseUrl }, + ]); + + await handleMetaCommand('record', ['start', dir], bm, shutdown); + const during = await bm.getActiveSession().page.context().cookies(); + expect(during.some(c => c.name === 'record_state_probe')).toBe(true); + + await handleMetaCommand('record', ['stop'], bm, shutdown); + const after = await bm.getActiveSession().page.context().cookies(); + expect(after.some(c => c.name === 'record_state_probe')).toBe(true); + }, 30000); + + test('starting while already recording reports the take it superseded', async () => { + const first = recordingDir('first'); + const second = recordingDir('second'); + + await handleMetaCommand('record', ['start', first], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + const restarted = await handleMetaCommand('record', ['start', second], bm, shutdown); + + // The first take is on disk. Telling the caller only about the new + // recording would strand it somewhere they were never given the path to. + const firstVideos = webmsIn(first); + expect(firstVideos.length).toBeGreaterThan(0); + for (const name of firstVideos) { + expect(restarted).toContain(path.join(first, name)); + } + expect(restarted).toContain(second); + expect(await handleMetaCommand('record', ['status'], bm, shutdown)).toContain(second); + + await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm); + await settle(); + await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(webmsIn(second).length).toBeGreaterThan(0); + }, 60000); + + test('--size sets the recorded frame size', async () => { + const dir = recordingDir('size'); + await handleMetaCommand('record', ['start', dir, '--size', '640x480'], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + await handleMetaCommand('record', ['stop'], bm, shutdown); + + const videos = webmsIn(dir); + expect(videos.length).toBeGreaterThan(0); + expect(videoDimensions(path.join(dir, videos[0]))).toEqual({ width: 640, height: 480 }); + }, 30000); + + test('a reused directory does not re-report the earlier take', async () => { + const dir = recordingDir('reused'); + + await handleMetaCommand('record', ['start', dir], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + const firstStop = await handleMetaCommand('record', ['stop'], bm, shutdown); + const afterFirst = webmsIn(dir); + expect(afterFirst.length).toBeGreaterThan(0); + + await handleMetaCommand('record', ['start', dir], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm); + await settle(); + const secondStop = await handleMetaCommand('record', ['stop'], bm, shutdown); + + // Both takes live in the same directory, so the second report must name + // only what the second take produced. + const listed = (report: string) => + report.split('\n').map(l => l.trim()).filter(l => l.endsWith('.webm')); + expect(webmsIn(dir).length).toBeGreaterThan(afterFirst.length); + expect(listed(firstStop).length).toBeGreaterThan(0); + expect(listed(secondStop).length).toBeGreaterThan(0); + for (const earlier of listed(firstStop)) { + expect(listed(secondStop)).not.toContain(earlier); + } + }, 60000); +}); + +// ─── Failure reporting ────────────────────────────────────────── + +describe('record failure reporting', () => { + test('a degraded flush is surfaced, not swallowed', async () => { + const dir = recordingDir('degraded'); + await handleMetaCommand('record', ['start', dir], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + + // The rebuild IS the flush, so recreateContext reporting a degraded rebuild + // means the flush degraded. Silently dropping it tells the caller their + // evidence is fine when their tabs were just reset. + const real = (bm as any).recreateContext.bind(bm); + (bm as any).recreateContext = async () => 'Context recreation failed: simulated. Browser reset to blank tab.'; + try { + const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(stopped).toContain('Context recreation failed'); + } finally { + (bm as any).recreateContext = real; + } + }, 30000); + + test('an unreadable recording directory is reported as a failure', async () => { + const dir = recordingDir('vanished'); + await handleMetaCommand('record', ['start', dir], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + fs.rmSync(dir, { recursive: true, force: true }); + + const stopped = await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(stopped).toMatch(/could not be read/i); + }, 30000); + + test('stop after a switch to headed mode still hands back the videos', async () => { + const dir = recordingDir('headed'); + await handleMetaCommand('record', ['start', dir], bm, shutdown); + await handleWriteCommand('goto', [baseUrl + '/snapshot.html'], bm); + await settle(); + + // handoff/connect close the recording context (flushing as they go) and + // leave the manager headed, where recreateContext() throws. The videos are + // already on disk, so stop has to hand them back rather than raise. + await (bm as any).context.close().catch(() => {}); + (bm as any).connectionMode = 'headed'; + try { + const result = await bm.stopRecording(); + expect(result.videos.length).toBeGreaterThan(0); + expect(bm.getRecordingDir()).toBeNull(); + } finally { + (bm as any).connectionMode = 'launched'; + await (bm as any).recreateContext(); + } + }, 30000); +}); + +// ─── Path policy ──────────────────────────────────────────────── + +describe('record path policy', () => { + test('a directory outside the safe roots is refused', async () => { + expect( + handleMetaCommand('record', ['start', '/etc/browse-record-should-not-exist'], bm, shutdown), + ).rejects.toThrow(/must be within/i); + expect(fs.existsSync('/etc/browse-record-should-not-exist')).toBe(false); + }); + + test('a rejected start leaves no directory behind', async () => { + const dir = path.join(TEMP_DIR, `browse-record-rejected-${Date.now()}`); + expect( + handleMetaCommand('record', ['start', dir, '--size', 'nonsense'], bm, shutdown), + ).rejects.toThrow(/expected WxH/); + expect(fs.existsSync(dir)).toBe(false); + }); + + test('no directory argument records under the temp dir', async () => { + const started = await handleMetaCommand('record', ['start'], bm, shutdown); + const match = /Recording → (\S+)/.exec(started); + expect(match).not.toBeNull(); + const dir = match![1]; + tempDirs.push(dir); + expect(dir).toContain('browse-record-'); + await handleMetaCommand('record', ['stop'], bm, shutdown); + }, 30000); +}); diff --git a/gstack/llms.txt b/gstack/llms.txt index 979acf3e7..3efb598c1 100644 --- a/gstack/llms.txt +++ b/gstack/llms.txt @@ -117,6 +117,7 @@ Run with `browse [args]`. Full reference: `browse/SKILL.md`. - `domain-skill save|list|show|edit|promote-to-global|rollback|rm `: Per-site notes the agent writes for itself. - `frame `: Switch to iframe context (or main to return) - `inbox [--clear]`: List messages from sidebar scout inbox +- `record start [dir] [--size WxH] | record stop | record status`: Record video of browser activity (interactive bug repros) - `skill list|show|run|test|rm [--arg k=v]... [--timeout=Ns]`: Run a browser-skill: deterministic Playwright script that drives the daemon over loopback HTTP. - `watch [stop]`: Passive observation — periodic snapshots while user browses