---
name: scrape
version: 1.0.0
description: |
  Pull data from a web page. First call on a new intent prototypes the flow
  via $B primitives and returns JSON. Subsequent calls on a matching intent
  route to a codified browser-skill and return in ~200ms. Read-only — for
  mutating flows (form fills, clicks, submissions), use /automate.
  Use when asked to "scrape", "get data from", "pull", "extract from", or
  "what's on" a page. (gstack)
allowed-tools:
  - Bash
  - Read
  - AskUserQuestion
triggers:
  - scrape this page
  - get data from
  - pull from
  - extract from
  - what is on
---

{{PREAMBLE}}

# /scrape — pull data from a page

One entry point for getting data off the web. Two paths under the hood:

1. **Match path** (~200ms) — if the user's intent matches an existing
   browser-skill's triggers, run it via `$B skill run <name>` and emit
   the JSON.
2. **Prototype path** (~30s) — no matching skill yet, so drive the page
   with `$B` primitives, return the JSON, and suggest `/skillify` so the
   next call lands on the match path.

Read-only by contract. If the intent implies writing (submitting forms,
clicking buttons that mutate state), refuse and route to `/automate`.

## Step 1 — Determine intent

The user's request after `/scrape` is the intent. If they did not include
one, ask once:

> "What do you want to scrape? Describe it in one line, e.g. 'top stories
> on Hacker News' or 'product names + prices on example.com/products'."

Do not ask multiple clarifying questions up front. Any further questions
go in the prototype path where they're cheaper.

## Step 2 — Refuse mutating intents

If the intent implies writes — verbs like *submit*, *post*, *send*, *log
in*, *click X*, *fill the form*, *delete*, *create*, *order*, *book* —
respond:

> "/scrape is read-only. For mutating flows, use /automate (browser-skills
> Phase 2 P0 in TODOS.md — not yet shipped). Until then, use $B click /
> $B fill / $B type directly."

Stop. Do not enter the match or prototype path.

## Step 3 — Match phase

List existing browser-skills:

```bash
$B skill list
```

For each skill, `$B skill show <name>` exposes the full SKILL.md including
`triggers:`, `description:`, and `host:`. Read these and judge whether the
user's intent semantically matches one of them.

A confident match means **all three** are true:

- The intent's domain matches the skill's `host` (or one of its hostnames)
- A `triggers:` phrase or the `description:` covers the same data the
  intent asks for
- The intent does not require args the skill does not declare in `args:`

If matched, parse any `--arg key=value` from the intent (or pass none for
zero-arg skills) and run:

```bash
$B skill run <name> [--arg key=value ...]
```

Emit the JSON the skill prints to stdout — but sanity-check it against the
intent first. A matched skill can return stale or empty data; if the result
is empty, malformed, or clearly does not cover what the intent asked for,
the skill is likely out of date, so fall through to the prototype path
instead of emitting it. Otherwise emit it and stop.

If matching is ambiguous (two skills could plausibly fit), pick the
narrower-tier one (project > global > bundled — `$B skill list` shows the
tier). If still ambiguous, fall through to the prototype path rather than
guess wrong.

## Step 4 — Prototype phase

No match. Drive the page using `$B` primitives:

1. `$B goto <url>` — navigate to the target. The user's intent usually
   names a host or a URL; use it directly.
2. `$B snapshot --text` (or `$B text`) — get a clean text view of the
   page to find selectors.
3. `$B html` — pull the raw HTML when you need to parse structured data
   (lists, tables, repeated rows).
4. `$B links` — when the intent is to gather URLs.
5. Iterate: try a selector, check the output, refine.

Assemble the result as JSON (one document, not pretty-printed). Use a
stable shape — typically `{ "items": [...], "count": N }` or similar — so
downstream consumers can treat it as data. Do not emit it yet — it has to
clear the evidence gate first.

## Step 5 — Verify the prototype result against the intent

A plausible-looking JSON shape is not proof you got the data right. Before
emitting the prototype result as the answer, check it against the intent:

1. **Decompose the intent into checkpoints.** Each datum, field, filter,
   count, or ranking the intent named is one checkpoint. "Product names and
   prices for items under $50, cheapest first" is four checkpoints: names
   present, prices present, every price under $50, order ascending by price.
2. **Back each checkpoint with evidence.** Evidence is a populated field in
   the result (not null, not an empty string, not a placeholder) or a
   capture that shows it on the page — `$B snapshot --text` for a value,
   `$B screenshot` for a visible state. A filter or ranking checkpoint is
   satisfied when the returned data itself demonstrates it — every row
   genuinely under $50, the array genuinely ordered by price — whether the
   page applied it or you filtered and sorted the extracted rows yourself.
   What fails is asserting an order or filter the data cannot back: a
   "cheapest first" you never actually sorted, or a "most reviewed" with no
   review-count field to sort on.
3. **Be harsh.** An empty `items: []` passes only if the page genuinely has
   nothing matching, and you confirm that from a capture rather than
   assuming it. Partial rows (half the records missing a price) or a count
   that disagrees with an explicit quantity in the intent fail the gate.

If every checkpoint is backed, emit the JSON and move to the nudge. If any
checkpoint cannot be backed, treat it as a prototype failure (see "When the
prototype fails") — never emit partial data as the answer.

## Step 6 — Skillify nudge

After a successful prototype, append exactly one line:

> "Say /skillify to make this a permanent skill (200ms on next call)."

That is the entire nudge. Do not nag, do not list pros, do not push.
Proactive surfacing is a Phase 3 knob (`gstack-config browser_skillify_prompts`),
not this skill's job.

## When the prototype fails

If the page loads but data extraction does not yield a sensible JSON shape
after 3-4 selector attempts:

- Report what you tried, what came back, and what's blocking (lazy-loaded,
  JS-rendered, paywalled, etc.).
- Do NOT write a partial result and call it done.
- Do NOT suggest /skillify on a broken prototype.
- Ask the user whether they want to (a) try a different selector, (b)
  switch to a different page, or (c) stop.

## What this skill does NOT do

- Mutating actions (use /automate when shipped, or $B primitives directly)
- Auth flows / cookie import (use /setup-browser-cookies first)
- Multi-page crawls (this is one-shot per call)
- Anything that requires the daemon to not be running

## Output discipline

The match path returns whatever JSON the matched skill emits. The
prototype path returns the JSON you construct, once it clears the Step 5
evidence gate. In both cases:

- One JSON document, on stdout.
- Stderr (or chat) is for logs and the skillify nudge.
- Do not embed prose around the JSON in the chat reply unless the user
  asked for an explanation — many `/scrape` callers pipe the output to
  `jq`.

{{LEARNINGS_LOG}}