browser-automation-skill 0.71.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +144 -0
- package/SECURITY.md +39 -0
- package/SKILL.md +206 -0
- package/bin/cli.mjs +55 -0
- package/install.sh +143 -0
- package/package.json +54 -0
- package/references/adapter-candidates.md +40 -0
- package/references/browser-mcp-cheatsheet.md +132 -0
- package/references/browser-stats-cheatsheet.md +155 -0
- package/references/chrome-devtools-mcp-cheatsheet.md +232 -0
- package/references/midscene-integration.md +359 -0
- package/references/obscura-cheatsheet.md +103 -0
- package/references/playwright-cli-cheatsheet.md +64 -0
- package/references/playwright-lib-cheatsheet.md +90 -0
- package/references/recipes/add-a-tool-adapter.md +134 -0
- package/references/recipes/agent-workflows/README.md +37 -0
- package/references/recipes/agent-workflows/cache-driven-bulk-operation.md +110 -0
- package/references/recipes/agent-workflows/flow-record-and-replay.md +102 -0
- package/references/recipes/agent-workflows/incremental-pattern-discovery.md +125 -0
- package/references/recipes/agent-workflows/login-then-scrape.md +100 -0
- package/references/recipes/anti-patterns-tool-extension.md +182 -0
- package/references/recipes/body-bytes-not-body.md +139 -0
- package/references/recipes/cache-write-security.md +210 -0
- package/references/recipes/fingerprint-rescue.md +154 -0
- package/references/recipes/model-routing.md +143 -0
- package/references/recipes/path-security.md +138 -0
- package/references/recipes/privacy-canary.md +96 -0
- package/references/recipes/visual-rescue-hook.md +182 -0
- package/references/stats-prices.json +42 -0
- package/references/stats-schema.json +77 -0
- package/references/tool-versions.md +8 -0
- package/scripts/browser-add-site.sh +113 -0
- package/scripts/browser-assert.sh +106 -0
- package/scripts/browser-audit.sh +68 -0
- package/scripts/browser-baseline.sh +135 -0
- package/scripts/browser-click.sh +100 -0
- package/scripts/browser-creds-add.sh +254 -0
- package/scripts/browser-creds-list.sh +67 -0
- package/scripts/browser-creds-migrate.sh +122 -0
- package/scripts/browser-creds-remove.sh +69 -0
- package/scripts/browser-creds-rotate-totp.sh +109 -0
- package/scripts/browser-creds-show.sh +82 -0
- package/scripts/browser-creds-totp.sh +94 -0
- package/scripts/browser-do.sh +630 -0
- package/scripts/browser-doctor.sh +365 -0
- package/scripts/browser-drag.sh +90 -0
- package/scripts/browser-extract.sh +192 -0
- package/scripts/browser-fill.sh +142 -0
- package/scripts/browser-flow.sh +316 -0
- package/scripts/browser-history.sh +187 -0
- package/scripts/browser-hover.sh +92 -0
- package/scripts/browser-inspect.sh +188 -0
- package/scripts/browser-list-sessions.sh +78 -0
- package/scripts/browser-list-sites.sh +42 -0
- package/scripts/browser-login.sh +279 -0
- package/scripts/browser-mcp.sh +65 -0
- package/scripts/browser-migrate.sh +195 -0
- package/scripts/browser-open.sh +134 -0
- package/scripts/browser-press.sh +80 -0
- package/scripts/browser-remove-session.sh +72 -0
- package/scripts/browser-remove-site.sh +68 -0
- package/scripts/browser-replay.sh +206 -0
- package/scripts/browser-route.sh +174 -0
- package/scripts/browser-select.sh +122 -0
- package/scripts/browser-show-session.sh +57 -0
- package/scripts/browser-show-site.sh +37 -0
- package/scripts/browser-snapshot.sh +176 -0
- package/scripts/browser-stats.sh +522 -0
- package/scripts/browser-tab-close.sh +112 -0
- package/scripts/browser-tab-list.sh +70 -0
- package/scripts/browser-tab-switch.sh +111 -0
- package/scripts/browser-upload.sh +132 -0
- package/scripts/browser-use.sh +60 -0
- package/scripts/browser-vlm.sh +707 -0
- package/scripts/browser-wait.sh +97 -0
- package/scripts/install-git-hooks.sh +16 -0
- package/scripts/lib/capture.sh +356 -0
- package/scripts/lib/common.sh +262 -0
- package/scripts/lib/credential.sh +237 -0
- package/scripts/lib/fingerprint-rescue.js +123 -0
- package/scripts/lib/flow.sh +448 -0
- package/scripts/lib/flow_record.sh +210 -0
- package/scripts/lib/mask.sh +49 -0
- package/scripts/lib/memory.sh +427 -0
- package/scripts/lib/migrate.sh +390 -0
- package/scripts/lib/migrators/README.md +23 -0
- package/scripts/lib/migrators/memory/v1_to_v2.sh +15 -0
- package/scripts/lib/migrators/recent_urls/README.md +13 -0
- package/scripts/lib/migrators/stats/README.md +24 -0
- package/scripts/lib/node/chrome-devtools-bridge.mjs +1812 -0
- package/scripts/lib/node/mcp-server.mjs +531 -0
- package/scripts/lib/node/mcp-tools.json +68 -0
- package/scripts/lib/node/playwright-driver.mjs +1104 -0
- package/scripts/lib/node/totp-core.mjs +52 -0
- package/scripts/lib/node/totp.mjs +52 -0
- package/scripts/lib/node/url-pattern-cluster.mjs +102 -0
- package/scripts/lib/node/url-pattern-resolver.mjs +77 -0
- package/scripts/lib/output.sh +79 -0
- package/scripts/lib/router.sh +342 -0
- package/scripts/lib/sanitize.sh +107 -0
- package/scripts/lib/secret/keychain.sh +91 -0
- package/scripts/lib/secret/libsecret.sh +74 -0
- package/scripts/lib/secret/plaintext.sh +75 -0
- package/scripts/lib/secret_backend_select.sh +57 -0
- package/scripts/lib/session.sh +153 -0
- package/scripts/lib/site.sh +126 -0
- package/scripts/lib/stats.sh +419 -0
- package/scripts/lib/tool/.gitkeep +0 -0
- package/scripts/lib/tool/chrome-devtools-mcp.sh +349 -0
- package/scripts/lib/tool/obscura.sh +249 -0
- package/scripts/lib/tool/playwright-cli.sh +155 -0
- package/scripts/lib/tool/playwright-lib.sh +106 -0
- package/scripts/lib/verb_helpers.sh +222 -0
- package/scripts/lib/visual-rescue-default.sh +145 -0
- package/scripts/regenerate-docs.sh +99 -0
- package/uninstall.sh +51 -0
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
# Recipe: Add a tool adapter
|
|
2
|
+
|
|
3
|
+
A 30-minute walkthrough for adding a new browser-automation adapter to the skill. Follow the **Path A** checklist for the initial commit; promote to default in a separate PR via **Path B**.
|
|
4
|
+
|
|
5
|
+
## When to use this recipe
|
|
6
|
+
|
|
7
|
+
Use this when adding a new browser-automation tool to the toolbox:
|
|
8
|
+
- `puppeteer`, `playwright-mcp`, `browserless`, etc.
|
|
9
|
+
|
|
10
|
+
Do NOT use this recipe for:
|
|
11
|
+
- Adding a verb (see `add-a-verb.md`).
|
|
12
|
+
- Changing routing precedence among existing tools (see `change-a-routing-rule.md`).
|
|
13
|
+
|
|
14
|
+
## Path A — Ship-without-promotion (zero edits to existing .sh files)
|
|
15
|
+
|
|
16
|
+
The adapter is reachable via `--tool=<name>` but is never the default for any verb. **This is the recommended way to introduce ANY new tool.** Soak-test in real workflows; promote later.
|
|
17
|
+
|
|
18
|
+
### Checklist
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
1. Create scripts/lib/tool/<tool>.sh — implement the contract:
|
|
22
|
+
- Identity (3 fns): tool_metadata, tool_capabilities, tool_doctor_check
|
|
23
|
+
- Verb dispatch (8): tool_open, tool_click, tool_fill, tool_snapshot,
|
|
24
|
+
tool_inspect, tool_audit, tool_extract, tool_eval
|
|
25
|
+
(return 41 / TOOL_UNSUPPORTED_OP for unsupported.)
|
|
26
|
+
- tool_metadata.name MUST equal the filename (lint enforces).
|
|
27
|
+
- tool_metadata.abi_version MUST equal BROWSER_SKILL_TOOL_ABI in common.sh.
|
|
28
|
+
- The adapter MUST `source "$(dirname "${BASH_SOURCE[0]}")/../output.sh"`
|
|
29
|
+
so verb output goes through emit_summary / emit_event (lint tier 3 enforces).
|
|
30
|
+
2. Create tests/stubs/<tool> — mock binary; logs argv to ${STUB_LOG_FILE}; returns canned JSON.
|
|
31
|
+
3. Create tests/fixtures/<tool>/ — JSON keyed by sha256(argv joined by NUL).
|
|
32
|
+
4. Create tests/<tool>_adapter.bats — contract conformance + happy-path tests.
|
|
33
|
+
5. Create references/<tool>-cheatsheet.md — usage notes.
|
|
34
|
+
6. Run scripts/regenerate-docs.sh — autogen edits references/tool-versions.md
|
|
35
|
+
and the marker block in SKILL.md.
|
|
36
|
+
7. Add CHANGELOG entry: [adapter] added <tool> (Path A — opt-in via --tool=<tool>)
|
|
37
|
+
8. Run tests/run.sh and tests/lint.sh — must be green.
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### What's NOT touched in Path A
|
|
41
|
+
|
|
42
|
+
| File | Action |
|
|
43
|
+
|---|---|
|
|
44
|
+
| `scripts/lib/router.sh` | UNTOUCHED — adapter is reachable via `--tool=<name>` only |
|
|
45
|
+
| `scripts/lib/common.sh` | UNTOUCHED |
|
|
46
|
+
| `scripts/lib/output.sh` | UNTOUCHED |
|
|
47
|
+
| `scripts/browser-doctor.sh` | UNTOUCHED — doctor walks `lib/tool/*.sh` automatically |
|
|
48
|
+
| `scripts/browser-<verb>.sh` (any) | UNTOUCHED |
|
|
49
|
+
| `references/routing-heuristics.md` | UNTOUCHED |
|
|
50
|
+
|
|
51
|
+
## Path B — Promote to default (run AFTER Path A ships)
|
|
52
|
+
|
|
53
|
+
Once the adapter has been validated via `--tool=<name>` in real workflows, you can promote it to default for one or more verbs.
|
|
54
|
+
|
|
55
|
+
### Checklist
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
1. Edit scripts/lib/router.sh — add a rule_<trigger> function and append it to ROUTING_RULES.
|
|
59
|
+
2. Update references/routing-heuristics.md — add a row matching the rule.
|
|
60
|
+
3. Update tests/router.bats — one positive + one negative case for the new rule.
|
|
61
|
+
4. Add CHANGELOG entry: [adapter] promoted <tool> to default for <trigger>.
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## File-by-file: what every contributor sees
|
|
65
|
+
|
|
66
|
+
| File | Path A | Path B |
|
|
67
|
+
|---|---|---|
|
|
68
|
+
| `scripts/lib/tool/<tool>.sh` | **CREATE** | (untouched) |
|
|
69
|
+
| `tests/stubs/<tool>` | **CREATE** | (untouched) |
|
|
70
|
+
| `tests/fixtures/<tool>/` | **CREATE** | (untouched) |
|
|
71
|
+
| `tests/<tool>_adapter.bats` | **CREATE** | (untouched) |
|
|
72
|
+
| `references/<tool>-cheatsheet.md` | **CREATE** | (untouched) |
|
|
73
|
+
| `scripts/lib/router.sh` | (untouched) | **EDIT** (one fn + one array append) |
|
|
74
|
+
| `scripts/lib/common.sh` | (untouched) | (untouched) |
|
|
75
|
+
| `scripts/lib/output.sh` | (untouched) | (untouched) |
|
|
76
|
+
| `scripts/browser-doctor.sh` | (untouched) | (untouched) |
|
|
77
|
+
| `scripts/browser-<verb>.sh` | (untouched) | (untouched) |
|
|
78
|
+
| `references/tool-versions.md` | **AUTOGEN** | (autogen) |
|
|
79
|
+
| `SKILL.md` | **AUTOGEN** (between markers) | (untouched) |
|
|
80
|
+
| `references/routing-heuristics.md` | (untouched) | **EDIT** (one row) |
|
|
81
|
+
| `tests/router.bats` | (untouched) | **EDIT** (positive + negative) |
|
|
82
|
+
| `CHANGELOG.md` | **EDIT** (one line) | **EDIT** (one line) |
|
|
83
|
+
|
|
84
|
+
**Path A is 5 creates + 2 autogen + 1 changelog line, with zero edits to .sh files in core.**
|
|
85
|
+
|
|
86
|
+
## Worked example: adding `puppeteer-via-bridge` in 30 minutes
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
# 1. Scaffold from playwright-cli (similar shape)
|
|
90
|
+
cp scripts/lib/tool/playwright-cli.sh scripts/lib/tool/puppeteer.sh
|
|
91
|
+
|
|
92
|
+
# 2. Edit puppeteer.sh:
|
|
93
|
+
# - sentinel guard: _BROWSER_TOOL_PUPPETEER_LOADED
|
|
94
|
+
# - readonly _BROWSER_TOOL_PUPPETEER_BIN="${PUPPETEER_BIN:-puppeteer}"
|
|
95
|
+
# - tool_metadata.name = "puppeteer", cheatsheet_path = "references/puppeteer-cheatsheet.md"
|
|
96
|
+
# - tool_capabilities: declare what puppeteer actually supports
|
|
97
|
+
# - tool_doctor_check: command -v puppeteer; install hint = "npm i -g puppeteer"
|
|
98
|
+
# - tool_open / tool_click / tool_fill / tool_snapshot / tool_inspect: shell to puppeteer
|
|
99
|
+
# - tool_audit / tool_extract / tool_eval: return 41
|
|
100
|
+
|
|
101
|
+
# 3. Stub + fixtures (copy & adapt)
|
|
102
|
+
cp tests/stubs/playwright-cli tests/stubs/puppeteer
|
|
103
|
+
mkdir tests/fixtures/puppeteer
|
|
104
|
+
|
|
105
|
+
# 4. Test (copy & adapt)
|
|
106
|
+
cp tests/playwright-cli_adapter.bats tests/puppeteer_adapter.bats
|
|
107
|
+
sed -i.bak 's/playwright-cli/puppeteer/g' tests/puppeteer_adapter.bats
|
|
108
|
+
rm tests/puppeteer_adapter.bats.bak
|
|
109
|
+
|
|
110
|
+
# 5. Cheatsheet
|
|
111
|
+
cp references/playwright-cli-cheatsheet.md references/puppeteer-cheatsheet.md
|
|
112
|
+
# (edit the cheatsheet content for puppeteer specifics)
|
|
113
|
+
|
|
114
|
+
# 6. Regen autogen docs
|
|
115
|
+
scripts/regenerate-docs.sh all
|
|
116
|
+
|
|
117
|
+
# 7. Test the lot
|
|
118
|
+
tests/run.sh
|
|
119
|
+
tests/lint.sh
|
|
120
|
+
|
|
121
|
+
# 8. CHANGELOG
|
|
122
|
+
echo "- [adapter] added puppeteer adapter (Path A — opt-in via --tool=puppeteer)" >> CHANGELOG.md
|
|
123
|
+
|
|
124
|
+
# 9. Commit
|
|
125
|
+
git add -A
|
|
126
|
+
git commit -m "feat(tool): puppeteer adapter (Path A — opt-in)"
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
## See also
|
|
130
|
+
|
|
131
|
+
- [Anti-patterns: tool extension](anti-patterns-tool-extension.md) — what NOT to do.
|
|
132
|
+
- [Tool adapter extension model spec](../../docs/superpowers/specs/2026-04-30-tool-adapter-extension-model-design.md) — the *why*.
|
|
133
|
+
- [Token-efficient adapter output spec](../../docs/superpowers/specs/2026-05-01-token-efficient-adapter-output-design.md) — `eN` refs, capture paths, single-line summaries.
|
|
134
|
+
- [Routing heuristics](../routing-heuristics.md) — current precedence table.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Agent-workflow recipes
|
|
2
|
+
|
|
3
|
+
End-to-end command sequences for common browser-automation tasks. Each
|
|
4
|
+
recipe assumes a **fresh `~/.browser-skill/`** (run `./install.sh` first if
|
|
5
|
+
needed) and walks through the full toolchain: site → session → action →
|
|
6
|
+
observation → cache build-up.
|
|
7
|
+
|
|
8
|
+
These are distinct from the **pattern recipes** in the parent directory
|
|
9
|
+
(`../privacy-canary.md`, `../path-security.md`, etc.), which codify
|
|
10
|
+
discipline ("when adding X, do Y, never Z"). Workflow recipes show
|
|
11
|
+
sequenced commands + expected output for actual user-facing tasks.
|
|
12
|
+
|
|
13
|
+
## Index
|
|
14
|
+
|
|
15
|
+
| Recipe | When to read it |
|
|
16
|
+
|---|---|
|
|
17
|
+
| [`login-then-scrape.md`](login-then-scrape.md) | First end-to-end task: register site, capture session, scrape pages. The "hello world" of the skill. |
|
|
18
|
+
| [`incremental-pattern-discovery.md`](incremental-pattern-discovery.md) | Build up the memory cache from a real session. Demonstrates PR #115/#125/#127 loop end-to-end. |
|
|
19
|
+
| [`flow-record-and-replay.md`](flow-record-and-replay.md) | Capture a manual interaction via `flow record`, replay it, diff against a baseline. |
|
|
20
|
+
| [`cache-driven-bulk-operation.md`](cache-driven-bulk-operation.md) | Process 50+ items with zero LLM tokens via the memory cache. The "ROI proof" workflow. |
|
|
21
|
+
|
|
22
|
+
## Convention
|
|
23
|
+
|
|
24
|
+
Each recipe:
|
|
25
|
+
- States the **goal** + **outcome** up front (one sentence each).
|
|
26
|
+
- Lists **prerequisites** (assumes clean `~/.browser-skill/`).
|
|
27
|
+
- Walks **numbered steps** with `bash` commands + abbreviated expected output.
|
|
28
|
+
- Ends with **verification** + **next-step** pointers.
|
|
29
|
+
|
|
30
|
+
Commands are runnable verbatim from any directory with `${CLAUDE_SKILL_DIR}`
|
|
31
|
+
set (or substituting the repo path if running standalone).
|
|
32
|
+
|
|
33
|
+
## When you're ready to ship
|
|
34
|
+
|
|
35
|
+
After working through 1-2 recipes end-to-end, the toolchain's full surface
|
|
36
|
+
is in muscle memory. Subsequent agent sessions can read SKILL.md + the
|
|
37
|
+
verb tables instead of stepping through workflows.
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# Workflow: cache-driven bulk operation
|
|
2
|
+
|
|
3
|
+
**Goal:** process 50+ items via the memory cache. Demonstrate the ROI of the Phase 11 cache + Phase 11 v2 observation log loop.
|
|
4
|
+
|
|
5
|
+
**Outcome:** after a one-time learning phase (~3 actions), 50 subsequent actions dispatch with zero LLM ref-resolution. Token cost drops from ~5K × 50 = 250K to ~200 × 50 = 10K (25× reduction; representative ballpark).
|
|
6
|
+
|
|
7
|
+
## Prerequisites
|
|
8
|
+
|
|
9
|
+
- Site registered + session captured ([`login-then-scrape.md`](login-then-scrape.md) steps 0-3).
|
|
10
|
+
- Patterns + archetypes set up for the target action ([`incremental-pattern-discovery.md`](incremental-pattern-discovery.md) steps 1-4).
|
|
11
|
+
- A list of 50+ URLs that share the same archetype (e.g. `https://app.acme.com/orders/1001` through `/orders/1050`).
|
|
12
|
+
|
|
13
|
+
## Steps
|
|
14
|
+
|
|
15
|
+
### 1. Confirm the cache is primed
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
# Pattern should exist for /orders/:id, archetype orders-id, with at least one
|
|
19
|
+
# intent recorded (e.g. "cancel order").
|
|
20
|
+
jq '.patterns[] | select(.url_pattern == "/orders/:id")' \
|
|
21
|
+
~/.browser-skill/memory/acme/patterns.json
|
|
22
|
+
# → 1 row
|
|
23
|
+
|
|
24
|
+
jq '.interactions[] | select(.intent == "cancel order")' \
|
|
25
|
+
~/.browser-skill/memory/acme/archetypes/orders-id.json
|
|
26
|
+
# → 1 row with success_count >= 1
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
If either is empty, run [`incremental-pattern-discovery.md`](incremental-pattern-discovery.md) first.
|
|
30
|
+
|
|
31
|
+
### 2. Generate the URL list
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
seq 1001 1050 | sed 's|^|https://app.acme.com/orders/|' > /tmp/order-urls.txt
|
|
35
|
+
wc -l /tmp/order-urls.txt
|
|
36
|
+
# → 50 /tmp/order-urls.txt
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### 3. Dispatch in a loop — zero LLM tokens after warm-up
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
bash scripts/browser-use.sh --set acme
|
|
43
|
+
exec 3</tmp/order-urls.txt
|
|
44
|
+
while IFS= read -r url <&3; do
|
|
45
|
+
bash scripts/browser-do.sh \
|
|
46
|
+
--site acme --verb click \
|
|
47
|
+
--intent "cancel order" \
|
|
48
|
+
--url "${url}" \
|
|
49
|
+
--as acme--admin 2>&1 | tail -1 | jq -c '{verb, mode, cache_hit, dispatch_rc, url}'
|
|
50
|
+
done
|
|
51
|
+
exec 3<&-
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Each iteration emits a one-line JSON summary. Watch `cache_hit:true` repeat 50 times.
|
|
55
|
+
|
|
56
|
+
### 4. Confirm ROI signal in doctor
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
bash scripts/browser-doctor.sh | grep "memory cache hit"
|
|
60
|
+
# → memory cache hit rate: 96% (50/52 events)
|
|
61
|
+
# (the 2 misses are the warm-up actions from step 1; hits scale linearly)
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Or query `events.jsonl` directly:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
jq -s '
|
|
68
|
+
{
|
|
69
|
+
total: length,
|
|
70
|
+
hits: ([.[] | select(.cache_hit == true)] | length),
|
|
71
|
+
rate_pct: (([.[] | select(.cache_hit == true)] | length) * 100 / length)
|
|
72
|
+
}
|
|
73
|
+
' ~/.browser-skill/memory/events.jsonl
|
|
74
|
+
# → {total: 52, hits: 50, rate_pct: 96}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Self-heal in motion
|
|
78
|
+
|
|
79
|
+
If the cached selector breaks mid-run (e.g. URL #25 is a different layout):
|
|
80
|
+
|
|
81
|
+
1. The dispatched `browser-click --selector ...` exits 11 (`EXIT_EMPTY_RESULT`).
|
|
82
|
+
2. `browser-do --intent` catches this on the D1 exit-code whitelist + calls `memory_record_failure`.
|
|
83
|
+
3. After 4 such failures, `disabled:true` flips + `self_heal_history[]` gains a `"disabled"` entry.
|
|
84
|
+
4. Subsequent iterations return `cache_miss reason:intent_not_cached` (disabled is indistinguishable from never-cached per D3).
|
|
85
|
+
5. Agent re-resolves the new selector + calls `browser-do record` → `disabled:false` + `"healed"` entry.
|
|
86
|
+
6. Loop resumes with cache hits.
|
|
87
|
+
|
|
88
|
+
Inspect the audit trail:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
jq '.interactions[] | select(.self_heal_history | length > 0)
|
|
92
|
+
| {intent, self_heal_history}' \
|
|
93
|
+
~/.browser-skill/memory/acme/archetypes/orders-id.json
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Numbers worth measuring
|
|
97
|
+
|
|
98
|
+
Before the dogfood loop, the comparison is theoretical. Run the script for 7 days against a real site you actually use. Then:
|
|
99
|
+
|
|
100
|
+
- **Cache hit rate per day:** trends upward as patterns are observed + recorded.
|
|
101
|
+
- **Wall-clock per action:** cache hit ≈ adapter dispatch only (50-200ms); cache miss ≈ full snapshot + agent reasoning (5-20s on cdt-mcp).
|
|
102
|
+
- **Token cost per action:** measure with `claude -p` harness (cache hit ≈ <500 input tokens; miss ≈ 3-10K).
|
|
103
|
+
|
|
104
|
+
The ROI claim ("zero LLM tokens on cache hit") is exact for the skill turn — the parent session may still consume tokens for context that's not skill-related.
|
|
105
|
+
|
|
106
|
+
## Don't
|
|
107
|
+
|
|
108
|
+
- **Don't run with `--auto-record` on an unverified corpus.** A bad heuristic match could pollute `patterns.json` with garbage. Validate the propose output first; auto-record after.
|
|
109
|
+
- **Don't assume cache hit means "did the right thing."** Cache hit means "skipped LLM ref-resolution + dispatched verb." Verb-level success (did the click actually cancel the order?) is the verb's exit code, not the cache lookup's status.
|
|
110
|
+
- **Don't run the loop in parallel against the same site.** `browser-do --intent` is single-shot; concurrent runs race on `memory_record_pattern`'s upsert. Serial is the design (PID-locking only on `browser-migrate`).
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Workflow: flow record and replay
|
|
2
|
+
|
|
3
|
+
**Goal:** capture a manual multi-step interaction via `playwright codegen`, replay it, diff against a baseline.
|
|
4
|
+
|
|
5
|
+
**Outcome:** a `.flow.yaml` file describes a deterministic action sequence; `replay` re-executes it + emits a per-step diff against a captured baseline.
|
|
6
|
+
|
|
7
|
+
## Prerequisites
|
|
8
|
+
|
|
9
|
+
- Site registered + session captured (run [`login-then-scrape.md`](login-then-scrape.md) steps 0-3 first).
|
|
10
|
+
- `playwright` installed (`npm i -g playwright @playwright/test && playwright install chromium`).
|
|
11
|
+
- A multi-step task in mind — e.g. "create a new task, assign to me, set priority high."
|
|
12
|
+
|
|
13
|
+
## Steps
|
|
14
|
+
|
|
15
|
+
### 1. Record the flow
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
bash scripts/browser-flow.sh record \
|
|
19
|
+
--site acme --as acme--admin \
|
|
20
|
+
--out create-task.flow.yaml
|
|
21
|
+
# → opens a browser via `playwright codegen`
|
|
22
|
+
# → perform the multi-step task in the real UI
|
|
23
|
+
# → close the browser; the regex-based JS→YAML mapper emits the .flow.yaml
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Password-canary write-side fires automatically (PR for Phase 9 part 1-iii): any field matching `/password/i` becomes `${secrets.password}` placeholder; the literal is dropped from the YAML.
|
|
27
|
+
|
|
28
|
+
Inspect the result:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
cat create-task.flow.yaml
|
|
32
|
+
# steps:
|
|
33
|
+
# - open: { url: "https://app.acme.com/tasks/new" }
|
|
34
|
+
# - fill: { ref: e3, text: "Ship Pick D recipes" }
|
|
35
|
+
# - click: { ref: e7 }
|
|
36
|
+
# - select: { ref: e12, value: "high" }
|
|
37
|
+
# - click: { ref: e15 }
|
|
38
|
+
# - assert: { selector: ".toast-success", text-contains: "Created" }
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
`${refs.NAME}` and `${var}` templating are available; see `docs/superpowers/specs/2026-05-10-phase-09-flow-runner-design.md` for the full schema.
|
|
42
|
+
|
|
43
|
+
### 2. Run the flow with whole-flow capture
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
bash scripts/browser-flow.sh run create-task.flow.yaml --capture
|
|
47
|
+
# → executes each step; emits one _kind:step JSON event per step
|
|
48
|
+
# → on success: summary line + capture_id NNN
|
|
49
|
+
# → captures all per-step events + final state to ~/.browser-skill/captures/NNN/
|
|
50
|
+
|
|
51
|
+
# Capture id is in the summary; e.g. capture_id: 042
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
### 3. Mark this run as a baseline
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
bash scripts/browser-flow.sh baseline save 042 --as after-redesign
|
|
58
|
+
# → ~/.browser-skill/baselines.json gains entry
|
|
59
|
+
# → captures/042/meta.json gets is_baseline:true (skip-rule honored by prune)
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### 4. Replay later + diff against baseline
|
|
63
|
+
|
|
64
|
+
Time passes. Site changes. Re-run + diff:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
bash scripts/browser-flow.sh run create-task.flow.yaml --capture
|
|
68
|
+
# → new capture_id 043
|
|
69
|
+
|
|
70
|
+
bash scripts/browser-replay.sh 043 --strict
|
|
71
|
+
# → per-step replay_diff events
|
|
72
|
+
# → exit 13 (ASSERTION_FAILED) on first divergent step under --strict;
|
|
73
|
+
# exit 0 with non-zero diff count under default mode
|
|
74
|
+
|
|
75
|
+
bash scripts/browser-flow.sh history diff 042 043
|
|
76
|
+
# → structured per-step diff, with duration_ms stripped before compare
|
|
77
|
+
# (Phase 9 part 1-iv strip-timing-from-semantic-comparison pattern)
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The diff stripping ensures timing-sensitive fields don't pollute the comparison — only semantic differences surface.
|
|
81
|
+
|
|
82
|
+
## Verification
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
bash scripts/browser-flow.sh history list --limit 5
|
|
86
|
+
# → newest-first table; 043 above 042; both with their summary fields
|
|
87
|
+
|
|
88
|
+
bash scripts/browser-flow.sh baseline list
|
|
89
|
+
# → after-redesign → capture 042
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Variations
|
|
93
|
+
|
|
94
|
+
- **Re-record after schema change:** the YAML's `${refs.NAME}` references break when the page restructures. Re-record, re-baseline.
|
|
95
|
+
- **Cross-environment replay:** capture against staging, replay against prod — drift the URL via `--var base=https://prod.example.com` if the flow uses `${base}` templating.
|
|
96
|
+
- **Multi-baseline:** keep one baseline per release (`baseline save NNN --as v1.2.3`); compare any new capture against any historical baseline.
|
|
97
|
+
|
|
98
|
+
## Don't
|
|
99
|
+
|
|
100
|
+
- **Don't commit `.flow.yaml` files with real secrets in them.** The recorder strips `/password/i` fields, but other sensitive content (API tokens in URL query strings, PII in text values) survives. Inspect every recorded YAML before committing.
|
|
101
|
+
- **Don't `baseline remove` a capture you still need to diff against.** The removal is a typed-phrase-confirmed delete; gone for good.
|
|
102
|
+
- **Don't expect deterministic replay across browser versions.** Playwright codegen-generated refs (`e3`, `e7`, etc.) are snapshot-relative; a major Chromium update may rearrange the accessibility tree. Re-record on upgrade.
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
# Workflow: incremental pattern discovery
|
|
2
|
+
|
|
3
|
+
**Goal:** demonstrate the full passive-observation → propose → cache-hit loop.
|
|
4
|
+
|
|
5
|
+
**Outcome:** after a few sessions of normal navigation, the skill auto-clusters visited URLs into patterns, stores them in `patterns.json`, and starts serving cache hits on subsequent agent actions.
|
|
6
|
+
|
|
7
|
+
## Prerequisites
|
|
8
|
+
|
|
9
|
+
- Site already registered (run [`login-then-scrape.md`](login-then-scrape.md) steps 0-3 if not).
|
|
10
|
+
- At least one cached archetype (or willingness to record one in step 3 below).
|
|
11
|
+
|
|
12
|
+
## The full loop in 5 steps
|
|
13
|
+
|
|
14
|
+
### 1. Navigate (passive observation writes `recent_urls.jsonl`)
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
bash scripts/browser-use.sh --set acme
|
|
18
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/orders/1001'
|
|
19
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/orders/1002'
|
|
20
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/orders/1003'
|
|
21
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/customers/jane-doe'
|
|
22
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/customers/john-roe'
|
|
23
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/customers/jim-roe'
|
|
24
|
+
|
|
25
|
+
cat ~/.browser-skill/memory/recent_urls.jsonl
|
|
26
|
+
# → 6 rows; each {ts, url, verb:"open", site:"acme", schema_version:1}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The tee is automatic on successful `browser-open` (PR #125 Pick A6).
|
|
30
|
+
|
|
31
|
+
### 2. Propose patterns from the observation log
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
bash scripts/browser-do.sh propose --site acme --from-recent
|
|
35
|
+
# → 2 _kind:proposal events:
|
|
36
|
+
# {url_pattern:"/orders/:id", archetype_id:"orders-id", count:3}
|
|
37
|
+
# {url_pattern:"/customers/:slug", archetype_id:"customers-slug", count:3}
|
|
38
|
+
# → summary: proposals:2 auto_recorded:0 skipped_known:0
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
`:id` from PR #97 (11-2-ii numeric heuristic); `:slug` from PR #127 (Pick A2). Neither row written to `patterns.json` yet — `propose` is read-only by default (C5 invariant from 11-2-ii).
|
|
42
|
+
|
|
43
|
+
### 3. Auto-record the proposals
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
bash scripts/browser-do.sh propose --site acme --from-recent --auto-record
|
|
47
|
+
# → same 2 proposals, but now auto_recorded:2 in summary
|
|
48
|
+
# → patterns.json gains 2 rows
|
|
49
|
+
|
|
50
|
+
cat ~/.browser-skill/memory/acme/patterns.json | jq '.patterns'
|
|
51
|
+
# → [
|
|
52
|
+
# {url_pattern:"/orders/:id", archetype_id:"orders-id", hit_count:1, ...},
|
|
53
|
+
# {url_pattern:"/customers/:slug", archetype_id:"customers-slug", hit_count:1, ...}
|
|
54
|
+
# ]
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Re-running with `--auto-record` is idempotent (filter-before-write pattern from PR #121).
|
|
58
|
+
|
|
59
|
+
### 4. Record an intent against the archetype
|
|
60
|
+
|
|
61
|
+
Need a cached selector for an action. Snapshot a real page + pick a ref:
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
bash scripts/browser-open.sh --url 'https://app.acme.com/orders/1001'
|
|
65
|
+
bash scripts/browser-snapshot.sh
|
|
66
|
+
# → eN list; find the "cancel order" button's ref, e.g. e12
|
|
67
|
+
|
|
68
|
+
# Record the intent → selector binding. browser-do record auto-derives
|
|
69
|
+
# the archetype-id from the URL pattern.
|
|
70
|
+
bash scripts/browser-do.sh record \
|
|
71
|
+
--site acme \
|
|
72
|
+
--intent "cancel order" \
|
|
73
|
+
--selector 'button[data-action="cancel"]' \
|
|
74
|
+
--url 'https://app.acme.com/orders/1001'
|
|
75
|
+
# → patterns.json + archetypes/orders-id.json updated
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### 5. Cache hit dispatch (zero LLM tokens)
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
bash scripts/browser-do.sh \
|
|
82
|
+
--site acme --verb click \
|
|
83
|
+
--intent "cancel order" \
|
|
84
|
+
--url 'https://app.acme.com/orders/1002'
|
|
85
|
+
# → cache_hit:true; dispatches click with --selector button[data-action="cancel"]
|
|
86
|
+
# → no snapshot needed; no LLM ref-resolution
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Same intent across different URLs of the same archetype → all served by one cached selector.
|
|
90
|
+
|
|
91
|
+
## Observation
|
|
92
|
+
|
|
93
|
+
The cache hit + write events both tee into `events.jsonl` (PR #115 Pick A1):
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
jq -c 'select(.cache_hit==true)' ~/.browser-skill/memory/events.jsonl | wc -l
|
|
97
|
+
# → count of cache hits across all browser-do --intent runs
|
|
98
|
+
|
|
99
|
+
bash scripts/browser-doctor.sh | grep "memory cache hit"
|
|
100
|
+
# → memory cache hit rate: X% (H/T events)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Run the loop on enough URLs and the hit-rate climbs. Design target: ≥70% after 20+ same-archetype actions.
|
|
104
|
+
|
|
105
|
+
## Self-heal
|
|
106
|
+
|
|
107
|
+
If a selector breaks (4 consecutive failures), `disabled:true` flips in the archetype JSON + an entry appends to `self_heal_history[]` (PR #119 Pick A5):
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
jq '.interactions[] | select(.self_heal_history | length > 0)' \
|
|
111
|
+
~/.browser-skill/memory/acme/archetypes/orders-id.json
|
|
112
|
+
# → array of {ts, event:"disabled"|"healed", fail_count, selector_at_time}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Next `browser-do --intent "cancel order"` returns `cache_miss reason:intent_not_cached` (disabled is indistinguishable from never-cached per D3); agent re-resolves + re-records via step 4, which appends a `"healed"` entry.
|
|
116
|
+
|
|
117
|
+
## Next steps
|
|
118
|
+
|
|
119
|
+
- Run [`cache-driven-bulk-operation.md`](cache-driven-bulk-operation.md) to see ROI at scale.
|
|
120
|
+
- Tweak the slug heuristic (`scripts/lib/node/url-pattern-cluster.mjs`) if your site uses non-standard patterns.
|
|
121
|
+
|
|
122
|
+
## Don't
|
|
123
|
+
|
|
124
|
+
- **Don't manually edit `patterns.json` or archetype JSONs.** Use `browser-do record` / `memory_record_pattern` (the lib API). Hand edits bypass the canonical-pattern compare (PR #123 Pick A4) and may create redundant rows.
|
|
125
|
+
- **Don't auto-record without reviewing.** First-time run on a new site: omit `--auto-record`, read the proposals, decide. `--auto-record` shines after the heuristics have been validated on the site.
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Workflow: login then scrape
|
|
2
|
+
|
|
3
|
+
**Goal:** register a site, capture an authenticated session, scrape multiple URLs from it.
|
|
4
|
+
|
|
5
|
+
**Outcome:** a single `obscura` adapter call returns JSON for N URLs, all using the same logged-in cookie state from a Playwright `storageState`. Zero LLM tokens after step 4.
|
|
6
|
+
|
|
7
|
+
## Prerequisites
|
|
8
|
+
|
|
9
|
+
- Clean `~/.browser-skill/` (or skip step 0 if already installed).
|
|
10
|
+
- At least one adapter installed (`chrome-devtools-mcp` recommended for login capture; `obscura` for bulk scrape).
|
|
11
|
+
- The target site has username+password login (TOTP optional).
|
|
12
|
+
|
|
13
|
+
## Steps
|
|
14
|
+
|
|
15
|
+
### 0. Install (one-time)
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
git clone https://github.com/xicv/browser-automation-skill ~/Projects/browser-automation-skill
|
|
19
|
+
cd ~/Projects/browser-automation-skill
|
|
20
|
+
./install.sh --with-hooks # --with-hooks installs the credential-leak pre-commit blocker
|
|
21
|
+
bash scripts/browser-doctor.sh
|
|
22
|
+
# → expect: "ok: all checks passed (4 adapter(s) ok)" if all 4 adapters installed
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
### 1. Register the site
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
bash scripts/browser-add-site.sh --name acme --url 'https://app.acme.com'
|
|
29
|
+
bash scripts/browser-use.sh --set acme
|
|
30
|
+
# → sticky "current site" set; subsequent verbs can omit --site
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### 2. Store credentials (stdin-only; never on argv)
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
printf '%s' 'your-password' | bash scripts/browser-creds-add.sh \
|
|
37
|
+
--site acme --as acme--admin \
|
|
38
|
+
--password-stdin \
|
|
39
|
+
--auth-flow single-step-username-password
|
|
40
|
+
# → keychain (macOS) or libsecret (Linux) by default; plaintext requires typed-phrase
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
If TOTP is required:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
printf '%s' 'BASE32SECRET' | bash scripts/browser-creds-add.sh \
|
|
47
|
+
--site acme --as acme--admin \
|
|
48
|
+
--totp-secret-stdin \
|
|
49
|
+
--auth-flow single-step-username-password-with-totp
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
### 3. Interactive login (one-time; captures `storageState`)
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
bash scripts/browser-login.sh \
|
|
56
|
+
--site acme --as acme--admin \
|
|
57
|
+
--interactive
|
|
58
|
+
# → opens a real browser; fill the login form; press Enter in the terminal
|
|
59
|
+
# → captures cookies + localStorage into ~/.browser-skill/sessions/acme--admin.json
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
After this, all subsequent verbs can pass `--as acme--admin` to reuse the session.
|
|
63
|
+
|
|
64
|
+
### 4. Bulk scrape
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
bash scripts/browser-extract.sh --site acme --as acme--admin \
|
|
68
|
+
--scrape \
|
|
69
|
+
'https://app.acme.com/orders/1001' \
|
|
70
|
+
'https://app.acme.com/orders/1002' \
|
|
71
|
+
'https://app.acme.com/orders/1003' \
|
|
72
|
+
--eval 'document.querySelector(".order-total")?.textContent' \
|
|
73
|
+
--format json
|
|
74
|
+
# → streams 3 JSON events (one per URL) + a summary line
|
|
75
|
+
# → routed to obscura adapter via the rule_scrape_flag router rule
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Verification
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
# Confirm session was captured + is mode 0600.
|
|
82
|
+
bash scripts/browser-list-sessions.sh --site acme
|
|
83
|
+
# → table includes acme--admin
|
|
84
|
+
|
|
85
|
+
# Confirm session file mode.
|
|
86
|
+
ls -la ~/.browser-skill/sessions/
|
|
87
|
+
# → expect: -rw------- (mode 0600) on acme--admin.json
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## Next steps
|
|
91
|
+
|
|
92
|
+
- Want to scrape 50+ URLs? See [`cache-driven-bulk-operation.md`](cache-driven-bulk-operation.md).
|
|
93
|
+
- Want to record + replay a multi-step interaction? See [`flow-record-and-replay.md`](flow-record-and-replay.md).
|
|
94
|
+
- The login flow auto-detects 2FA prompts; for non-interactive re-login, see `browser-login.sh --help`.
|
|
95
|
+
|
|
96
|
+
## Don't
|
|
97
|
+
|
|
98
|
+
- **Don't pass `--password` or `--totp` on argv.** Always use `--password-stdin` / `--totp-secret-stdin`. The pre-commit hook + tests/argv_leak.bats enforce this; bypassing leaks secrets into `ps`, shell history, and the Claude transcript.
|
|
99
|
+
- **Don't commit `~/.browser-skill/`** to git. `.gitignore` blocks the pattern; verify with `git check-ignore ~/.browser-skill/credentials/foo.json`.
|
|
100
|
+
- **Don't store credentials with `--backend plaintext`** unless the typed-phrase confirms understanding. Keychain (macOS) / libsecret (Linux) is the default — let it stay default.
|