@automatebrowser/mcp 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +17 -0
- package/.claude-plugin/plugin.json +10 -0
- package/CHANGELOG.md +167 -0
- package/LICENSE +202 -0
- package/README.md +2092 -0
- package/dist/chunk-R5JREBXQ.js +211 -0
- package/dist/chunk-WAXVZKT5.js +5363 -0
- package/dist/cli.js +135 -0
- package/dist/index.js +287 -0
- package/dist/relay.js +919 -0
- package/package.json +80 -0
- package/skills/automate-browser/SKILL.md +181 -0
- package/skills/automate-browser/references/capture-and-diagnostics.md +486 -0
- package/skills/automate-browser/references/page-interaction.md +237 -0
- package/skills/automate-browser/references/reading-and-extraction.md +113 -0
- package/skills/automate-browser/references/sessions-and-state.md +186 -0
- package/skills/automate-browser/references/tabs-and-multi-agent.md +156 -0
- package/skills/automate-browser/references/tool-reference.md +236 -0
- package/skills/automate-browser/references/troubleshooting.md +136 -0
|
@@ -0,0 +1,237 @@
|
|
|
1
|
+
# Interacting with a page
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
| Section | What it answers |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [Refs, not selectors](#refs-not-selectors) | How you address an element |
|
|
8
|
+
| [Finding an element cheaply](#finding-an-element-cheaply) | `browser_find` vs a full snapshot |
|
|
9
|
+
| [The interaction tools](#the-interaction-tools) | click, type, hover, drag, select, clear, scroll |
|
|
10
|
+
| [Filling a form](#filling-a-form-in-one-call) | `browser_fill_form` |
|
|
11
|
+
| [Clicking what a snapshot cannot name](#clicking-what-a-snapshot-cannot-name) | Canvases, maps, PDFs |
|
|
12
|
+
| [Waiting](#waiting-for-the-page-to-catch-up) | `wait_for` vs `wait` vs settle options |
|
|
13
|
+
| [Actionability](#actionability-why-a-click-refuses) | Why a click refused before running |
|
|
14
|
+
| [Stale refs](#when-a-ref-goes-stale) | Recovering from a re-render |
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Refs, not selectors
|
|
19
|
+
|
|
20
|
+
Every interaction addresses an element by a **`ref`** handle plus a human-readable `element`
|
|
21
|
+
description, never a raw CSS selector:
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
browser_click { element: "the Sign in button", ref: "e17" }
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Refs come from `browser_snapshot` or `browser_find`. The `element` string is not decoration — it is
|
|
28
|
+
what appears in the error if the click refuses, and what the user sees in the audit log. Describe the
|
|
29
|
+
thing as a person would.
|
|
30
|
+
|
|
31
|
+
## Finding an element cheaply
|
|
32
|
+
|
|
33
|
+
`browser_snapshot` is the **map, not the data**. It returns the page's interactive elements with
|
|
34
|
+
their refs. Read it once to learn the page's shape, then stop — it is the most expensive call in the
|
|
35
|
+
set.
|
|
36
|
+
|
|
37
|
+
When you already know what you are after, `browser_find` is far smaller:
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
browser_find { text: "Add to basket" }
|
|
41
|
+
browser_find { role: "button", max: 5 }
|
|
42
|
+
browser_find { selector: "form#checkout input" }
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
It returns matching elements with refs you can act on immediately. Reach for a full snapshot only
|
|
46
|
+
when you genuinely need to survey an unfamiliar page.
|
|
47
|
+
|
|
48
|
+
`browser_snapshot { verbose: true }` returns the fuller tree; `{ filePath }` writes it to disk instead
|
|
49
|
+
of into your context.
|
|
50
|
+
|
|
51
|
+
## The interaction tools
|
|
52
|
+
|
|
53
|
+
| Tool | Required | Notes |
|
|
54
|
+
|---|---|---|
|
|
55
|
+
| `browser_click` | `element`, `ref` — or `x`, `y` | `dblClick: true` for double-click |
|
|
56
|
+
| `browser_type` | `element`, `ref`, `text`, `submit` | `submit` is required — say whether to press Enter |
|
|
57
|
+
| `browser_hover` | `element`, `ref` | For menus that open on hover |
|
|
58
|
+
| `browser_select_option` | `element`, `ref`, `values` | `values` is an array, even for one |
|
|
59
|
+
| `browser_drag` | `startElement`, `startRef`, `endElement`, `endRef` | Both ends need a description |
|
|
60
|
+
| `browser_clear` | `ref` | Empties an input properly, better than typing over |
|
|
61
|
+
| `browser_press_key` | `key` | `"Enter"`, `"Control+A"`, `"Escape"` — no ref, goes to the page |
|
|
62
|
+
| `browser_scroll` | — | `{ ref }` scrolls to an element, or `{ dx, dy }`, or `{ to: "bottom" }` |
|
|
63
|
+
|
|
64
|
+
## Filling a form in one call
|
|
65
|
+
|
|
66
|
+
Do not click-and-type field by field. `browser_fill_form` takes them all at once and is dramatically
|
|
67
|
+
faster and less brittle:
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
browser_fill_form { fields: [
|
|
71
|
+
{ element: "email", ref: "e3", value: "a@b.com" },
|
|
72
|
+
{ element: "country", ref: "e7", value: "United Kingdom" }
|
|
73
|
+
] }
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
It handles text inputs, selects, checkboxes and radios. Submit separately, so a failed fill never
|
|
77
|
+
half-submits a form.
|
|
78
|
+
|
|
79
|
+
**Read the count, not the error flag.** A fill where some fields landed and some did not is neither
|
|
80
|
+
a success nor a failure, and the flag alone cannot say so. It returns `outcome: "partial"`, the
|
|
81
|
+
`Filled 2/3 field(s)` line with a `✗ ref: reason` for each field that refused, and the same thing
|
|
82
|
+
structured as `{ filled, total, errors }`. Only a fill where **nothing** landed sets `isError`.
|
|
83
|
+
Acting on the flag without reading the count is how a form gets submitted a third empty.
|
|
84
|
+
|
|
85
|
+
**A checkbox or radio takes a boolean and nothing else** — `"true"` or `"false"` (also `1`/`0`,
|
|
86
|
+
`on`/`off`, `yes`/`no`, `checked`/`unchecked`). Any other value is refused for those two, as a
|
|
87
|
+
per-field error; the rest of the batch still fills. **Pick one option of a group by setting THAT
|
|
88
|
+
option's own ref to `"true"`** — not by passing the option's label, which is how a `<select>` is
|
|
89
|
+
filled and is refused here. `"false"` on a radio clears it and leaves the group with nothing selected;
|
|
90
|
+
it never promotes a sibling.
|
|
91
|
+
|
|
92
|
+
**A form that spans frames still goes in ONE call.** This is the one tool that may mix prefixes: a
|
|
93
|
+
checkout puts the card number inside an embedded widget and the address in the page around it, and
|
|
94
|
+
splitting that up would defeat the tool. Pass the fields in the order you want them written and that
|
|
95
|
+
is the order they are written, even when the batch crosses back and forth.
|
|
96
|
+
|
|
97
|
+
Three things follow from that, and they are the ones worth relying on:
|
|
98
|
+
|
|
99
|
+
- **Nothing is written until every ref parses.** A bad prefix anywhere refuses the whole call with the
|
|
100
|
+
form untouched, rather than stopping halfway through with no way to tell how far it got.
|
|
101
|
+
- **Failures are per field, and name the ref you passed.** A frame that has gone fails only its own
|
|
102
|
+
fields; the rest of the batch still reports its own result. Read `errors`, not just the count.
|
|
103
|
+
- **Submit is still separate.** Nothing here changes that.
|
|
104
|
+
|
|
105
|
+
`browser_clear` follows the same rules. Both reach a field inside a shadow root, so a design-system
|
|
106
|
+
input with a perfectly good ref is fillable.
|
|
107
|
+
|
|
108
|
+
## Clicking what a snapshot cannot name
|
|
109
|
+
|
|
110
|
+
A canvas, an embedded map, a PDF viewer — nothing in the accessibility tree to point at. Click by
|
|
111
|
+
coordinate instead:
|
|
112
|
+
|
|
113
|
+
```
|
|
114
|
+
browser_click { x: 420, y: 310 }
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Coordinates are viewport pixels. The reply names what was actually under the point, so you can tell a
|
|
118
|
+
hit from a miss. Pass **either** a ref or coordinates, never both; giving both is refused before
|
|
119
|
+
anything runs.
|
|
120
|
+
|
|
121
|
+
## Waiting for the page to catch up
|
|
122
|
+
|
|
123
|
+
In order of preference:
|
|
124
|
+
|
|
125
|
+
1. **`include: "snapshot"`** on the action itself — the fresh page comes back in the same reply, with
|
|
126
|
+
no second call. On `browser_navigate`, `browser_click` and `browser_type` **only**; elsewhere it is
|
|
127
|
+
rejected. The plain `includeSnapshot: true` reaches further — those three plus `browser_hover`,
|
|
128
|
+
`browser_select_option` and `browser_drag`, but **not** `press_key`, `fill_form`, `clear` or
|
|
129
|
+
`scroll`.
|
|
130
|
+
2. **`browser_wait_for`** — waits for a real condition: `{ selector }`, `{ text }`, `{ urlPattern }`,
|
|
131
|
+
or `{ state }`. This is the right tool when a site swaps content without navigating.
|
|
132
|
+
3. **`settleMs` / `waitUntil`** on the action — tune how long it waits for the page to go quiet after
|
|
133
|
+
acting.
|
|
134
|
+
4. **`browser_wait { time }`** — a blind sleep. Last resort. It is either too short and flaky or too
|
|
135
|
+
long and slow, and usually both on different days.
|
|
136
|
+
|
|
137
|
+
### The defaults these three already have
|
|
138
|
+
|
|
139
|
+
They are **not** in the tool schemas, deliberately — a sentence in a schema is re-sent on every
|
|
140
|
+
request, and saying these six times over cost 10% of the entire tool budget. Read them here, once:
|
|
141
|
+
|
|
142
|
+
| | Navigation (`browser_navigate`, back/forward) | Interactions (click, type, hover, …) |
|
|
143
|
+
|---|---|---|
|
|
144
|
+
| `includeSnapshot` | **true** — a snapshot comes back unasked | **false** — the reply stays lean |
|
|
145
|
+
| `waitUntil` | **`load`** | **`auto`** |
|
|
146
|
+
| `settleMs` | 15 s cap for navigate, 10 s for back/forward | **2000** |
|
|
147
|
+
|
|
148
|
+
So you rarely need to set any of them. Set `includeSnapshot: false` on a navigation whose page you
|
|
149
|
+
are not about to touch; set `waitUntil: "none"` when you want the reply immediately and will wait for
|
|
150
|
+
a real condition yourself; and prefer an explicit `browser_snapshot` when what you actually want is
|
|
151
|
+
fresh refs, or `browser_eval` when one value would answer the question.
|
|
152
|
+
|
|
153
|
+
### Reading a navigation's `settled`
|
|
154
|
+
|
|
155
|
+
`settled: true` means **the navigation you asked for** finished — not that the tab says "complete",
|
|
156
|
+
which straight after a reload still describes the page you are leaving. Three things follow:
|
|
157
|
+
|
|
158
|
+
- A `#fragment` jump, or a history step that stays inside one document, settles as soon as the url
|
|
159
|
+
changes. No load event is coming for one of those.
|
|
160
|
+
- A transition that never starts — a link that turns out to be a download, an unanswered "Leave site?"
|
|
161
|
+
prompt, a forward entry that was not there — gives up after **about a second** with `settled: false`.
|
|
162
|
+
The tab is asked before that is believed, so a navigation the browser is merely slow to begin gets
|
|
163
|
+
its full budget: Chrome re-attempts a page it has refused before after roughly **3 seconds**, and
|
|
164
|
+
that used to come back as a page that never loaded.
|
|
165
|
+
- **`settled: false` is not a failure.** It means the load was not seen to finish inside what you
|
|
166
|
+
allowed. Read `urlAfter` and `navigated` for what actually happened, then take a fresh snapshot
|
|
167
|
+
before you use any ref.
|
|
168
|
+
|
|
169
|
+
`settleMs` only ever shortens the wait (15 s for `browser_navigate`, 10 s for back/forward). The one
|
|
170
|
+
exception to `waitUntil: "none"` returning immediately is `initScript`: the script has to stay
|
|
171
|
+
installed until the new document is built, so that combination waits for the page to commit.
|
|
172
|
+
|
|
173
|
+
### When a navigation did not happen at all
|
|
174
|
+
|
|
175
|
+
`browser_navigate` no longer claims success regardless. If the tab is not where you asked it to go,
|
|
176
|
+
the reply opens with `Did NOT reach <url> — after 2.0s the tab was still on <old url>`, and that line
|
|
177
|
+
appears **inside the snapshot reply too**, at the top. Believe it: the snapshot underneath it is the
|
|
178
|
+
OLD page, and every ref in it belongs to that page. Do not act on them as though you had arrived.
|
|
179
|
+
|
|
180
|
+
Read it as an observation over a window, not a verdict. The server re-asks the tab for up to 2 s
|
|
181
|
+
before saying anything, because a page the browser is slow to commit lands roughly 700 ms late, and
|
|
182
|
+
that wait is paid only when the first answer already said nothing had moved. It is a race against a
|
|
183
|
+
still-moving browser, so at the boundary it can warn about a page that arrives a moment later, or stay
|
|
184
|
+
quiet about one that bounces back — 14 of 16 real navigations were reported correctly.
|
|
185
|
+
|
|
186
|
+
It stays silent wherever an unchanged url is correct: a reload, `waitUntil: "none"`, a navigation to
|
|
187
|
+
the page already open, and a redirect that lands somewhere other than the url you typed. So the line
|
|
188
|
+
appearing means something went wrong; its absence is not a guarantee that nothing did.
|
|
189
|
+
|
|
190
|
+
## Actionability: why a click refuses
|
|
191
|
+
|
|
192
|
+
Every interaction waits for the element to be genuinely ready — visible, enabled, not moving, and not
|
|
193
|
+
covered by something else — before acting. A refusal names **which check failed and what was in the
|
|
194
|
+
way**:
|
|
195
|
+
|
|
196
|
+
> `NOT_ACTIONABLE: "Submit" is covered by "Cookie consent banner"`
|
|
197
|
+
|
|
198
|
+
That is usually the real bug, not a timing problem. Dismiss the overlay rather than retrying or
|
|
199
|
+
padding the wait.
|
|
200
|
+
|
|
201
|
+
**The one exception, and it will catch you: an element that fades in with a CSS transition never
|
|
202
|
+
becomes visible in your background tab.** Chrome does not advance transitions in a tab it is not
|
|
203
|
+
drawing, so the opacity stays at its starting value indefinitely — the script sets the target, the
|
|
204
|
+
animation never runs, and you get `failed the "visible" check` no matter how long you wait. Measured
|
|
205
|
+
2026-09-01. Padding the timeout cannot help.
|
|
206
|
+
|
|
207
|
+
When a click refuses as invisible and the element is one a human would see appear — a modal, a
|
|
208
|
+
dropdown, a toast, anything revealed on interaction — that is this, not a slow page. Either drive the
|
|
209
|
+
element's final state directly (`browser_eval` to read it, or act on what the fade reveals), or
|
|
210
|
+
`browser_switch_tab` to bring the tab forward, accepting that you are taking the user's focus.
|
|
211
|
+
|
|
212
|
+
## When a ref goes stale
|
|
213
|
+
|
|
214
|
+
Refs survive small re-renders and recover themselves once if the page swapped the element out
|
|
215
|
+
underneath you. When one truly cannot be found you get `STALE_REF`, and the recovery is always the
|
|
216
|
+
same: take a fresh `browser_find` or `browser_snapshot` and use the new ref.
|
|
217
|
+
|
|
218
|
+
**Do not cache refs across a navigation.** A new document means new refs, always.
|
|
219
|
+
|
|
220
|
+
**Frames:** an embedded widget is a separate document. **Every** frame's refs are prefixed `f1:`,
|
|
221
|
+
`f2:` and so on — same-origin ones too, and a `srcdoc` frame as well. Pass them through unchanged;
|
|
222
|
+
they work like any other ref. In the page tree an `<iframe>` shows as a bare `- iframe` marker and its
|
|
223
|
+
contents appear below under their own `- frame <url>` heading, never inline.
|
|
224
|
+
|
|
225
|
+
**But one INTERACTION acts inside one frame.** For `browser_click`, `browser_hover`, `browser_type`,
|
|
226
|
+
`browser_select_option`, `browser_drag` and `browser_eval`, every ref in the call has to carry the
|
|
227
|
+
same prefix, and **no prefix means the top page** — it is not a wildcard that joins whatever frame the
|
|
228
|
+
other ref named. `browser_drag { startRef: "f3:e9c4", endRef: "e1a2" }` is a mismatch, not a
|
|
229
|
+
shorthand, and you get `BAD_ARGS` before anything is clicked, typed or dragged. There is no
|
|
230
|
+
cross-frame drag; do the work one frame at a time.
|
|
231
|
+
|
|
232
|
+
`browser_fill_form` and `browser_clear` are the exception and may mix frames freely — see
|
|
233
|
+
[Filling a form in one call](#filling-a-form-in-one-call).
|
|
234
|
+
|
|
235
|
+
Copy a prefix exactly as the snapshot printed it. A half-written one — `f3:` alone, `fx:e1a2` — is
|
|
236
|
+
refused rather than guessed at, because guessing means acting on a same-named element in the wrong
|
|
237
|
+
document and telling you it worked.
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# Reading a page and extracting data
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
| Section | What it answers |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [Pick the cheapest tool](#pick-the-cheapest-tool-that-answers-the-question) | read_page vs find vs get_html vs eval |
|
|
8
|
+
| [The extraction loop](#the-extraction-loop) | The shape of a scraping task |
|
|
9
|
+
| [Extract with a function](#extract-with-a-function-not-an-expression) | Using `browser_eval` properly |
|
|
10
|
+
| [Big results](#big-results-go-to-a-file) | Keeping output out of your context |
|
|
11
|
+
| [Pagination](#pagination) | Walking multiple pages |
|
|
12
|
+
| [What will bite you](#what-will-bite-you) | Late rendering, frames, infinite lists |
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Pick the cheapest tool that answers the question
|
|
17
|
+
|
|
18
|
+
| You want | Use | Why not the others |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| The visible text of the page | `browser_read_page` | Cheapest. Already stripped of markup. `format: "markdown"` keeps headings and links; `maxLength` caps it. |
|
|
21
|
+
| A few elements matching text, role or selector | `browser_find` | Returns `ref`s you can act on, and is far smaller than a snapshot |
|
|
22
|
+
| A table, repeated cards, any structure | `browser_eval` with a **function** | Returns real JSON. Do the mapping in the page, not in your head. |
|
|
23
|
+
| The raw markup of one element | `browser_get_html { ref }` | Whole-page HTML is almost never what you want |
|
|
24
|
+
| The page's interactive shape | `browser_snapshot` | The most expensive call here — for finding things to click, not for reading |
|
|
25
|
+
|
|
26
|
+
`browser_read_page` is the default answer to "what does this page say". Reach past it only when you
|
|
27
|
+
need structure it has thrown away.
|
|
28
|
+
|
|
29
|
+
## The extraction loop
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
browser_navigate → browser_read_page (does the answer just fall out?)
|
|
33
|
+
→ browser_find (locate the container)
|
|
34
|
+
→ browser_eval (function) (map it to JSON)
|
|
35
|
+
→ next page
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Take a snapshot only if you cannot find what you need without one.
|
|
39
|
+
|
|
40
|
+
## Extract with a function, not an expression
|
|
41
|
+
|
|
42
|
+
`browser_eval` takes either an `expression` (a quick one-liner) or a `function` plus `args` of element
|
|
43
|
+
refs. The function form is what you want for anything structured, because a ref passed in `args`
|
|
44
|
+
arrives as a real element:
|
|
45
|
+
|
|
46
|
+
```js
|
|
47
|
+
// browser_eval {
|
|
48
|
+
// function: "(row) => ({ name: row.querySelector('.name').innerText, price: row.querySelector('.price').innerText })",
|
|
49
|
+
// args: ["e4k2"]
|
|
50
|
+
// }
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
For a whole table, one call beats one call per row:
|
|
54
|
+
|
|
55
|
+
```js
|
|
56
|
+
// browser_eval {
|
|
57
|
+
// function: "() => [...document.querySelectorAll('table tbody tr')].map(r => [...r.cells].map(c => c.innerText.trim()))"
|
|
58
|
+
// }
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Return **plain JSON-serialisable values**. DOM nodes do not survive the trip. Passing both
|
|
62
|
+
`expression` and `function` is refused, and `args` without `function` is refused — both before
|
|
63
|
+
anything runs.
|
|
64
|
+
|
|
65
|
+
## Big results go to a file
|
|
66
|
+
|
|
67
|
+
`browser_eval`, `browser_snapshot` and `browser_screenshot` all take `filePath`. A five-thousand-row
|
|
68
|
+
table pasted into your reply costs more than the whole task did. Write it, then read the file with
|
|
69
|
+
your normal file tools.
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
browser_eval { function: "...", filePath: "./out/rows.json" }
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Paths are sandboxed to the folders your editor advertises, plus any in `AUTOMATE_BROWSER_WORKSPACE`,
|
|
76
|
+
plus the temp directory. A refusal names the directories it *would* have accepted, so you do not have
|
|
77
|
+
to guess. Editors that advertise none (Cline, Zed, Windsurf, Gemini CLI, Codex) leave only the
|
|
78
|
+
server's working directory — if that is the refusal you hit, tell the user to set that variable to the
|
|
79
|
+
folder they want written, rather than suggesting they turn the sandbox off. Writing to the temp
|
|
80
|
+
directory always works and needs no configuration at all.
|
|
81
|
+
|
|
82
|
+
## Pagination
|
|
83
|
+
|
|
84
|
+
Prefer the URL over the button when the site puts the page number in the URL — one call instead of
|
|
85
|
+
three, and it cannot get stuck mid-animation:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
browser_navigate { url: ".../results?page=2" }
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Otherwise click and wait for the content to change, not for a fixed delay:
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
browser_click { element: "next page", ref: "e9", include: "snapshot" }
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`include` returns the fresh page in the same reply. Use `browser_wait_for` when the site swaps content
|
|
98
|
+
without navigating.
|
|
99
|
+
|
|
100
|
+
## What will bite you
|
|
101
|
+
|
|
102
|
+
- **A page that renders after load** returns nothing on a first read. `browser_wait_for` a selector or
|
|
103
|
+
some text, then read.
|
|
104
|
+
- **Content behind a scroll** may not be in the DOM at all on an infinite list. Scroll, then read, and
|
|
105
|
+
repeat — do not assume one read got everything.
|
|
106
|
+
- **Refs go stale on a re-render.** You get `STALE_REF` and the tool names the fix. Never cache a ref
|
|
107
|
+
across a navigation.
|
|
108
|
+
- **An embedded widget** — a map, a payment field — is a separate frame, whether or not it shares
|
|
109
|
+
the page's origin. Reads reach inside it. Its contents appear under their own `- frame <url>`
|
|
110
|
+
heading rather than inline, and its refs are prefixed `f1:` and so on. Pass them through unchanged.
|
|
111
|
+
An `<iframe>` in the tree itself is just a marker; look below for its block.
|
|
112
|
+
- **Rate limits and terms of use are yours to respect.** This drives the user's real, logged-in
|
|
113
|
+
browser: whatever you do is done as them, from their address, with their account.
|
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
# Sessions, logins and browser state
|
|
2
|
+
|
|
3
|
+
## Contents
|
|
4
|
+
|
|
5
|
+
| Section | What it answers |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [Start from "already signed in"](#start-from-already-signed-in) | Why you usually do not log in at all |
|
|
8
|
+
| [Credentials](#never-type-credentials-you-were-not-given) | The hard line |
|
|
9
|
+
| [2FA and CAPTCHA](#2fa-captcha-and-consent-screens) | How to stop cleanly |
|
|
10
|
+
| [Cookies and storage](#cookies-and-storage) | Reading and setting session state, and what is redacted |
|
|
11
|
+
| [Dialogs](#dialogs) | `alert`, `confirm`, `prompt`, "Leave site?" |
|
|
12
|
+
| [Uploads and downloads](#uploads-and-downloads) | Files in and out |
|
|
13
|
+
| [Restricted pages](#restricted-pages) | What refuses automation outright |
|
|
14
|
+
| [Signed-in vs signed-out bugs](#when-something-works-signed-in-and-fails-signed-out) | Diagnosing the difference |
|
|
15
|
+
| [Recipe: a session that will not stick](#recipe-it-says-i-am-signed-out--the-cookie-is-not-sticking) | Six calls, cheapest first |
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Start from "already signed in"
|
|
20
|
+
|
|
21
|
+
This drives the user's **real** browser. If they are signed in, **you are signed in** — navigating to
|
|
22
|
+
the page is usually the whole job. Do not go hunting for a login form first; check whether the page
|
|
23
|
+
you want simply loads.
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
browser_navigate { url: "https://app.example.com/settings" }
|
|
27
|
+
browser_read_page
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Landing on a login screen is the *signal* that a session is missing. It is not an invitation to type
|
|
31
|
+
credentials you were never given.
|
|
32
|
+
|
|
33
|
+
## Never type credentials you were not given
|
|
34
|
+
|
|
35
|
+
If a flow needs a password and none was supplied: stop and say so. Do not read one out of a password
|
|
36
|
+
manager, a `.env` file, the page, or the repository. Ask, or hand the step back.
|
|
37
|
+
|
|
38
|
+
## 2FA, CAPTCHA and consent screens
|
|
39
|
+
|
|
40
|
+
A one-time code cannot be produced by you, and a CAPTCHA is a deliberate wall. The right move is a
|
|
41
|
+
clean pause:
|
|
42
|
+
|
|
43
|
+
1. Say exactly what the page is asking for.
|
|
44
|
+
2. Say that the tab is ready and waiting.
|
|
45
|
+
3. Stop, and let the user finish that step themselves.
|
|
46
|
+
4. Resume with `browser_snapshot` when they say they are through.
|
|
47
|
+
|
|
48
|
+
If the tab is one you opened in the background, use `browser_switch_tab` to bring it to them — this is
|
|
49
|
+
precisely the "show me" case that tool exists for.
|
|
50
|
+
|
|
51
|
+
Do not retry a CAPTCHA, and never click "resend code" repeatedly. That locks accounts.
|
|
52
|
+
|
|
53
|
+
## Cookies and storage
|
|
54
|
+
|
|
55
|
+
| Tool | Use |
|
|
56
|
+
|---|---|
|
|
57
|
+
| `browser_get_cookies` | Confirm a session cookie exists for this origin |
|
|
58
|
+
| `browser_set_cookie` | Restore a session you were explicitly given |
|
|
59
|
+
| `browser_storage` | `localStorage` / `sessionStorage` — where single-page apps keep tokens |
|
|
60
|
+
|
|
61
|
+
Values are **redacted by default** in what comes back, because they are the keys to the account.
|
|
62
|
+
`revealValues: true` exists; treat anything you do see as a secret and never write it to a file, a
|
|
63
|
+
commit, or a message.
|
|
64
|
+
|
|
65
|
+
**The same applies to request and response headers.** `browser_get_network_request` returns both
|
|
66
|
+
header sets with the body, and hides the value of `authorization`, `proxy-authorization`, `cookie`
|
|
67
|
+
and `set-cookie`, plus any name containing `token`, `api-key`, `apikey`, `secret`, `password` or
|
|
68
|
+
`credential` — case-insensitively, and the same `revealValues: true` opts out. Names are always
|
|
69
|
+
kept and the result counts what it withheld, so `<redacted>` means "an auth header you cannot see",
|
|
70
|
+
never "no auth header". That distinction is usually the whole answer when a request returns `401`.
|
|
71
|
+
You rarely need the real value: whether the header was **sent** is the bug, not what was in it.
|
|
72
|
+
|
|
73
|
+
Cookies are scoped to the URL of the tab you are driving, so navigate to the origin first.
|
|
74
|
+
|
|
75
|
+
A cookie session is shared across tabs of the same browser profile, so a tab you open is already
|
|
76
|
+
signed in. What is **not** shared is `sessionStorage` — a token kept there dies with its tab.
|
|
77
|
+
|
|
78
|
+
## Dialogs
|
|
79
|
+
|
|
80
|
+
`browser_handle_dialog { action: "accept" | "dismiss", promptText }` answers `alert`, `confirm` and
|
|
81
|
+
`prompt`. It also answers the native "Leave site?" prompt, which page JavaScript cannot even see.
|
|
82
|
+
|
|
83
|
+
This matters more than it sounds: **an unanswered dialog freezes the page**, so anything injected into
|
|
84
|
+
it never runs and your call waits out its entire timeout. A call that hangs and then times out is a
|
|
85
|
+
dialog until proven otherwise.
|
|
86
|
+
|
|
87
|
+
For a navigation you already know will trigger one, arm it in advance — this needs advanced mode:
|
|
88
|
+
|
|
89
|
+
```
|
|
90
|
+
browser_navigate { url: "...", handleBeforeUnload: "accept" }
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
## Uploads and downloads
|
|
94
|
+
|
|
95
|
+
- **`browser_upload_file { ref, filePaths }`** needs advanced mode — call
|
|
96
|
+
`browser_advanced_mode { enable: true }` first, or you get `ADVANCED_MODE_REQUIRED`. The `ref` must
|
|
97
|
+
point at the file input itself.
|
|
98
|
+
- A file input inside a **same-origin** frame or a shadow root works.
|
|
99
|
+
- One inside a **cross-origin** frame (`f3:` prefixed) is **refused**, and says the frame is why.
|
|
100
|
+
The debugger session it drives does not extend into another origin's document. Nothing about the
|
|
101
|
+
ref is wrong, so a fresh snapshot will not help — there is no upload path into that frame.
|
|
102
|
+
- **`browser_downloads`** lists what the browser has downloaded, **with the path on disk**, so you can
|
|
103
|
+
read the file afterwards. `wait: true` blocks until an in-flight transfer finishes rather than
|
|
104
|
+
returning a half-written file.
|
|
105
|
+
|
|
106
|
+
## Restricted pages
|
|
107
|
+
|
|
108
|
+
`chrome://` URLs, the extension store, the PDF viewer and other browser-internal pages refuse
|
|
109
|
+
injection — you get `RESTRICTED_PAGE`. There is no workaround. Open a normal `http(s)` page.
|
|
110
|
+
|
|
111
|
+
## When something works signed in and fails signed out
|
|
112
|
+
|
|
113
|
+
The failure is usually silent — a redirect, or a button that does nothing. Two cheap checks:
|
|
114
|
+
|
|
115
|
+
- `browser_click { ..., include: "console, network" }` — a 401 or 403 shows up in the network block,
|
|
116
|
+
in the same reply as the click.
|
|
117
|
+
- `browser_issues` — the only tool that sees failures producing **no console error at all**: blocked
|
|
118
|
+
third-party cookies, content-security-policy blocks, mixed content, CORS. A dropped third-party
|
|
119
|
+
cookie is a very common cause of "it works when I do it by hand".
|
|
120
|
+
|
|
121
|
+
## Recipe: "it says I am signed out" / "the cookie is not sticking"
|
|
122
|
+
|
|
123
|
+
Six calls, cheapest first. Do not start by setting a cookie — you almost never need to.
|
|
124
|
+
|
|
125
|
+
**1. Be on the origin.** Cookies are scoped to the URL of the tab you are driving, so a
|
|
126
|
+
`browser_get_cookies` from the wrong page is an empty answer that means nothing.
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
browser_navigate { url: "https://app.example.com" }
|
|
130
|
+
browser_get_cookies
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
**2. Is the cookie there at all?** Three different answers, three different bugs:
|
|
134
|
+
|
|
135
|
+
| What you see | What it means |
|
|
136
|
+
|---|---|
|
|
137
|
+
| No cookie for the origin | It was never set, or it was set on a different domain |
|
|
138
|
+
| The cookie is there, page still logged out | The app is not reading it — look at storage, step 3 |
|
|
139
|
+
| The cookie is there, requests still 401 | It is not being *sent* — look at issues, step 4 |
|
|
140
|
+
|
|
141
|
+
Values come back **redacted**. You do not need to reveal them to answer any of the three.
|
|
142
|
+
|
|
143
|
+
**3. Single-page apps usually keep the token somewhere else.**
|
|
144
|
+
|
|
145
|
+
```
|
|
146
|
+
browser_storage
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
`localStorage` survives the tab; **`sessionStorage` dies with it**. A session that works until you
|
|
150
|
+
open a second tab is almost always a `sessionStorage` token.
|
|
151
|
+
|
|
152
|
+
**4. Check what the console cannot see.**
|
|
153
|
+
|
|
154
|
+
```
|
|
155
|
+
browser_issues
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
**A dropped third-party cookie is the single most common cause of "it works when I do it by hand"**,
|
|
159
|
+
and it produces no console error whatsoever. Same for a CSP block on the auth iframe.
|
|
160
|
+
|
|
161
|
+
**5. Catch the status code on the action itself.**
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
browser_click { ref: "e7", element: "Save", include: "console, network" }
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
The 401 or 403 arrives in the same reply as the click, so you never have to guess which request the
|
|
168
|
+
button made.
|
|
169
|
+
|
|
170
|
+
**6. To see what a stranger sees, use a private window — never the user's own session.**
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
browser_new_tab { incognito: true, url: "https://app.example.com" }
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Its own cookie jar, its own empty storage, and closing it ends the session with nothing to clean up.
|
|
177
|
+
This needs the one-off "Allow in Incognito" setting a **person** has to switch on; without it you get
|
|
178
|
+
`INCOGNITO_BLOCKED`. Ask, and do not retry.
|
|
179
|
+
|
|
180
|
+
**Never diagnose by clearing the user's cookies or storage.** That signs them out of a browser they
|
|
181
|
+
are actually using, and it destroys the evidence you were sent to look at.
|
|
182
|
+
|
|
183
|
+
## Leave the session as you found it
|
|
184
|
+
|
|
185
|
+
Do not sign the user out, clear their storage, or revoke sessions to "clean up". You are a guest in
|
|
186
|
+
the browser they actually use.
|